If you're a retention marketer, you've probably had this debate with your team at least once: do we go with a gamified popup — the scratch-off, the spin-to-win, the mystery reveal — or a simpler, preference-based popup that just asks the shopper what they're looking for?
It's a fair question. Gamification feels like it should win. It's fun, it's novel, it taps into the same dopamine loop as a slot machine. But "feels like it should win" and "actually wins" are two different things, and the only way to know which one you're dealing with is to test it.
We test popups like this every day at Justuno — across our own experiments and the thousands of live A/B tests running inside customer accounts — and we like to publish what we find, especially when the result repeats itself as consistently as this one has. We just wrapped another head-to-head: a Shopper Preference popup (an activation style that asks visitors a quick question about what they're shopping for) against a Scratch Off popup (a gamified reveal-a-prize mechanic). Same traffic split, same offer, same list-growth goal. The results landed exactly where they usually do.
The test
Metric | Variant A — Shopper Preference | Variant B — Scratch Off |
|---|---|---|
Sessions | 10,050 | 9,833 |
Result | Winning — +10.93% over baseline | Baseline (lowest performer) |
Email opt-in rate | 8.29% | 7.47% |
Email opt-ins | 684 | 601 |
Email-to-SMS opt-in rate | 68.42% | 63.56% |
SMS opt-ins | 468 | 382 |
Influenced sessions | 1,910 | 1,224 |
Shopper Preference didn't just edge out Scratch Off on the headline number. It led on every metric that matters to a retention marketer: more email opt-ins, a meaningfully higher email-to-SMS carry-through rate, and over 500 more influenced sessions downstream. And this was on a near-identical sample size, so it isn't a traffic-skew artifact — it's a real behavioral difference.
Zoom into the daily trend and the two variants trade places constantly day to day (that's normal, and exactly why you don't call a test after three good days). But look at the shape over the full three-week window: Shopper Preference holds a steadier, more consistent line, while Scratch Off's peaks are followed by deeper valleys. Consistency compounds over a 20,000-session test, and that's what tipped this one.
<img src="/images/omMi9KxADQmZuJ4xewMQTUVE8.png?scale-down-to=2048&width=2844&height=1670">
This isn't a one-off — it's a pattern
Here's the part that actually matters more than any single test: we've run this exact matchup — a preference/interest-based activation against a scratch-off mechanic — many times across many different brands, and the outcome keeps landing the same way. Shopper Preference wins, per usual.
That consistency is worth sitting with. A single test can be noise. A pattern that holds up across different verticals, offers, and audiences is a signal.
Our working theory: a scratch-off asks a visitor to play a game to unlock a discount, which is a great novelty the first time someone sees it and a recognizable gimmick every time after. A shopper preference popup instead does something a gamified popup can't — it collects a real signal about intent ("I'm shopping for myself" vs. "I'm shopping for a gift," for example) while asking for the email. That makes the exchange feel less like "give me your email to win a prize" and more like "help me show you the right thing," which is a lower-friction, higher-trust ask. It also hands your flows a segmentation data point on the way in, which the scratch-off never does.
That trust carries forward, too — it's likely why Shopper Preference's email-to-SMS opt-in rate (68.42%) beat Scratch Off's (63.56%) by nearly 5 points. A visitor who just told you something true about themselves is more willing to hand over a phone number a few seconds later than one who just spun a wheel.
Popup FAQs retention marketers keep asking us
Do gamified popups (scratch-off, spin-to-win, mystery box) actually convert better than a standard opt-in form? Not reliably, based on what we see in testing. Gamified mechanics often produce a short-term novelty lift the first time a segment sees them, but that lift tends to fade with repeat exposure, and in head-to-head tests against preference-based or value-forward popups, gamification frequently comes in second. The only way to know for your audience is to test both.
What's the best type of popup for growing an email list right now? There's no single "best" popup — it depends on your traffic and offer — but in our repeated testing, activation styles that ask the shopper something about themselves (like Shopper Preference) consistently outperform pure-discount or pure-game mechanics. They convert well because the exchange feels personalized rather than transactional.
Does the popup type also affect SMS opt-in rates, or just email? It affects both, and often more on the SMS side than people expect. In this test, the email-to-SMS opt-in rate gap (68.42% vs. 63.56%) was proportionally larger than the email opt-in rate gap. Whatever builds trust at the email step tends to carry through to the SMS ask that follows it.
How long should I run a popup A/B test before trusting the results? Long enough to see the daily leaderboard flip multiple times and still land on a consistent winner. In the chart above, Variant A and Variant B swap the daily lead constantly — if we'd called this test on day 4 or day 9, we'd have picked the wrong winner. Run it until the pattern, not a single good day, points to a winner, and until you've hit a sample size that gives you statistical confidence for your traffic volume.
What's a "good" email opt-in rate benchmark for a popup? It varies widely by industry, traffic source, and offer, but the two variants here (7.47% and 8.29%) sit in a healthy, typical range for an on-site popup with an incentive. If you're seeing opt-in rates well under that, the offer, targeting, or copy is usually the first place to look — before assuming the format itself is the problem.
Should I stop using gamified popups altogether? Not necessarily. Gamification can still work well for specific goals — cart recovery, exit-intent last chances, or reactivating a fatigued list with something novel. The mistake is defaulting to it as your primary list-growth mechanic without testing it against simpler, preference-driven alternatives. Let your own data decide, not the assumption that "fun" always wins.
How do I run a test like this on my own site? Split your popup traffic evenly across the two activation styles, hold every other variable constant (targeting, timing, offer), and let it run until you have enough sessions per variant to trust the daily trend rather than a single spike. This is exactly the kind of test we run for customers inside Justuno every day — if you want to see how your own traffic responds to Shopper Preference vs. a gamified format, it's a test worth putting on your roadmap.
The takeaway
Gamified popups get a lot of attention because they're visually fun and easy to demo. But when we let the data run — 20,000 sessions this time, and the same result the many times before it — preference-based activation keeps winning on email opt-ins, SMS carry-through, and downstream influenced sessions. If your list-growth strategy leans heavily on scratch-offs and spin-to-wins, it's worth running this exact test on your own traffic before assuming the game is doing the job.
We'll keep testing and keep sharing what we find.
