Pop-up & Welcome Flow A/B Test

The Headline

Pop-up A/B went live May 14. New customer acquisition is up sharply while paid spend is down — blended NC CPA dropped 40%.

Blended NC CPA
£59.77
-39.6% vs £99.03
Blended NC ROAS
1.22x
+78.3% vs 0.69x
NC Revenue
£40,045
+182.2% vs £14,191
New Customers
548
+162.2% vs 209
Blended CPA (all)
£11.11
+1.4% vs £10.96
Blended ROAS (all)
4.94x
+1.8% vs 4.86x

Revenue Mix — New vs Returning

Bar width = total revenue, segments = NC (blue) and Returning (grey). Total revenue is up 61.2% — driven by shifting returning revenue — but NC's share of the pie is bigger.

New customer revenueReturning revenue
Test (May 9-9)
NC £40,045
Returning £121,897
£161,942
Prior (Apr 20-8)
NC £14,191
Returning £86,295
£100,487

New Customer Orders by Channel

Triple Whale linearAll. Klaviyo is the story — pop-up + welcome flow are now contributing meaningful NC volume.

Test (2026-05-09 to 2026-06-09)Prior (2026-04-20 to 2026-05-08)
Meta / Facebook
Test
245 orders · £20,059
Prior
90 orders · £6,616
Google Ads
Test
157 orders · £9,958
Prior
63 orders · £4,038
Klaviyo
Test
52 orders · £3,973
Prior
13 orders · £870
Organic / Social
Test
61 orders · £3,804
Prior
28 orders · £1,858
Direct
Test
19 orders · £1,012
Prior
10 orders · £537

NC CPA by Channel

What each paid channel costs per new customer (channel spend ÷ TW-attributed NC orders). Meta and Google held flat; the blended improvement comes from Klaviyo orders being effectively free.

Test — channelTest — blendedPrior
Meta / Facebook
Test
£107.41
Prior
£192.83
Google Ads
Test
£41.00
Prior
£53.05
Blended paid (Meta + Google)
Test
£59.77
Prior
£99.03

Pop-up A/B Test — £5 Off vs Starter Pack

The pop-up is running the same A/B test as the welcome flow: £5 off vs Starter Pack push. Klaviyo is splitting traffic between the two variants across both the sitewide newsletter pop-up and a separate version that targets the Ocean / Sea-Salt product page. The A/B variants went live May 14 (27 days of data). Submit rate = submits ÷ views.

£5 off variantStarter Pack variantPre-test (May 9-13)
Mobile (combined: sitewide + Ocean-page pop-ups)
£5 offsitewide v26062800 + ocean v26062755
8.41% · 456 / 5,425
Starter Packsitewide v26062801 + ocean v26062756
9.53% · 544 / 5,711
Desktop (combined: sitewide + Ocean-page pop-ups)
£5 offsitewide v26062796 + ocean v26062744
6.27% · 70 / 1,117
Starter Packsitewide v26062797 + ocean v26062746
8.69% · 93 / 1,070
Both devices · A/B test totals (May 14-9, 27 days)
£5 offall 4 £5 variants
8.04% · 526 / 6,542
Starter Packall 4 Starter variants
9.39% · 637 / 6,781
Pre-test versions (May 9-13, before A/B went live)
Older variantsv25704807 + v25712685 (mob), v25704810 + v25712759 (desk)
5.24% · 197 / 3,757

The Ocean-page-targeted pop-up (forms YARErD mobile and Rtb8ye desktop) is a separate set of forms that only fires on the Sea Salt / Ocean product page. It runs the same £5 vs Starter A/B and has low traffic on its own, so it is rolled into the device totals above.

Opt-in rate over time — £5 off vs Starter Pack

Daily submit rate (submits ÷ views) per A/B arm, all forms combined. Dashed line marks A/B launch (May 14).

£5 offStarter Pack
0.0% 4.5% 9.1% 13.6% 18.2% A/B live May 9May 14May 19May 24May 29Jun 3Jun 8Jun 9 Opt-in rate Date

Pop-up → New Customer funnel

Qualified views
17,080
Submits
1,360
Klaviyo NC orders (TW)
52
+300% vs 13
Submit → order
3.8%

Welcome Flow A/B — £5 Off vs Starter Pack Push

Flow QVnbAQ: same A/B test as the pop-up. The £5 arm pushes a £5-off welcome discount; the Starter Pack arm pushes the Refillable Starter Pack as the entry product. Each arm has 7 messages (5 emails + 2 SMS). The £5 arm includes an extra "discount expires" reminder pair.

£5 off arm LEADING

£ per entrant£14.80
Entered flow452
Total opens (across flow)853
Total clicks (across flow)189
Conv rate (of entrants)20.6%
Conversions93 · £6,689
Messages7 (5 email + 1 SMS + 1 reminder)

Starter Pack arm

£ per entrant£11.93
Entered flow560
Total opens (across flow)1,110
Total clicks (across flow)291
Conv rate (of entrants)14.8%
Conversions83 · £6,683
Messages7 (5 email + 2 SMS)

"You're in 💫" — first welcome email head-to-head

The single most-important email in the flow. The £5 arm wins on per-recipient revenue and conversion rate; the Starter arm has slightly higher open rate but converts fewer per recipient.

£5 off · TTcTZb LEADING

£ per recipient£12.47
Recipients452
Open rate74.0%
Click rate23.8%
Conv rate17.3%
Conversions77 · £5,638

Starter Pack · VASnYT

£ per recipient£9.09
Recipients560
Open rate75.5%
Click rate34.5%
Conv rate12.4%
Conversions70 · £5,091

Welcome flow trends — £5 off vs Starter Pack arm

Each arm tracked separately. Daily entrants is raw daily volume; revenue per entrant and conversion rate are cumulative (running totals) so small-sample days don't distort the trend.

Daily entrants

First-email recipients per day · £5 arm 452 total · Starter arm 560 total
£5 off armStarter arm
0 12 24 35 47 A/B live May 9May 14May 19May 24May 29Jun 3Jun 8Jun 9 Entrants / day Date

Revenue per entrant (cumulative)

£5 arm current £14.80 · Starter arm current £11.93
£5 off armStarter arm
£0.0 £5.0 £9.9 £14.9 £19.8 A/B live May 9May 14May 19May 24May 29Jun 3Jun 8Jun 9 £ / entrant Date

Conversion rate (cumulative)

Conversions ÷ entrants, both running totals · £5 arm current 20.6% · Starter arm current 14.8%
£5 off armStarter arm
0.0% 6.8% 13.5% 20.3% 27.1% A/B live May 9May 14May 19May 24May 29Jun 3Jun 8Jun 9 Conversion rate Date

Reading the Result

What's Next

Sources: Shopify Orders (NC by customer.created_at), Triple Whale linearAll, Meta Marketing API, Google Ads API v20, Klaviyo Forms API + flow-values-reports. Window: 2026-05-09 → 2026-06-09 vs 2026-04-20 → 2026-05-08. Generated 2026-06-10.