Test and measure
Reading A/B test results
What revenue per visitor means, when PageStax calls a test ready, and why it sometimes withholds a winner.
Open a test from A/B tests to see its Results. PageStax is deliberately strict about calling a winner: a result you can't trust is worse than no result.
Revenue per visitor#
Revenue / visitor is total revenue from the visitors who saw a version, divided by the number of those visitors. It rewards both more orders and bigger orders, so it's the fairest single number for most tests.
Every figure counts unique visitors, not page views. The Lift column shows how much better or worse a challenger did than the control (A), with a range — the interval — the true difference probably falls in.
Look whenever you like#
Results are always-valid: you can check as often as you want without making a false winner more likely. You don't need to wait for a set date before looking.
What the status means#
| Reading | What it tells you |
|---|---|
| Collecting data | Too few visitors so far to read anything. |
| Needs more traffic | The interval still includes zero — the versions may be the same. |
| Significant — keep running | A challenger is ahead, but the test hasn't reached its planned visitors. |
| Ready to decide | Enough visitors, a healthy split, and the interval has cleared zero. |
| Uneven traffic split | The split drifted from what you set, so the test paused itself. Shoppers keep seeing A. |
Why a winner is withheld#
Declare winner stays disabled until the test is ready. Hover it to see why — for example Not ready to declare a winner. The interval still includes zero. The reasons are:
- too few visitors to read anything yet;
- the interval still includes zero, so the difference could be chance;
- the test hasn't reached its planned visitors;
- an uneven traffic split — if visitors aren't split the way you set, something is wrong with how the test is being served, and any "winner" can't be trusted.
PageStax also only calls a winner on the metric you chose for the test.
Declare a winner#
When the test reads Ready to decide, press Declare winner, check the summary, and confirm.
- In a section test, the losing section is removed and the winner stays.
- In a page test, the winning version becomes the main page.
Afterwards, check Pages: if the page shows Draft changes, push it live so every shopper sees the winner.