The A/B test sample size formula tells you how many visitors each variant needs before a result can be trusted. Skip it and you will either stop tests too early and ship false winners, or run them for months without learning anything. If you have read our guide on what makes a good A/B test, sample size is the step that turns a good idea into a reliable answer.
This article gives the formula, explains every input, works through three ecommerce examples by hand and finishes with a lookup table you can use without a calculator.
Before you start: the four inputs that set sample size
Sample size is not a mysterious number from a tool. It is the output of four decisions you make before launch:
- Baseline conversion rate (p1): the current rate of the metric you will judge the test on, from the last 4 to 8 weeks of data. Example: 3% of visitors place an order.
- Minimum detectable effect (MDE): the smallest lift worth detecting, usually stated as a relative change. A 10% relative MDE on a 3% baseline means detecting a move from 3.0% to 3.3%.
- Significance level (α): the false positive rate you accept, almost always 0.05 (95% confidence).
- Statistical power (1 − β): the chance of detecting a real effect of size MDE, usually 0.80.
Change any of the four and the sample size changes. The MDE has the biggest effect, because it enters the formula squared.
1. The A/B test sample size formula
For a two-variant test on a conversion rate (a two-proportion z-test), the sample size per variant is:
n = [z(1−α/2) × √(2 × p̄ × (1 − p̄)) + z(1−β) × √(p1(1 − p1) + p2(1 − p2))]² ÷ (p2 − p1)²
Where:
- p1 = baseline conversion rate
- p2 = p1 × (1 + relative MDE)
- p̄ = (p1 + p2) ÷ 2, the pooled rate
- z(1−α/2) = 1.96 for 95% confidence, two-sided
- z(1−β) = 0.84 for 80% power
Many people use a simpler shortcut, often called Lehr’s rule: n ≈ 16 × p(1 − p) ÷ (p2 − p1)². The 16 comes from 2 × (1.96 + 0.84)². It gives a slightly higher, rough number that is good for planning but not for the final decision.
2. Worked example: 3% baseline, 10% relative lift
A store converts 3% of visitors and wants to detect a 10% relative lift.
- 1. p1 = 0.030, p2 = 0.030 × 1.10 = 0.033, p̄ = 0.0315
- 2. First term: 1.96 × √(2 × 0.0315 × 0.9685) = 1.96 × 0.2470 = 0.4841
- 3. Second term: 0.84 × √(0.0291 + 0.0319) = 0.84 × 0.2470 = 0.2075
- 4. Numerator: (0.4841 + 0.2075)² = 0.4783
- 5. Denominator: (0.033 − 0.030)² = 0.000009
- 6. n = 0.4783 ÷ 0.000009 ≈ 53,150 visitors per variant (53,211 with unrounded z-values)
That is about 106,400 visitors in total for an A/B test with two variants. A 0.3 percentage point difference is small compared with random noise, and this is what it costs to see it.
3. Worked example: the same store chasing a 20% lift
Keep the 3% baseline but set the relative MDE to 20% (3.0% to 3.6%). The same formula returns about 13,900 visitors per variant. Doubling the MDE cut the required sample by roughly four times, because the effect size sits squared in the denominator.
This is the most practical lesson in this article. A small store cannot detect small lifts. It should test bolder changes, such as a new offer or a different page layout, that can plausibly move conversion by 20% or more.
4. Worked example: 5% add-to-cart rate, 10% lift
Tests higher in the funnel need fewer visitors because the baseline rate is higher. With a 5% baseline and a 10% relative MDE, the minimum sample size is about 31,200 per variant, compared with 53,200 at 3%. Using add-to-cart as the primary metric on a product page test can make an experiment feasible on modest traffic, as long as you keep orders as a guardrail.
Read more: How to Choose the Right Metrics for A/B Testing Success
5. Sample size per variant lookup table
Visitors per variant at 95% confidence and 80% power, two-sided, using the formula above:
| Baseline rate | 5% MDE | 10% MDE | 15% MDE | 20% MDE |
| 1% | 637,010 | 163,095 | 74,193 | 42,693 |
| 2% | 315,206 | 80,682 | 36,693 | 21,109 |
| 3% | 207,938 | 53,211 | 24,193 | 13,914 |
| 5% | 122,124 | 31,234 | 14,193 | 8,158 |
Read across a row to see how fast the requirement falls as the MDE grows. Read down a column to see why tests on rare events, such as a 1% purchase rate, need very large traffic.
Size Your Next Test Before You Launch It
Plan the sample, run the experiment and read the result in one place, without writing code. Use every CustomFit.ai feature free for 14 days.
6. A/B test sample size and duration
Sample size becomes test duration once you divide by traffic:
Duration (days) = (n per variant × number of variants) ÷ daily visitors entering the test
If 53,211 per variant are needed, there are two variants and 6,000 visitors a day see the tested page, the test needs 53,211 × 2 ÷ 6,000 ≈ 18 days. Round up to full weeks (21 days) so every weekday appears equally often.
Two rules keep duration honest:
- Run at least one full week, ideally two, even if the sample is reached sooner, to cover weekday and weekend behavior.
- If the duration is longer than 6 to 8 weeks, cookie deletion and seasonality start to bias the result. Raise the MDE, test a higher-traffic page or choose a higher-frequency metric instead.
Our guide on how long to run an A/B test covers seasonality and business cycles in more detail.
7. How many visitors for an A/B test with more than two variants?
Every extra variant adds another full sample. An A/B/C/D test with a 3% baseline and 10% MDE needs about 53,200 × 4 = 212,800 visitors before any correction. If you correct for multiple comparisons (for example a Bonferroni α of 0.05 ÷ 3 = 0.0167), each comparison needs roughly 30% more. That is why most low-traffic stores should run one variant against control.
8. Sample size for revenue per visitor
Conversion rate is a yes-or-no metric, so its variance comes straight from p(1 − p). Revenue per visitor is different: most visitors spend nothing and a few spend a lot, so the standard deviation is often two to four times the mean. Use the continuous version of the formula:
n per variant = 2 × σ² × (z(1−α/2) + z(1−β))² ÷ d²
Example: revenue per visitor averages $2.40 with a standard deviation of $14. To detect a 10% lift (d = $0.24) at 95% confidence and 80% power: 2 × 196 × 7.84 ÷ 0.0576 ≈ 53,400 visitors per variant. Capping extreme orders at the 99th percentile (winsorizing) often cuts σ by a third and the required sample by more than half.
9. One-sided or two-sided?
A one-sided test needs about 20% fewer visitors because z(1−α) is 1.645 instead of 1.96. It only answers “is B better?”, not “is B worse?”. Most teams keep two-sided tests, because a variant that hurts conversion is exactly what you want to catch.
10. Mistakes that make sample size meaningless
- Peeking: checking results every day and stopping on the first significant reading can push the real false positive rate above 20%. Fix the sample in advance, or use a sequential method designed for early stopping.
- Choosing the MDE after the fact: the MDE is a business decision about what lift matters, set before launch.
- Counting sessions instead of users: if you randomize by user, count users. Mixing units breaks the variance assumptions in the formula.
- Ignoring revenue variance: revenue per visitor has far higher variance than conversion rate and needs a different formula based on the metric’s standard deviation, n = 2σ²(z(1−α/2) + z(1−β))² ÷ d².
- Ignoring sample ratio mismatch: if a 50/50 split arrives as 52/48, the sample is biased and the size calculation no longer protects you.
More pitfalls are covered in common A/B testing mistakes and how to avoid them. For the significance side of the same math, see this explainer on statistical significance in A/B testing.
Sample size checklist
- Measure the baseline rate for the exact metric and page in the test.
- Set the MDE from business value, not from hope.
- Use 95% confidence and 80% power unless you have a reason to change.
- Calculate n per variant, then multiply by the number of variants.
- Convert to days with real traffic and round up to full weeks.
- Do not stop early, and check the traffic split before you read results.
If you’d rather not do the arithmetic by hand, the A/B testing sample size calculator guide walks through the same inputs.
Pick One Test and Start Testing
Start with one bold change on a high-traffic page. Try CustomFit.ai free for 14 days, cancel anytime.
Where to start
Pull your baseline conversion rate for the page you want to test, then read the lookup table above. If the row and column you need demand more traffic than you get in four weeks, increase the MDE or move the primary metric higher in the funnel before you build anything.
Summary
The A/B test sample size per variant depends on four inputs: the baseline rate, the MDE, the significance level and power. At 95% confidence and 80% power, a 3% baseline needs about 53,000 visitors per variant to detect a 10% lift and about 14,000 for a 20% lift. Fix the number before launch, run full weeks and never stop on the first significant day.
FAQs
What is the formula for A/B test sample size?
For conversion rates, n per variant = [1.96 × √(2p̄(1 − p̄)) + 0.84 × √(p1(1 − p1) + p2(1 − p2))]² ÷ (p2 − p1)², at 95% confidence and 80% power. p1 is the baseline rate, p2 the rate after the minimum detectable effect, and p̄ their average.
What is the minimum sample size for an A/B test?
There is no universal minimum; it depends on baseline rate and MDE. As a rough guide, a 3% conversion rate needs about 14,000 visitors per variant to detect a 20% relative lift and about 53,000 for a 10% lift, at 95% confidence and 80% power.
How many visitors do I need for an A/B test?
Multiply the sample size per variant by the number of variants. A two-variant test on a 5% baseline with a 10% relative MDE needs about 31,200 per variant, or roughly 62,500 visitors in total. Divide by daily traffic to estimate duration.
How does the minimum detectable effect change sample size?
Sample size scales with the inverse square of the effect. Halving the MDE roughly quadruples the visitors you need, and doubling it cuts the requirement by about four times. That is why low-traffic stores should test bold changes that can move metrics by 20% or more.
How long should an A/B test run?
Divide the total sample by daily visitors in the test, then round up to full weeks. Run at least one, ideally two, full weeks to capture weekday patterns, and avoid running much beyond six to eight weeks because cookie churn and seasonality bias the result.
