Math · A/B testing
A/B test sample size calculator.
How many visitors does each variant need, and how many days is that? Enter your baseline conversion rate, the smallest lift worth detecting, and your traffic. Everything runs in your browser.
To detect a lift from a 10% conversion rate to 11% (a 10% relative improvement) with 80% power at a 5% two-sided significance level, each variant needs 14,751 visitors, or 29,502 across two variants. At 2,000 visitors a day that is about 15 days.
Test setup
Your result
Fill the form and press Calculate.
The formula
With a baseline rate p1, a rate to detect p2 = p1 + δ, and the pooled rate p̄ = (p1 + p2) / 2, the visitors needed in each variant are:
n = (zα √(2p̄(1−p̄)) + zβ √(p1(1−p1) + p2(1−p2)))2 / δ2, rounded up.
Here zα is z1−α/2 for a two-sided test (1.960 at 5%) or z1−α for a one-sided test (1.645 at 5%), and zβ is the normal value for the power (0.842 at 80%). The pooled rate in the first term matches the pooled two-proportion z-test used on the significance calculator, so a test sized here has the chosen power for that test. This is the standard normal-approximation power calculation for that test; the sources below confirm the test and the reference tool, not this exact expression.
Worked example
Baseline 10%, detect a 10% relative lift (so p2 = 11%), 5% two-sided, 80% power: p̄ = 10.5%, and n = (1.960 × 0.4335 + 0.842 × 0.4335)2 / 0.012 ≈ 14,750.8 (from unrounded square roots), rounded up to 14,751 per variant (29,502 in total, 15 days at 2,000 a day). Raising power to 90% needs 19,747; a 1% significance level needs 21,950; a one-sided test needs 11,620. Detecting a 20% relative lift (10% to 12%) needs 3,841, roughly a quarter of the traffic, because n scales with 1/δ2.
How this compares with Evan Miller's calculator
Evan Miller's sample size calculator is the tool many people check against. Its published code uses the baseline rate p1 instead of the pooled rate p̄ for the null-hypothesis variance, which gives a smaller number: 1,030 against 1,094 here for a move from 20% to 25% at 80% power and 5% significance, and 14,313 against 14,751 for 10% to 11%. The pooled version is used here because it has the chosen power for the pooled z-test; the other is a few percent optimistic for that test.
Using the number
This is a fixed-sample plan: decide the sample size before the test and judge the result when you reach it. Evan Miller contrasts Bayesian formulas with the traditional frequentist ones that need a pre-ordained sample size, and points to the "peeking" problem. Stopping a fixed-sample test the moment it looks significant is the usual way that problem arises. Running whole weeks so each weekday is represented equally is a common rule of thumb, not a published standard. The calculation assumes independent visitors, one conversion event per visitor and a conversion rate that stays steady during the test.
Sources, opened 8 October 2026: NIST/SEMATECH e-Handbook of Statistical Methods, 7.3.3 How can we determine whether two processes produce the same proportion of defectives? (pooled z statistic, z1−α/2 for two-sided and z1−α for one-sided tests); NIST/SEMATECH e-Handbook, 1.3.6.7.1 Cumulative Distribution Function of the Standard Normal Distribution (1.645, 1.960); Evan's Awesome A/B Tools, Sample Size Calculator (its linked script sample-size-fixed.js, whose num_subjects function is compared above); Evan Miller, Formulas for Bayesian A/B Testing (1 October 2015) (the pre-ordained sample size and peeking remarks); NIST/SEMATECH e-Handbook, 7.1.3.1 Critical values and p values (common significance levels 0.1, 0.05 and 0.01).
Frequently asked questions
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and the smallest lift you want to detect. A 10% baseline and a 10% relative lift (10% to 11%) needs 14,751 visitors per variant at 80% power and a 5% two-sided significance level. A 20% relative lift on the same baseline needs 3,841.
What is a good minimum detectable effect?
Pick the smallest lift that would change a business decision. Smaller effects need far more traffic: halving the detectable relative lift needs roughly four times as many visitors, because the sample size scales with one over the effect squared.
How long should an A/B test run?
Divide the total visitors needed (per variant times number of variations) by your daily test traffic and round up. At 29,502 total visitors and 2,000 visitors a day that is 15 days. Many teams round up to whole weeks, which is a rule of thumb rather than a published standard.
What power and significance level should I use?
80% power and a 5% significance level are common conventions, not laws. NIST notes that significance levels of 0.1, 0.05 and 0.01 are common. Higher power or a stricter level needs more visitors.
Is the number per variant or in total?
Per variant. With two variants (control and one challenger) the total is twice the number shown, and the calculator multiplies by the number of variations you enter.
Why does this differ from Evan Miller's calculator?
His published code uses the baseline rate rather than the pooled rate for the null variance, so it returns a smaller number. For 20% to 25% at 80% power and 5% significance it gives 1,030 and this calculator gives 1,094. The pooled version matches the pooled z-test used to judge results.
Planning and learning aid
Normal-approximation planning estimate for a single comparison with one primary metric. It does not adjust for multiple variations, multiple metrics, repeated looks at the data, or non-independent traffic.