Math · A/B testing
Bayesian A/B test calculator.
What is the probability that your variant really beats the control? Enter visitors and conversions for each. Exact beta-binomial maths with a uniform prior, in your browser.
If A converts 100 of 1,000 visitors and B converts 120 of 1,000, the chance that B's true conversion rate is higher than A's is 92.3% (and 7.7% that A is higher). The posterior rates are 10.08% for A (95% credible interval 8.29% to 12.02%) and 12.08% for B (10.13% to 14.16%).
Test results
Your result
Fill the form and press Calculate.
The model
Treat each variant's true conversion rate as unknown, with a uniform Beta(1, 1) prior (every rate equally likely before seeing data). After S conversions and F non-conversions the belief about the rate is Beta(S + 1, F + 1). Write α = S + 1 and β = F + 1 for each variant. The probability that B's rate beats A's has a closed form (Evan Miller, 2015):
Pr(pB > pA) = ∑i=0αB−1 B(αA + i, βA + βB) / ( (βB + i) B(1 + i, βB) B(αA, βA) )
where B is the beta function. The same page gives three equivalent sums over the other parameters; the calculator evaluates whichever has the fewest terms, in logarithms to avoid overflow. Credible intervals are the middle 90, 95 or 99% of each variant's Beta posterior, found from the inverse of the regularised incomplete beta function. The posterior mean is α / (α + β).
Worked example
A: 100 of 1,000, so αA = 101 and βA = 901. B: 120 of 1,000, so αB = 121 and βB = 881. The sum gives Pr(pB > pA) = 0.9232. Posterior means are 101 / 1,002 = 10.08% and 121 / 1,002 = 12.08%, slightly above the raw 10.0% and 12.0% because the uniform prior adds one pseudo-success and one pseudo-failure. The ratio of the means is a 19.8% relative lift. With only a tenth of the data (10 of 100 against 12 of 100) the same rates give just 67.1%: the probability reflects how much evidence there is, not just the gap between rates.
Reading the result
The output is a probability about the true rates given your data and prior, which is not the same as a frequentist p-value. It does not say how large the improvement is (use the credible intervals for that) and it does not tell you when to stop. Decide in advance what probability, and what minimum size of improvement, would make you ship B. This calculator uses the uniform prior only; a different prior would shift small-sample results.
Sources, opened 8 October 2026: Evan Miller, Formulas for Bayesian A/B Testing (1 October 2015) (the beta-binomial model, the non-informative prior, the Pr(pB > pA) sum and its three equivalent forms, the advice to work in logarithms). The credible intervals are standard quantiles of the Beta posterior and are not taken from that page.
Frequently asked questions
What does "probability that B beats A" mean?
It is the probability, given your data and a uniform prior, that the true conversion rate of B is higher than the true rate of A. 92.3% means that under this model B is better in about 92 of 100 equally plausible worlds consistent with the data.
Is this the same as a p-value?
No. A p-value is the probability of data at least as extreme as yours if there were no difference. The Bayesian probability is about the rates themselves. The two answer different questions and will not match exactly. See the A/B test significance calculator for the p-value.
What prior does this calculator use?
A uniform Beta(1, 1) prior for each variant, the non-informative starting point in Evan Miller's formulas. Posterior rates are therefore (conversions + 1) / (visitors + 2).
What probability is enough to pick B?
There is no universal threshold. Choose one before the test based on the cost of being wrong, and also look at the credible interval to see how large the improvement could be.
Do I still need a sample size?
Evan Miller writes that Bayesian formulas do not require a pre-ordained sample size to give a valid result, unlike the traditional fixed-sample formulas. Very small samples still give uncertain answers, so check the credible intervals.
What is a credible interval?
The range that contains the true conversion rate with the stated probability under the model: a 95% credible interval of 8.29% to 12.02% for A means a 95% chance the true rate lies in that range. It is different from a frequentist confidence interval.
Planning and learning aid
Statistical estimate under a beta-binomial model with a uniform prior and independent visitors. It does not model multiple variants, repeated metrics, seasonality or tracking errors.