Statistics & Probability
Two-Proportion A/B Test & Difference Interval Calculator
Compare binary event rates for two independent groups using a pooled two-sided z test and an unpooled difference interval.
How the calculation works
pa = events_a/n_a. pb = events_b/n_b. pool = (events_a+events_b)/(n_a+n_b). pooled_se = sqrt(pool*(1-pool)*(1/n_a+1/n_b)). unpooled_se = sqrt(pa*(1-pa)/n_a+pb*(1-pb)/n_b). Group A observed event rate = pa*100 %. Group B observed event rate = pb*100 %. Difference B − A = (pb-pa)*100 percentage points. Pooled-null z statistic = (pb-pa)/pooled_se . Two-sided p-value = 2*normal_cdf(-abs((pb-pa)/pooled_se)) . Wald difference lower endpoint = (pb-pa-normal_quantile((1+confidence/100)/2)*unpooled_se)*100 percentage points. Wald difference upper endpoint = (pb-pa+normal_quantile((1+confidence/100)/2)*unpooled_se)*100 percentage points. Relative lift B versus A = (pb/pa-1)*100 if pa > 0 else "Undefined when group A has zero observed rate" %.
Worked example
Example inputs — Group A events: 10; Group A observations: 100; Group B events: 10; Group B observations: 100; Two-sided interval confidence (%): 95. Expected results — Group A observed event rate: 10 %; Group B observed event rate: 10 %; Difference B − A: 0 percentage points; Pooled-null z statistic: 0; Two-sided p-value: 1; Wald difference lower endpoint: -8.315422946 percentage points; Wald difference upper endpoint: 8.315422946 percentage points; Relative lift B versus A: 0 %.
Assumptions and limits
Independent groups, one binary outcome per observation, and fixed-sample normal approximations. The hypothesis test pools rates under equal population proportions; the displayed Wald interval uses an unpooled standard error, so its decisions need not exactly match the test. Wald intervals may extend outside feasible bounds and perform poorly for small or extreme counts. Repeated peeking, multiple comparisons, clustering and nonrandom assignment are not corrected. Relative lift is undefined when group A has no observed events.
Reference and further reading
Read the underlying subject reference. Calculith is an independent implementation; the reference does not endorse or certify this tool.
Related calculators
- Sample & Population Variance Calculator
- Quartiles, IQR & Tukey Fences Calculator
- Adjusted Skewness & Excess Kurtosis Calculator
- Z-Score & Normal Percentile Calculator
- Normal Interval Probability & Quantile Calculator
Example values are editable. Results are estimates where assumptions apply. Calculations run in your browser. Updated 2026-11-16.