Statistics & Probability

Two-Proportion A/B Test & Difference Interval Calculator

Compare binary event rates for two independent groups using a pooled two-sided z test and an unpooled difference interval.

How the calculation works

pa = events_a/n_a. pb = events_b/n_b. pool = (events_a+events_b)/(n_a+n_b). pooled_se = sqrt(pool*(1-pool)*(1/n_a+1/n_b)). unpooled_se = sqrt(pa*(1-pa)/n_a+pb*(1-pb)/n_b). Group A observed event rate = pa*100 %. Group B observed event rate = pb*100 %. Difference B − A = (pb-pa)*100 percentage points. Pooled-null z statistic = (pb-pa)/pooled_se . Two-sided p-value = 2*normal_cdf(-abs((pb-pa)/pooled_se)) . Wald difference lower endpoint = (pb-pa-normal_quantile((1+confidence/100)/2)*unpooled_se)*100 percentage points. Wald difference upper endpoint = (pb-pa+normal_quantile((1+confidence/100)/2)*unpooled_se)*100 percentage points. Relative lift B versus A = (pb/pa-1)*100 if pa > 0 else "Undefined when group A has zero observed rate" %.

Worked example

Example inputs — Group A events: 10; Group A observations: 100; Group B events: 10; Group B observations: 100; Two-sided interval confidence (%): 95. Expected results — Group A observed event rate: 10 %; Group B observed event rate: 10 %; Difference B − A: 0 percentage points; Pooled-null z statistic: 0; Two-sided p-value: 1; Wald difference lower endpoint: -8.315422946 percentage points; Wald difference upper endpoint: 8.315422946 percentage points; Relative lift B versus A: 0 %.

Assumptions and limits

Independent groups, one binary outcome per observation, and fixed-sample normal approximations. The hypothesis test pools rates under equal population proportions; the displayed Wald interval uses an unpooled standard error, so its decisions need not exactly match the test. Wald intervals may extend outside feasible bounds and perform poorly for small or extreme counts. Repeated peeking, multiple comparisons, clustering and nonrandom assignment are not corrected. Relative lift is undefined when group A has no observed events.

Reference and further reading

Read the underlying subject reference. Calculith is an independent implementation; the reference does not endorse or certify this tool.

Related calculators

Browse all 1000 calculators →

Example values are editable. Results are estimates where assumptions apply. Calculations run in your browser. Updated 2026-11-16.