McNemar's test for paired proportions

Use McNemar’s test to compare two proportions when both come from the same units, with a yes/no outcome in each measurement: the same participants measured twice, matched pairs, or two tests applied to the same items. The test looks only at the pairs where the two measurements disagree, because only those pairs carry information about a change.

A sensible way to use it: choose McNemar when the data are paired binary, report the effect as the difference in proportions or the ratio of the discordant counts, and when the number of discordant pairs is small, consider the mid-p version, or the exact version if you want a guaranteed error rate, over the classic chi-square form. For how McNemar fits among the other contingency-table tests, see chi-square or Fisher’s exact test; this guide goes deeper on the paired case.

When the pairing is real

The data are paired when each observation in one measurement has a specific partner in the other.

• Before and after on the same units: the same participants classified as pass or fail before and after training.
• Matched designs: each case is matched to a control, and exposure is recorded for both members of the pair.
• Two raters or two tests on the same items: do two raters, or two assays, call the same samples positive or negative?

The data are not paired when two separate groups are compared, even if the groups are the same size or share a characteristic. Ten treated plots and ten untreated plots, with nothing linking one plot to one other plot, are two independent samples, and the independent-groups tests apply. The pairing has to come from the design, not from the layout of the spreadsheet.

Discordant pairs: the only cells that matter

Cross-classify the two measurements into a 2 x 2 table of pairs, with cells a, b, c and d:

	Second: yes	Second: no
First: yes	a	b
First: no	c	d

Cells a and d are concordant pairs: both measurements agree. Cells b and c are discordant: the unit changed, or the two tests disagreed. Concordant pairs are the same under both measurements, so they say nothing about whether the two proportions differ.

The null hypothesis is that, among the discordant pairs, a change in one direction is as likely as a change in the other: b = c in expectation. This is equivalent to the two marginal proportions being equal, since the first-measurement proportion is (a + b) / n and the second is (a + c) / n, and a drops out of the difference. McNemar (1947) built the test on exactly this reduction.

Why a plain chi-square is wrong for paired data

A standard chi-square test of independence on measurement-by-outcome treats the two measurements as separate samples. With 60 participants measured twice, it sees 120 observations that it assumes to be independent. They are not: each participant appears twice, and a person who passes once tends to pass again. Counting each participant twice overstates the information, and ignoring the positive correlation within pairs usually inflates the standard error of the difference, so the plain test tends to be too conservative and misses real changes.

The right table has one row per pair, with the cells above. The wrong table has one row per measurement occasion. The two have different totals (60 pairs against 120 observations) and answer different questions.

Worked example

A synthetic study classifies 60 participants as pass or fail on a screening task before and after a training session. No real participants are involved; the data were generated in R 4.6.1.

	After: pass	After: fail
Before: pass	30	14
Before: fail	4	12

Thirty participants passed both times and 12 failed both times. Of the 18 discordant pairs, 14 went from pass to fail and 4 from fail to pass. The proportion passing was 44 / 60 = 73.3% before and 34 / 60 = 56.7% after, a drop of 16.7 percentage points.

The wrong analysis stacks the two occasions into a table of 120 observations (44 pass and 16 fail before; 34 pass and 26 fail after) and runs a chi-square test: X-squared = 2.97, p = 0.085 with Yates’ correction, or p = 0.056 without. It finds no clear change.

McNemar’s test uses only the 14 and 4:

Version	Statistic	p
Continuity-corrected chi-square	4.50, 1 df	0.034
Uncorrected chi-square	5.56, 1 df	0.018
Exact binomial (14 of 18)	-	0.031
Mid-p	-	0.019

The correct paired analysis gives p of 0.02 to 0.03 under all four versions, while the unpaired test, which ignores the pairing, does not reach 0.05. Pairing removes the between-participant variability that the wrong analysis leaves in the noise. The four versions are close here but not identical, which brings up the choice between them.

Exact, mid-p and continuity-corrected versions

All the versions test b against c. They differ in how they get a p-value.

• Uncorrected chi-square: (b - c)^2 / (b + c), compared with a chi-square distribution on 1 degree of freedom. It relies on a large-sample approximation.
• Continuity-corrected chi-square: subtracts 1 from the absolute difference before squaring (Edwards, 1948). The correction guards against over-rejection with discrete counts but is conservative, giving larger p-values and lower power.
• Exact conditional (binomial): conditional on the b + c discordant pairs, b follows a binomial distribution with probability one half under the null. The p-value is an exact tail probability, so the test never exceeds its nominal level, but it is conservative for the same reason as the correction.
• Mid-p: the exact test with half the probability of the observed outcome counted in the tail. It removes much of the conservatism and still performs well in small samples.

Fagerland, Lydersen and Laake (2013) compared these versions by simulation and concluded that the exact conditional test is too conservative, and that the mid-p and asymptotic tests perform better. With many discordant pairs the versions converge. A common rule of thumb is that the chi-square approximation becomes questionable below roughly 25 discordant pairs, which is the point at which to switch to an exact or mid-p version.

The paired odds ratio b/c

The ratio b / c, here 14 / 4 = 3.5, is an effect measure for the paired design. It is the odds that a change goes in the first direction rather than the second: pass-to-fail changes are 3.5 times as common as fail-to-pass changes among units that changed.

Two cautions. It is a conditional odds ratio, based on discordant pairs only, so it is not the odds ratio you would compute from the margins of the table (which would ignore the pairing), and it should not be compared with one. And it is undefined when c = 0, so a zero cell leaves no ratio to report. A confidence interval for b / c comes from the binomial distribution on b of b + c; it is not part of the basic test output.

What to report

• The paired 2 x 2 table with all four counts, including b and c.
• The version of the test used (corrected, uncorrected, exact or mid-p) and its p-value.
• The two marginal proportions and their difference, with a confidence interval where you can.
• The ratio b / c, if an odds-type summary is wanted.
• The number of pairs and how the pairing was defined.

How Celeus handles this

Celeus can suggest McNemar’s test when it finds two binary columns that look like a before-and-after pair sharing the same two categories, and it requires that the columns share those two categories. It runs the continuity-corrected chi-square test, not an exact or mid-p version, and reports the chi-square, degrees of freedom, p-value and the ratio b / c, without a confidence interval for that ratio. It also checks the discordant pair count and reports when there are fewer than 25, since the chi-square approximation may be unreliable there; it does not switch methods, so compute an exact or mid-p version separately when the count is low.