Paired vs unpaired t-test: which design you have

Whether two sets of measurements are paired is decided by how the data were collected, not by what the numbers look like. If every value in one set has exactly one partner in the other, fixed by the design, use a paired t-test; if not, use an unpaired one, and Welch’s version is the sensible default.

The trouble starts near the boundary: the same people surveyed in two years but not all of them, controls matched on age bands, three visits instead of two. This post gives a three-question check, walks through the designs people misclassify, and runs one public dataset through a deterministic engine as a paired t-test, to show what the pairing buys and what ignoring it costs. For the mechanics of every t-test, see t-tests explained; this post is about recognising the design.

What makes two measurements a pair

A pair is two measurements that share a source. Whatever made one of them high or low, the person, the specimen, the plot of land, the day, also acted on the other. Pairing comes in four common shapes:

• The same unit measured twice. Before and after an intervention, two conditions in a crossover study, a baseline and a follow-up visit.
• Two parts of one unit. Left and right eye, two halves of a leaf, one shoe sole material on each foot, two treatments on two sites of the same mouth.
• Two methods on the same specimen. Two assays run on one blood sample, two raters scoring the same image, two instruments reading the same item.
• Units matched one to one before the outcome. Twins split between two conditions, or each case matched to one control on stated characteristics when the study was set up.

In each shape you can draw a line from every value in one column to exactly one value in the other, and the reason for that line comes from the design. Equal group sizes are not pairing. A correlation between the columns is not pairing either: you check the design, not the data.

Independent groups are everything else: different units with no row-level link, such as participants randomised to two arms or two batches of material.

The three-question design check

Answer these in order. If all three are yes, the data are paired. If any is no, they are not, whatever the spreadsheet looks like.

1. Does each value in one set come from the same unit, or a deliberately matched unit, as one value in the other? This is the substance of pairing: a shared source. “Same participants at both time points” is yes. “Same clinic, different patients” is no.

2. Is the link one to one and recorded? Each unit must contribute exactly one value to each set, and the data must carry an identifier that rebuilds the pairs. If two of your 40 people have no follow-up value, you have 38 pairs and two unpaired values, not two groups of 40 and 38. Without identifiers, pairs cannot be analysed as pairs.

3. Was the pairing fixed before the outcome was measured? Matching has to come from the design. Sorting two independent groups and lining them up by rank, or matching after the fact on the outcome, manufactures a correlation that was never there and makes the paired test far too optimistic.

The test follows. Paired data get a test on the within-pair differences, the paired t-test for a mean difference. Independent data get a two-sample test, with Welch’s t-test as the default for means, since it does not assume equal variances. Both tests assume independent units and roughly normal data: the differences when paired, each group when not. Choosing between the paired t-test and its rank-based alternative is a separate question, covered in the guide to the paired t-test or Wilcoxon signed-rank test.

Designs that are easy to misclassify

Repeated surveys of a population. A department surveys its staff in two consecutive years. Many people answer both times, but the responses are anonymous. Question 2 fails: with no identifier, the two years are analysed as independent samples. The overlap makes that approximation imperfect, which belongs in the limitations, but guessing pairs cannot repair it.

Frequency matching. A case-control study recruits controls so that their age distribution matches the cases’, band by band, without tying any control to a specific case. Question 1 fails: no control has a designated case. Individually matched designs, one control picked for one named case, are paired. If the protocol says “matched on age” without more, find out which before you analyse.

Missing partners. Some units have only one of the two measurements. The complete pairs are still pairs, and a paired test uses only them. If many units are incomplete, or the reason could be related to the outcome, dropping them can bias the result. A mixed model uses every available measurement; the guide to repeated-measures ANOVA or a mixed model explains when that is worth it.

Three or more measurements per unit. Baseline, three months and six months is repeated measures, not three pairs. A paired t-test for every pair of time points multiplies the tests and throws away the structure.

Clustered data that look paired. Two eyes per person is a pair. Twenty pupils from each of two classrooms is not: there is no pupil-to-pupil link between the classes, and the pupils are clustered within them, which needs a model that accounts for it.

A paired yes/no outcome. If both measurements are binary, such as positive or negative on two tests run on the same samples, the data are paired but the t-test does not apply. McNemar’s test is the paired test for proportions.

Two methods measuring the same thing. Two instruments on the same specimens are paired, and a paired t-test answers whether one reads higher on average. It does not answer whether the two methods agree closely enough to be interchangeable, which is usually the real question. That needs a Bland-Altman analysis.

If your design still does not fit, the guide to choosing a statistical test starts from the outcome type and how the observations are linked.

Worked example: a paired t-test on shoe sole wear

The R package MASS ships a small dataset called shoes, taken from Box, Hunter and Hunter’s textbook Statistics for Experimenters. Ten boys each wore shoes with sole material A on one foot and material B on the other, and the wear of each sole was measured. Every boy contributes one A value and one B value, so the pairs are boys, and the three questions all answer yes: same unit, one to one with the boy as the identifier, and the pairing fixed by the design, since each boy wore both materials at once. The book presents it as a randomized paired comparison, with the foot for material A chosen at random for each boy.

Here are the data, with the within-boy difference added:

 boy    A    B A_minus_B
   1 13.2 14.0      -0.8
   2  8.2  8.8      -0.6
   3 10.9 11.2      -0.3
   4 14.3 14.2       0.1
   5 10.7 11.8      -1.1
   6  6.6  6.4       0.2
   7  9.5  9.8      -0.3
   8 10.8 11.3      -0.5
   9  8.8  9.3      -0.5
  10 13.3 13.6      -0.3

Before any test, note that wear varies a great deal from boy to boy, and within a boy the two materials are close, with B showing more wear than A in eight of the ten boys. A boy who wears one sole heavily wears the other heavily too. That shared variation is what a paired analysis removes.

Seeded and run in R; the paired t-test was chosen from the design before the run. The output and the R script that reproduces it appear below the article.

Quantity	Value
Pairs analysed	10 boys
Mean difference, A minus B	-0.41
95% confidence interval for the mean difference	-0.687 to -0.133
Paired t	-3.349 on 9 degrees of freedom
p-value (two-sided)	0.00854
Cohen’s d_z with its 95% interval	-1.059 (-1.826 to -0.2562)

Material A showed less wear than material B by 0.41 on average, in the dataset’s units of wear. The 95% interval runs from -0.687 to -0.133: the data are compatible with true average differences from 0.133 to 0.687 in favour of A. The paired t is -3.349 on 9 degrees of freedom, one fewer than the number of pairs, and the two-sided p-value is 0.00854. The interval excludes zero, the same information as p falling below 0.05. Whether a difference of that size matters is a question about shoes, not statistics.

What ignoring the pairing does

Now treat the same twenty numbers as if they came from twenty different boys. In R that is a one-word change: drop paired = TRUE.

library(MASS)
t.test(shoes$A, shoes$B, paired = TRUE)  # the analysis the design calls for
t.test(shoes$A, shoes$B)                 # the same numbers, pairing ignored (Welch)

The engine run on this page is the paired analysis only, so this post quotes no numbers for the second call. Run it and you will see the same estimated difference, a confidence interval several units wide that comfortably includes zero, and a p-value far above 0.05. The data have not changed. Only the question about how they were collected has.

The reason is the standard error. The unpaired test estimates it from the spread within each column, which here is dominated by how differently the boys wear their shoes. The paired test works on the within-boy differences, whose spread is small because the boy-to-boy variation cancels when you subtract; its t is the mean difference divided by its standard error, the SD of the differences over the square root of the number of pairs. Formally, the variance of a difference between two correlated measurements is the sum of their variances minus twice their covariance. The unpaired test leaves out the covariance term. When partners are strongly positively correlated, as here, that term is large, and leaving it out inflates the standard error many times over.

Two consequences follow. First, an unpaired analysis of paired data is not a safe, conservative choice. It answers a question about two unrelated groups, and with correlated pairs it can miss a clear effect entirely, as the second call does. Its non-significant result does not show that the materials wear the same. Second, when partners are barely correlated, pairing removes little variance and the paired test gains little. That is a consideration for planning the design, never a reason to pick whichever test gives the smaller p-value. Decide from the three questions before you look at either result.

The reverse error, pairing values that are not pairs (question 3), is worse: the paired test can then report an interval far too narrow, and no output warns you.

The effect size is not on the unpaired scale

The engine reports Cohen’s d as -1.059, with a 95% interval from -1.826 to -0.2562. For a paired test this d is d_z: the mean difference divided by the standard deviation of the differences. Its interval comes from the differences alone, so no equal-variance assumption is involved.

By the usual labels, a d of about 1 is large. Yet the difference between materials is small next to the variation between boys. Both are true, because they use different denominators: d_z measures the difference against the spread of the within-boy differences, which the pairing has made small, while d_av, which divides by the average standard deviation of the A and B wear values themselves, would be much smaller here.

Dunlap and colleagues showed in 1996 that an effect size computed from a paired test statistic, without accounting for the correlation between the measurements, is overestimated relative to the standardised difference on the original scale. Lakens’ 2013 primer sets out the options for within-subject designs, d_z, d_av and d_rm, and notes that d_z exceeds the other two whenever the correlation between the measurements is above 0.5. If your effect size will be compared with independent-groups studies or pooled in a meta-analysis, report d_av or d_rm as well, and always name which d you report.

The assumption check and the sensitivity figure, read in context

Normality of the differences (Shapiro-Wilk): passed. W was 0.961 with p = 0.801. The check was run on the ten within-boy differences, which is the right place: a paired t-test makes no assumption about the raw A and B columns. With ten pairs, though, a normality test has little power, so “passed” means a departure was not detected, not that the differences are normal; the listing above is at least as informative. Had they been badly skewed or had one extreme value, the Wilcoxon signed-rank guide explains the alternative.

Sample size: passed. Ten pairs clears the engine’s minimum-size floor, a basic sanity check that says nothing about the assumptions or power.

Sensitivity. The engine flagged the study as small. With 10 pairs, a two-sided test at the 0.05 level reaches 80% power only for effects of d_z = 0.996 or larger, which the engine rounds to 1.00 in its warning; detecting d_z = 0.50 with 80% power would need 34 pairs. This describes what the design could reliably detect; it is not “observed power” computed from the result, which would add nothing to the p-value. These figures are on the d_z scale too, so they depend on how strongly the pairs are correlated: when planning a paired study, the expected correlation matters as much as the expected difference, and both should come from prior knowledge, not from an estimate like this one.

How to report it

Name the design and the test, give the mean difference with its interval, then the test statistic, degrees of freedom and p-value, and an effect size with its interval and its denominator. For the example:

Sole wear was compared between materials A and B worn by the same 10 boys, one material on each foot. A paired t-test gave a mean difference (A minus B) of -0.41 (95% CI -0.687 to -0.133), t(9) = -3.349, p = 0.00854, Cohen’s d_z (standardised by the SD of the within-boy differences) = -1.059 (95% CI -1.826 to -0.2562).

Say in the methods how the pairs were formed, for example “paired by participant identifier”, and give the number of units dropped for a missing partner.

Rerunning the example

Below the article is the R script exactly as the engine generated it, with the seed and the bundled data path, and the output it printed. The script reads the shoes data from a data/ folder as two columns, A and B, one row per boy, and on the same package versions reproduces the paired t-test and the sensitivity figure. The effect size and the normality check come from the engine itself; effectsize::cohens_d(data$A, data$B, paired = TRUE) and shapiro.test(data$A - data$B) reproduce them. The unpaired call shown earlier is not in that script.

The guide to reproducible statistical analysis explains what a reviewer needs to rerun a result like this. For independent groups and the choice between means and ranks, see the t-test or Mann-Whitney guide.

# Results

- Test: Paired t-test
- n: 10
- statistic: -3.349
- p-value: 0.00854
- effect size: Cohen's d = -1.059 (95% CI -1.826 to -0.2562)
- 95% CI: [-0.687, -0.133]
- seed: 20261021

## Assumption checks
- normality (Shapiro-Wilk): passed=TRUE (no evidence against normality)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=10: Cohen's d = 0.996 (severely underpowered)
- to detect a conventional medium effect: 34 pairs to detect d=0.50
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Paired t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/paired-vs-unpaired-t-test.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("A", "B")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(data[["A"]], data[["B"]], paired = TRUE)
print(result)

# Sensitivity: minimum detectable effect at n=10 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t.test(n = 10, sig.level = 0.05, power = 0.8, type = "paired")
print(power)