Regression to the mean in pre-post studies

Regression to the mean is the tendency of people selected for an extreme score to score closer to the average the next time they are measured, with no treatment and no real change. In a pre-post study it looks exactly like an improvement. A control group chosen in the same way, or an ANCOVA on the baseline score, removes it from the treatment estimate; a before-and-after comparison of one selected group cannot.

This post builds the effect from scratch in a seeded simulation where the true treatment effect is zero and every person’s true score is fixed, then shows how large the apparent improvement is and how the analysis of a randomised comparison makes it disappear. It is written for readers planning a thesis or reading someone else’s before-and-after result. For the wider choice between analyses, the guide to statistics for a dissertation starts from the study design.

What regression to the mean is, and what it is not

Francis Galton first described it in 1886, when he noticed that the adult children of unusually tall parents were on average shorter than their parents, and the children of unusually short parents were taller (Galton, 1886). Nothing was pulling heights back. His data simply showed that a second, imperfectly related measurement of an extreme group lies nearer the average.

Bland and Altman (1994) state the general rule: regression towards the mean occurs whenever an extreme group is selected on one variable and another variable is then measured on that group, and the second group mean is closer to the mean of all subjects than the first. The effect is larger when the correlation between the two variables is weaker.

Three things it is not:

• Not a real change in the people. In the simulation below, every true score stays fixed between the two sittings.
• Not the gambler’s fallacy. No process is “due” to correct itself. The only ingredient is selection on a value that contained some luck.
• Not specific to health. It appears wherever two imperfectly related measurements are paired: exam scores, symptom scales, sensor readings, survey ratings.

Why selecting extreme scores produces it

An observed score is a stable part plus a random part: what the person is really like on that day-to-day average, plus measurement error and momentary noise. Choose the lowest scorers in a large cohort and you choose people with a low stable part, and also people whose random part happened to be unusually negative. On a retest, the stable part comes with them but the random part is drawn again, and on average it is zero.

For two measurements with the same spread and a bivariate normal relationship, the expected second score of someone who scored a certain distance from the mean is that distance multiplied by the correlation between the two measurements, r (Barnett, van der Pols and Dobson, 2005). With r = 1 nothing regresses; with r = 0 the whole gap vanishes. In the simulation below the stable part has a standard deviation of 10 and the error a standard deviation of 7, so the correlation between sittings is 10² divided by 10² + 7², that is 100/149, about 0.67 (calculated from the simulation settings, not reported by the engine): about one third of the distance to the cohort mean is expected to disappear at retest. Barnett and colleagues give the group version: the expected shift is the baseline SD times (1 − r) times a factor for how extreme the cut-off is.

Barnett and colleagues add the practical point. The effect grows with measurement error, and it shows up when follow-up is examined only in a subgroup selected on the baseline value. Screening for entry into a study is exactly that.

Worked example: no treatment, and a large improvement anyway

The data are synthetic and describe no real study or person. A screening cohort of 2000 people is simulated with fixed true scores and independent error at each sitting. The 80 lowest scorers at the first sitting are kept and split at random into a control arm and an “intervention” arm of 40 each. The intervention does nothing: the true effect is exactly zero. The second sitting is a fresh draw of error on the same true scores.

Three seeded analyses were run in R, each as a single planned test with the method chosen before the run.

The naive reading: one group, before and after

A researcher with no control group would compare the 80 post-test scores with the 80 pre-test scores, a paired t-test on the change (see the guide to paired t-tests and Wilcoxon tests for the method).

Quantity	Value
Participants analysed	80
Mean change, post minus pre (95% CI)	9.438 (7.182 to 11.69)
Paired t test	t(79) = 8.327, p < 0.001
Cohen’s d (95% CI)	0.931 (0.6662 to 1.192)
Shapiro-Wilk on the changes	p = 0.5600

Read in isolation, that is a large and highly significant improvement of about nine points, with an interval that excludes zero comfortably. It is also entirely artefact. Nothing was done to anyone, and the true scores did not change. A thesis that reported this as the effect of an intervention would be reporting regression to the mean.

The contrast: the same cohort, no selection on the pre-test

Drawing 80 people at random from the same cohort and running the same paired test removes the selection and leaves everything else unchanged.

Quantity	Value
Participants analysed	80
Mean change, post minus pre (95% CI)	-0.8375 (-2.942 to 1.267)
Paired t test	t(79) = -0.7921, p = 0.431
Cohen’s d (95% CI)	-0.08856 (-0.3078 to 0.1313)

The interval includes zero and is centred near it. The measurement error is still there at both sittings; what changed is that nobody was chosen for an extreme first value. The improvement in the first table is therefore caused by the selection, not by the measure being noisy in general.

How a control group and ANCOVA remove it

Back in the selected sample, the randomised comparison asks a different question: do the two arms differ at the post-test among people with the same pre-test score? Both arms were selected the same way, so both regress by the same amount, and the pull towards the mean is the same for the same starting score. ANCOVA models the post-test score from the arm and the pre-test score, so the pre-test score absorbs the regression and the arm effect is what is left (Vickers and Altman, 2001). The ANCOVA guide covers the mechanics, and the pre-test post-test design post explains when to prefer it to change scores.

Quantity	Value
Participants analysed	80 (40 per arm, no missing values)
Adjusted mean post-test, control (95% CI)	34.04 (30.88 to 37.19)
Adjusted mean post-test, intervention (95% CI)	34.19 (31.03 to 37.34)
Group F test, adjusted for pre-test	F(1, 77) = 0.004229, p = 0.948
Partial eta-squared for the group (95% CI)	5.492e-05 (0 to 0.01854)
Homogeneity of regression slopes	F = 0.05667, p = 0.8125
Normality of residuals (Shapiro-Wilk)	W = 0.9876, p = 0.6390

The two adjusted means are 34.19 and 34.04: essentially the same. The group effect size is close to zero and its interval runs from 0 to 0.01854, which says that the arm explains almost none of the post-test variation left after the pre-test is accounted for. The result is compatible with no difference, which is the truth in this simulation. The engine does not print an interval for the difference between the adjusted means. On the example data, confint(pairs(emmeans::emmeans(lm(post ~ pre + arm, data = data), "arm"))) gives it, and it is the number to report beside the test. The script below the article reproduces the ANCOVA table, the adjusted means and the partial eta-squared. It does not rerun the slopes or normality checks, or the two paired comparisons, which are not shown below the article.

One disclosure: the first seed we tried gave p = 0.046 for the arm effect, although the true effect is zero. A 0.05 threshold flags about 1 in 20 null comparisons like this one by chance, and the example above uses the next seed.

Compare the two readings of the same people. The one-group analysis said the scores had risen by about nine points. The comparison with a control group, built from the same participants, found no difference between treated and untreated. That the whole 9-point rise is regression to the mean follows from the design: the true effect was set to zero.

The assumption checks

Homogeneity of regression slopes: no difference detected. The interaction of arm and pre-test gave F = 0.05667, p = 0.8125. That means a difference in slopes was not detected, not that the slopes are equal; at 40 per arm the test has limited power, so look at a plot of post against pre by arm.

Normality of residuals: no departure detected. Shapiro-Wilk gave W = 0.9876, p = 0.6390; as with any non-significant assumption test, that is an absence of detected departure, not evidence of normality.

Selection and slope estimation. Because the sample was chosen from one end of the pre-test range, the slope of post on pre is estimated from a narrow range of pre-test values. The model is fitted within the selected range, so it remains usable, but the slope is estimated less precisely than it would be in an unselected sample.

Multiplicity: each of the three analyses is one planned test of one question, so no correction applies. They are three views of one simulated cohort; the seed disclosure above is the only time a run was repeated.

Spotting it in your own study

Before trusting any pre-post change, ask four questions:

1. Were participants chosen because of the baseline value? Screening for high blood pressure, low test scores, high symptom scores or poor performance is selection on an extreme.
2. How well does the measure repeat? A noisy measure regresses more. A reported test-retest correlation tells you how much of the gap to expect to vanish.
3. Is there a comparison group selected in the same way? If not, the change is a description, and the apparent effect cannot be separated from regression (Morton and Torgerson, 2003).
4. Was allocation random after selection? Randomising after selection gives the two arms the same regression, so comparing them is fair. Comparing a selected group with an unselected one is not.

Morton and Torgerson warn that mistaking regression to the mean for a treatment effect leads to wrongly crediting a treatment with an improvement that chance would have produced anyway. With a single selected group and no control, the honest options are to say so, report the change as descriptive, and avoid causal wording.

How to report it

For the worked example, a methods and results sentence would read:

Participants were the 80 lowest scorers at screening, randomised 1:1 to the two arms. Post-test scores were compared between arms with ANCOVA adjusting for the pre-test score. Adjusted mean post-test scores were 34.19 (95% CI 31.03 to 37.34) in the intervention arm and 34.04 (95% CI 30.88 to 37.19) in the control arm, F(1, 77) = 0.004229, p = 0.948; partial eta-squared 5.492e-05 (95% CI 0 to 0.01854); difference in adjusted means [estimate, 95% CI]. A pre-test-by-arm interaction was not detected (p = 0.8125).

If you must present a one-group change, add a sentence naming the selection and saying that part of the change is expected from regression to the mean, and cite one of the sources above for the mechanism.

# Results

- Test: ANCOVA (analysis of covariance)
- n: 80
- statistic: 0.004229
- p-value: 0.948
- effect size: partial eta-squared = 5.492e-05 (95% CI 0 to 0.01854)
  (two-sided CI at the run's confidence level)
- seed: 20261024

## Assumption checks
- homogeneity of regression slopes: passed=TRUE (no evidence a covariate's effect differs by group; the common-slope ANCOVA model is appropriate)
- normality (Shapiro-Wilk): passed=TRUE (no evidence against normality)

## Power
- Power is not calculated for this test.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: ANCOVA (analysis of covariance)  |  engine 0.31.1  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261024)
.data_path <- "data/regression-to-the-mean.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("post", "arm", "pre")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- { .d <- data; .d[["arm"]] <- as.factor(.d[["arm"]]); .fit <- stats::lm(`post` ~ `pre` + `arm`, data = .d); .an <- stats::anova(.fit); .em <- emmeans::emmeans(.fit, specs = "arm"); list(anova = .an, emmeans = summary(.em)) }
print(result)

# Effect size (partial_eta_sq) and its confidence interval.
effect <- { .es_d <- data; .es_d[["arm"]] <- as.factor(.es_d[["arm"]]); .es_an <- stats::anova(stats::lm(`post` ~ `pre` + `arm`, data = .es_d)); effectsize::F_to_eta2(.es_an[["F value"]][nrow(.es_an) - 1L], .es_an[["Df"]][nrow(.es_an) - 1L], .es_an[["Df"]][nrow(.es_an)], ci = 0.95, alternative = "two.sided") }
print(effect)