ANCOVA (analysis of covariance): a practical guide

Analysis of covariance, or ANCOVA, compares the means of two or more groups on a continuous outcome while adjusting for a continuous covariate. The covariate is most often a baseline measurement of the same outcome: a pre-test score, a starting weight, a reading taken before an intervention. Adjusting for it removes the part of the outcome differences that the covariate already explains, which usually makes the comparison more precise and corrects for groups that happened to start in different places.

For a randomised pre/post design, a good default is ANCOVA on the follow-up measurement with the baseline as the covariate. It tends to be more precise than comparing follow-up scores alone, and it avoids the problems of comparing change scores, which are explained below.

What ANCOVA (analysis of covariance) does

ANCOVA is a linear model with a group factor and a covariate. It fits one straight line of outcome against covariate, with the same slope in every group, and gives each group its own height. The group effect is the vertical distance between those parallel lines: how different the groups would be if they all had the same covariate value. It is a one-way ANOVA with a covariate added, so the group test is an F-test on that adjusted difference, and the same family of follow-up comparisons applies.

The covariate earns its place by soaking up outcome variation. If the covariate and outcome are strongly correlated, the residual variation shrinks, and the group comparison gains power. If they are barely correlated, adjusting buys little.

Assumptions

• Linear relationship between covariate and outcome, within each group. A scatter plot of outcome against covariate, by group, shows it quickly.
• Homogeneity of regression slopes: the covariate has the same effect on the outcome in every group. This is the assumption specific to ANCOVA, and it is tested by adding a covariate-by-group interaction and asking whether it improves the model.
• Independent observations, one outcome value per unit. Repeated measurements on the same units need a different model; see repeated-measures ANOVA or a mixed model.
• Residuals roughly normal with similar variance across groups, as in ANOVA. With moderate group sizes the method is forgiving, but strong skew or very unequal spread in small groups deserves a look.
• The covariate is not affected by the treatment. A baseline measured before the intervention is safe. A variable that the intervention changes, such as a mid-course measurement, is a mediator, and adjusting for it distorts the group effect.

Adjusted means

The group means ANCOVA reports are adjusted means, also called estimated marginal means: the predicted mean outcome for each group at the overall mean value of the covariate. When groups differ on the covariate, their raw means mix two things, the group effect and the head start. The adjusted means remove the second.

That is why the adjusted and raw differences can disagree, sometimes by a lot. If one group started higher and the covariate predicts the outcome, the raw difference exaggerates the group effect, and the adjusted difference is smaller. Report the adjusted difference together with its confidence interval, not just the p-value.

When the slopes differ

If the interaction test says the slopes differ between groups, the covariate matters more in one group than another, and “the” group effect depends on the covariate value. A single adjusted difference is then misleading, because it averages over a gap that widens or narrows along the covariate. Options in the literature include describing the group difference at several chosen covariate values, or fitting separate slopes and reporting where the groups differ. As with any test, a non-significant interaction does not prove the slopes are equal, so look at the plot and the size of the slope difference, not only the p-value.

Worked example: two teaching methods

A synthetic class of 16 students is split into 8 taught by a standard method and 8 by a new method, with a pre-test and a post-test score for each. The groups were not perfectly balanced at the start. The data are synthetic, generated in R 4.6.1.

Standard pre	Standard post	New pre	New post
48	52	58	66
52	60	63	71
55	58	66	80
60	67	70	75
62	61	72	74
58	66	68	81
65	64	75	79
50	55	61	70

The new-method group started higher: mean pre-test 66.6 against 56.3. Its mean post-test was 74.5 against 60.4, a raw difference of 14.1 points (Welch 95% CI 8.5 to 19.8). Much of that gap was there before teaching began.

Three analyses give three answers:

Analysis	Estimated effect of the new method	95% CI	p
Post-test only	14.1	8.5 to 19.8	0.0001
Change score (post minus pre)	3.75	-0.5 to 8.0	0.077
ANCOVA (post-test, adjusting for pre-test)	7.1	1.7 to 12.4	0.013

The ANCOVA used a slope of 0.68 points of post-test per point of pre-test. Its adjusted means are 63.9 for the standard method and 71.0 for the new method, a difference of 7.1. The homogeneity-of-slopes test is clearly non-significant (F = 0.01 on 1 and 12 degrees of freedom, p = 0.91), so one common slope is reasonable.

The post-test-only analysis credits the new method with the head start the students already had. The change-score analysis goes the other way. A slope of 0.68, below 1, means a high pre-test score is partly luck that does not repeat, and change scores treat the pre-test as if its slope were exactly 1. That over-corrects, and costs power here. ANCOVA estimates the slope from the data and lands between them. This is the usual pattern, and the reason Vickers and Altman (2001) recommend ANCOVA over analysing change scores, with Senn (2006) giving the fuller argument. The three answers are not equally valid; they answer different questions, and only ANCOVA’s adjusted comparison asks how students with the same pre-test score would differ.

One caution about the design. In a randomised class, the baseline gap is chance imbalance, and ANCOVA corrects for it. If students instead chose their method for reasons related to the outcome, ANCOVA can adjust only for what was measured, and cannot repair the comparison. Randomisation is what makes the adjusted difference a causal estimate.

Which one to report

• Randomised pre/post design: ANCOVA on the follow-up with baseline as the covariate, reporting the adjusted difference with its confidence interval.
• Follow-up only: loses precision and ignores baseline imbalance. Use it only when no baseline exists.
• Change scores: acceptable, but usually less efficient than ANCOVA, and misleading when the groups differ at baseline.
• Slopes clearly differ: do not report a single adjusted difference without saying so. Describe the effect at specific covariate values, or report separate slopes.
• Three or more groups: ANCOVA gives an overall test of the group effect. Pairwise adjusted contrasts between the groups need multiplicity control; see multiple comparisons.

Decide the analysis before looking at the data. Picking whichever of the three gives the smallest p-value is not a defensible procedure.

How Celeus handles this

For ANCOVA, Celeus tests homogeneity of regression slopes with a covariate-by-group interaction test and reports the result with a note in the assumptions list. It reports the adjusted (estimated marginal) mean for each group.