ANOVA test explained: one-way, Welch and two-way

An ANOVA test asks whether the means of a continuous outcome differ across groups by more than random variation would explain. There are four versions in everyday use (the classic one-way ANOVA, Welch’s ANOVA, the two-way ANOVA and the repeated-measures ANOVA), and the right one follows from how the data were collected: how many factors, whether the same units were measured more than once, and whether the groups can be assumed to share one variance.

This guide explains what an ANOVA computes, gives the design questions that pick the version, and works one example in full: Welch’s ANOVA on R’s built-in PlantGrowth data, run through a deterministic engine, with the effect size and its interval, the pairwise follow-up, the assumption checks and a sensitivity power figure. It builds on the guide to t-tests, which covers the two-group case.

What an ANOVA test does

The name stands for analysis of variance, and it describes the method rather than the question. The question is about means. The method answers it by splitting the variation in the outcome into two parts.

The first part is the variation between groups: how far each group’s mean sits from the overall mean. If the groups really differ, this part is large. The second part is the variation within groups: how much individual values scatter around their own group’s mean. That is the background noise, the variation the grouping cannot explain.

The F statistic is the ratio of the two, each divided by its degrees of freedom to make it an average (a mean square). When the group means are truly equal, the between-groups mean square estimates the same noise as the within-groups one, so F tends to sit near 1. When the means differ, the numerator grows and F moves above 1. This ratio is the classic F. Welch’s version, covered below, does not pool the within-groups variation: it weights each group by its own variance. The p-value is how often an F at least this large would turn up if all the population means were equal and the assumptions held.

The null hypothesis is that every group has the same population mean. The alternative is that at least one mean differs. A significant F does not say which groups differ, or that all of them do. That needs a follow-up, covered below.

Why not run several t-tests?

With three groups you could run three t-tests, one per pair. The problem is that each test carries its own chance of a false positive, and the chances accumulate across the tests. With many groups the chance that at least one pair comes out significant by accident becomes substantial. ANOVA asks one question about all the groups at once, at one error rate, and the pairwise follow-up then uses a procedure built to control the error rate across all the pairs.

Which ANOVA: the design decides

Answer these questions about how the data were collected before opening any software.

1. How many factors were varied? One grouping variable, such as treatment condition, points to a one-way ANOVA. Two grouping variables crossed with each other, such as treatment and dose, points to a two-way ANOVA.

2. Were the same units measured more than once? If each subject, plant or sample appears in every condition, or at several time points, the measurements are related and the design is repeated measures. If every value comes from a different unit, the groups are independent.

3. For independent groups, can you justify one shared variance? The classic one-way ANOVA pools the groups’ variances into one estimate. Welch’s ANOVA estimates each group’s variance separately. As with t-tests, Welch is the safer default.

Your design	Test	In R
One factor, independent groups (default)	Welch’s ANOVA	oneway.test(y ~ group, data = d)
One factor, independent groups, equal variances justified by design	Classic one-way ANOVA	oneway.test(y ~ group, data = d, var.equal = TRUE)
Two crossed factors, independent units	Two-way ANOVA	aov(y ~ a * b, data = d)
One factor measured within the same units	Repeated-measures ANOVA	afex::aov_ez(...), see below

The engine has a reference page for each: Welch’s ANOVA, the classic one-way ANOVA, the two-way ANOVA and the repeated-measures ANOVA. If your outcome is not continuous, or the design fits no row, the guide to choosing a statistical test starts from the outcome type instead.

One-way ANOVA: classic F or Welch

The classic one-way ANOVA assumes that every group has the same population variance. When that holds, it is the most efficient test. When it does not, how much it matters depends on the group sizes. With equal group sizes and moderately different spreads, the classic F keeps its error rate reasonably close to the stated level. With unequal group sizes it can go badly wrong: too many false positives when the smaller groups have the larger spread, too little power in the reverse case.

Welch proposed an alternative in 1951 that weights each group by its own variance and adjusts the denominator degrees of freedom, which are usually not a whole number. Delacre and colleagues compared the two by simulation in 2019 and found that under realistic departures from equal variances the classic F can give severely biased results, while Welch’s version held up better across a range of realistic scenarios. They recommend Welch’s ANOVA by default when comparing means.

Choosing between the two with a preliminary Levene test is the same mistake as with t-tests: the pre-test has little power in small samples, exactly where unequal spread does the most damage. Specify the test from the design. The one-way ANOVA, Welch or Kruskal-Wallis guide also covers the rank-based Kruskal-Wallis test and what it does and does not compare.

Worked example: Welch’s ANOVA on PlantGrowth

R ships a small dataset called PlantGrowth, from Dobson’s 1983 textbook. It records the dried weight of 30 plants: 10 grown under a control condition (ctrl) and 10 under each of two treatment conditions (trt1 and trt2). Each of the 30 values is the weight of a different plant in one condition, so the three groups are independent. This is a one-way between-groups design, not a repeated-measures one.

The research question is: does mean dried weight differ between the three conditions? Before any test, plot the individual weights by condition: spread and outliers show there first.

Seeded and run in R; Welch’s ANOVA was chosen before the run. Welch’s version, rather than the classic F, was specified for the reason given above: nothing in how the plants were grown guarantees one shared variance. With equal group sizes the two usually agree closely. The full output and the R script that reproduces it appear below the article.

The result

Quantity	Value
Plants analysed	30 (10 per condition)
Welch’s F	5.181 on 2 and 17.13 degrees of freedom
p-value	0.0174
Eta squared with its 95% interval	0.2641 (0.011 to 0.49)

Reading it

Start with the test. Welch’s F is 5.181, on 2 numerator degrees of freedom (three groups minus one) and 17.13 denominator degrees of freedom. The fractional denominator is Welch’s adjustment for unequal variances at work. The p-value is 0.0174, so at the conventional 0.05 level the data are poorly compatible with all three conditions having the same mean weight. That is all the omnibus test says: at least one condition differs.

Then the effect size. Eta squared is the share of the total variation in weight that the grouping accounts for, the between-groups sum of squares divided by the total. Here it is 0.2641, so the condition accounts for about a quarter of the variation in this sample. Two cautions apply. It is computed from the classic sums of squares, so it does not use Welch’s weighting, and its interval is built from the classic F statistic (not Welch’s F) and the noncentral F distribution, which assume equal variances. Treat the interval as approximate. And it runs from 0.011 to 0.49: compatible with the condition explaining almost nothing and with it explaining about half the variation. With 30 plants, that width is the honest summary of what the study can say about size.

Eta squared, like other effect sizes computed from sample averages, also tends to overstate the population value. Lakens’ 2013 primer explains the less biased omega squared, and when the partial and generalised versions apply.

Which groups differ: Games-Howell

A significant omnibus test calls for a pairwise follow-up. After Welch’s ANOVA the matching procedure is Games-Howell, which compares each pair with its own variances, like a Welch t-test, and adjusts the p-values for the three comparisons together. Games and Howell compared three pairwise methods by simulation in 1976, across a range of group sizes and variance conditions. With unequal group sizes of six and up, only the separate-variances statistic this procedure uses kept both the error rate for each comparison and the error rate across the family of comparisons under control.

Comparison	Adjusted p	Hedges’ g with its 95% interval
ctrl vs trt1	0.4746	0.5102 (-0.3521 to 1.359)
ctrl vs trt2	0.1129	-0.9139 (-1.793 to -0.01252)
trt1 vs trt2	0.0237	-1.289 (-2.213 to -0.3371)

Hedges’ g is the first group’s mean minus the second’s, divided by the pooled standard deviation of those two groups, with a small-sample correction. A negative value means the second group is heavier. Read together, the signs order the conditions: trt1 lightest, ctrl in the middle, trt2 heaviest.

Only one pair is significant after adjustment: trt1 against trt2, with an adjusted p of 0.0237 and a g of -1.289. Control against each treatment is not.

The ctrl against trt2 row needs care. Its g interval, -1.793 to -0.01252, just excludes zero, yet its adjusted p is 0.1129. The two are not in conflict. The interval is for one comparison on its own, with no adjustment for the other two, and it pools the two groups’ variances. The p-value is adjusted for all three comparisons and does not pool. Report the adjusted p-values for the question of which pairs differ, and the g intervals as per-pair effect sizes, saying they are unadjusted.

Reading the assumption checks

Normality by group (Shapiro-Wilk): no departure detected. The statistics were W = 0.957 (p = 0.747) for ctrl, W = 0.930 (p = 0.452) for trt1 and W = 0.941 (p = 0.564) for trt2. With 10 plants per group the test has little power, so this is not evidence that the weights are normally distributed. It only means the test found nothing to flag.

Variance homogeneity (Levene, median-centred): no difference detected. The statistic was 1.119 with p = 0.341. For the same reason, that is not evidence that the variances are equal. It also did not choose the test: Welch’s ANOVA was specified before the run and remains valid whether or not the variances are equal.

Neither check could validate the analysis on its own. Independence comes from the design, each plant being a separate unit, not from any test on the data.

Power: what this study could detect

The engine reports a sensitivity figure, which describes the design, not the result. Using the standard power formula for the classic F test, which serves as an approximation for Welch’s test, with three groups of 10 at the 0.05 level, the study had 80% power only for effects of Cohen’s f = 0.5997 or larger, which is a very large effect. The engine flags the study as underpowered on this basis. Detecting a conventional medium effect of f = 0.25 with 80% power would need 53 plants per group.

This is not “observed power”, which is a restatement of the p-value. Its practical meaning here is twofold. The non-significant comparisons, ctrl against each treatment, are inconclusive rather than evidence of no difference. And a small study reaches significance only when its estimate happens to be large, so the significant effects above more likely overstate the true effect than understate it. To size a follow-up study, start from the smallest effect that would matter in your field, not from these estimates.

Two-way ANOVA: two factors at once

When two factors are varied together, a two-way ANOVA estimates the effect of each and their interaction. R’s ToothGrowth data, the worked example in the t-test guide, is the standard case: two vitamin C delivery methods crossed with three doses, 10 animals in each of the six combinations.

# Two-way ANOVA on ToothGrowth: delivery method, dose and their interaction.
tg <- ToothGrowth
tg$dose <- factor(tg$dose)   # dose is a factor here, not a numeric slope
fit <- aov(len ~ supp * dose, data = tg)
summary(fit)

Read the interaction first. If the effect of delivery method depends on dose, a single “delivery effect” averaged over doses describes no dose in particular, and the useful results are the simple effects, the delivery difference at each dose. If the interaction is small and imprecise, the main effects are easier to interpret.

Two practical points. With equal numbers in every cell, as here, the order of the terms does not change the sums of squares. With unequal cells it does, and summary(aov()) gives sequential sums of squares that depend on the order; use car::Anova() with type II or III sums of squares instead and say which. For an effect size, partial eta squared is the usual choice for each term. The two-way ANOVA guide walks through the full analysis, and the ANCOVA guide covers adding a continuous covariate.

Repeated-measures ANOVA: the same units, measured again

When every unit is measured under each condition or at each time point, the measurements on one unit are related, and treating them as independent groups is an error. A repeated-measures ANOVA separates the stable differences between units from the variation within them, which makes it more sensitive than an independent-groups analysis whenever measurements on the same unit are positively correlated, as they usually are.

# Repeated-measures ANOVA on R's ChickWeight data: the same chicks weighed on days 0, 2, 4 and 6.
# Long format, one row per chick per day. Diet is ignored here to keep one within-subject factor.
d <- subset(ChickWeight, Time <= 6)
d$Time <- factor(d$Time)   # day is the within-chick factor here, not a numeric slope
fit <- afex::aov_ez(id = "Chick", dv = "weight", within = "Time", data = d)
fit   # Greenhouse-Geisser corrected by default, with generalised eta squared (ges)
# afex warns that it removed one chick with missing days: see the paragraph below on missing data.

It adds an assumption the other versions do not have: sphericity, that the variances of the differences between every pair of conditions are equal. Real repeated data, where measurements close in time correlate more strongly than distant ones, often violate it, and the uncorrected F then produces too many false positives. Greenhouse and Geisser’s 1959 adjustment shrinks the degrees of freedom to compensate, giving a conservative test. Report the corrected result routinely rather than deciding from a sphericity test, for the same reason Levene’s test should not pick between classic and Welch.

The classic repeated-measures ANOVA also needs every unit measured in every condition. A unit with one missing measurement drops out entirely. When measurements are missing, spaced unevenly in time, or the question is about trajectories, a linear mixed model uses all the available data. The repeated-measures ANOVA or mixed model guide explains when to switch.

How to report an ANOVA test

A complete report names the test, gives F with both degrees of freedom, the p-value, an effect size with its interval, and the follow-up with its adjustment. For the worked example:

Mean dried weight differed between the three conditions (Welch’s ANOVA, F(2, 17.13) = 5.18, p = 0.017; eta squared from the classic sums of squares = 0.26, 95% CI 0.01 to 0.49, 10 plants per condition). In Games-Howell comparisons, trt2 plants were heavier than trt1 plants (adjusted p = 0.024; Hedges’ g = 1.29, unadjusted 95% CI 0.34 to 2.21). Neither treatment differed significantly from the control (adjusted p = 0.47 and 0.11), comparisons the study had little power to resolve.

Points to get right:

• Name the version. “An ANOVA” is ambiguous. Say classic, Welch’s, two-way or repeated-measures, and for the last, which correction.
• Give both degrees of freedom, with decimals for Welch’s denominator.
• Report the p-value exactly, rather than “p < 0.05”.
• Report an effect size with its interval, and say which one: eta squared, partial eta squared or generalised eta squared are not interchangeable.
• Name the post hoc procedure and its adjustment.
• Describe non-significant pairs as inconclusive when the study could only detect large effects.

Common mistakes

Reading a significant F as “all the groups differ”. It says at least one mean differs. The pairwise follow-up decides which.

Running post hoc tests without adjustment. Unadjusted pairwise t-tests after an ANOVA bring back the inflated error rate the ANOVA was meant to avoid. The multiple comparisons guide explains the corrections.

Treating “not significant” as “no difference”. Greenland and colleagues list this among the common misinterpretations of tests. In the worked example, control against trt1 is inconclusive, not shown to be equal.

Reading the p-value as the probability that the null hypothesis is true. A p of 0.0174 is the probability of an F at least this large if the means were equal and the assumptions held. It is not the chance that they are equal.

Running an independent-groups ANOVA on repeated measurements. If the same units appear in several conditions, the independence assumption fails and the result can be badly wrong. Check the design before choosing.

Choosing classic or Welch with Levene’s test, or switching tests on one Shapiro-Wilk result. Specify the version from the design and read the checks as diagnostics.

Rerunning the example

Below the article you will find the R script exactly as the engine generated it, with the seed, and the output it printed. The script reads its data from a data/ folder. On the same package versions it reproduces Welch’s F and p-value, eta squared with its interval, the Games-Howell adjusted p-values and the sensitivity figure. The Hedges’ g values come from the engine itself; effectsize::hedges_g() on each pair of groups reproduces them. The Shapiro-Wilk and Levene checks come from the engine too: shapiro.test() within each group and car::leveneTest(weight ~ group, data = data) reproduce them.

To see how much the variance assumption matters here, add oneway.test(weight ~ group, data = data, var.equal = TRUE) to the script and compare the classic F with Welch’s. The guide to reproducible statistical analysis explains what a reviewer needs to rerun a result like this one.

# Results

- Test: Welch's ANOVA
- n: 30
- statistic: 5.181
- p-value: 0.0174
- effect size: eta-squared = 0.2641 (95% CI 0.01099 to 0.4902)
  (two-sided CI at the run's confidence level)
- seed: 20261021

## Assumption checks
- normality by group (Shapiro-Wilk): passed=TRUE
- variance homogeneity (Levene): passed=TRUE (no evidence of unequal variances)

## Post-hoc (Games-Howell; correction: single-step (Games-Howell))
- ctrl vs trt1: adj p = 0.4746 | Hedges' g 0.5102 [-0.3521, 1.359]
- ctrl vs trt2: adj p = 0.1129 | Hedges' g -0.9139 [-1.793, -0.01252]
- trt1 vs trt2: adj p = 0.0237 | Hedges' g -1.289 [-2.213, -0.3371]

groups: ctrl (ab), trt1 (a), trt2 (b) - groups sharing a letter are not significantly different at the adjusted threshold.

## Power (sensitivity, not observed power)
- minimum detectable effect at n=30: Cohen's f = 0.5997 (severely underpowered)
- to detect a conventional medium effect: 53 per group to detect f=0.25
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Welch's ANOVA  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/anova-explained.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("weight", "group")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- oneway.test(`weight` ~ `group`, data = data, var.equal = FALSE)
print(result)

# Effect size (Eta2) and its confidence interval.
effect <- { .es_d <- data; .es_d[["group"]] <- as.factor(.es_d[["group"]]); .es_d[["weight"]] <- .es_d[["weight"]] - stats::median(.es_d[["weight"]], na.rm = TRUE); effectsize::eta_squared(stats::aov(`weight` ~ `group`, data = .es_d), partial = FALSE, ci = 0.95, alternative = "two.sided") }
print(effect)

# Post-hoc pairwise comparisons (Games-Howell; correction: single-step (Games-Howell)).
data[["group"]] <- as.factor(data[["group"]])
posthoc <- { .ph_d <- data; .ph_d[["group"]] <- as.factor(.ph_d[["group"]]); PMCMRplus::gamesHowellTest(`weight` ~ `group`, data = .ph_d) }
print(posthoc)

# Sensitivity: minimum detectable effect at n=30 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.anova.test(k = 3, n = 10, sig.level = 0.05, power = 0.8)
print(power)