Factorial designs and blocking: 2x2 and beyond

A factorial design crosses two or more factors so that every level of one appears with every level of the other. A 2x2 factorial design has two factors with two levels each, four conditions in all; a 2x3 design has six. You analyse it with a two-way ANOVA that tests each factor and their interaction, and you read the interaction first. Blocking is a different tool: a block groups similar units that you cannot assign, and the analysis removes its variation instead of testing it.

The examples use real data: a 2x3 factorial on guinea pig tooth growth, plus companion runs on a blocked pea field trial and on unequal cells. The two-way ANOVA guide covers the test itself; this post is about the design. For design choices in a thesis as a whole, see the dissertation statistics guide.

What a factorial design is

A factor is a variable you set on purpose, such as a drug, a dose, a teaching method or a fertiliser. Each factor has levels: the drug or a placebo, three doses, two methods. A factorial design runs every combination of levels, and each combination is a cell. The notation lists the number of levels of each factor: 2x2 means two factors with two levels each, 2x3 means one factor with two levels and one with three, and 2x2x2 means three two-level factors and eight cells.

Each unit, whether a person, an animal or a plot, is randomised to one cell: a between-subjects factorial, the case covered here. If the same person goes through several conditions, use a repeated-measures analysis.

Why cross the factors instead of running two studies

Fisher (1935) made the case for varying several factors in one experiment. In a factorial design every unit contributes to every main effect: in a 2x2 study of a drug and an exercise programme, all participants inform the drug comparison and all of them inform the exercise comparison. Two separate two-arm trials would need about twice as many people for the same precision. Montgomery, Peters and Little (2003) describe the factorial trial as a way to test more than one intervention without that larger sample.

The second reason is the interaction: only crossed factors can show whether the effect of one depends on the level of the other.

The efficiency argument holds when the factors act roughly independently. When they interact, a main effect averages over levels of the other factor, and the useful quantities become the effects within each level (simple effects), each from fewer units. Plan the sample size for the comparison you will actually report.

A 2x3 factorial on tooth growth

The ToothGrowth data in R come from a study of vitamin C and tooth growth in guinea pigs (Crampton, 1947). Sixty animals were each given one of two delivery methods, orange juice (OJ) or ascorbic acid (VC), at one of three daily doses, 0.5, 1 or 2 mg, with 10 animals per cell. The outcome is odontoblast length (the cells that form the tooth), measured once per animal.

The analysis fits both factors and their interaction, treats dose as three categories rather than a number, and uses Type II sums of squares (how to find a p-value from a test statistic recomputes its p-values by hand). The engine reported:

Read the interaction first

Read the interaction row first, even though it is listed last, because it decides how the other two rows can be read. Its effect-size interval, 0.001 to 0.295, is compatible with anything from a negligible to a substantial interaction. The cell means, each with its 95% interval, show its shape:

The table suggests orange juice gives longer teeth than ascorbic acid at 0.5 and 1 mg, with nearly identical means at 2 mg (26.06 against 26.14). Those differences at each dose (simple effects) were not tested here; they are what the means show. Read that way, the delivery-method main effect, F(1, 54) = 15.57, averages a sizeable gap at low doses with almost none at the top dose, so “orange juice works better” without the dose qualifier would misdescribe the data. The means rise with dose under both methods, so the dose main effect (partial eta squared 0.773) reads more directly.

The cell-mean intervals are unadjusted, useful for seeing the pattern. Formal tests of the method difference at each dose are three simple-effect comparisons and need a correction for multiplicity; the multiple comparisons guide explains the options.

Check the assumptions on the residuals

A two-way ANOVA assumes independent observations, residuals that are roughly normal, and equal variance across the cells. Independence comes from the design (one measurement per animal). The engine checked the other two: Shapiro-Wilk on the residuals gave p = 0.669, and Levene’s test across the six cells gave F = 1.709, p = 0.148. Neither shows evidence of a problem, which is not proof that the assumptions hold, and with equal cells the F-test is fairly robust to modest departures. The homogeneity of variance post covers what to do when the variances do differ.

Report it

One way to write the result, filled from this run: “Tooth length was analysed with a two-way ANOVA (delivery method x dose, Type II sums of squares, n = 60, 10 per cell). The method by dose interaction was significant, F(2, 54) = 4.107, p = 0.0219, partial eta squared = 0.132, 95% CI 0.001 to 0.295; Table 2 gives cell means with 95% CIs.” Give every cell’s size and mean, and each term’s effect size with its interval; the eta squared post explains partial eta squared.

Blocking: a factor you do not test

A block is a group of units more alike than units in general: plots in one strip of a field, samples processed on one day, patients at one clinic, litter mates. The block is a property of the unit, not assigned. You randomise the treatments within each block, so each block holds every treatment (a randomised complete block design). Box, Hunter and Hunter (2005) summarise the principle as blocking what you can and randomising what you cannot.

A block enters the model like a factor, but it plays a different role:

• Purpose. A factor’s effect is the research question. A block’s effect is nuisance variation you want out of the error term.
• Assignment. Units are randomised to factor levels. Blocks are not assigned; treatments are randomised within them.
• Interaction. A factorial analysis estimates the interaction between factors. A block analysis normally assumes the treatment effect is the same in every block and fits the block as a main effect only.

The npk pea trial with and without its blocks

The npk data in R are a field trial on peas: 24 plots in 6 blocks of 4, with nitrogen, phosphate and potassium each applied or not (Venables and Ripley, 2002). Within every block, 2 plots received nitrogen and 2 did not, so nitrogen is balanced inside each block. Yield is in pounds per plot. To isolate what blocking does, the companion runs look at nitrogen alone, analysed twice on the same data.

Ignoring the blocks, a one-way ANOVA of yield on nitrogen gave F(1, 22) = 6.061, p = 0.0221, eta squared 0.216 (95% CI 0.002 to 0.482). Adding the block, the model became yield on block plus nitrogen with nitrogen tested after the block, and gave F(1, 17) = 9.36, p = 0.0071, partial eta squared 0.355 (95% CI 0.037 to 0.617).

The blocked run’s adjusted nitrogen means, 52.07 pounds per plot without nitrogen and 57.68 with it, are exactly the raw means the first analysis compares, because nitrogen is balanced within blocks. What changed is the error term. The blocks absorbed plot-to-plot variation the first analysis left in the residual, so the same difference became more precise despite the 5 degrees of freedom the block cost (22 down to 17). The larger partial eta squared reflects that smaller error term, not a larger effect, so the two are not comparable. Had the blocks been alike, the lost degrees of freedom would have weakened the test.

The block-by-nitrogen interaction, checked as an assumption (the engine labels it homogeneity of regression slopes), gave p = 0.477: no evidence that nitrogen worked differently by block.

Two caveats: phosphate and potassium are in neither model, so both residuals still contain their effects. And each block holds only 4 of the 8 nutrient combinations, so the three-way nutrient interaction is confounded with blocks.

A blocked factorial has the model block plus the full factorial of the treatments (in R, yield ~ block + N * P), with the treatment terms tested after the block.

What an unbalanced design changes in the output

A design is balanced when every cell has the same number of units; real data often are not. The companion run removed 12 of the 60 ToothGrowth animals: the last 6 from orange juice at 0.5 mg (rows 35 to 40 of the data as R ships it) and the last 6 from ascorbic acid at 2 mg (rows 25 to 30). The cell sizes became 4, 10 and 10 for orange juice and 10, 10 and 4 for ascorbic acid, 48 animals in all.

The same two-way ANOVA on the unbalanced data gave:

The interaction’s effect-size interval starts at exactly 0 although p = 0.0444 is below 0.05. No contradiction: it is a two-sided 95% interval from the noncentral F distribution, so its lower end clears zero only for p below half of 0.05. In the balanced run (p = 0.0219) it was 0.001.

Four things changed.

The factors are no longer independent. With 10 animals per cell, delivery method and dose were unrelated in the data. After the removals, orange juice holds mostly higher-dose animals and ascorbic acid mostly lower-dose ones, so a raw orange-juice average leans toward the high-dose result. Compare cell means, not raw group averages.

The sums of squares depend on the type. When the factors are correlated, part of the outcome’s variation could be credited to either factor. Type I credits it to whichever factor is entered first. Type II tests each main effect after the other main effect, and Type III after everything including the interaction. The engine uses Type II, which Langsrud (2003) recommends for unbalanced data when the interaction is small. In a balanced design all three agree (Type III only with sum-to-zero contrasts); here they would not, so say which you used. Only the main-effect rows change; the interaction test is the same under all three.

The intervals are no longer equal. In the balanced run every cell mean had the same standard error, 1.148. In the unbalanced run the cells with 10 animals have 1.115 (slightly smaller, because the pooled residual spread happens to fall) and the cells with 4 have 1.763, so the orange juice 0.5 mg cell now reads 16.00 (12.44 to 19.56), an interval about half as wide again as before. Comparisons involving small cells inherit that imprecision.

The cell means moved. The orange juice 0.5 mg mean went from 13.23 to 16.00 and the ascorbic acid 2 mg mean from 26.14 to 25.375, because the four animals left in each of those cells are not typical of the ten. Here the rows were dropped by position, assuming the row order within each cell is unrelated to tooth length, so the shift is chance. When units go missing for reasons related to the outcome, the same kind of shift is bias, and no choice of sums of squares fixes it.

The delivery-method row grew from F(1, 54) = 15.57 to F(1, 42) = 22.31 despite losing 12 animals. Three things contributed: both shrunken cells moved in orange juice’s favour (at 0.5 mg its mean rose while ascorbic acid’s stayed at 7.98; at 2 mg ascorbic acid’s fell below orange juice’s 26.06); the residual spread happened to fall; and with unequal cells the Type II main-effect test depends on the cell sizes, unlike Type III’s unweighted cell averages. The run reports Type II only, so it cannot separate them.

Plan equal allocation, and when cells end up unequal, report every cell’s size, name the type of sums of squares, and say why units are missing.

How Celeus runs these analyses

Celeus’s two-way ANOVA fits both factors and their interaction with Type II sums of squares, reports an F-test and partial eta squared with its interval per term and every cell’s mean with a confidence interval, and checks residual normality and equal variance. The suggested analyses can offer it when a dataset has a continuous outcome and two categorical columns. It analyses complete cases only.

The two-way ANOVA takes exactly two factors and has no block term, so a blocked factorial with two treatment factors is not a single analysis in Celeus today. A blocked design with one treatment factor is: choose the ANCOVA method by hand with the block column as the adjustment variable, as in the pea example. A block column stored as numbers must be declared categorical in the codebook or given text labels, or it is fitted as a straight-line trend instead of separate blocks.

Each run ships with a reproduce.R script that refits the model in plain R; the main tooth growth run’s script is below. The two-way ANOVA, ANCOVA and one-way ANOVA reference pages list what each method reports.

Choosing your design

Use a factorial design when you want two or more factors and their interaction, and can randomise units to every combination. Add blocks when units come in natural groups that differ (sites, days, batches). Keep cells equal, and size the study for the comparison you will report. For the wider choice of test, see how to choose a statistical test; for an interaction with a continuous predictor, see interaction effects in regression.

# Results

- Test: Two-way ANOVA (factorial)
- n: 60
- statistic: 4.107
- p-value: 0.0219
- effect size: partial eta-squared = 0.132 (95% CI 0.001466 to 0.295)
  (two-sided CI at the run's confidence level)
- seed: 20261101

## Assumption checks
- normality (Shapiro-Wilk): passed=TRUE (no evidence against normality)
- variance homogeneity (Levene): passed=TRUE (no evidence of unequal variances)
- sample size: passed=TRUE (adequate n)

## Term tests
| Term | Test | Statistic | df | Residual df | p-value | Partial eta-squared (95% CI) |
|---|---|---|---|---|---|---|
| supp | F | 15.57 | 1 | 54 | < 0.001 | 0.2238 (0.05865 to 0.4021) |
| dose | F | 92 | 2 | 54 | < 0.001 | 0.7731 (0.6617 to 0.8382) |
| supp:dose | F | 4.107 | 2 | 54 | 0.0219 | 0.132 (0.001466 to 0.295) |

## Power
- Power is not calculated for this test.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Two-way ANOVA (factorial)  |  engine 0.41.7  |  catalog c40-dc08
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261101)
.data_path <- "data/factorial-design.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("len", "supp", "dose")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- { .d <- data; .d[["supp"]] <- as.factor(.d[["supp"]]); .d[["dose"]] <- as.factor(.d[["dose"]]); .fit <- stats::lm(`len` ~ `supp` * `dose`, data = .d); .an <- car::Anova(.fit, type = 2); .em <- emmeans::emmeans(.fit, specs = c("supp", "dose")); list(anova = .an, emmeans = summary(.em)) }
print(result)

# Effect size (partial_eta_sq) and its confidence interval.
effect <- { .es_d <- data; .es_d[["supp"]] <- as.factor(.es_d[["supp"]]); .es_d[["dose"]] <- as.factor(.es_d[["dose"]]); .es_an <- car::Anova(stats::lm(`len` ~ `supp` * `dose`, data = .es_d), type = 2); effectsize::F_to_eta2(.es_an[["F value"]][seq_len(nrow(.es_an) - 1L)], .es_an[["Df"]][seq_len(nrow(.es_an) - 1L)], .es_an[["Df"]][nrow(.es_an)], ci = 0.95, alternative = "two.sided") }
print(effect)