Two-way ANOVA (factorial)

Shown in the app as two_way_anova. Part of the method reference, under Comparing groups.

When to use

A continuous outcome and TWO categorical factors (2+ levels each): tests each factor's main effect and whether the two factors INTERACT (whether one factor's effect on the outcome depends on the level of the other).

What you provide

Outcome: one numeric column.
The two factors: two categorical columns.

Options

Missing data

Complete case (listwise deletion)

Multiple comparisons

Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.

Confidence level

Every interval in the result is reported at the level you choose:

Other methods under Comparing groups:

Guides that discuss this method

Two-way ANOVA: interaction, main effects, post hoc

How to read the output

Two-way ANOVA compares a numeric outcome across the combinations of two categorical factors, for example plant height under three fertilisers and two watering schedules, or the number of yarn breaks by wool type and loom tension. Each factor needs at least two levels, and only rows with a value for the outcome and both factors are used.

The interaction test. The headline result is the test of the interaction: does the effect of one factor on the outcome depend on the level of the other? It is an F-test with two degrees-of-freedom values (numerator and denominator), and it has no single point estimate or confidence interval. The effect size is partial eta-squared for the interaction, from 0 to 1: the interaction’s sum of squares as a proportion of itself plus the residual sum of squares.

Main effects. The test for each factor on its own is part of the same ANOVA table, which the reproducible R script prints in full, but it is not the headline. These tests use Type II sums of squares: each factor is adjusted for the other factor but tested without the interaction term. When the interaction is substantial, a main effect averages over effects that differ from one level of the other factor to the next, so it is hard to interpret. Celeus does not warn about this, so read the interaction first. Langsrud (2003) sets out why Type II rather than Type III tests suit unbalanced designs.

Cell means. The table of results lists the estimated mean outcome, with its confidence interval, for every combination of the two factors. These are the means themselves, not differences between them, so they show the pattern the interaction test is about.

Checks and limits. Celeus checks the normality of the residuals (Shapiro-Wilk), equal variance across the cells (Levene’s test, in the median-centred form of Brown and Forsythe), and warns when there are fewer than 10 observations. It does not run a post hoc test or adjust p-values for multiple comparisons after a two-way ANOVA. If you compare individual cells or factor levels yourself, treat them as a family of tests; see multiple comparisons.

Sources: Fisher RA (1925), Statistical Methods for Research Workers, Oliver and Boyd. Langsrud O (2003), Statistics and Computing 13(2):163-167, doi:10.1023/A:1023260610025. Brown MB, Forsythe AB (1974), Journal of the American Statistical Association 69(346):364-367, doi:10.1080/01621459.1974.10482955.

Worked examples

Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.

datasets::warpbreaks

Source: Published dataset distributed with an R package
Licence: base R (GPL)
Reference: Tippett (1950), Technological Applications of Statistics

Quantity	Celeus	Independent computation	Published value	Agrees
Test statistic	4.18907	4.18907	-	Yes
Degrees of freedom	2,48	2,48	-	Yes
p-value	0.0210442	0.0210442	-	Yes
Test statistic for each model term	3.76529, 8.49805, 4.18907	3.76529, 8.49805, 4.18907	-	Yes
p-value for each model term	0.058213000, 0.000692621, 0.021044200	0.058213000, 0.000692621, 0.021044200	-	Yes
Partial eta-squared for each model term	0.0727377, 0.2614940, 0.1486060	0.0727377, 0.2614940, 0.1486060	-	Yes
Effect size confidence interval	0.00214424, 0.32360900	0.00214424, 0.32360900	-	Yes

datasets::ToothGrowth (dose stored as the numbers 0.5, 1, 2)

Source: Published dataset distributed with an R package
Licence: base R (GPL)
Reference: Crampton (1947), Journal of Nutrition 33(5):491-504 (the data); independent computation

Quantity	Celeus	Independent computation	Published value	Agrees
Test statistic	4.10699	4.10699	-	Yes
Degrees of freedom	2,54	2,54	-	Yes
p-value	0.0218603	0.0218603	-	Yes
Test statistic for each model term	15.57200, 92.00000, 4.10699	15.57200, 92.00000, 4.10699	-	Yes
p-value for each model term	2.31183e-04, 4.04629e-18, 2.18603e-02	2.31183e-04, 4.04629e-18, 2.18603e-02	-	Yes
Partial eta-squared for each model term	0.223825, 0.773109, 0.132028	0.223825, 0.773109, 0.132028	-	Yes
Effect size confidence interval	0.00146557, 0.29497200	0.00146556, 0.29497200	-	Yes

datasets::warpbreaks (tension L and H, first four looms per wool x tension cell)

Source: Published dataset distributed with an R package
Licence: base R (GPL)
Reference: Tippett (1950), Technological Applications of Statistics (the data); independent computation

Quantity	Celeus	Independent computation	Published value	Agrees
Test statistic	0.686157	0.686157	-	Yes
Degrees of freedom	1,12	1,12	-	Yes
p-value	0.423644	0.423644	-	Yes
Test statistic for each model term	3.238240, 1.508570, 0.686157	3.238240, 1.508570, 0.686157	-	Yes
p-value for each model term	0.0971136, 0.2429030, 0.4236440	0.0971136, 0.2429030, 0.4236440	-	Yes
Partial eta-squared for each model term	0.2125070, 0.1116750, 0.0540871	0.2125070, 0.1116750, 0.0540871	-	Yes
Effect size confidence interval	0.00000, 0.39443	0.00000, 0.39443	-	Yes

datasets::warpbreaks without rows 1, 2, 3, 20 and 40 (unbalanced cells)

Source: Published dataset distributed with an R package
Licence: base R (GPL)
Reference: Tippett (1950), Technological Applications of Statistics (the data); independent computation

Quantity	Celeus	Independent computation	Published value	Agrees
Test statistic	6.18233	6.18233	-	Yes
Degrees of freedom	2,43	2,43	-	Yes
p-value	0.00436585	0.00436585	-	Yes
Test statistic for each model term	3.87832, 8.25136, 6.18233	3.87832, 8.25136, 6.18233	-	Yes
p-value for each model term	0.055378100, 0.000926896, 0.004365850	0.055378100, 0.000926896, 0.004365850	-	Yes
Partial eta-squared for each model term	0.0827317, 0.2773440, 0.2233310	0.0827317, 0.2773440, 0.2233310	-	Yes
Effect size confidence interval	0.0504248, 0.3806400	0.0504248, 0.3806400	-	Yes
Confidence level of the effect size interval	0.9	0.9	-	Yes

datasets::warpbreaks with the outcome removed on rows 5, 23 and 41

Source: Published dataset distributed with an R package
Licence: base R (GPL)
Reference: Tippett (1950), Technological Applications of Statistics (the data); independent computation

Quantity	Celeus	Independent computation	Published value	Agrees
Test statistic	2.93958	2.93958	-	Yes
Degrees of freedom	2,45	2,45	-	Yes
p-value	0.0631148	0.0631148	-	Yes
Test statistic for each model term	4.05590, 6.48209, 2.93958	4.05590, 6.48209, 2.93958	-	Yes
p-value for each model term	0.05001940, 0.00335883, 0.06311480	0.05001940, 0.00335883, 0.06311480	-	Yes
Partial eta-squared for each model term	0.0826791, 0.2236590, 0.1155520	0.0826791, 0.2236590, 0.1155520	-	Yes
Effect size confidence interval	0.000000, 0.289467	0.000000, 0.289467	-	Yes
Number of observations	51	51	-	Yes

Validation report

References

Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh.