F-test vs t-test: what each one compares

“F-test or t-test?” has no single answer, because “F-test” is not one test. It is the name for any test whose statistic follows an F distribution, and two of them meet a two-group comparison: the ANOVA F, which compares means and on two groups is the t-test in another form, and the variance-ratio F, which compares spreads and answers a different question entirely.

This post separates the two, shows on one dataset that the two-group ANOVA F is exactly Student’s t squared, says precisely why that identity does not carry over to Welch’s t, and explains why the variance-ratio F should not be used to pick between Student’s and Welch’s t-test. The example uses R’s built-in ToothGrowth data.

What a t-test compares

A two-sample t-test compares two group means. It divides the difference between them by its standard error and reads the ratio against a t distribution. The two versions differ in that standard error. Student’s version pools the two sample variances into one estimate, which assumes the groups share a population variance. Welch’s version keeps the two variances separate and adjusts the degrees of freedom to match, so it does not need that assumption. The t-tests explained post covers both, with the one-sample and paired versions.

What an F-test compares

An F statistic is a ratio of two variance estimates. Under its null hypothesis it follows an F distribution with two degrees of freedom, one for the numerator and one for the denominator. Which variances go into the ratio decides what the test is about.

• The variance-ratio F test divides one group’s sample variance by the other’s. It asks whether two populations have the same spread. In R it is var.test(). It says nothing about means.
• The one-way ANOVA F divides the variance between group means (the between-group mean square) by the variance within groups (the within-group mean square). It asks whether all group means are equal. With three or more groups it is the usual omnibus test; with two, it asks the same question as the t-test. The one-way ANOVA, Welch and Kruskal-Wallis guide covers the general case.
• Levene’s test is a third F you will meet in the same output: an ANOVA F run on each value’s absolute distance from its group centre. It is another test of equal spread, and Brown and Forsythe found that versions centred on a more robust location estimate, such as the median, hold up better than the original mean-centred test when the data are not normal.
• The regression F tests a group of regression terms at once: all predictors together, or a full model against a smaller nested one. Each single coefficient gets its own t-test. When the group is one term, its F is that coefficient’s t squared, the same identity shown below.

So “F-test vs t-test” is really two questions: against the ANOVA F, on two groups they are the same test; against the variance-ratio F, they test different things.

	Two-sample t-test	ANOVA F	Variance-ratio F
Question	Are two means equal?	Are all group means equal?	Are two variances equal?
Statistic	Mean difference over its standard error	Between-group over within-group mean square	One sample variance over the other
Groups	Two	Two or more	Two
Direction	Signed; one- or two-sided	Unsigned; always positive	One- or two-sided on the ratio
Main assumptions	Independence, roughly normal groups; Student’s adds equal variances	Independence, normal residuals, equal variances (Welch’s ANOVA drops the last)	Independence and normality, with little tolerance for departures

The worked example: one comparison, four runs

ToothGrowth records the length of odontoblasts, the cells responsible for tooth growth (tooth length for short), in 60 guinea pigs, 30 given vitamin C as orange juice (OJ) and 30 as ascorbic acid (VC), each at one of three doses. Every animal appears once, so the two delivery groups are independent. Pooling over dose, as here, answers only the marginal question about delivery method; the full design is a two-way layout, covered in the two-way ANOVA guide.

The same two groups went through four engine runs: Student’s t-test (the example shown at the end of this post), the classic one-way ANOVA, Welch’s t-test and Welch’s one-way ANOVA.

Run	Statistic	Degrees of freedom	p-value
Student’s t-test	t = 1.915	58	0.0604
Classic one-way ANOVA	F = 3.668	1 and 58	0.0604
Welch’s t-test	t = 1.915	55.31	0.0606
Welch’s one-way ANOVA	F = 3.668	1 and 55.31	0.0606

The group means were 20.66 for OJ and 16.96 for VC, a difference of 3.7 units.

F equals Student’s t squared

Squaring Student’s t of 1.9153 gives 3.668, the ANOVA F, and the two p-values agree at 0.0604. That is not a coincidence of this dataset. With two groups, the between-group mean square in ANOVA is a function of the squared difference in means, and the within-group mean square is exactly the pooled variance that Student’s t uses. The ratio works out to t squared. An F with 1 numerator degree of freedom and ν denominator degrees of freedom is the distribution of the square of a t with ν degrees of freedom, so the two tests give the same p-value for a two-sided question. The identity holds for any group sizes, balanced or not.

data(ToothGrowth)
t_student <- t.test(len ~ supp, data = ToothGrowth, var.equal = TRUE)
f_classic <- oneway.test(len ~ supp, data = ToothGrowth, var.equal = TRUE)
unname(t_student$statistic)^2   # equals f_classic$statistic

One thing the t-test gives that the F does not is a direction and an interval. Student’s 95% interval for the OJ minus VC difference ran from -0.167 to 7.567. The ANOVA F is always positive and two-sided by construction, so for two groups the t-test is the more informative way to report the same result.

Why the identity does not carry over to Welch

Welch’s t-test and the classic ANOVA F answer the same question under different assumptions, and they are not each other’s square in general. Precisely:

1. Student’s t squared equals the classic ANOVA F, always. Both rest on the pooled within-group variance.
2. Welch’s t squared equals Welch’s ANOVA F with two groups. Both use separate variances and the same Welch-Satterthwaite degrees of freedom: here F = 3.668 on 1 and 55.31, with the same p-value of 0.0606 as Welch’s t.
3. Welch’s t squared generally differs from the classic ANOVA F when group sizes differ. With unequal groups the separate-variance standard error differs from the pooled one unless the two sample variances happen to be equal, so the two statistics differ.

ToothGrowth hides point 3 because its groups are balanced. With 30 animals in each, the separate-variance and pooled standard errors are algebraically identical, so Welch’s t equals Student’s t (both 1.915) and all four statistics square up. The degrees of freedom still differ, 55.31 against 58, so the p-values differ slightly: 0.0606 against 0.0604. The intervals moved too: Welch’s 95% interval for the difference was -0.171 to 7.571.

Why the variance-ratio F should not choose between Student and Welch

A common recipe runs var.test() first and uses Student’s t if it is not significant, Welch’s t if it is. The recipe has two problems.

First, it does not protect what it is meant to protect. Markowski and Markowski evaluated the variance-ratio F as a preliminary test for the two-sample t-test. With normal data and equal group sizes, they found the t-test insensitive to unequal variances, so no preliminary test was needed. With unequal sizes, the F test had a small chance of detecting many of the variance differences that made the t-test perform poorly. Zimmerman reached the same verdict for the two-stage procedure with Levene’s test: it fails to protect the significance level, and a separate-variances test used unconditionally whenever sample sizes are unequal is the better protection.

Second, the variance-ratio F is sensitive to non-normality, which Markowski and Markowski confirmed when sampling from other distributions. A test of spread that reacts to the shape of the data is a poor gatekeeper for a test of means.

The alternative is simple: use Welch’s t-test by default. Delacre, Lakens and Leys recommend it as the default because it controls the false-positive rate better when variances differ and loses little robustness when Student’s assumptions hold. Use Student’s t when there is a reason from the design, not from a variance test, to expect equal spread.

If you want to describe the spread anyway, report the two standard deviations and, if you test them, read the result correctly. On ToothGrowth the engine’s Levene check (median-centred) gave F = 1.214 with p = 0.275: no difference in spread was detected. That is not evidence that the variances are equal.

var.test(len ~ supp, data = ToothGrowth)   # ratio of the two sample variances: spread, not means

Reading the rest of the output

Effect size. Cohen’s d was 0.4945, with a 95% interval from -0.02151 to 1.006. That d divides the mean difference by the pooled standard deviation, and its interval comes from the noncentral t distribution, which assumes equal variances and normal data. The ANOVA run reports the same result as eta squared, 0.05948, with a 95% interval from 0 to 0.2076: the share of the total variation in tooth length in this sample that lies between the two delivery groups.

Assumption checks. Shapiro-Wilk flagged non-normality in the OJ group (p = 0.0236) but not in VC (p = 0.428). A non-significant check in VC is not evidence that VC is normal either. How much this matters for a test of means depends on the sample size and on how skewed the data are, which the t-tests explained post discusses. One possible explanation to check, not a finding of this run, is that each group pools three dose levels. The engine’s sample-size check passed, but that check is a minimum floor, not a judgement that the design is adequate.

What the result says. With p = 0.0604, the data are compatible with no difference in mean tooth length between the delivery methods, and with anything from a VC advantage of 0.167 units to an OJ advantage of 7.567 units. The result is inconclusive, not evidence of no difference. The sensitivity figure makes the limit concrete: at 30 per group and a two-sided 0.05 level, the design has 80% power only for effects of about d = 0.7356 or larger. Any study planning should start from the smallest effect that matters in the field, not from this estimate.

Reporting it. With Welch’s version as the default, in APA style: Welch’s t(55.31) = 1.92, p = .061, mean difference 3.70, 95% CI [-0.17, 7.57], Cohen’s d (pooled SD) = 0.49, 95% CI [-0.02, 1.01]. Student’s version would read t(58) = 1.92, p = .060, and the two-group ANOVA F(1, 58) = 3.67, p = .060: the same result without the direction or the interval.

Which test to use

• Two independent groups, a question about means: Welch’s t-test by default, or Student’s t where the design justifies equal variances. The two-group ANOVA adds nothing that Student’s t does not already give.
• Three or more groups: a one-way ANOVA, Welch’s ANOVA by default when spreads may differ, or Kruskal-Wallis for ranks. Running pairwise t-tests instead inflates the false-positive rate; see the multiple comparisons guide.
• A question about spread itself: a test of equal variances, preferably the median-centred Levene test because the variance-ratio F depends heavily on normality.
• Non-normal data with a question about the distribution as a whole: see t-test vs Mann-Whitney.

For the wider decision, the guide to choosing a statistical test starts from the design. Method notes: Student’s t-test, Welch’s t-test, one-way ANOVA and Welch’s ANOVA.

Rerunning the example

The script below is the engine’s own reproduce script for the Student’s t-test run. It recomputes the t statistic, its p-value and interval, and the sensitivity power figure from the bundled data. It does not recompute Cohen’s d, the assumption checks or the three companion runs; the R lines above recompute the ANOVA F and the variance ratio from R’s own copy of the data.

# Results

- Test: Student's t-test
- n: 60
- statistic: 1.915
- p-value: 0.0604
- effect size: Cohen's d = 0.4945 (95% CI -0.02151 to 1.006)
- 95% CI: [-0.167, 7.567]
- seed: 20261022

## Assumption checks
- normality by group (Shapiro-Wilk): passed=FALSE
- variance homogeneity (Levene): passed=TRUE (no evidence of unequal variances)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=60: Cohen's d = 0.7356
- to detect a conventional medium effect: 64 per group to detect d=0.50
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Student's t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/f-test-vs-t-test.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("len", "supp")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(`len` ~ `supp`, data = data, var.equal = TRUE)
print(result)

# Sensitivity: minimum detectable effect at n=60 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t2n.test(n1 = 30, n2 = 30, sig.level = 0.05, power = 0.8)
print(power)