Two-sample t-test: formula, assumptions, example

The two-sample t-test compares the means of two independent groups: different units in each group, one measurement per unit. It divides the difference between the group means by its standard error and asks how unusual that ratio would be if the population means were equal.

This post works the formula by hand on a real dataset, R’s MASS::cats (heart weight in 47 female and 97 male cats), and then confirms every step with the output of a deterministic engine run. Because the groups are unequal in size and their spreads differ, the same data also show what separates Student’s version from Welch’s, which is the one decision every two-sample t-test needs and that is best made before the data are seen. If you are still deciding which kind of t-test fits your design, the overview of t-tests starts there; if you are choosing between a t-test and a rank test, the t-test or Mann-Whitney guide covers that question.

When a two-sample t-test fits

Three conditions make a design a two-sample t-test design.

• Two groups, each made of different units. Patients in one arm and patients in the other, plots under one treatment and plots under another, female and male animals. If the same units were measured twice, or units were deliberately matched in pairs, the data are paired and the paired t-test is the right tool.
• One value per unit, and units that do not influence each other. Several measurements per animal, pupils within classes or samples within batches break this, because the values within a cluster are more alike than the test assumes.
• A continuous outcome, and a question about means. If the outcome is ordinal, or the question is whether values in one group tend to be larger, a rank-based comparison may fit better.

With three or more groups, run one analysis of all of them rather than a t-test for every pair. When the design itself is the open question, the guide to choosing a statistical test works through it from the type of outcome.

The formula, step by step

Write the two groups’ sample sizes as n1 and n2, their means as x̄1 and x̄2, and their standard deviations as s1 and s2. The null hypothesis is that the two population means are equal, and the usual alternative is two-sided: that they differ in either direction. A one-sided alternative is justified only when the direction was fixed in the plan beforehand, as the mistakes section below explains. Student’s two-sample t-test (Student, 1908) takes four steps.

1. Pool the two variances. Student’s version assumes the two populations have the same variance, so it combines the two sample variances into one estimate, weighting each by its degrees of freedom:

pooled SD:  sp = sqrt( ((n1 - 1) s1^2 + (n2 - 1) s2^2) / (n1 + n2 - 2) )

2. Compute the standard error of the difference. The difference between two means wobbles from sample to sample. Its standard error grows with the spread and shrinks with both sample sizes:

standard error:  SE = sp * sqrt(1/n1 + 1/n2)

3. Divide the difference by its standard error. That ratio is the t statistic, a signal-to-noise ratio: the observed difference measured in units of its own sampling noise.

t = (x̄1 - x̄2) / SE

4. Compare t with the t distribution on n1 + n2 - 2 degrees of freedom. The two-sided p-value is the probability of a t at least this far from zero if the population means were equal and the assumptions held. The 95% confidence interval for the difference is the estimate plus or minus the 97.5th percentile of that t distribution times the standard error.

Welch’s version changes two things

Welch’s t-test (Welch, 1947) drops the equal-variance assumption. It uses each group’s own variance in the standard error, and it replaces the n1 + n2 - 2 degrees of freedom with an approximation that depends on how much each group contributes to that standard error:

Welch SE:  SE_W = sqrt( s1^2/n1 + s2^2/n2 )
Welch df:  (s1^2/n1 + s2^2/n2)^2 / ( (s1^2/n1)^2/(n1 - 1) + (s2^2/n2)^2/(n2 - 1) )

The difference in means is the same in both versions. Only the standard error, and therefore t, the degrees of freedom, the p-value and the interval, can change. With equal group sizes the two standard errors are identical, and the versions differ only through the degrees of freedom. With unequal group sizes they can differ a lot, as the worked example shows.

The assumptions, and what a check can and cannot tell you

• Independence. Every value comes from a different unit, and the units do not influence each other. No test on the data can confirm this; it follows from how the data were collected.
• An approximately normal sampling distribution of each mean. This holds when the data in each group are roughly normal, and also for moderately skewed data once the groups are reasonably large, because means become close to normally distributed as samples grow. With small, strongly skewed or outlier-prone groups, look at the data before relying on a t-test; the t-test or Mann-Whitney guide covers the alternatives.
• Equal population variances, for Student’s version only. Welch’s version does not need it.

Checking these with tests has limits. A Shapiro-Wilk or Levene test that is not significant means a departure was not detected, which in a small sample says little, because these tests have little power there. In a large sample they detect departures too small to matter. Letting a preliminary Levene test pick between Student’s and Welch’s version is a specific trap. In Zimmerman’s simulations, running the variance test first and then the t-test it pointed to did not hold the false-positive rate at its nominal level and often made it worse; with unequal group sizes, the best protection came from running the separate-variances test every time, whatever the variance test said (Zimmerman, 2004).

Worked example: heart weight in female and male cats

The cats data in R’s MASS package record the heart weight in grams and the body weight in kilograms of 144 adult cats, all over 2 kg, used in experiments with the drug digitalis; the data come from Fisher (1947). There are 47 female and 97 male cats, each measured once, so the two groups are independent and of very different sizes. The research question for this example is the simple one: does mean heart weight differ between female and male cats?

Seeded and run in R; Student’s t-test was chosen before the run, because this post teaches the pooled formula by hand and that formula is Student’s. That is a teaching choice, not a recommendation for these data: the engine’s own variance check flagged unequal variances, and the version to plan and report here is Welch’s, for the reasons set out below. Welch’s test was run on the same data as a second engine run, for the comparison in the next section. The Student run’s complete output, and its R script, follow the article.

The hand calculation

Start from what the engine reports: mean heart weight 9.202 g in the 47 female cats and 11.32 g in the 97 male cats, a difference (female minus male) of -2.121 g, and Cohen’s d of -0.9515. Cohen’s d in this form is the difference in means divided by the pooled standard deviation, so the pooled SD of step 1 can be read back from it as |difference| / |d|. The square root uses the two group sizes, the standard error is the pooled SD times that root, and the critical value is the 97.5th percentile of the t distribution on 142 degrees of freedom, which qt(0.975, 142) returns in R. The arithmetic, using the rounded figures:

pooled SD           sp  = 2.121 / 0.9515                 = 2.229 g
sqrt(1/n1 + 1/n2)       = sqrt(1/47 + 1/97)              = 0.1777
standard error      SE  = 2.229 * 0.1777                 = 0.3961 g
t                       = -2.121 / 0.3961                = -5.35
degrees of freedom      = 47 + 97 - 2                    = 142
97.5th percentile of t on 142 df   qt(0.975, 142)        = 1.977
margin              1.977 * 0.3961                       = 0.783 g
95% CI              -2.121 - 0.783 to -2.121 + 0.783     = -2.904 to -1.338 g

The engine’s result

Quantity	Student’s t-test
Cats analysed	144 (47 female, 97 male)
Mean heart weight, female	9.202 g
Mean heart weight, male	11.32 g
Difference, female minus male	-2.121 g
95% confidence interval for the difference	-2.904 to -1.338 g
t on degrees of freedom	-5.354 on 142
p-value (two-sided)	< 0.001
Cohen’s d (pooled SD) with its approximate 95% interval	-0.9515 (-1.315 to -0.5845)

The hand calculation agrees with the engine to the rounding used: t of -5.354 on 142 degrees of freedom and an interval from -2.904 to -1.338. The same relationship runs the other way too. Lakens (2013) gives d = t × sqrt(1/n1 + 1/n2) for Student’s test, and multiplying 5.354 by sqrt(1/47 + 1/97) returns the engine’s d of 0.9515, so the effect size and the test statistic are two views of one calculation. The critical-value route gives the same verdict as the p-value: |t| = 5.354 is far beyond 1.977, the 97.5th percentile in the block above, so the difference is significant at the two-sided 0.05 level.

Read in plain terms: on average, the female cats’ hearts weighed about 2.1 g less than the male cats’ in this sample. The data are compatible with a population difference anywhere from about 1.3 g to 2.9 g, and they are hard to reconcile with no difference at all (p < 0.001).

Student versus Welch on the same data

Here is the same comparison run as Welch’s t-test.

Quantity	Student’s t-test	Welch’s t-test
Difference, female minus male	-2.121 g	-2.121 g
95% confidence interval	-2.904 to -1.338 g	-2.764 to -1.477 g
t	-5.354	-6.518
Degrees of freedom	142	140.61
p-value (two-sided)	< 0.001	< 0.001

The estimate is identical, as it must be. What changed is the standard error. Student’s version pooled the two variances, and because the larger group, the 97 males, also had the larger spread, the pooled estimate was pulled toward that larger variance and then applied to both groups. Welch’s version used each group’s own variance, which gave a smaller standard error here, a larger t and a narrower interval.

That direction is not a rule. Ruxton (2006) reports published simulation results with unequal group sizes and unequal variances in which Student’s test had a false-positive rate over three times its nominal level when the larger variance went with the smaller group, and under a quarter of it when the larger variance went with the larger group, while the unequal-variance test never strayed far from the nominal 5%. Those figures belong to the settings he tabulates, not to every dataset, but the direction of the bias follows from how the pooling works. These cats are the second case, so Student’s version was conservative here. Swap which group is larger and the same pooling would make it too liberal, producing significant results more often than the stated 5%.

On this dataset both versions lead to the same conclusion, so the choice did not change the answer. The reason to prefer Welch’s version is that you cannot know in advance which case you are in, and Welch’s version costs little when the variances are equal. Delacre, Lakens and Leys (2017) found that it controls the false-positive rate better when variances differ and loses little robustness when they are equal, and recommend it as the default. The decision belongs in the analysis plan, before the data are seen. Use Student’s version only when equal variances follow from the design, and note that nothing in a comparison of female and male animals guarantees them.

Reading the assumption checks

The engine ran three checks alongside the test.

Normality by group (Shapiro-Wilk): no departure detected. In the female cats W was 0.979 with p = 0.543; in the male cats W was 0.977 with p = 0.0904. Neither test detected a departure from normality. That is not evidence that the data are normal, only that these tests did not flag them, and with 47 and 97 cats the group means are close to normally distributed under moderate departures anyway.

Variance homogeneity (Levene): unequal variances detected. Levene’s statistic was 14.92 with p = 0.00017, and the engine flagged the run and suggested Welch’s t-test. Read that as a description of these data, not as the rule that picks the test. The choice of Welch’s version should come from the design and the analysis plan; here, the check simply makes visible why Student’s pooled standard error differs from Welch’s.

Sample size: passed. With 144 cats the sample clears the engine’s minimum-size floor. That floor is a basic sanity check. It does not show that the assumptions hold, and it says nothing about power.

The engine also noted two heart-weight values beyond Tukey’s outlier fence, which is set from the interquartile range. That is a descriptive flag, not an error, and nothing was removed. In a real analysis, check whether such values are genuine before deciding anything about them.

The effect size, and what its interval assumes

Cohen’s d was -0.9515, so the female cats’ mean heart weight was a little under one pooled standard deviation below the males’. Its approximate 95% interval runs from -1.315 to -0.5845.

Two caveats matter here. First, the denominator is the pooled standard deviation, and with 97 males and 47 females the pool is weighted toward the males’ larger spread. A d computed against the females’ SD alone would be larger, so always say which SD a d uses. Second, the interval for d is computed the standard way, from the noncentral t distribution, which assumes equal variances. With the variance difference these data show, treat that interval as approximate. Lakens (2013) explains which version of d fits which design.

The engine also reports a sensitivity figure for the design: with 47 and 97 cats, a two-sided test at the 0.05 level had 80% power only for effects of d = 0.5013 or larger. This is a property of the sample sizes, computed before looking at the result, not “observed power”, which adds nothing to the p-value. For a new study, size it for the smallest effect that would matter in your field, not for the d estimated here.

What this comparison does not answer: body size

Heart weight scales with body size, and Fisher’s original analysis of these data was about exactly that relationship between a part and the whole. The t-test above compares heart weight between the sexes without accounting for body weight. It answers a marginal question: how do female and male cats in this sample differ in heart weight, whatever their size? It does not say whether the sexes differ in heart weight at the same body weight.

That second question needs a model with body weight as a covariate. Student’s t-test is itself a linear regression of heart weight on a female-or-male indicator, with the same t (up to its sign) and p-value, so the adjusted analysis is that regression with body weight added. The ANCOVA guide explains how to set one up and what it assumes. Before running any two-sample t-test on your own data, list the variables that could explain part of the difference and decide whether your question is the marginal one or the adjusted one.

How to report it

For these data, report Welch’s version. Name it, and give both group means, the mean difference and its confidence interval, t and its degrees of freedom, the p-value, and Cohen’s d with its interval, saying which SD the d uses and that its interval is approximate when the variances differ. For example:

Mean heart weight was 9.20 g in female cats (47 cats) and 11.32 g in male cats (97 cats). Welch’s two-sample t-test gave a difference of -2.12 g (95% CI -2.76 to -1.48), t(140.61) = -6.52, p < 0.001, Cohen’s d (pooled SD) = -0.95 (approximate 95% CI -1.32 to -0.58).

If you report Student’s version, say so and give 142 whole degrees of freedom. Giving each group’s mean, standard deviation and sample size as well lets a reader recompute either version of the test.

Common mistakes

• Running a two-sample test on paired data. Before-and-after values on the same units are not two independent groups. Treated as independent groups, paired values lose the information the pairing carries, and the interval usually comes out far wider than it should.
• Letting a variance test choose the version. Decide on Welch’s version in the plan; a preliminary Levene test makes the error rate worse, not better.
• Reading “not significant” as “no difference”. A large p-value says zero is among the differences the data cannot rule out; it does not show the means are equal. The confidence interval lists the other differences the data cannot rule out either.
• Switching to a one-sided test once the direction is known. One-sided testing needs the direction written into the plan beforehand; picking it after looking simply halves the p-value without any justification.
• Mislabelling the output. Fractional degrees of freedom mean Welch’s version was run. Reporting them as Student’s test, or rounding them to a whole number, hides which test produced the result.

Running it in R

data(cats, package = "MASS")

# Welch's two-sample t-test (R's default)
t.test(Hwt ~ Sex, data = cats)

# Student's two-sample t-test (pooled variance)
t.test(Hwt ~ Sex, data = cats, var.equal = TRUE)

# The hand calculation from the summary statistics
n  <- table(cats$Sex)
m  <- tapply(cats$Hwt, cats$Sex, mean)
s  <- tapply(cats$Hwt, cats$Sex, sd)
sp <- sqrt(((n[1] - 1) * s[1]^2 + (n[2] - 1) * s[2]^2) / (sum(n) - 2))
se <- sp * sqrt(1 / n[1] + 1 / n[2])
(m[1] - m[2]) / se              # Student's t

Other software does not all share R’s Welch default. In Python, SciPy’s scipy.stats.ttest_ind() runs Student’s version unless you pass equal_var=False. Excel’s T.TEST() has no default: its required type argument is 2 for Student’s version and 3 for Welch’s. The engine’s own run used the same t.test() call with var.equal = TRUE. Its reference pages describe Student’s t-test and Welch’s t-test as the engine runs them.

Rerunning the example

After the article comes the engine’s R script, unedited, with its seed and the path to the bundled data, followed by what it printed. That script recomputes two things: Student’s t-test and the sensitivity figure. The script does not recompute Cohen’s d, its interval or the assumption checks; those are the engine’s own output, and effectsize::cohens_d(Hwt ~ Sex, data = data), car::leveneTest(Hwt ~ factor(Sex), data = data) and shapiro.test() in each group give the same figures. It does not recompute the Welch figures either, which come from a separate engine run on the same data. For what a reviewer needs in order to rerun a result like this, see the guide to reproducible statistical analysis.

# Results

- Test: Student's t-test
- n: 144
- statistic: -5.354
- p-value: < 0.001
- effect size: Cohen's d = -0.9515 (95% CI -1.315 to -0.5845)
- 95% CI: [-2.904, -1.338]
- seed: 20261022

## Assumption checks
- normality by group (Shapiro-Wilk): passed=TRUE
- variance homogeneity (Levene): passed=FALSE (evidence of unequal variances)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=144: Cohen's d = 0.5013
- to detect a conventional medium effect: 64 per group to detect d=0.50
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Student's t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/two-sample-t-test.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("Hwt", "Sex")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(`Hwt` ~ `Sex`, data = data, var.equal = TRUE)
print(result)

# Sensitivity: minimum detectable effect at n=144 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t2n.test(n1 = 47, n2 = 97, sig.level = 0.05, power = 0.8)
print(power)