Celeus

T-tests explained: one-sample, paired and Welch

What a t-test does, how to choose the one-sample, paired, Student or Welch version, and a worked Welch example in R with Cohen's d and its interval.

Updated . Written by Matt Gencer.

Key takeaway

There are four versions in everyday use (one-sample, paired, Student's two-sample and Welch's two-sample), and choosing between them is a question about how your data were collected, not about which one gives the smallest p-value.

On this page

A t-test asks whether a mean, or a difference between two means, is larger than the random variation you would expect from data of that size. There are four versions in everyday use (one-sample, paired, Student’s two-sample and Welch’s two-sample), and choosing between them is a question about how your data were collected, not about which one gives the smallest p-value.

This guide explains what every t-test computes, gives three design questions that pick the right version, shows the base R code for each, and then works one example in full: Welch’s t-test on R’s built-in ToothGrowth data, run through a deterministic engine, with the effect size, its confidence interval, the assumption checks and a sensitivity power figure. The script below reproduces the test and the sensitivity figure, and the closing section shows how to rerun it.

What a t-test does

Every t-test has the same shape. You take a difference you care about, you divide it by an estimate of how much that difference would wobble from sample to sample, and you ask how unusual the resulting ratio is.

The difference is the signal. For a one-sample test it is the sample mean minus a reference value. For a paired test it is the mean of the within-pair differences. For a two-sample test it is the mean of one group minus the mean of the other.

The wobble is the standard error. It grows with the spread of the data and shrinks with the square root of the sample size. Two groups whose means differ by three units can give a decisive result if the measurements are tight and plentiful, and a useless one if they are noisy and few.

The ratio is the t statistic. When there is truly no difference, t tends to sit close to zero, and its sampling distribution is the t distribution with a given number of degrees of freedom. The degrees of freedom reflect how much information went into the estimate of the spread. With few observations the t distribution has heavier tails than the normal distribution, so a larger t is needed before the result counts as unusual. As the sample grows, the t distribution approaches the normal one. The distribution was published in 1908 by an author signing as Student, which is why the tests built on it carry that name, and why the equal-variance two-sample test is usually called Student’s to set it apart from Welch’s.

From the estimate, its standard error and the degrees of freedom you get two things:

  • A p-value: how often a t at least this far from zero would turn up if the true difference were exactly zero and the assumptions held.
  • A confidence interval: the range of true differences that are compatible with the data at the chosen level, usually 95%. It is built from the same ingredients, the estimate plus or minus a multiple of its standard error.

Of the two, the interval carries more information. The p-value collapses the result into one number about one hypothesis, zero difference. The interval shows the size of the difference, its direction and how precisely it was estimated, and it tells you at once whether differences that would matter in your field are still on the table.

A third quantity belongs in every report: an effect size, such as Cohen’s d, which expresses the difference in standard deviation units. It lets a reader compare results across studies that measured the outcome on different scales, and it puts the result on the scale a sample size calculation uses.

Hypotheses, tails and critical values

Every t-test sets a null hypothesis, usually that the true difference is zero, against an alternative. In a two-tailed test the alternative is a difference in either direction. It is R’s default and the right choice for almost every research question. A one-tailed test names one direction, such as “the new method gives higher values”, and puts the whole error rate on that side. It is justified only when the direction was fixed before the data were seen and a result in the other direction would be treated exactly like no result. In R, alternative = "greater" or "less" requests it, and the interval becomes one-sided too; conf.level changes the confidence level.

Older textbooks reach the decision another way: look up the critical value of t for the chosen level and degrees of freedom in a table, and call the result significant when the observed t lies further from zero. That is the same decision as comparing the p-value with the level, expressed on the t scale. Software reports the p-value and the interval instead, which carry more information than a yes-or-no comparison with a table.

The formulas

In every version, t is the estimate divided by its standard error. With x̄ for a sample mean, s for a standard deviation, n for a sample size and d̄ for the mean of the within-pair differences:

One-sample:  t = (x̄ - μ0) / (s / √n)                     df = n - 1
Paired:      t = d̄ / (s_d / √n)                          df = n - 1   (n pairs)
Student:     t = (x̄1 - x̄2) / (s_p · √(1/n1 + 1/n2))      df = n1 + n2 - 2
             s_p² = ((n1 - 1)·s1² + (n2 - 1)·s2²) / (n1 + n2 - 2)
Welch:       t = (x̄1 - x̄2) / √(s1²/n1 + s2²/n2)
             df = (s1²/n1 + s2²/n2)² / [ (s1²/n1)² / (n1 - 1) + (s2²/n2)² / (n2 - 1) ]

The Welch degrees of freedom come from the Welch-Satterthwaite approximation, which is why Welch output shows a fractional value. Each two-sided confidence interval is the estimate plus or minus the critical t for those degrees of freedom times the standard error.

Which t-test: three design questions

Before opening any software, answer three questions about how the data were collected. The answers, not the data values, pick the test.

1. How many means are you comparing, and with what? One group against a fixed number, such as a published norm, a target value or a theoretical value, points to a one-sample test. Two sets of measurements points to a paired or a two-sample test. Three or more groups means a t-test is the wrong tool; see the section on when to use something else.

2. Is each value in one set matched to a specific value in the other? If the same units were measured twice (before and after, left and right, two methods on the same sample), or the units were deliberately matched in pairs (twins, matched case and control), the data are paired. If the two sets come from different, unrelated units, they are independent. The test for this is simple: could you draw a line from each value in one column to exactly one partner in the other, for a reason that comes from the design? If yes, the data are paired.

3. For two independent groups, can you justify assuming the two groups have the same variance? Almost never with confidence, which is why Welch’s version is the default. Student’s version assumes equal variances; Welch’s does not. (F-test or t-test explains why testing the variances first does not settle it.)

Your design Test What it estimates In base R
One group against a reference value One-sample t-test Mean minus the reference value t.test(x, mu = m)
Two measurements on the same units, or matched pairs Paired t-test Mean of the within-pair differences t.test(after - before)
Two independent groups (default) Welch’s two-sample t-test Difference between the group means t.test(y ~ group, data = d)
Two independent groups, equal variances justified by design Student’s two-sample t-test Difference between the group means t.test(y ~ group, data = d, var.equal = TRUE)

The engine that ran the worked example below has a reference page for each of these methods: Welch’s t-test, Student’s t-test and the paired t-test. If your design does not fit any row of the table, the guide to choosing a statistical test starts from the outcome type and the design instead.

The one-sample t-test

The one-sample t-test compares the mean of one group with a value fixed before the data were seen. The question is whether the sample mean differs from that value by more than sampling variation would explain.

Typical uses are comparing a sample with a published reference mean, checking whether a measurement process is centred on its target, or testing a mean against a theoretical value. What matters is that the reference value comes from outside the data. A value taken from the same sample, such as “the mean of the first half”, is not a reference value, and testing against it is circular.

# One-sample t-test: is the mean height of the 15 women in R's women data
# different from a reference value set in advance?
reference_height <- 64   # inches; chosen before looking at the data
t.test(women$height, mu = reference_height)

The output gives the sample mean, t, its degrees of freedom (the number of observations minus one), the p-value and a confidence interval for the mean. Report the mean with its interval and say what the reference value is and where it came from. An effect size for this design is the difference from the reference value divided by the sample standard deviation.

The assumptions are that the observations are independent of each other and that the sample mean is close to normally distributed. The second holds when the data themselves are roughly normal, and also for skewed data once the sample is moderately large, because the sampling distribution of a mean becomes close to normal as the sample grows. With a very small, strongly skewed sample, consider whether the mean is the right summary at all.

The paired t-test is a one-sample test on differences

The paired t-test is the most misunderstood of the four, and the reason is that it is not really a two-group test. It computes the difference within each pair and then runs a one-sample t-test on those differences against zero.

That is why pairing is so powerful. Much of the variation between units, such as some people being taller, some samples more concentrated or some sites busier, appears in both measurements of a pair and cancels when you subtract. What is left is the variation of the change itself, which is usually far smaller. The same mean difference can therefore be clear in a paired analysis and invisible in an unpaired one.

# Paired t-test on R's sleep data: each of 10 people took both drugs,
# so the two columns are paired by person (the ID column).
wide <- reshape(sleep, idvar = "ID", timevar = "group", direction = "wide")
diffs <- wide$extra.2 - wide$extra.1

t.test(diffs)                     # one-sample test on the differences
t.test(wide$extra.2, wide$extra.1, paired = TRUE)  # the same test, same result

Both calls give identical results, which is the clearest way to see what the paired test is. The assumptions follow: the pairs are independent of each other, and the mean of the differences is close to normally distributed. The two original measurements do not need to be normal; only the differences matter.

Two errors are common. The first is analysing paired data as two independent groups, which throws away the pairing and usually widens the interval a great deal. The second is the reverse: pairing values that were not collected as pairs, for example by sorting two independent groups and matching them by rank. Pairing is a fact about the design. It cannot be created afterwards.

When the differences are badly skewed or contain extreme values in a small sample, the Wilcoxon signed-rank test answers a related but different question. The paired t-test or Wilcoxon signed-rank guide explains what each one tests and when the switch is worth making.

Two independent groups: Student or Welch

For two independent groups there are two versions, and the difference between them is a single assumption.

Student’s t-test pools the two groups’ variances into one estimate. That is only correct when the two populations have the same variance. Its degrees of freedom are the total sample size minus two.

Welch’s t-test estimates each group’s variance separately and adjusts the degrees of freedom to reflect how much information each group contributes. The adjusted degrees of freedom are usually not a whole number, which is the easiest way to recognise Welch output.

When the variances really are equal, the two tests give almost the same answer and Welch loses very little. When the variances differ and the group sizes are unequal, Student’s test can go badly wrong. When the smaller group has the larger spread, it produces far more false positives than its nominal level. Welch keeps its error rate close to the stated level in both situations. That asymmetry is the whole argument: Welch costs almost nothing when it is not needed and protects you when it is. R’s t.test() runs Welch by default, and Delacre, Lakens and Leys recommend it as the default in their 2017 paper.

A common practice is to run Levene’s test first and pick Student or Welch based on its result. Do not do this. A test of equal variances has little power in small samples, so it misses real differences exactly where they do the most damage, and it flags trivial differences in large samples, where they barely matter. The two-step procedure also changes the error rate of the test that follows. Zimmerman’s 2004 simulations found that this two-stage procedure fails to protect the significance level and usually makes things worse, and that using the separate-variances (Welch) test unconditionally gives the best protection when the group sizes are unequal.

Use Student’s test only when equal variances follow from the design, for example when both groups come from the same measurement process under randomisation and there is a strong reason the treatment would shift the mean without changing the spread. Even then, Welch gives you a result very close to Student’s, so the choice rarely matters when the assumption holds. The t-test or Mann-Whitney guide covers the other common decision for two independent groups, whether to compare means or ranks.

Student’s test is also a special case of linear regression. Regressing the outcome on a two-level group indicator, lm(y ~ group, data = d), gives a slope equal to the difference in means and the same p-value as t.test(..., var.equal = TRUE), with the same t up to its sign, which depends on which group is subtracted from which. That is the bridge to the models in the section on when a t-test is the wrong tool: adding a second factor or a covariate (the ANCOVA guide covers the covariate case) is one more term in the same model.

Defaults differ between packages, so check which two-sample test yours runs. R’s t.test() runs Welch unless told otherwise. Python’s scipy.stats.ttest_ind() runs Student’s test unless you pass equal_var=False. Excel’s T.TEST() takes a type argument for paired, equal-variance or unequal-variance tests and returns only the p-value, with no estimate or interval.

Worked example: Welch’s t-test on ToothGrowth

R ships a small dataset called ToothGrowth from a 1947 nutrition experiment by Crampton. Sixty guinea pigs each received vitamin C by one of two delivery methods, orange juice (OJ) or ascorbic acid (VC), at one of three doses. The outcome is odontoblast length, the length of the cells responsible for tooth growth, which this guide shortens to “length”. Each animal contributes one measurement, so the 30 animals in each delivery group are different animals: this is a two-sample design, never a paired one.

The research question for this example is deliberately the simple one: does mean length differ between the two delivery methods? We return below to what that question leaves out.

Seeded and run in R; the method, Welch’s t-test, was chosen before the run. The full output and the R script that reproduces it appear below the article.

The result

Quantity Value
Animals analysed 60 (30 per delivery method)
Mean length, orange juice (OJ) 20.66
Mean length, ascorbic acid (VC) 16.96
Difference in means, OJ minus VC 3.70
95% confidence interval for the difference -0.171 to 7.571
Welch’s t 1.915 on 55.31 degrees of freedom
p-value (two-sided) 0.0606
Cohen’s d with its 95% interval 0.4945 (-0.02151 to 1.006)

Reading it

Start with the estimate. The orange juice group’s mean length was 3.7 units higher than the ascorbic acid group’s. That is the answer to the question asked, and it is the number a reader most needs.

Next, the interval. The 95% confidence interval for the difference runs from -0.171 to 7.571, from slightly below zero to about 7.6. The data are therefore compatible with no difference at all, and also with a difference roughly twice the size of the estimate. That width is the honest summary of how much these 60 animals can tell us.

Then the test statistic and the p-value. Welch’s t is 1.915 on 55.31 degrees of freedom; the fractional degrees of freedom are Welch’s adjustment at work. The two-sided p-value is 0.0606. At the conventional 0.05 level, the result is not statistically significant, which is the same information as the interval including zero.

Then the effect size. Cohen’s d is 0.4945, so the difference is just under half a standard deviation, using the pooled standard deviation of the two groups. With equal group sizes, that is the same as averaging the two variances, so it does not depend on the equal-variance assumption. Its interval is computed the standard way for d, which does assume equal variances, so treat it as approximate. Its 95% interval runs from -0.02151 to 1.006, which again spans zero and also spans effects that most fields would call large. The effect size is what allows this result to be compared with other studies. To plan a new one, size it for the smallest effect that would matter, not for this estimate, whose interval is far too wide to plan from.

What this result does not show is that the two delivery methods are equivalent. “Not significant” is not “no difference”. The engine reports a sensitivity figure to make that concrete: with 30 animals per group, a two-sided test at the 0.05 level reached 80% power only for effects of d = 0.74 or larger. Smaller true effects, including a conventional medium effect of d = 0.50, had less than an 80% chance of reaching significance at this size. Detecting an effect of d = 0.50 with 80% power would need 64 animals per group. This is a sensitivity analysis of the design, not “observed power” computed from the result, which adds nothing to the p-value.

The correct conclusion is inconclusive: a positive difference favouring orange juice is estimated, the data are compatible with anything from no effect to a large one, and the study was not sized to tell them apart.

Reading the assumption checks

The engine ran three assumption checks alongside the test. Reading them in context is more useful than treating each as a pass or fail gate.

Normality by group (Shapiro-Wilk): failed. In the orange juice group, W was 0.918 with p = 0.0236, which is evidence that the values are not normally distributed. In the ascorbic acid group, W was 0.966 with p = 0.428, with no detectable departure from normality. Because one group failed, the engine flagged the outcome as non-normal and suggested a rank-based alternative, the Mann-Whitney test.

That flag is a prompt to think, not an instruction to switch. Two facts matter here.

First, a likely reason for the non-normality is visible in the design. Each delivery group pools 10 animals at each of three doses. If length depends on dose, each group is a mixture of three clusters of values at different levels, which does not look like one bell curve, and the Shapiro-Wilk test may notice. The ascorbic acid group pools three doses too and passed, so this is a candidate explanation, not a demonstrated one. This analysis did not test a dose effect, so read that as a likely explanation to check, not a finding: the second factor may be showing through, rather than the measurement itself being non-normal.

Second, a t-test does not need the raw values to be normal. It needs the sampling distribution of the difference in means to be close to normal, and with 30 observations per group and a moderate departure like this one, that is usually a reasonable approximation. Lumley and colleagues reviewed this for large data sets in 2002; the same central limit reasoning applies, with less margin, to moderate samples like this one and moderate departures like this one. Switching to Mann-Whitney would change the question from a difference in means to a comparison of ranks, and it would do nothing about the pooling over dose either.

Variance homogeneity (Levene): passed. Levene’s statistic was 1.214 with p = 0.275 (the median-centred form), so the test did not detect unequal variances. Given how little power such tests have, that is not evidence that the variances are equal. As explained above, this check is reported for information. It did not choose the test: Welch was specified before the run, and it remains valid whether or not the variances are equal.

Sample size: passed. With 60 animals, the sample clears the engine’s minimum-size floor. That floor is a basic sanity check. It does not show that the test’s assumptions hold, and it says nothing about power, which the sensitivity figure above covers.

The general lesson: assumption checks are diagnostics that explain the data. When one fails, ask why before you change the method. Here the most plausible answer points to the design, which leads to the next section.

What this t-test does not answer: the dose factor

ToothGrowth was not designed as a two-group comparison. It is a two-factor experiment: two delivery methods crossed with three doses (0.5, 1 and 2 mg a day), with 10 animals in each of the six combinations.

The Welch test above pools over dose. It answers a marginal question, the difference between delivery methods averaged over the three doses used in this experiment. That is a legitimate question, but it has two weaknesses.

The first is precision. Any variation due to dose is left in the within-group spread, where it inflates the standard error and widens the interval. To the extent that dose explains part of that spread, a model that accounts for dose removes it and estimates the delivery difference more precisely.

The second is that the delivery difference may depend on dose. If orange juice helps at low doses but makes no difference at the highest dose, an average over doses describes no dose in particular. That possibility is an interaction, and a two-group t-test cannot see it.

The analysis that matches the full design is a two-way ANOVA with delivery method, dose and their interaction. The two-way ANOVA guide explains how to set it up and read it. A t-test on ToothGrowth is a teaching example of the t-test; it is not the analysis a researcher would report for this experiment.

The broader point applies to your own data. Before running a t-test, list every factor in the design. If there is a second factor that was varied on purpose, or a clustering of units, a t-test on two columns is answering a narrower question than the one the study was built for.

How to report a t-test

A complete report gives the reader the test, the estimate with its interval, the test statistic with its degrees of freedom, the p-value and an effect size with its interval. For the worked example, a results sentence would read:

Mean odontoblast length was 20.66 in the orange juice group and 16.96 in the ascorbic acid group (30 animals each). Welch’s t-test gave a difference of 3.70 (95% CI -0.17 to 7.57), t(55.31) = 1.92, p = 0.061, Cohen’s d (pooled SD) = 0.49 (95% CI -0.02 to 1.01). The difference was not statistically significant, and the interval is compatible with no difference and with a moderately large one.

Points to get right:

  • Name the test. “A t-test” is ambiguous; say Welch’s, Student’s, paired or one-sample.
  • Give Welch degrees of freedom with decimals. They are not a whole number, and rounding them to a whole number hides which test was used.
  • Report the p-value exactly, to two or three decimals, rather than “p > 0.05” or “n.s.”. Use “p < 0.001” only for very small values.
  • Put the interval next to the estimate. The interval is the main result; the p-value supports it.
  • Report an effect size with its interval. Lakens’ 2013 primer explains the choices for t-tests and when each version of d is appropriate.
  • Say what you assumed and checked, briefly, in the methods section: independence from the design, and how you judged the distribution.
  • State a non-significant result as inconclusive when the interval includes meaningful effects, and add the sensitivity figure if you have it.

Common mistakes

Treating “not significant” as “no difference”. A p-value above 0.05 never shows the means are equal. The interval in the worked example includes differences of over seven units. Greenland and colleagues include it in their list of 25 common misinterpretations of P values, confidence intervals and power.

Reading the p-value as the probability that the null hypothesis is true. It is the probability of data at least this extreme if the null hypothesis were true, a different quantity. A p of 0.0606 does not mean a 6% chance that there is no difference.

Saying a test “proves” a difference, or that you “accept the null”. A test can only show that the data are or are not compatible with a hypothesis at a chosen level. Neither direction is proof.

Analysing paired data as independent. If the same units were measured twice, the unpaired test ignores the pairing and usually reports a much wider interval than the data justify.

Choosing between Student and Welch with Levene’s test. Use Welch by default; the preliminary test makes things worse, not better.

Testing normality on the pooled outcome, or switching tests on a single Shapiro-Wilk result. Normality is judged within each group, or on the paired differences, and with moderate samples the t-test tolerates moderate departures. A failed check is a reason to look at the data and the design, as the worked example illustrates.

Running many t-tests instead of one model. Comparing three or more groups with a t-test for every pair inflates the chance of at least one false positive, and every extra comparison adds to it. Use an analysis designed for several groups, and adjust for multiple comparisons when you do follow-up tests. The multiple comparisons guide explains the corrections and when each fits.

Deciding on a one-sided test after seeing the data. A one-sided test is justified only when the direction was fixed in advance and an effect in the other direction would be treated the same as no effect. Choosing it afterwards halves the p-value for no reason.

Ignoring a second factor. As the ToothGrowth example shows, a t-test on two columns can hide a factor that was part of the design.

When a t-test is the wrong tool

A t-test fits one continuous outcome and at most two means. Outside that, other methods answer the question better.

  • Three or more groups: a one-way ANOVA, Welch’s ANOVA when spreads may differ, or Kruskal-Wallis for ranks (Dunn’s test follows it), then post hoc tests to say which groups differ. The one-way ANOVA, Welch and Kruskal-Wallis guide compares them.
  • Two or more factors, such as delivery method and dose: a two-way ANOVA or a regression model.
  • A question about ranks or an ordinal outcome: the Mann-Whitney test for independent groups, or the Wilcoxon signed-rank test for paired data.
  • A question about means, but a small and badly skewed sample: a bootstrap confidence interval for the difference in means keeps the question about means and leans less on the normal approximation, though bootstrap intervals also tend to be too narrow in small samples, so they are no rescue for very few observations.
  • Several outcomes measured together on the same units: Hotelling’s T² or a MANOVA tests them jointly, rather than one t-test per outcome.
  • A binary outcome: compare proportions with a chi-square or Fisher’s exact test, or model the outcome with logistic regression.
  • Repeated measurements over time on the same units: a mixed model, rather than a series of paired tests.
  • Clustered units, such as pupils within schools or samples within batches: a model that accounts for the clustering, since a t-test assumes every observation is independent.

Rerunning the example

Below the article you will find the R script exactly as the engine generated it, with the seed and the bundled data path, and the output it printed. The script reads its data from a data/ folder; the data file is offered for download with it. On the same package versions it reproduces the Welch test and the sensitivity figure. The effect size and the assumption checks come from the engine itself: effectsize::cohens_d(len ~ supp, data = data), car::leveneTest(len ~ factor(supp), data = data) and shapiro.test() within each group reproduce them.

The script uses base R’s t.test() with var.equal = FALSE, which is Welch’s test, and the pwr package for the sensitivity figure. If you want to check whether dose matters, add a two-way model to the same script and compare its delivery estimate and interval with the ones above. The guide to reproducible statistical analysis explains what a reviewer needs in order to rerun a result like this one.

Worked example

Data: Crampton EW (1947). The growth of the odontoblasts of the incisor tooth as a criterion of the vitamin C intake of the guinea pig. Journal of Nutrition 33(5):491-504. https://doi.org/10.1093/jn/33.5.491. Seed: 20261021.

Run on Celeus engine 0.30.0, seed 20261021, public data.

Output

# Results

- Test: Welch's t-test
- n: 60
- statistic: 1.915
- p-value: 0.0606
- effect size: Cohen's d = 0.4945 (95% CI -0.02151 to 1.006)
- 95% CI: [-0.171, 7.571]
- seed: 20261021

## Assumption checks
- normality by group (Shapiro-Wilk): passed=FALSE
- variance homogeneity (Levene): passed=TRUE (no evidence of unequal variances)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=60: Cohen's d = 0.7356
- to detect a conventional medium effect: 64 per group to detect d=0.50
- alpha = 0.05, target power = 0.8
R script
# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Welch's t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/t-tests-explained.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("len", "supp")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(`len` ~ `supp`, data = data, var.equal = FALSE)
print(result)

# Sensitivity: minimum detectable effect at n=60 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t2n.test(n1 = 30, n2 = 30, sig.level = 0.05, power = 0.8)
print(power)

Common questions

What is a t-test?

A t-test compares a mean, or a difference between two means, with the amount of random variation you would expect in data of that size. It divides the observed difference by its standard error and compares that ratio with the t distribution to give a p-value and a confidence interval.

When do I use a t-test?

Use a t-test when the outcome is a continuous measurement and the question is about a mean: one group against a reference value, two related measurements on the same units, or two independent groups. With three or more groups, a second factor or a binary outcome, a different method fits better.

How do I do a t-test in R?

Base R's t.test() runs all of them. Give it one vector and mu for a one-sample test, a formula such as outcome ~ group for two independent groups (Welch by default), and the vector of within-pair differences for a paired test. Report the estimate and its interval, not only the p-value.

What is the difference between a one-sample, two-sample and paired t-test?

A one-sample test compares one mean with a fixed value. A two-sample test compares the means of two independent groups. A paired test compares two measurements on the same units by testing whether the mean of their differences is zero, which makes it a one-sample test on the differences.

Does a non-significant t-test mean there is no difference?

No. A p-value above 0.05 means the data are compatible with no difference, not that the difference is zero. Look at the confidence interval: if it also includes differences large enough to matter, the study is inconclusive, and a sensitivity power figure shows which effects it could reliably have detected.

What are the assumptions of a t-test?

The observations are independent of each other, apart from any pairing built into the design, and the mean (or the mean of the paired differences) is close to normally distributed, which holds for roughly normal data and, as samples grow, for moderate departures too; strong skew or outliers need larger samples. Student's two-sample version also assumes equal variances; Welch's does not, which is why it is the safer default.

Should I use a one-tailed or a two-tailed t-test?

Use a two-tailed test unless the direction of the effect was fixed before the data were collected and a result in the other direction would be treated exactly like no effect. Choosing a one-tailed test in the direction the data already point halves the p-value without any justification, so a two-tailed test is the safe choice whenever the direction was not committed to in advance.

Sources

  1. Student (1908). The probable error of a mean. Biometrika 6(1):1-25. doi:10.2307/2331554
  2. Welch BL (1947). The generalization of 'Student's' problem when several different population variances are involved. Biometrika 34(1-2):28-35. doi:10.1093/biomet/34.1-2.28
  3. Delacre M, Lakens D, Leys C (2017). Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology 30(1):92-101. doi:10.5334/irsp.82
  4. Zimmerman DW (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology 57(1):173-181. doi:10.1348/000711004849222
  5. Lumley T, Diehr P, Emerson S, Chen L (2002). The importance of the normality assumption in large public health data sets. Annual Review of Public Health 23:151-169. doi:10.1146/annurev.publhealth.23.100901.140546
  6. Lakens D (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. doi:10.3389/fpsyg.2013.00863
  7. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology 31(4):337-350. doi:10.1007/s10654-016-0149-3

Methods discussed

Try it on your own data

Upload a dataset and Celeus suggests a method, checks its assumptions, runs it in R, and gives you the script to rerun it.

Try it in Celeus