One-sample t-test: comparing a mean to a reference

A one-sample t-test asks whether the mean of one set of measurements differs from a single value fixed outside the data. Most of the work is not the calculation, which is one line of R, but deciding whether your reference value really is fixed and known, and reading the result honestly when the data have a few extreme values.

This post covers when the test fits, how to run it in base R with an effect size, how it relates to the paired t-test, and what to do when the data are not normal. The worked example uses real data and a published reference value: Simon Newcomb’s 1882 light measurements, compared with the value now considered correct. For how the one-sample test sits beside the paired, Student and Welch versions, see the t-tests overview.

When a one-sample t-test fits

The design has three parts. There is one group of units, each measured once. The outcome is a continuous measurement. And there is one number you want to compare the group’s mean with, chosen before you looked at the data.

That number is the whole test. Everything depends on where it came from, and good reference values fall into a few kinds:

• A value known by definition or by a far more precise method, such as the certified concentration of a reference material, or a physical constant measured to many more digits than your instrument can reach.
• A limit or target written down in advance, such as a regulatory threshold or the nominal fill weight a production line is set to.
• A published norm from a reference population so large that its own uncertainty is negligible next to yours.

What fails is a reference with its own sampling error that is not small. If the “reference” is the mean of another study of 40 people, it is not a fixed number, it is a second sample, and the question is a two-group comparison. A one-sample test treats the reference as known exactly, so it understates the uncertainty and gives intervals that are too narrow. The guide to choosing a statistical test covers the two-group designs.

Two other conditions matter. The observations must be independent: several readings on the same person are not independent units. And the reference must not come from the same data, such as the mean of an earlier part of the series, which is circular.

What the test computes

The null hypothesis is that the true mean equals the reference value; the alternative, for the usual two-sided test, is that it differs in either direction. The test statistic is a signal over a noise: the signal is the sample mean minus the reference value, and the noise is the standard error of the mean (the sample standard deviation over the square root of n). The ratio is compared with a t distribution on n minus 1 degrees of freedom. Student’s 1908 paper derived that distribution for exactly this problem, a mean from a small sample with an estimated spread.

Why t and not z. A z test divides by a standard error built from a known population standard deviation, which you almost never have. Estimating the spread from the same data adds uncertainty, so the ratio has heavier tails than a normal curve, and the t distribution accounts for that. As the degrees of freedom grow the two curves become nearly indistinguishable, which is why a t-test on a large sample gives almost the same answer as a z test would.

The confidence interval is more useful than the p-value here, because it is on the scale of the measurement: it shows how far from the reference the true mean could plausibly be.

# One-sample t-test of a numeric vector x against a fixed reference value
reference <- 100                     # replace with your documented value
t.test(x, mu = reference)            # two-sided, 95% interval for the mean

# Effect size: Cohen's d for one sample, (mean - reference) / sd, with its interval
effectsize::cohens_d(x, mu = reference)

The effect size for one sample is Cohen’s d: the difference between the mean and the reference, divided by the sample standard deviation. Lakens’ 2013 primer sets out this and the related versions of d for t-tests. It expresses the difference in standard deviation units, which makes it comparable across studies, but it inherits whatever inflates or shrinks that standard deviation, a point the worked example makes concrete.

A one-sample test is a paired test against a constant

The overview explains that a paired t-test is a one-sample test on the within-pair differences. The reverse is just as true, and it is useful in practice. If you put the reference value in a second column, the same number on every row, a paired t-test of your measurements against that column is mathematically the one-sample t-test, not an approximation of it. The reason is arithmetic. Each paired difference is a measurement minus the reference, so the differences have mean equal to the sample mean minus the reference, and exactly the same standard deviation as the measurements, because subtracting a constant shifts values without changing their spread. The paired statistic, mean difference over its standard error on n minus 1 degrees of freedom, is therefore the one-sample statistic: the same t, degrees of freedom, p-value and interval for the difference.

d <- data.frame(x = x, reference = reference)
t.test(d$x, d$reference, paired = TRUE)   # same t, df and p as t.test(x, mu = reference)

That is how the worked example below was run. The engine that ran it offers the paired t-test and not a separate one-sample method, so the reference value sits in its own column in the data file. The reference value then travels with the data and appears in the script. The output reports the estimate and interval as a difference from the reference, and labels the test “Paired t-test”.

Worked example: Newcomb’s 1882 light measurements

In 1882 Simon Newcomb recorded a series of measurements of the time light took to traverse a known distance. R’s MASS package ships his “Third Series” of 66 measurements as the dataset newcomb, in a coded unit: each value is the passage time in nanoseconds minus 24,800, so a recorded 28 means 24,828 nanoseconds.

The reference value is 33.02 in the same units. The MASS documentation for the dataset, which takes the measurements from Stigler’s 1977 study of robust estimators, records 33.02 as the value “now considered” true for this series. It is a fixed, published number from outside the data, which is what a one-sample test needs, and the test treats it as known exactly.

The question is whether the mean of Newcomb’s measurements differs from that value. Seeded and run in R; the test was chosen before the run.

The result

Quantity	Value
Measurements analysed	66
Mean difference, measurement minus 33.02	-6.808
95% confidence interval for the difference	-9.449 to -4.166
t on 65 degrees of freedom	-5.147
p-value (two-sided)	< 0.001
Cohen’s d with its 95% interval	-0.6336 (-0.8961 to -0.3669)

Reading it

Start with the estimate. On average, Newcomb’s measurements sat 6.81 units below the reference value, about 6.8 nanoseconds of passage time.

The interval runs from -9.45 to -4.17. The data are compatible with a mean anywhere from about 4 to about 9 units below the reference, and not with a mean at the reference. The p-value says the same thing in a single number: t = -5.147 on 65 degrees of freedom, p < 0.001. If Newcomb’s process had been centred on 33.02, a mean this far from it would be very unusual in 66 independent measurements with this spread.

What the result means is narrow and worth stating exactly. It says the measurements, taken together, centre below the accepted value by more than their scatter explains. It does not say why. A shared offset, such as a calibration error that affects every reading the same way, would produce exactly this pattern, and a t-test cannot tell a shared offset from any other reason the mean is low. It says nothing about any value Newcomb himself published.

Cohen’s d is -0.6336, with a 95% interval from -0.8961 to -0.3669. Here d is the mean difference divided by the standard deviation of the measurements; because the reference column is constant, that is the same as the standard deviation of the differences. Its interval comes from the noncentral t distribution, which assumes the measurements are normal, so with the data below treat it as approximate. Keep in mind what drives it: the two extreme values discussed next sit far below the rest, so they pull the mean down and also inflate the standard deviation. The d reported here describes all 66 values, those two included.

The engine also reports a sensitivity figure for the design: with 66 observations, a two-sided test at the 0.05 level has 80% power only for effects of d = 0.3501 or larger, and a conventional medium effect of d = 0.50 would need 34 observations (the output says “34 pairs”, because each measurement is paired with the reference). That is a statement about the design, not “observed power” from this result, nor a basis for sizing a new study.

Reading the assumption checks

Normality (Shapiro-Wilk): failed, clearly. The engine tests the differences from the reference, which here are the measurements shifted by a constant, so the check is on the measurements themselves. W was 0.5912 with p < 0.001. The engine flagged the outcome as non-normal and suggested the Wilcoxon signed-rank test as the alternative.

The reason is visible in the data. The engine’s outlier note found two values beyond Tukey’s fence: Newcomb’s series contains a reading of -44 and one of -2, while the other 64 lie between 16 and 40. Values that far from the rest are the likely reason the check failed.

What does that mean for the t-test? The test needs the sampling distribution of the mean to be close to normal, not the raw values. With 66 observations, a mild skew would hardly matter; Lumley and colleagues review why the t-test holds up for non-normal data in large samples. Two gross outliers in 66 are a stronger departure than a mild skew, and there is no general rule that n = 66 is enough for any shape. Here the outliers pull the mean further from the reference and also widen the standard error, so they push in opposite directions, and you should not assume either one wins.

Sample size: passed. The 66 measurements clear the engine’s minimum-size floor. That floor is a sanity check, not evidence that the assumptions hold.

When the data are not normal: the Wilcoxon signed-rank check

The standard fallback is the Wilcoxon signed-rank test (Wilcoxon, 1945) against the same value. It ranks the absolute differences from the reference and asks whether positive and negative differences are balanced. It is far less affected by a few extreme values, because an outlier’s rank is limited no matter how far out it is.

It answers a related but different question. Under its assumption that the differences are symmetric around their centre, it tests whether that centre is at the reference value. It is not a test of the mean. Its estimate, the Hodges-Lehmann pseudo-median, is not the mean either.

# One-sample Wilcoxon signed-rank test against the same reference value,
# with the Hodges-Lehmann estimate and its interval
wilcox.test(MASS::newcomb, mu = 33.02, conf.int = TRUE)

Use it as a sensitivity check, not as a replacement chosen after seeing which p-value you prefer. If the t-test and the signed-rank test point the same way, the conclusion does not hinge on the extreme values. If they disagree, report both and say why, because the disagreement is itself a finding about the data. The same choice for paired data is covered in depth in the paired t-test or Wilcoxon signed-rank guide.

Do not delete the two outliers to make the normality check pass. Removing data because it fails a test, rather than for a documented reason fixed in advance, changes the question and makes the p-value hard to interpret.

One-sided or two-sided

A one-sided test fits only when the direction was fixed before the data were seen and a result in the other direction would be treated the same as no difference, for example a check that a contaminant is below a limit, where being far below is simply a pass. In R, add alternative = "less" or alternative = "greater". Choosing the direction after looking at the data halves the p-value for no reason. The Newcomb question is two-sided: a mean above or below the reference would both be informative.

How to report it

Give the sample size, the reference value and where it came from, the mean difference with its interval, t with its degrees of freedom, the exact p-value, and an effect size with its interval and the standard deviation it uses. For the worked example:

Newcomb’s 66 measurements of the passage time of light were compared with the value now considered correct for this series, 33.02 in the dataset’s units (as recorded in the MASS documentation for these data, from Stigler, 1977). The mean differed from it by -6.81 (95% CI -9.45 to -4.17), t(65) = -5.15, p < 0.001, Cohen’s d (sample SD) = -0.63 (95% CI -0.90 to -0.37). Two values lay far below the rest, and the data failed a Shapiro-Wilk test of normality.

Common mistakes with a one-sample t-test

A reference value with no source. If you cannot say where the number came from in one sentence, with a citation, the test has no meaning to a reader. A round number chosen because it looks typical is not a reference value.

A reference that is really another sample. A mean from a modest published study has its own uncertainty. Compare the two samples instead, using their summary statistics if that is all you have.

Reading p as the probability that the mean equals the reference. It is the probability of a result at least this extreme if the mean did equal the reference. Greenland and colleagues list this among the common misinterpretations of P values.

Treating a non-significant result as agreement with the reference. A p-value above 0.05 means the data are compatible with the reference value, not that the mean equals it. Read the interval: if it also contains differences that would matter, the result is inconclusive.

Testing many reference values one after another. Each extra test adds a chance of a false positive. Fix the comparison in advance, or adjust for the number of tests you ran.

Rerunning the example

The R script below the article is the one the engine generated. It reads a data file holding Newcomb’s 66 values and the reference value on every row; saving data.frame(passage_time = MASS::newcomb, reference_value = 33.02) as that CSV rebuilds it. It runs the paired call shown above, and on the same package versions it reproduces the t-test and the sensitivity figure. The effect size and the normality check come from the engine itself: effectsize::cohens_d(x, y, paired = TRUE) and shapiro.test(x - y) on the two columns reproduce them. The signed-rank code above is not part of that run. The guide to reproducible statistical analysis explains what a reviewer needs in order to rerun a result like this one.

# Results

- Test: Paired t-test
- n: 66
- statistic: -5.147
- p-value: < 0.001
- effect size: Cohen's d = -0.6336 (95% CI -0.8961 to -0.3669)
- 95% CI: [-9.449, -4.166]
- seed: 20261021

## Assumption checks
- normality (Shapiro-Wilk): passed=FALSE (evidence of non-normality)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=66: Cohen's d = 0.3501
- to detect a conventional medium effect: 34 pairs to detect d=0.50
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Paired t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/one-sample-t-test.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("passage_time", "reference_value")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(data[["passage_time"]], data[["reference_value"]], paired = TRUE)
print(result)

# Sensitivity: minimum detectable effect at n=66 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t.test(n = 66, sig.level = 0.05, power = 0.8, type = "paired")
print(power)