Do you need to test for normality before a t-test?

Many analysis guides tell you to run a Shapiro-Wilk test on your data before a t-test, use the t-test if the result is not significant and switch to a non-parametric test if it is. Usually you should not. The pre-test answers a different question from the one the t-test needs answered, its answer depends more on your sample size than on whether the departure matters, and using it as a gate changes the behaviour of the test that follows.

This post explains what the t-test assumes, why a normality pre-test points the wrong way at both small and large sample sizes, and what to check instead. One engine-run example has 200 units per group and a mild skew, where Shapiro-Wilk correctly finds the skew and the t-test is still the right analysis. A short simulation at the end shows the pattern for your sample size.

The short answer

Do not use a normality test as a switch. Decide on the analysis from the design and the question before you look at the data, then look at the data within each group, with a plot rather than a p-value, to see whether anything is badly wrong. A formal test of normality is a reasonable tool for a different job: checking whether a distribution is normal when that is itself the question. Shapiro and Wilk introduced their test in 1965, and it does that job well. The alternatives (Lilliefors-corrected Kolmogorov-Smirnov, Anderson-Darling, D’Agostino’s skewness and kurtosis test) share the sample-size problem described below.

What a t-test actually assumes

A two-sample t-test compares two means. Its p-value and confidence interval are exact when the observations in each group are independent and normally distributed. But exactness is not the practical requirement. What the calculation needs is that the difference in sample means has a sampling distribution close to normal, so that dividing it by its standard error gives something close to a t distribution.

The difference in means is an average, and averages become close to normal as the sample grows, whatever the shape of the raw data, provided the variance is finite. That is the central limit theorem, and it is why Lumley and colleagues could write in 2002 that the belief that the t-test is valid only for normally distributed outcomes is widespread but incorrect: in large samples it is valid for any distribution, which they demonstrated by simulation in extremely non-normal data.

Two features of the two-sample design add to that robustness. When both groups come from the same distribution and have the same size, as they do when there is no difference of any kind, the two sample means have identical distributions, so their difference is symmetric about zero and skew in the raw data largely cancels. And Welch’s version of the test does not need the two variances to be equal. The robustness is weakest when the groups are small, the skew is strong, and the groups differ in size or in shape.

So the useful question is not “are my data normal?” (real data rarely are) but “is the departure large enough, for a sample this size, to distort a test about means?” A normality test cannot answer that.

Why a normality pre-test points the wrong way

A Shapiro-Wilk test, like any significance test, gets more sensitive as the sample grows. That is the opposite of what you need from a check on a t-test’s assumptions.

In small samples, it misses departures that matter. With 10 observations per group, the test has little power to detect even a clear skew. That is true of any significance test at small n; the simulation at the end shows it. A non-significant result means only that non-normality was not detected, not that the data are normal. Yet small samples are where the t-test leans most on normality, because the central limit theorem has had little to work with. The check is weakest where it is needed most.

In large samples, it flags departures that do not matter. With a few hundred observations per group, the test detects a mild skew that has almost no effect on a test about means, because by then the sampling distribution of the mean is very close to normal. The check is strongest where it is needed least.

So the pre-test’s verdict is driven largely by your sample size, and both the significant result at a large size and the non-significant one at a small size can send you the wrong way.

The two-stage procedure changes the test you run

There is a second, subtler problem. When you run a t-test only on the samples that passed a normality test, you are no longer running a plain t-test. You are running a two-stage procedure, and the samples that reach the t-test are a selected subset.

Rochon, Gondan and Kieser studied this in 2012 with a simulation of the common two-stage rule: a Shapiro-Wilk test first, then Student’s t-test if it passed and a Mann-Whitney test if it failed, on equal-sized samples from exponential, uniform and normal distributions. They found that the preliminary test seriously altered the conditional Type I error rates of both the t-test and the Mann-Whitney test, that is, their error rates among the samples routed to each. The overall Type I error rate and power of the whole procedure stayed within acceptable limits in the examples they studied, but they noted that the procedure might be considered incorrect from a formal perspective.

Schucany and Ng reached a sharper conclusion for the one-sample t-test in 2006. Screening samples with a Shapiro-Wilk test did not correct the wrong significance levels in non-normal populations, and for exponential data it made them worse. They recommended graphical diagnostics over a formal pretest.

Worked example: a mild skew with 200 units per group

To make the large-sample half of the argument concrete, here is one analysis run through a deterministic engine with a fixed seed. The data are synthetic and describe no real study: two independent groups, control and treated, with 200 units each and one strictly positive measurement per unit. Both groups were generated from the same right-skewed shape with different centres, the way many positive measurements look.

Welch’s t-test was chosen before the run.

The result

Quantity	Value
Units analysed	400 (200 per group)
Mean, control	51.46
Mean, treated	55.28
Difference in means, control minus treated	-3.82
95% confidence interval for the difference	-6.33 to -1.304
Welch’s t	-2.986 on 396.68 degrees of freedom
p-value (two-sided)	0.003
Cohen’s d with its 95% interval	-0.2986 (-0.4955 to -0.1013)
Shapiro-Wilk, control	W = 0.944, p < 0.001
Shapiro-Wilk, treated	W = 0.971, p < 0.001
Levene (median-centred)	p = 0.321

The treated group’s mean was 3.82 units higher, with a 95% interval for the control minus treated difference from -6.33 to -1.304, so the data are compatible with a treated mean higher by anything from a little over one unit to over six. Cohen’s d uses the pooled standard deviation of the two groups; with equal group sizes that is the square root of the average of the two variances, so the estimate itself does not rely on equal variances, but its interval is computed the standard way for d, which does assume them, so treat the interval as approximate.

Reading the normality check

The engine’s Shapiro-Wilk check failed in both groups, with p-values below 0.001, and the engine flagged the outcome as non-normal and suggested a rank-based alternative, the Mann-Whitney test. A separate note reported values beyond Tukey’s conventional outlier fence, which is what a long right tail looks like; nothing was removed.

The check is correct: these data are skewed, because they were generated that way. The question is what that changes, and here the answer is very little.

• The sample size is large. With 200 per group, the sampling distribution of each mean is close to normal for a mild skew like this one, which is the regime Lumley and colleagues describe.
• The design is balanced and the shapes match. Both groups share one shape and one size, which is the case where skew does the least damage to a two-sample test. Levene’s test did not detect unequal spread (p = 0.321), which is not evidence that the spreads are equal; Welch’s test does not need them to be.
• The question is about means. Switching to Mann-Whitney would not repair the t-test; it would change the question. Fagerland showed in 2012 that in large skewed samples the rank test answers a different question, and recommended t-tests and their confidence intervals for large studies even with heavily skewed data.

So the flag is a prompt to look, not an instruction to switch. Here, looking confirms a mild, matched skew at a large sample size, and the Welch result stands. If the measurement were heavily skewed and the means were a poor summary, that would be a reason to rethink the question, not a reason to let a p-value choose the test.

The engine’s sample-size check also passed. That check is a floor, a basic sanity check, and says nothing about whether the assumptions hold. A sensitivity figure is a separate thing: with 200 per group, the design had 80% power at the two-sided 0.05 level for effects of d = 0.28 or larger. That describes the design, not this result.

What to check instead

Replace the gate with three habits.

Choose the analysis before you see the outcome data. The design and the question decide it: two independent groups and a question about means point to Welch’s t-test. The guide to choosing a statistical test starts from the design for this reason.

Look at the data within each group. A normal Q-Q plot or a histogram per group, or of the within-pair differences for a paired design, shows the size and kind of departure, which a p-value does not. In a Q-Q plot, points that bend away from the line at one end suggest skew, points that curl at both ends suggest heavy or light tails, and a staircase suggests rounded or discrete values. For a t-test the shape that matters is within each group (the residuals around each group mean), not the two groups pooled; for regression and ANOVA, it is the residuals. The parametric test assumptions guide covers what to look for in each assumption.

Ask what departure would actually matter at your sample size. A mild skew at 200 per group is not worth a second thought. A strong skew with 8 per group, or one group much smaller than the other with different shapes, is. In that situation, consider:

• a log scale, when the measurement is positive and a ratio of geometric means answers the question;
• a bootstrap or permutation interval for the difference in means, which keeps the question about means;
• a rank test such as Mann-Whitney, which is valid without normality but compares the distributions rather than the means. The t-test or Mann-Whitney guide explains that difference, and the paired t-test or Wilcoxon guide covers paired data.

When normality really does matter

The t-test’s robustness has conditions, and outside them the shape of the data deserves attention whatever a formal test says.

• Small samples with a strong skew or heavy tails. The central limit theorem has not yet done its work.
• Unequal group sizes with different shapes. The skew no longer cancels in the difference.
• One-sided tests. Skew makes one tail of the t statistic’s distribution heavier and the other lighter. In a two-sided test those errors partly offset each other; a one-sided test uses only one tail, so nothing offsets them.
• Very small significance levels. The normal approximation is least accurate far out in the tails, so the gap between the nominal and actual error rate is larger, relative to the level, at 0.001 than at 0.05.
• Anything about individual values rather than means, such as a prediction interval or a reference range. These depend on the shape of the data itself, and no sample size rescues them.

The points about one-sided tests and small significance levels are reasoning, not findings from a cited study; the simulation below can check them (use a one-sided alternative or a cut-off below 0.05).

See the pattern yourself

The simulation below draws two groups from the same skewed distribution, so there is no true difference, and records two rates at each sample size: how often Shapiro-Wilk rejects normality, and how often the Welch t-test gives p below 0.05. The first rate is the pre-test’s sensitivity; the second is the t-test’s actual false-positive rate. Run it and compare the two rates at each size; then change the distribution to one closer to your own data. (In Python, scipy.stats.shapiro runs the pre-test and scipy.stats.ttest_ind(a, b, equal_var=False) the Welch test, since its default is Student’s; statsmodels.api.qqplot draws a Q-Q plot.)

# Two groups from the same right-skewed distribution: no true difference.
set.seed(2026)
n_sims <- 2000
skewed <- function(n) rlnorm(n, meanlog = 0, sdlog = 0.25)

rates_at <- function(n) {
  sw_reject <- t_reject <- logical(n_sims)
  for (i in seq_len(n_sims)) {
    a <- skewed(n); b <- skewed(n)
    sw_reject[i] <- shapiro.test(a)$p.value < 0.05
    t_reject[i] <- t.test(a, b)$p.value < 0.05   # Welch by default
  }
  c(n_per_group = n, shapiro_rejects = mean(sw_reject), t_false_positive = mean(t_reject))
}

t(sapply(c(10, 30, 200), rates_at))

What to look for: the rejection rate of the pre-test should climb steeply with sample size while the t-test’s false-positive rate stays near the nominal 5% at every size. Simulation results are estimates. With 2000 repetitions, a rate near 5% carries a margin of about one percentage point (twice its binomial standard error, 2 x sqrt(0.05 x 0.95 / 2000), or 0.97 points), and a rate near 50% about two points. Try a heavier skew, such as sdlog = 1, with 10 per group to see where the t-test itself starts to drift from 5%.

How to write it up

Report what you decided and why, not a normality p-value as a licence. A methods sentence might read:

Group means were compared with Welch’s two-sample t-test, chosen before analysis because the question concerned means and the variances were not assumed equal. Distributions within each group were inspected graphically; both showed a mild right skew, which, with 200 units per group and equal group sizes, was not expected to affect inference about the means.

If a reviewer asks for a normality test, report it with that context rather than letting it decide the analysis. The t-tests explained post shows how to report the test itself, and the method reference for Welch’s t-test lists exactly what the engine computes.

Rerunning the example

Below the article you will find the R script exactly as the engine generated it, with the seed, and the output it printed. The script reads its data from a data/ folder; the data file is offered for download with it. On the same package versions it reproduces the Welch test and the sensitivity figure. The effect size and the assumption checks come from the engine itself: effectsize::cohens_d(measurement ~ group, data = data), car::leveneTest(measurement ~ factor(group), data = data) and shapiro.test() within each group reproduce them. The guide to reproducible statistical analysis explains what a reviewer needs to rerun a result like this one.

# Results

- Test: Welch's t-test
- n: 400
- statistic: -2.986
- p-value: 0.003
- effect size: Cohen's d = -0.2986 (95% CI -0.4955 to -0.1013)
- 95% CI: [-6.33, -1.304]
- seed: 20261022

## Assumption checks
- normality by group (Shapiro-Wilk): passed=FALSE
- variance homogeneity (Levene): passed=TRUE (no evidence of unequal variances)
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=400: Cohen's d = 0.2808
- to detect a conventional medium effect: 64 per group to detect d=0.50
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Welch's t-test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/test-for-normality-before-t-test.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("measurement", "group")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- t.test(`measurement` ~ `group`, data = data, var.equal = FALSE)
print(result)

# Sensitivity: minimum detectable effect at n=400 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.t2n.test(n1 = 200, n2 = 200, sig.level = 0.05, power = 0.8)
print(power)