Mean vs median: when the mean misleads

The mean and the median both answer the question “what is a typical value?”, and they agree when the distribution is symmetric. On right-skewed data the mean usually sits above the median, and the better summary depends on the question: the mean speaks for totals and averages across all units, the median for the typical unit.

The sections below define the mean, the median and skew, then work through one seeded synthetic example in which two groups are ordered one way by their means and the opposite way by their medians. The same example shows how a single extreme value moves each summary, and why Welch’s t-test and the Mann-Whitney test can give different answers on the same data. The data cleaning checklist places this summary step among the other checks before a test.

Mean

The mean is the sum of the values divided by their number. Geometrically it is the balance point of the data: the distances of the values below it exactly cancel the distances of the values above it. Every value enters with the same weight, so a value ten times larger than the rest has ten times the pull.

Equal weighting is both the mean’s strength and its weakness. Because the mean times the number of units equals the total, it is the right summary when the total matters, such as the total cost of a set of projects or the total waiting time of a set of requests. Because every value counts at full size, a single extreme value can move it a long way.

The standard deviation is built on the same arithmetic, from squared distances to the mean, so one extreme value inflates the standard deviation as well as the mean. Mean and standard deviation belong together, and both are fragile in the same way.

A seven-value toy vector shows the pull. One value of 95 sits among values between 12 and 20, and the code compares the mean and median with and without it:

x <- c(12, 14, 15, 16, 18, 20, 95)
mean(x)
median(x)
x2 <- x[-7]
c(mean = mean(x2), median = median(x2))

[1] 27.14286
[1] 16
    mean   median 
15.83333 15.50000 

With the 95 present the mean is far above all but one of the values, while the median stays inside the main group. Dropping that value changes the mean by more than eleven units and the median by half a unit.

Median

The median is the middle value after sorting: half the values lie at or below it and half at or above it. With an even number of values it is the average of the two middle ones. Because it depends on the order of the values and not their size, it is unchanged when the largest value grows from 95 to 950.

The interquartile range is the matching measure of spread: the distance between the first and third quartiles, which holds the middle half of the values. Quartile definitions are not unique. Hyndman and Fan (1996) list nine ways statistical packages compute a sample quantile, so state which one you used. The engine’s summary says it uses R’s quantile type 7, and R’s quantile() with its default argument gives identical quartiles:

identical(quantile(x), quantile(x, type = 7))
quantile(x)

[1] TRUE
  0%  25%  50%  75% 100% 
12.0 14.5 16.0 19.0 95.0 

The median is not a better mean. Ignoring the size of the extremes also hides them, and in some questions the extremes are the point. If a few requests stall for months, the median waiting time can look healthy while the total waiting time grows. The median describes the typical unit; it says nothing about how bad the worst ones are, which is a job for the upper quartile, the maximum, or the mean.

Skew

Skew is asymmetry. A right-skewed (positively skewed) distribution has a long tail of large values and a short tail on the left; a left-skewed distribution is the mirror image. Quantities bounded at zero and built from multiplied effects, such as durations, costs and concentrations, are often right-skewed. The log-normal distribution is the standard model: the logarithm of the value is normal.

For a log-normal distribution with log-mean μ and log-standard-deviation σ, the median is e^μ and the mean is e^(μ + σ²/2), so the mean exceeds the median by a factor that grows with σ. For the two simulated groups below, recomputed from their generating parameters rather than taken from the engine, team A has a population median of about 20 and mean of about 23, and team B a population median of about 16 and mean of about 26. The heavier tail of team B raises its mean while its median is lower.

A gap between mean and median is a quick sign of skew, but not a definition. Von Hippel (2005) shows that the textbook rule “the mean lies on the tail side of the median” fails for some discrete and multimodal distributions, so look at the distribution itself. A histogram or dot plot per group shows the tail faster than any statistic.

A sample skewness coefficient puts a number on asymmetry, but there are several formulas that differ in small samples (Joanes and Gill 1998). If you report one, name the formula or the software function, and do not use it to decide between tests: the plot and the question decide.

Worked example: two teams and their completion times

The data are seeded synthetic values, not real requests. Each of 120 data requests was completed once by one of two teams, 60 per team, and the outcome is days to completion. The groups are independent units in a two-sample design, with a continuous positive outcome and no censoring. The generating values give team A a moderate tail and team B a lower typical time with a much heavier tail, so the population values themselves order the teams one way by mean and the other way by median (see the skew section above).

The seed was then chosen, by trying a few, so that this sample shows the same pattern; with another seed a sample of 60 per team need not. The example illustrates what can happen, not how often it happens in real data. The question is whether the two teams differ in how long requests take.

Mean and median disagree

The engine’s descriptive summary, run on the same 120 rows, gives these figures per team:

Team A is only mildly skewed: its mean of 23.08 and its median of 21.4 are close, and its standard deviation is modest. Team B is skewed: its mean of 27.28 sits well above its median of 16.2, its standard deviation is more than three times that of team A, and its longest request took 172.7 days.

The means say team B is slower, 27.28 days against 23.08. The medians say team B is faster, 16.2 days against 21.4. Both statements are true of this sample. They describe different things: the mean carries the heavy tail of team B, and the median describes its typical request. Which one matters depends on what the reader will do with the answer, which is why the summary must name the question.

One extreme value

A second run adds a single request to team A that took 400 days, leaving the other 120 rows unchanged. Team A now has 61 requests, and its summary moves as follows:

One value out of 61 raises the mean from 23.08 to 29.26 and the standard deviation more than fivefold, while the median moves from 21.4 to 22 and the interquartile range from 12.2 to 12.6. The order of the two team means also flips: team A’s mean of 29.26 now exceeds team B’s 27.28.

The 400 is a deliberately planted value, and the example does not say it should be removed. Whether a real extreme value is corrected, excluded or kept is a separate decision with its own rule, set out in how to handle outliers before a statistical test, and the IQR rule post shows how a value gets flagged. The point here is only that the choice of summary decides how much one value counts.

Two tests, two questions

On the original 120 rows the engine ran both tests. Welch’s t-test compares the group means. It gives a mean difference (A minus B) of -4.197 days with a 95% confidence interval from -12.71 to 4.317, t = -0.9833 on 69.59 degrees of freedom, p = 0.329 and Cohen’s d = -0.1795 (95% CI -0.5377 to 0.1794). Shapiro-Wilk flagged non-normality, so the engine ran its pre-specified sensitivity test, the Mann-Whitney test, beside Welch’s. Levene’s test also detected unequal variances (p = 0.0052), which Welch’s test already allows for. The interval is wide in both directions and the engine puts the smallest effect this sample could detect with 80% power at d = 0.5157, so the mean comparison is inconclusive, not evidence of equal means.

The Mann-Whitney test asks a different question: whether values in one group tend to be larger than values in the other. It gives p = 0.0478, a rank-biserial correlation of 0.2097 (95% CI 0.005423 to 0.3972) and a Hodges-Lehmann estimate of 4.5 days (95% CI 0.09995 to 8.5), positive because requests in team A tend to take longer. The engine flagged p = 0.0478 as borderline, and the lower end of both intervals sits close to zero.

The 4.5 is the Hodges-Lehmann estimate of the shift between the groups, the median of all pairwise differences. It is not the difference between the two medians, which would be 21.4 minus 16.2.

When the test is used to compare medians or means, Fagerland and Sandvik (2009) show by simulation that small differences in variance and moderate skew, worse still when the two groups differ in skew, push its type I error rate well away from the nominal level. Here the teams differ in both, so read the method reference before describing the result as a difference in medians.

The two results do not contradict each other, because the two tests ask about different features of skewed data. Choose the question before looking: the guide to the t-test or Mann-Whitney U test covers the choice, and the Welch’s t-test post covers the unequal-variance case. Running both and reporting whichever fits is test shopping; report both, name the primary one in advance, and describe each in terms of what it compares.

Which summary to report

For a comparison, match the test to the summary. A difference in means with a confidence interval (Welch’s t-test) answers a question about means. A rank-based test answers a question about tendency to be larger. When ratios are the natural scale, as with durations and costs, analysing the logarithm compares geometric means; choose that for a reason about the measurement, as the test for normality post advises, not to make a result look better.

Reporting both

A reporting sentence for the example could read:

Completion time was right-skewed in team B. Team A: mean 23.08 days (SD 9.522), median 21.4 (IQR 12.2); team B: mean 27.28 (SD 31.66), median 16.2 (IQR 20.525); 60 requests per team. The difference in means (A minus B) was -4.197 days (95% CI -12.71 to 4.317; Welch’s t-test, p = 0.329). A rank-based sensitivity analysis gave p = 0.0478 (Hodges-Lehmann shift 4.5 days, 95% CI 0.09995 to 8.5).

The standard error and margin of error post explains why the standard deviation, not the standard error, belongs in a sentence that describes the data.

Common mistakes

• Reporting only the mean for a long-tailed outcome. A reader sees a typical value that few units resemble.
• Reporting only the median when the total matters. The tail disappears from a cost or time total that it dominates.
• Using a gap between mean and median as the definition of skew. The gap is a sign; the distribution is the evidence.
• Calling the Mann-Whitney test a test of medians. It compares distributions through ranks.
• Running both tests and quoting the one with the smaller p value. Name the primary question first.
• Pairing the median with the standard deviation, or the mean with the IQR. Match centre and spread: mean with SD, median with IQR.

Rerunning the example

The script below the article reproduces the Mann-Whitney run on the 120-row data file. The companion runs for Welch’s t-test, the descriptive summary and the summary with the extra request use the same seed and code, so they regenerate the same 120 rows, and they are not part of the downloadable script. By hand, mean(), sd(), median() and IQR() give the summary figures, t.test(days ~ team) the Welch result (its printed heading reads “Welch Two Sample t-test”) and wilcox.test(days ~ team, conf.int = TRUE) the rank-based one. The method reference pages for the descriptive summary, Welch’s t-test and the Mann-Whitney U test describe each, and the guide to preparing data for statistics places the summary step before the test.

# Results

- Test: Mann-Whitney U test
- n: 120
- statistic: 2178
- p-value: 0.0478
- effect size: rank-biserial correlation = 0.2097 (95% CI 0.005423 to 0.3972)
- 95% CI (Hodges-Lehmann difference in location): [0.09995, 8.5]
- seed: 1

## Assumption checks
- normality by group (Shapiro-Wilk): context only, passed=FALSE (evidence of non-normality - normality is not assumed by this rank-based test)
- sample size: passed=TRUE (adequate n)

## Power
- Power is not calculated for this test. The matching parametric test gives a rough guide.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Mann-Whitney U test  |  engine 0.53.2  |  catalog c40-dc08
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(1)
.data_path <- "data/mean-vs-median.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("days", "team")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- wilcox.test(`days` ~ `team`, data = data, conf.int = TRUE, conf.level = 0.95)
print(result)