Effect sizes for Mann-Whitney and Wilcoxon tests

A Mann-Whitney U test or a Wilcoxon signed-rank test gives you a p-value, and a p-value does not tell a reader how large the difference is. The effect size that matches these tests is the rank-biserial correlation: the proportion of comparisons that favour one group minus the proportion that favour the other, reported with its confidence interval.

This post computes it on R’s InsectSprays data with a deterministic engine run, explains its interval, separates it from Z divided by the square root of N, and shows the paired version. For the wider picture of effect sizes across tests, start with effect sizes explained. Three or more independent groups (the Kruskal-Wallis test) need a different rank-based effect size and are outside this post.

Which effect size goes with which rank test

Test	Design	Effect size	Range
Mann-Whitney U	two independent groups	rank-biserial correlation (or the probability of superiority)	-1 to 1 (0 to 1)
Wilcoxon signed-rank	two measurements on the same units	matched-pairs rank-biserial correlation	-1 to 1

A survey of articles in one psychology journal found effect sizes reported for fewer than half of the analyses and no confidence interval for any effect size (Fritz, Morris and Richler, 2012).

What the rank-biserial correlation measures

Take every pair you can make from one value in group A and one value in group B. For each pair, ask which value is larger. The rank-biserial correlation is the proportion of pairs where A is larger minus the proportion where B is larger, with a tie counted as half a pair for each side (Kerby, 2014, calls this the simple difference formula; Cureton, 1956, gave the general formula that allows ties).

That matches what the Mann-Whitney test examines. The U statistic is the number of pairs in which the first group’s value is larger, plus half the ties. So:

• the probability of superiority is U divided by the number of pairs: the chance that a randomly chosen value from the first group beats one from the second, with ties split. McGraw and Wong (1992) proposed this reading but estimated it from means and variances; Vargha and Delaney (2000) gave a version for any ordinal variable, discrete or continuous;
• the rank-biserial correlation is twice that probability minus 1, which moves it onto a scale from -1 to 1. For two groups it equals Cliff’s delta, the chance that the first group’s value is higher minus the chance that the second’s is; ties add equally to both halves of the rank-biserial count, so they cancel.

A value of 0 means neither group tends to be higher, which is 0.5 on the probability scale. A value of -1 means every value in the first group is below every value in the second. The sign depends on which group you list first (swap them and the sign flips, and the probability of superiority becomes 1 minus its old value), so always say which group the comparison favours, in words.

It is not a difference in medians or in means; the t-test vs Mann-Whitney guide explains why.

Worked example: insect counts under sprays C and D

R’s InsectSprays data hold insect counts from 72 field plots, 12 plots per spray, each plot treated with one of six sprays and counted once. The plots under spray C and spray D are different plots, so the groups are independent: a two-sample design. The counts are small, tied whole numbers.

The engine ran the Mann-Whitney test on sprays C and D only, through a recorded filter that the script applies too. The engine asks that a subgroup chosen after looking be treated as exploratory, and this one was: we picked C and D after looking at all six sprays, because the pair illustrates the method. Read the result as a demonstration of the effect size, not evidence about these sprays.

The engine output, with C as the first group:

• n = 24 (12 plots per spray)
• W = 20, two-sided p = 0.00184, from an exact test (the result records exact = TRUE even though the counts are tied)
• rank-biserial correlation r = -0.72, 95% CI -0.88 to -0.41
• Hodges-Lehmann shift estimate -3, with an interval from -4 to -1 (98.5% coverage, see below)

In R, W for the first group is the U statistic, so you can check the effect size by hand. Of the 144 pairs, spray C’s plot had the higher count in 20, after counting ties as half. The rank-biserial correlation is 2 times 20 divided by 144, minus 1, which gives -0.72. In words: comparisons where C has the lower count outnumber those where D does by about 72 percentage points.

The Hodges-Lehmann estimate is the median of all 144 pairwise differences, C minus D. Its interval, -4 to -1, has an achieved coverage of 98.5%, not the 95% requested. An exact interval’s ends must be pairwise differences, and with whole-number counts many are equal, so the coverage moves in jumps. Report it as a 98.5% interval.

The engine’s only assumption check for this test, sample size, passed; that is a floor, not evidence of validity. The design matters more: 24 independent plots, each in one spray.

Counts also have a natural model: a Poisson or negative binomial regression estimates a rate ratio (“how many times as many insects”), not “which group tends to be higher”.

How the interval is built, and when it is rough

The engine uses effectsize::rank_biserial(). In the pinned version, the interval is a normal approximation on Fisher’s z scale: the estimate is transformed with the inverse hyperbolic tangent, a standard error of the square root of (n1 + n2 + 1) divided by (3 n1 n2) is added and subtracted, and the limits are transformed back.

That standard error is the spread r would have, without ties, if the two groups had the same distribution (it follows from the null variance of U). It depends only on the group sizes, not on the observed effect or the ties; the paired version’s depends only on the number of non-zero differences. With heavy ties, an estimate near -1 or 1, or small groups, the approximation is rougher. Read -0.88 to -0.41 as approximate, and cross-check with a bootstrap interval if precise limits matter.

The interval is the useful part: the data are compatible with anything from a moderate to a very large tendency for C plots to have fewer insects. The p-value of 0.00184 says only that a W this far from its expected value would be unusual if the two sprays’ counts had the same distribution.

The other r: Z divided by the square root of N

Many tutorials give the Mann-Whitney effect size as r equals Z divided by the square root of the total sample size, with cut-offs of 0.1, 0.3 and 0.5. This is a different quantity from the rank-biserial correlation, and the two usually differ on the same data.

• It depends on how Z was computed. Z comes from the normal approximation to the test statistic, with or without a continuity correction and a tie correction. Software differs in these choices, so the same data can give different values. An exact test, like this example’s, produces no Z, so this post quotes no Z over root N.
• It has no direct reading. The rank-biserial correlation is a difference of two proportions you can count. Z over root N is a rescaled test statistic.
• It has no standard interval. Software that reports it rarely gives a confidence interval with it.

If asked for “r”, report the rank-biserial correlation and name it, or report both and label each.

A one-sided test does not change the rank-biserial estimate, which comes from the same pairs. The p-value changes, and so does the interval if you request a one-sided one (alternative = "less" or "greater" in rank_biserial()).

The Wilcoxon signed-rank test: the matched-pairs version

For paired data the same idea uses ranks of the within-pair differences. Rank the absolute differences, leave out any pair whose difference is exactly zero (the test leaves them out too), and take the sum of the ranks of the positive differences minus the sum of the ranks of the negative ones, divided by the total of all ranks (Kerby, 2014).

A second engine run applied the Wilcoxon signed-rank test to the anorexia data in R’s MASS package, a published teaching data set of weights of young women treated for anorexia. We restricted it to the 17 in the family therapy group, weight after against before; the engine flags that restriction too. Each patient was weighed twice, so the columns are paired. No difference was exactly zero.

• V = 142, p < 0.001, from an exact test
• matched-pairs rank-biserial r = 0.86, approximate 95% CI 0.63 to 0.95
• Hodges-Lehmann median of the paired differences 7.65 (pounds), with an interval from 3.45 to 11.2 at an achieved coverage of 95.5%

V is the sum of the ranks of the positive differences. With 17 pairs the ranks add up to 153, so the negative differences hold 11, and r is 142 minus 11, divided by 153. In words: increases in weight outweigh decreases by a wide margin in this group.

Two cautions. This is a change within one group, so it says nothing about whether the therapy caused it; only a comparison with the data set’s control group could address that. These are research data used to show a calculation, not guidance on treatment. And the signed-rank test assumes the differences are roughly symmetric. The engine’s Shapiro-Wilk check on the differences detected no departure from normality, but it tests normality, not symmetry, and has little power with 17 pairs, so plot the differences too. The paired t-test vs Wilcoxon guide covers when the signed-rank test is the right choice at all.

Small, medium and large

The engine labels both results “strong”, using cut-offs of 0.1, 0.3 and 0.5 on the absolute value. Such labels are conventions, not findings. Whether an effect matters depends on the outcome and earlier studies, so report the number and its interval and argue its importance in your field’s terms.

How to report it

Built from the main example:

Insect counts were lower on plots treated with spray C than on plots treated with spray D (Mann-Whitney U = 20, n = 24, exact two-sided p = 0.00184). The rank-biserial correlation was r = -0.72 (95% CI -0.88 to -0.41), so in comparisons of one C plot with one D plot, the C plot had the lower count far more often than the higher one. The pair was chosen from six sprays after inspecting the data, so this comparison is exploratory.

Name the effect size, give its direction in words and its interval, and say how the comparison was chosen.

Common mistakes

• Calling Z over root N the rank-biserial correlation.
• Leaving out the direction: a negative r means nothing until the reader knows which group was first.
• Reporting only a p-value. A small p-value can come with a small effect in a large sample.
• Reading a non-significant result as “no effect”. Check the interval: if it includes effects that would matter, the study is inconclusive.
• Planning a sample size from this study’s estimate: one small study’s effect is a noisy guide.

Rerunning the example

The engine example below carries the exact R script for the spray comparison; the paired run is not packaged with this page. The script reproduces the W statistic, the p-value and the Hodges-Lehmann estimate with its interval, but not the rank-biserial correlation. To get that, add one line after the filter in the script, where data holds the two sprays:

effectsize::rank_biserial(data$count[data$spray == "C"], data$count[data$spray == "D"])

This is the same function the engine calls. For the paired version, pass the two columns with paired = TRUE, as in effectsize::rank_biserial(after, before, paired = TRUE). The formula interface refuses paired = TRUE, so use two vectors. For choosing the test itself, see choosing a statistical test; for the parametric counterpart, t-tests explained.

# Results

- Test: Mann-Whitney U test
- n: 24
- statistic: 20
- p-value: 0.00184
- effect size: rank-biserial correlation = -0.7222 (95% CI -0.8818 to -0.4143)
- 98.5039% CI: [-4, -1]
- seed: 20261022

## Assumption checks
- sample size: passed=TRUE (adequate n)

## Power
- Power is not calculated for this test. The matching parametric test gives a rough guide.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Mann-Whitney U test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/effect-size-mann-whitney.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)

# Analysis population filter (recorded in the spec): spray in C, D
data <- data[which(data[["spray"]] %in% c("C", "D")), , drop = FALSE]
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("count", "spray")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- wilcox.test(`count` ~ `spray`, data = data, conf.int = TRUE, conf.level = 0.95)
print(result)