Cramer's V and phi: effect sizes for chi-square

A chi-square test tells you whether two categorical variables look associated. It does not tell you how strongly. With a few thousand observations, an association too weak to matter will still produce a tiny p-value, so a chi-square result needs an effect size beside it. For tables, the usual choices are phi for a 2x2 table and Cramer’s V for anything larger.

This post covers what both measure, why the small, medium and large labels change with table size, what the bias-corrected V fixes, and how to read V’s one-sided interval. The main example is a 3x4 table from a public survey, run through a deterministic engine, with a 2x2 example for phi. For effect sizes in general, start with the effect sizes overview.

What Cramer’s V measures

The chi-square statistic grows with the strength of the association and with the number of observations: double the sample at the same proportions and it doubles. That makes it a poor measure of strength.

Cramer’s V, introduced by Harald Cramer in 1946, removes both dependencies. Take the statistic, scale it by n, scale it again by one less than the smaller of the two table dimensions, and take the square root:

V = sqrt( X² / (n x (k - 1)) ), where k is the smaller of the number of rows and the number of columns.

The first division removes the sample size; the second makes the largest possible value 1 whatever the table’s shape. V is 0 when the sample proportions are identical in every row, and 1 when knowing one variable tells you the other exactly (as far as the table’s shape allows).

Three properties matter in practice:

• V is symmetric. Swapping rows and columns gives the same value. It says nothing about which variable might depend on the other.
• V has no direction. It is always between 0 and 1, so it cannot say which categories go together. The table of counts, row percentages or standardized residuals show that.
• V is not a correlation. Do not read 0.3 as you would a Pearson r of 0.3; even the conventional labels for V depend on k.

Phi on a 2x2 table

For a 2x2 table, k - 1 is 1 and V reduces to sqrt(X² / n). That quantity is phi, or rather its absolute value. Phi can carry a sign, positive when the two “yes” categories go together and negative when they oppose, so it is the one member of the family with a direction.

A companion run on the Mroz data (753 married women from the US Panel Study of Income Dynamics) cross-tabulates whether a woman attended college against whether she was in the labour force. The engine reports V = 0.1401, with a one-sided 95% interval from 0.08016 up to 1. V gives the size of the association but not its direction. The risk difference from the same run does: the labour-force participation proportion is 0.1543 higher among women who attended college (95% CI 0.07867 to 0.2299). This is an association in observational data, not an effect of college.

A detail trips people up when checking by hand: R’s chisq.test() applies Yates’ continuity correction to every 2x2 table by default, and the engine reports the test as R runs it, so its statistic, 14.16 here, is the corrected one. V and phi use the uncorrected Pearson statistic, as effectsize, and the engine through it, compute them. Plug 14.16 into the formula and you get sqrt(14.16 / 753), about 0.137, slightly below the 0.1401 reported. Tables larger than 2x2 get no correction, so the issue disappears there.

For a 2x2 table, the odds ratio or the risk difference is usually more useful to a reader than phi, because each has a direct interpretation; the odds ratio vs relative risk guide covers which one fits which design.

Worked example: education and voting intention

The main example uses the Chile data shipped with R’s carData package: a national survey from April and May 1988 asking how respondents intended to vote in that year’s plebiscite, where yes was a vote for Pinochet. We cross-tabulate education (primary, secondary, post-secondary) against stated voting intention (yes, no, undecided, will abstain). That is a 3x4 table, so k - 1 = 2.

Each respondent is one row, classified once on each variable, so this is a test of independence. The data frame carries no sampling weights, so the analysis treats respondents as independent draws; a design-based analysis of the original survey could give different standard errors.

The engine ran the chi-square test with the method fixed in advance:

• Rows used: 2522 of 2700. The 178 rows missing education or voting intention were dropped (complete cases). If missingness is related to either variable, the result describes only the respondents who answered both.
• Expected counts: the smallest expected cell count is 32.48, so every cell clears the engine’s floor of 5. That floor is a minimum for the chi-square approximation, not a judgment that the analysis is valid in every other respect.
• Test: X²(6) = 135.8, p < 0.001.
• Effect size: Cramer’s V = 0.1641, one-sided 95% CI 0.1377 to 1.

The p-value says the observed table would be very unusual if education and voting intention were independent in the population sampled. It does not say the association is strong. V does: 0.1641 with k - 1 = 2. The engine’s plain-language note calls it weak, using fixed cut-offs it applies to every correlation-type measure; by Cohen’s convention adjusted for this table shape, covered next, it sits between small and medium.

In a write-up, one sentence carries it: “Education and voting intention were associated, X²(6, N = 2522) = 135.8, p < 0.001, Cramer’s V = 0.1641 (one-sided 95% CI 0.1377 to 1).” Add the version of V (classic here) and the table’s shape.

The run also reports a sensitivity figure: for this sample size and 6 degrees of freedom, the smallest effect the test would detect with 80% power at alpha 0.05 is Cohen’s w = 0.0735. Since V = w / sqrt(k - 1), that is a V of about 0.052 (0.0735 / sqrt(2)), so this sample could detect associations much weaker than the one observed. That describes the design, not how likely this result is to be real.

Why small, medium and large depend on the table

The cut-offs most people quote for V, 0.10, 0.30 and 0.50, come from Jacob Cohen’s book on power analysis. Cohen set them for a different index, w, which is sqrt(X² / n) without the division by k - 1. For a 2x2 table, w, phi and V coincide, so the cut-offs carry over unchanged. For larger tables they do not.

The relationship is w = V x sqrt(k - 1). Holding Cohen’s w benchmarks fixed and solving for V gives cut-offs that shrink as the smaller side of the table grows: each one is Cohen’s value divided by sqrt(k - 1). Kim (2017, Table 2) tabulates them, citing Cohen. They are published conventions, not engine output. Kim labels the rows “degrees of freedom”, but the values follow from k - 1, not the test’s (r - 1)(c - 1):

k - 1 (smaller dimension minus 1)	Small	Medium	Large
1 (any 2 x c table)	0.10	0.30	0.50
2 (smaller side has 3 categories)	0.07	0.21	0.35
3	0.06	0.17	0.29
4	0.05	0.15	0.25

So a V of 0.25 is between small and medium on a 2x2 table and large on a 5x5 table. Our example, with three education levels, falls in the second row: 0.1641 is above the small value and below the medium one.

Cohen offered these as conventions for when nothing better is known, not as a scale of importance. A previous study or a minimum size that would change a decision is a better yardstick.

The bias-corrected V

The classic V is a consistent estimator but a biased one in finite samples: it tends to overstate the association, and the overstatement grows with the number of cells and shrinks with the sample size. Under independence the population V is 0, yet the sample V is almost never exactly 0, because chance alone produces some chi-square.

Bergsma (2013) proposed a simple correction. It subtracts (r - 1)(c - 1) / (n - 1), roughly what phi-squared averages under independence, from the sample phi-squared, floors the result at zero, and shrinks the table dimensions in the denominator to match. In simulations it outperformed the classic estimator for tables larger than 2x2 and performed comparably on 2x2.

Software differs on the default, which is the real practical trap:

library(effectsize)
tab <- table(carData::Chile$education, carData::Chile$vote)
cramers_v(tab)                  # bias-corrected (adjust = TRUE is the default)
cramers_v(tab, adjust = FALSE)  # the classic V

The engine reports the classic V, so its value can be checked against the formula above. Here the subtracted term is 6 (that is, 2 x 3) divided by n - 1 with n in the thousands, tiny next to the sample phi-squared, so the versions barely differ. On a small sample spread over many cells they can differ a lot, and a corrected V of zero beside a clearly positive classic V is possible. Name the one you report.

Reading the one-sided interval

The interval for V in the example runs from 0.1377 to 1. The upper end is not an estimate; it is the ceiling of the scale. Effectsize, and the engine through it, returns a one-sided interval by default because V cannot be negative and the chi-square test only rejects for large statistics.

The lower bound is the informative number: values of V below 0.1377 are not compatible with the data at this level. Because the bound comes from inverting a noncentral chi-square distribution, 5% in one tail gives the same cut point as 10% split over two, so a one-sided 95% lower bound equals the lower limit of a two-sided 90% interval. If a journal asks for a two-sided 95% interval, ask the software for one (alternative = "two.sided") rather than relabelling the one-sided result.

Common mistakes

• Reporting the p-value as the strength of the association. A p below 0.001 here came with a V between small and medium.
• Applying 2x2 cut-offs to a larger table. Read the row of the table above that matches k - 1.
• Mixing corrected and uncorrected statistics on a 2x2 table, or classic and bias-corrected V across studies.
• Using V when expected counts are small. Then the chi-square approximation itself is in doubt; see the chi-square vs Fisher’s exact test guide.
• Treating V as causal. Association in survey or observational data says nothing about what causes what.

When V is the wrong measure

V suits two nominal variables measured on independent observations. Three situations call for something else:

• Ordered categories. V ignores order: shuffling the education levels leaves it unchanged. That is acceptable here because voting intention has no order; for two ordinal variables, Kendall’s tau-b or gamma uses the order and gives a direction.
• Paired or repeated categorical data. The same people classified twice break the independence the chi-square test assumes. Use McNemar’s test for a 2x2 table, or kappa when the question is agreement.
• Prediction, not association. Goodman-Kruskal’s lambda and Theil’s U are asymmetric: they ask how much knowing one variable helps guess the other. The contingency coefficient C is symmetric but its maximum falls below 1 and depends on the table.

Where V does fit, report it with its interval, its version and the table shape.

If you are unsure whether a chi-square test fits your design at all, the test chooser walks through the alternatives. The chi-square test reference lists what the engine computes for this method.

Rerunning the example

The reproduce script attached below reruns the chi-square test and the sensitivity calculation and gives the engine’s statistic and p-value. It does not compute V; for that, add effectsize::cramers_v(table(data$education, data$vote), adjust = FALSE) after the script’s own lines. The seed is fixed and the package versions are pinned, which is what makes the run repeatable; the reproducible analysis guide explains how that works.

# Results

- Test: Chi-squared test of association
- n: 2522
- statistic: 135.8
- p-value: < 0.001
- effect size: Cramer's V = 0.1641 (95% CI 0.1377 to 1)
  (one-sided CI (upper bound fixed at 1))
- rows used: 2522 of 2700 (178 dropped)
- seed: 20261022

## Assumption checks
- expected cell counts >= 5: passed=TRUE (all expected counts >= 5)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=2522: Cohen's w = 0.0735
- to detect a conventional medium effect: 152 total to detect w=0.30
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Chi-squared test of association  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/cramers-v.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("education", "vote")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- chisq.test(table(data[["education"]], data[["vote"]]))
print(result)

# Sensitivity: minimum detectable effect at n=2522 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.chisq.test(N = 2522, df = 6, sig.level = 0.05, power = 0.8)
print(power)