Chi-square test explained, with a worked R example
What the chi-square test of independence computes, how to read expected counts, Cramer's V and residuals, and a worked R example on real survey data.
Updated . Written by Matt Gencer.
On this page
A chi-square test asks whether the counts in a table of categories are close to the counts you would expect if some hypothesis were true. The version researchers use most, the chi-square test of independence, asks whether two categorical variables measured on the same people, samples or items are associated. It is quick to run and easy to misread, because a significant result says nothing about how strong the association is, where in the table it sits, or whether a third variable produced it.
This guide explains what the statistic computes, where expected counts and degrees of freedom come from, and which assumptions matter. It then works one example in full: education and voting intention in a 1988 national survey from Chile, run through a deterministic engine, with Cramer’s V and its interval, the expected-count check and a sensitivity figure. A short companion run on a 2 x 2 table shows the continuity correction and the 2 x 2 effect measures, and a final section shows, with runnable code, how a collapsed table can mislead.
What a chi-square test asks
“Chi-square test” and “χ² test” are the same name; the Greek letter is just the symbol. The name belongs to a family of tests whose statistic follows a chi-square distribution, which also includes the likelihood-ratio test and a test for the variance of a normal population. This guide covers Pearson’s version, the one people mean for count data. Three chi-square tests are in everyday use, and they share one formula but answer different questions.
The goodness-of-fit test has one categorical variable and a hypothesised set of proportions fixed in advance. It asks whether the observed counts in each category depart from those proportions by more than chance would explain. A geneticist checking a Mendelian ratio, or a survey team checking that a sample’s regional mix matches the census, is asking a goodness-of-fit question.
The test of independence has two categorical variables recorded on every unit in one sample. Neither margin was fixed by the design: you sampled people and recorded both things about each of them. It asks whether knowing one variable tells you anything about the other.
The test of homogeneity has one categorical outcome measured in two or more groups whose sizes were fixed by the design, for example a set number of people recruited at each of several sites. It asks whether the distribution of the outcome is the same in every group.
The independence and homogeneity tests are computed identically, from the same cross-table, and give the same statistic and p-value. The difference is in what you may conclude. With fixed group sizes, the group proportions in the sample say nothing about the population, so you can compare outcome distributions across groups but not estimate how common each group is. Saying which design you have is part of stating the hypothesis correctly.
| Question | Variables | What is fixed by design | Null hypothesis |
|---|---|---|---|
| Goodness of fit | One categorical variable | The total, and the hypothesised proportions | The category proportions equal the stated ones |
| Independence | Two categorical variables on the same units | Only the total sample size | The two variables are independent |
| Homogeneity | One outcome across fixed groups | The size of each group | The outcome distribution is the same in every group |
Most of this guide is about the test of independence, because it is the one people mean when they say “a chi-square test” about a cross-table. The goodness-of-fit version is a one-line call in R:
# Goodness of fit: do 4 categories follow proportions fixed in advance?
observed <- c(a = 210, b = 190, c = 340, d = 260) # your counts
expected_props <- c(0.2, 0.2, 0.35, 0.25) # set before seeing the data
chisq.test(observed, p = expected_props)
The proportions must come from outside the data: a theory, a census, a previous study. If you estimate them from the same sample, the degrees of freedom change and the default call gives the wrong p-value.
How the statistic is built
Every chi-square test compares observed counts with expected counts. The work is in getting the expected counts right; the statistic itself is a simple sum. In order, a test of independence goes like this:
- State the hypotheses. The null hypothesis is that the two variables are independent; the alternative is that they are associated in some way, without saying how.
- Lay out one row per unit, or a table of counts, and cross-tabulate the two variables.
- Compute the expected counts from the margins.
- Sum (observed minus expected) squared over expected across the cells.
- Find the degrees of freedom and the upper-tail p-value.
- Check the expected counts and the design assumptions.
- Report the effect size, look at where the departure lies, and state the conclusion in plain words.
The sections below take these steps one at a time.
Expected counts under independence
For a test of independence, the expected count in a cell is what you would see if the two variables were unrelated, given the row and column totals you observed. If a fraction p of the sample is in one row and a fraction q is in one column, independence says that p times q of the sample should be in the cell where they cross. Written as a formula:
expected count = (row total x column total) / grand total
The margins are taken as given. The test does not ask whether a row is common or rare; it asks whether the way rows and columns combine departs from what their totals alone would produce.
tab <- table(df$row_var, df$col_var) # observed counts, one unit per row of df
fit <- chisq.test(tab)
fit$expected # expected counts under independence
If your data arrive already counted, one row per combination of categories with a count column, build the table with xtabs(count ~ row_var + col_var, data = counts) instead of table(). Both give the same cross-table; what matters is that every unit is counted once.
The statistic
For every cell, take the difference between the observed and the expected count, square it, and divide by the expected count. Add those terms over all cells. That sum is Pearson’s chi-square statistic, introduced by Karl Pearson in 1900.
Dividing by the expected count puts cells on a comparable scale: the same difference matters much more in a cell expecting a handful of units than in a cell expecting thousands. Squaring makes every departure count, whatever its direction. So the statistic is zero only when every observed count equals its expected count, and it grows as the table departs from independence anywhere.
That last point has a consequence. The chi-square test is an omnibus test: it is sensitive to any pattern of departure and is not aimed at a particular one. Cochran made this point in 1954: because the deviations are squared and added without regard to any expected pattern, the usual chi-square tests can be insensitive when the real alternative has a specific shape, such as a steady trend across ordered categories. If you expect such a shape in advance, a test built for it (a trend test, or an ordinal model) has more power.
Degrees of freedom
The statistic is compared with a chi-square distribution whose degrees of freedom are (rows minus 1) times (columns minus 1). The reason is that once the margins are fixed, only that many cells can vary freely; the rest are determined by the totals. A 2 x 2 table has 1 degree of freedom, and a 3 x 4 table has 6. Pearson’s original paper counted the degrees of freedom differently for contingency tables, and Fisher showed in 1922 that the (rows minus 1) times (columns minus 1) count is the correct one.
From statistic to p-value
The p-value is the probability, if the two variables were truly independent and the assumptions held, of a statistic at least as large as the one observed. It is always the upper tail: there is no “direction” for a chi-square test of a table larger than 2 x 2, because the statistic squares away every sign. In R, pchisq(statistic, df, lower.tail = FALSE) gives it from the statistic and the degrees of freedom.
The chi-square distribution is an approximation. The counts are discrete, the distribution is continuous, and the approximation is good when the expected counts are reasonably large. That is the root of the most famous rule about this test.
Assumptions, and which ones matter
The test rests on a short list of conditions. Most of them are facts about the design, so check them before you look at any output.
Each unit is counted once, in exactly one cell. The table must be a cross-tabulation of units, one entry per person, sample or item. If a person can tick several answers to a “select all that apply” question, or appears in the table at two time points, the counts are not independent, and the test is invalid.
The units are independent of each other. The chi-square distribution assumes something close to simple random sampling. Clustered data, such as pupils within classes, patients within clinics or repeated visits by the same person, usually carry less information than their count suggests, and the plain test then tends to report p-values that are too small. Rao and Scott showed in 1981 that stratification and clustering change the large-sample distribution of Pearson’s statistic, related that change to the survey’s design effects, and proposed a simple correction based on them. For data from a complex survey design, use a design-based test rather than the plain one.
The cells contain counts. Not percentages, means or rates. A chi-square test run on percentages treats every row as if it had 100 units, which gives the wrong answer whenever the real row totals differ from 100.
The categories were defined in advance. Merging or splitting categories after looking at the table, until the result becomes significant, invalidates the p-value in the same way as any other choice made after seeing the data.
The expected counts are large enough. This is the condition the next section is about, and the only one you check from the data rather than from the design.
There is no normality assumption: the approximation concerns the statistic, not the data.
Expected counts and the rule of 5
The classic advice is to avoid the chi-square test when any expected count is below 5. Its usual source is Cochran’s 1954 paper, and what Cochran recommended was more nuanced than the classroom version: stricter for small 2 x 2 tables, more lenient for larger tables, where a few expected counts between 1 and 5 were acceptable. Campbell’s 2007 simulations found that, for 2 x 2 tables, a slightly modified statistic (Pearson’s statistic times (N minus 1) divided by N) kept its error rate well whenever every expected count was at least 1. Campbell also notes that, by Cochran’s own account, the figure 5 was chosen arbitrarily.
Three points are worth holding on to:
- The rule is about expected counts, never observed ones. An observed zero in a cell whose expected count is 30 is strong evidence of association, not a problem with the test.
- A low expected count is a property of the margins. A rare row crossed with a rare column will have a small expected count, however large the total.
- When the approximation is doubtful, an exact test costs little. Fisher’s exact test, or a Monte Carlo p-value from
chisq.test(tab, simulate.p.value = TRUE), avoids the approximation.
How to choose between the chi-square and Fisher’s exact test, what Fisher’s test conditions on, and why paired tables need McNemar’s test instead are covered in depth in the chi-square or Fisher’s exact test guide. The engine that ran the worked example below has a reference page for the chi-square test and for Fisher’s exact test.
Worked example: education and voting intention in Chile
The carData package for R ships a survey called Chile. It comes from a national survey conducted in April and May 1988 by FLACSO/Chile, ahead of that year’s Chilean plebiscite. The data frame has 2700 respondents, and among other things it records each respondent’s education (primary, secondary or post-secondary) and voting intention (yes, no, undecided, or will abstain).
The research question for this example is the one a test of independence answers: is voting intention associated with education level among the respondents? Each respondent appears once and falls into exactly one combination of education and intention, and neither margin was fixed by the survey design, so this is the independence form of the test, on a 3 x 4 table.
Seeded and run in R; the method, Pearson’s chi-square test, was chosen before the run. The full output and the R script that reproduces it appear below the article.
Rows missing either variable were dropped before the table was built. Of the 2700 respondents, 178 lacked education or voting intention, so the analysis used 2522. Dropping them is only harmless if the missing answers are unrelated to the association being tested; a respondent who declined to state an intention may differ from one who stated it, so the result describes the respondents who answered both questions.
The result
| Quantity | Value |
|---|---|
| Respondents analysed | 2522 of 2700 (178 dropped for missing values) |
| Table | 3 education levels x 4 voting intentions |
| Pearson’s chi-square | 135.85 on 6 degrees of freedom |
| p-value | < 0.001 |
| Cramer’s V with its one-sided 95% interval | 0.1641 (0.1377 to 1) |
| Smallest expected count | 32.48 |
| Minimum detectable effect at 80% power | Cohen’s w = 0.0735 |
Reading it
Start with the check that decides whether the p-value can be trusted. The smallest expected count in the table was 32.48, so every expected count was far above any version of the rule of 5. The large-sample approximation is not in question here.
Then the statistic and the p-value. The chi-square statistic was 135.85 on 6 degrees of freedom, (3 minus 1) times (4 minus 1). The p-value is below 0.001. Under independence, a statistic this large would almost never turn up, so the data are clearly incompatible with education and voting intention being unrelated among these respondents.
That is all the p-value says. With 2522 respondents even a modest association produces a tiny p-value, so the strength of the association needs an effect size.
Effect size: Cramer’s V and its interval
Cramer’s V rescales the chi-square statistic so that it runs from 0, meaning no association, to 1, meaning one variable perfectly predicts the other. It divides the statistic by the sample size and by the smaller of (rows minus 1) and (columns minus 1), then takes the square root, so that tables of different sizes and samples land on one scale.
In this example V was 0.1641. The engine described that as a weak association. Its 95% interval is one-sided: the lower bound is 0.1377 and the upper bound is fixed at 1. That is the default of the effectsize package the engine uses, and it follows from the test: the chi-square test asks only whether V is above zero, and V cannot exceed 1, so the interval answers the matching question of how small the association could plausibly be. Here it is unlikely to be smaller than about 0.14.
Three cautions apply to V:
- Labels depend on the table. Conventional cut-offs for “weak”, “moderate” and “strong” differ with the number of categories, because the same strength of relationship gives a smaller V in a larger table. Treat any label as a rough guide and compare V with values from studies that used tables of the same shape.
- V is biased upward in small samples. Bergsma described a bias-corrected version in 2013. With 2522 respondents the correction is small, but in a small table with a small sample the uncorrected V can look noticeably stronger than the true association. The engine reports the uncorrected V; recent versions of the effectsize package return the corrected one by default, so pass
adjust = FALSEto match it. - V is symmetric and has no direction. It says how much the two variables go together, not which category is more common where. That is the job of the residuals.
What the sample size could detect
The engine reports a sensitivity figure for the design. With 2522 respondents and 6 degrees of freedom, a test at the 0.05 level would reach 80% power for effects as small as Cohen’s w = 0.0735. Detecting a conventional medium effect, w = 0.30, with the same power would need only 152 respondents in total.
That figure is about the design, not about this result: it is not “observed power”, which adds nothing to the p-value. What it shows is that this survey could detect very small associations reliably. In a sample this size, even an association too small to matter in most fields is very likely to be significant, so the effect size and the pattern in the table carry the substance of the result.
What this result does not show
The test shows an association between education and stated voting intention among these respondents. It does not show that education changed anyone’s intention. Education is related to age, income, region and many other things, and any of them could produce or shape the association. A chi-square test on two variables cannot separate those explanations, and the section on Simpson’s paradox below shows how badly a collapsed table can mislead.
The test also treats the 2522 respondents as independent draws. The documentation that ships with the data describes it as a national survey but does not describe its sampling design. If respondents were sampled in clusters, as national surveys often are, the plain chi-square p-value would usually be too small, and a design-based test would be needed. For this example, which is about the method, the independence assumption is stated rather than verified.
Where the association is: standardized residuals
A significant chi-square test on a 3 x 4 table says the 12 cells, taken together, depart from independence. It does not say which cells. To find out, look at the residuals cell by cell.
The adjusted (standardized) residual for a cell is the difference between the observed and expected count, divided by its estimated standard error. Haberman’s 1973 paper applied this kind of residual analysis to the hypothesis of independence in r x c tables. Under independence, each adjusted residual is approximately standard normal, so values beyond about 2 in absolute size mark cells with clearly more (positive) or fewer (negative) units than independence predicts.
library(carData)
tab <- table(Chile$education, Chile$vote) # rows with a missing value are left out by table()
fit <- chisq.test(tab)
round(fit$expected, 1) # expected counts under independence
round(fit$stdres, 2) # adjusted residuals: beyond about 2 in size is notable
Read the residuals as a description of the table, not as a second set of significance tests. A 3 x 4 table has 12 residuals, and with 12 looks at the data a value beyond 2 can appear by chance even under independence. If you plan to make formal claims about particular cells, decide which cells in advance, or adjust for the number of comparisons; the multiple comparisons guide covers the options.
Two other displays help. A mosaic plot, mosaicplot(tab, shade = TRUE) in base R, draws each cell with an area proportional to its count and shades it by its residual. And row percentages, prop.table(tab, 1), show how the distribution of voting intention differs between education levels.
The code above runs on the same data the engine used, and fit$statistic from it matches the engine’s chi-square statistic. The engine output does not list cell residuals, so this guide does not quote them; run the code to see which cells drive the result.
The 2 x 2 table: continuity correction and effect measures
A 2 x 2 table is the most common chi-square table and has two features of its own. A companion engine run shows both, on the Mroz data in carData: 753 married women from the 1975 Panel Study of Income Dynamics, cross-classified by whether the woman attended college and whether she was in the labour force. Each woman is in exactly one of the four cells.
| Quantity | Value |
|---|---|
| Women analysed | 753 |
| Chi-square with Yates’ correction | 14.16 on 1 degree of freedom |
| p-value | < 0.001 |
| Smallest expected count | 91.50 |
| Cramer’s V (equal to phi here) with its one-sided 95% interval | 0.1401 (0.08016 to 1) |
| Risk ratio, college vs no college, for being in the labour force | 1.294 (95% CI 1.145 to 1.462) |
| Risk difference | 0.1543 (95% CI 0.07867 to 0.2299) |
| Odds ratio | 1.916 (95% CI 1.372 to 2.677) |
The continuity correction. For a 2 x 2 table, R’s chisq.test() applies Yates’ continuity correction by default, which subtracts a half from each absolute difference between observed and expected counts before squaring. Yates proposed it in 1934 to bring the continuous approximation closer to the discrete distribution. Campbell’s 2007 comparison of the error rates of seven two-sided tests for 2 x 2 tables found that the best policy, whenever every expected count was at least 1, was the “N minus 1” chi-square, not Yates’s version. With expected counts as large as these (the smallest was 91.50), the correction changes the statistic only slightly and does not change the conclusion. If you report a 2 x 2 chi-square, say whether the correction was used; chisq.test(tab, correct = FALSE) gives the uncorrected statistic.
A detail that trips readers: Cramer’s V (for a 2 x 2 table the same as the phi coefficient) is computed from the uncorrected statistic, so squaring V and multiplying by n does not give back the corrected 14.16. Neither number is wrong; they simply use different versions of the statistic.
Effect measures with intervals. In a 2 x 2 table, V is a poor summary for most readers. The comparison of two proportions has natural effect measures. Women who attended college were in the labour force 1.294 times as often as those who did not (95% CI 1.145 to 1.462), a risk difference of 0.1543, or about 15 percentage points (95% CI 0.07867 to 0.2299). The odds ratio was 1.916 (95% CI 1.372 to 2.677). The odds ratio is always further from 1 than the risk ratio, and the gap widens as the outcome becomes more common; the clear gap here reflects that labour-force participation was far from rare in this sample. Report the one that matches your question and design; the odds ratio or relative risk guide explains when each fits. Risk ratios and risk differences assume the groups were sampled by exposure or cross-sectionally, as here; in a case-control design only the odds ratio is interpretable.
As with the main example, these are associations in observational data. The data set also records age, young and older children, family income and the husband’s college attendance, any of which could be related to both variables, and this table adjusts for none of them. A logistic regression with those covariates is the analysis that asks whether the association survives adjustment.
Simpson’s paradox: what a collapsed table hides
The classic teaching example is UCBAdmissions, which ships with R. It records graduate applications to the six largest departments at the University of California, Berkeley, in 1973, classified by admission decision, applicant sex and department. It is stored as a three-way count table, so the code below works on counts directly rather than on one row per applicant.
# Collapsed over department: admission by sex
collapsed <- margin.table(UCBAdmissions, c(1, 2))
collapsed
prop.table(collapsed, 2) # admission rate for each sex
chisq.test(collapsed) # a clear association in the collapsed table
# Department by department
for (d in dimnames(UCBAdmissions)$Dept) {
dept <- UCBAdmissions[, , d]
cat("\nDepartment", d, "\n")
print(round(prop.table(dept, 2), 3)) # admission rate for each sex within the department
}
# Who applied where: applications by sex and department
prop.table(margin.table(UCBAdmissions, c(2, 3)), 1)
Run it and the collapsed table shows a clear association: a higher proportion of male applicants were admitted. Bickel, Hammel and O’Connell analysed the full Berkeley graduate admissions data for fall 1973 in their 1975 paper (the R data set keeps only the six largest departments) and described that aggregate pattern as clear but misleading. Department by department, few departments showed a significant departure from the expected admissions of women, and about as many appeared to favour women as men. The aggregate pattern arose because women applied more often to departments that were harder for applicants of either sex to enter.
This is Simpson’s paradox: an association in a collapsed table can be absent, weaker or reversed within every level of a third variable. The chi-square test on the collapsed table is computed correctly. It answers the question it was given, whether admission and sex are associated across all applicants pooled together, and that question was the wrong one for judging the admission decisions, which were made department by department.
The lesson for your own tables:
- List the variables that could sit behind both of yours before running the test, and look at the table within levels of each.
- A stratified analysis keeps the third variable in view. The Cochran-Mantel-Haenszel test,
mantelhaen.test(UCBAdmissions)in base R, tests the association within strata and combines the evidence, under the assumption that the association is similar across strata. A logistic regression with the third variable as a covariate is the more flexible alternative. - The data cannot tell you which table is right. Whether to stratify depends on what the third variable is in the causal story. Department is where the admission decision was made, so stratifying by it makes sense here. Stratifying by a variable that is itself a consequence of the one you are studying can create a misleading association instead of removing one.
How to report a chi-square test
A complete report names the test, gives the table or its key percentages, the statistic with its degrees of freedom and sample size, the p-value, an effect size with its interval, and the expected-count check. For the worked example, a results sentence would read:
Voting intention was associated with education level among the 2522 respondents with complete data, chi-square(6, N = 2522) = 135.85, p < 0.001. The association was weak, Cramer’s V = 0.16 (one-sided 95% CI from 0.14). All expected counts were at least 5 (smallest 32.48).
Points to get right:
- Report the p-value exactly when it is not tiny, and as p < 0.001 when it is. Never write “p = 0.000”.
- Report an effect size. Cramer’s V for a general table; for a 2 x 2 table, a risk difference, risk ratio or odds ratio with its interval is usually more informative.
- Show where the association is, with row or column percentages, residuals or a mosaic plot, rather than leaving the reader with one omnibus number.
- State how missing data were handled and how many units were dropped.
- For a 2 x 2 table, say whether Yates’ correction was used.
- Keep the language associational unless the design was a randomised experiment.
Common mistakes
Counting a unit more than once. Multiple-response questions and repeated measurements on the same people produce tables whose cells are not independent. Paired yes or no outcomes, such as the same people before and after, need McNemar’s test, and repeated measurements over time need a model for repeated measures.
Treating a significant result as a strong one. With thousands of units, a trivial association is significant. The worked example’s p-value is far below 0.001, and its V describes a weak association. Report both.
Treating a non-significant result as evidence of independence. A p-value above 0.05 means the data are compatible with independence, not that the variables are unrelated. Small tables can miss substantial associations; the effect size interval and a sensitivity figure show what the study could have detected. Greenland and colleagues list this among their 25 common misinterpretations of P values, confidence intervals and power.
Reading the p-value as the probability that the variables are independent. It is the probability of a statistic at least this large if they were independent, a different quantity.
Ignoring the order of ordered categories. With an ordinal variable, such as education level or a rating scale, the chi-square test treats the categories as unordered and spreads its power over every possible pattern. If the question is about a trend, a test or model that uses the order is more powerful, as Cochran noted in 1954.
Testing every pair of categories after a significant result without adjustment. Breaking a large table into many 2 x 2 tables and testing each one multiplies the chance of a false positive. Plan the comparisons, or adjust for them.
When the chi-square test is the wrong tool
The chi-square test of independence fits two unordered categorical variables, recorded once on each of many independent units, with adequate expected counts. Outside that, other methods answer the question better.
- Small expected counts: Fisher’s exact test or a Monte Carlo p-value. The chi-square or Fisher’s exact test guide explains the choice.
- Paired or matched binary data: McNemar’s test, which uses only the pairs that changed.
- An ordered variable and a trend question: a test for trend or an ordinal regression model.
- A third variable to adjust for: a stratified analysis such as the Cochran-Mantel-Haenszel test, or a logistic regression with covariates.
- A binary outcome you want to model, with several predictors, continuous ones included: logistic regression.
- A continuous outcome: compare means with a t-test or ANOVA instead of cutting the outcome into categories, which throws information away. The t-tests guide explains the two-group case.
- Clustered or survey data: a design-based chi-square test or a model that accounts for the clustering.
If you are unsure which family your question belongs to, the guide to choosing a statistical test starts from the outcome type and the design.
Rerunning the example
Below the article you will find the R script exactly as the engine generated it, with the seed and the bundled data path, and the output it printed. The script reads its data from a data/ folder, which holds the Chile data frame as the engine received it. On the same package versions it reproduces the chi-square statistic, its p-value and the sensitivity figure.
The script does not recompute Cramer’s V or the expected-count check, which come from the engine itself. effectsize::cramers_v(table(data$education, data$vote), adjust = FALSE) and min(chisq.test(table(data$education, data$vote))$expected) reproduce them. The residual code in the section above runs on the same data and shows where the association lies. The guide to reproducible statistical analysis explains what a reviewer needs in order to rerun a result like this one.
Worked example
Data: FLACSO/Chile (1988). National survey of voting intentions before the 1988 Chilean plebiscite, distributed as the Chile data frame in the carData R package (Fox J, Weisberg S, Price B). https://CRAN.R-project.org/package=carData. Seed: 20261022.
Run on Celeus engine 0.30.0, seed 20261022, public data.
Output
# Results - Test: Chi-squared test of association - n: 2522 - statistic: 135.8 - p-value: < 0.001 - effect size: Cramer's V = 0.1641 (95% CI 0.1377 to 1) (one-sided CI (upper bound fixed at 1)) - rows used: 2522 of 2700 (178 dropped) - seed: 20261022 ## Assumption checks - expected cell counts >= 5: passed=TRUE (all expected counts >= 5) ## Power (sensitivity, not observed power) - minimum detectable effect at n=2522: Cohen's w = 0.0735 - to detect a conventional medium effect: 152 total to detect w=0.30 - alpha = 0.05, target power = 0.8
R script
# Celeus - reproduce.R (run from this package folder; bundled data in data/)
# Test: Chi-squared test of association | engine 0.30.0 | catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.
set.seed(20261022)
.data_path <- "data/chi-square-test-explained.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE) # needs the readr package
clean_cells <- function (df)
{
ctrl_collapse <- "[\r\n\t\v\f]+"
zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
clean_text <- function(x) {
out <- x
for (ch in zero_width_chars) out <- gsub(ch, "", out,
fixed = TRUE)
out <- gsub(ctrl_collapse, " ", out)
for (ch in uni_space_chars) out <- gsub(ch, " ", out,
fixed = TRUE)
trimws(gsub(" +", " ", out))
}
changed_cols <- character(0)
cells_cleaned <- 0L
retyped <- character(0)
for (j in seq_along(df)) {
col <- df[[j]]
if (!is.character(col))
next
cleaned <- clean_text(col)
n_changed <- sum(!is.na(col) & !is.na(cleaned) & col !=
cleaned, na.rm = TRUE)
if (n_changed > 0L) {
changed_cols <- c(changed_cols, names(df)[j])
cells_cleaned <- cells_cleaned + n_changed
}
df[[j]] <- cleaned
non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$",
non_blank))) {
parsed <- suppressWarnings(as.numeric(cleaned))
was_blank <- is.na(cleaned) | !nzchar(cleaned)
lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
if (!any(lost)) {
finite <- parsed[!is.na(parsed)]
all_whole <- all(finite == floor(finite))
in_range <- all(abs(finite) <= .Machine$integer.max)
df[[j]] <- if (all_whole && in_range)
as.integer(parsed)
else parsed
retyped <- c(retyped, names(df)[j])
}
}
}
names(df) <- clean_text(names(df))
if (length(changed_cols)) {
attr(df, "ingest_clean_columns") <- changed_cols
attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
}
if (length(retyped))
attr(df, "ingest_clean_retyped") <- retyped
df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("education", "vote")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- chisq.test(table(data[["education"]], data[["vote"]]))
print(result)
# Sensitivity: minimum detectable effect at n=2522 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.chisq.test(N = 2522, df = 6, sig.level = 0.05, power = 0.8)
print(power)Common questions
What is a chi-square test?
A chi-square test compares the counts you observed in each category with the counts you would expect if a hypothesis were true. The most common version, the test of independence, asks whether two categorical variables measured on the same units are associated, using the counts in their cross-table.
When should I use a chi-square test?
Use it when both variables are categorical, each unit is counted once in exactly one cell, the units are independent of each other, and the expected count in each cell is large enough for the approximation. For paired data, ordered categories with a trend question or very small tables, another test fits better.
How do I calculate a chi-square test by hand?
For each cell, compute the expected count as the row total times the column total divided by the grand total. Then add up (observed minus expected) squared divided by expected over every cell, and compare that sum with a chi-square distribution with (rows minus 1) times (columns minus 1) degrees of freedom.
What does a significant chi-square test tell me?
Only that the counts are unlikely under independence. It does not say how strong the association is, which cells drive it, or whether one variable causes the other. Report an effect size such as Cramer's V with its interval, and look at standardized residuals to see where the departure from independence lies.
Can a chi-square test be misleading?
Yes. A third variable can create, hide or reverse an association in a collapsed table, which is Simpson's paradox. Very large samples make trivial associations significant, and clustered or repeated observations break the independence the test relies on. The design, not the p-value, decides what the table can show.
What is the difference between a chi-square goodness-of-fit test and a test of independence?
A goodness-of-fit test has one categorical variable and compares its counts with proportions fixed in advance. A test of independence has two categorical variables recorded on the same units and asks whether they are associated. Both use the same observed-minus-expected sum, but the expected counts and degrees of freedom come from different places.
What are the null and alternative hypotheses of a chi-square test of independence?
The null hypothesis is that the two variables are independent, so the cell proportions are what the row and column totals alone would produce. The alternative is that they are associated in some unspecified way. The test is omnibus, so a significant result does not say which cells or which pattern is responsible.
Sources
- Pearson K (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine 50(302):157-175. doi:10.1080/14786440009463897
- Fisher RA (1922). On the interpretation of chi-square from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 85(1):87-94. doi:10.2307/2340521
- Yates F (1934). Contingency tables involving small numbers and the chi-square test. Supplement to the Journal of the Royal Statistical Society 1(2):217-235. doi:10.2307/2983604
- Cochran WG (1954). Some methods for strengthening the common chi-square tests. Biometrics 10(4):417-451. doi:10.2307/3001616
- Haberman SJ (1973). The analysis of residuals in cross-classified tables. Biometrics 29(1):205-220. doi:10.2307/2529686
- Bickel PJ, Hammel EA, O'Connell JW (1975). Sex bias in graduate admissions: data from Berkeley. Science 187(4175):398-404. doi:10.1126/science.187.4175.398
- Rao JNK, Scott AJ (1981). The analysis of categorical data from complex sample surveys: chi-squared tests for goodness of fit and independence in two-way tables. Journal of the American Statistical Association 76(374):221-230. doi:10.1080/01621459.1981.10477633
- Campbell I (2007). Chi-squared and Fisher-Irwin tests of two-by-two tables with small sample recommendations. Statistics in Medicine 26(19):3661-3675. doi:10.1002/sim.2832
- Bergsma W (2013). A bias-correction for Cramer's V and Tschuprow's T. Journal of the Korean Statistical Society 42(3):323-328. doi:10.1016/j.jkss.2012.10.002
- Mroz TA (1987). The sensitivity of an empirical model of married women's hours of work to economic and statistical assumptions. Econometrica 55(4):765-799. doi:10.2307/1911029
- Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology 31(4):337-350. doi:10.1007/s10654-016-0149-3
Methods discussed
Try it on your own data
Upload a dataset and Celeus suggests a method, checks its assumptions, runs it in R, and gives you the script to rerun it.
Try it in Celeus