Fisher's exact test
Shown in the app as fisher_exact. Part of the method reference, under Association and correlation.
When to use
Association between two categorical variables, small expected counts.
What you provide
- The two categorical variables: two categorical columns.
Options
Missing data
- Complete case (listwise deletion)
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Alternative hypothesis
Two-sided unless you choose otherwise. Choose a one-sided test only when the direction was specified before the data were seen; the result then says so.
- Two-sided: the first differs from the second in either direction
- One-sided: the first is lower than the second
- One-sided: the first is higher than the second
Related methods
Alternatives that take the same inputs:
Other methods under Association and correlation:
Guides that discuss this method
- Chi-square or Fisher's exact test: which to use?
- How to choose a statistical test for your data
- Odds ratio or relative risk: which to report?
How to read the output
These methods ask whether two categorical variables are associated, for example survey respondents’ region and their preferred answer, or a seed treatment and whether the seed germinated. The data are summarised as a table of counts.
Chi-squared test. The test compares the observed counts with the counts expected if the two variables were independent. A small p-value means the observed pattern would be unusual under independence. It does not say which cells drive the association or how strong it is. The approximation behind the test is unreliable when expected counts are small, which is what the expected-count check looks at (Cochran, 1954). Fisher’s exact test is the usual alternative then. For a 2 by 2 table the p-value includes the Yates continuity correction (R’s default), so a hand calculation without it will give a slightly different p-value.
Fisher’s exact test. The p-value is computed exactly from the table rather than from an approximation, so it holds for small counts. For a two-by-two table the estimate is an odds ratio with its confidence interval. For a larger table the exact computation can be too large to finish; the run then says so and the chi-squared test is the alternative, rather than quietly reporting a different test’s p-value. A test of a larger table is non-directional, so its Methods text never calls it two-sided.
Strength of association. The effect size for both is Cramér’s V, from 0 (no association) to 1 (perfect association). It is the classic, unadjusted V computed from the chi-squared statistic without a continuity correction, and it comes with a one-sided confidence interval whose upper bound is fixed at 1, because V cannot exceed 1. For a two-by-two table Celeus also reports the odds ratio, the risk ratio and the risk difference. The risk ratio and risk difference are meaningful only when the groups were sampled by the first variable (for example, by treatment); if they were sampled by the outcome, only the odds ratio can be interpreted.
Options. Every interval of a run (Cramér’s V, the odds ratio and the two-by-two measures) uses one confidence level: 90%, 95% (the default) or 99%. Fisher’s exact test on a two-by-two table can also be run one-sided, when the direction was decided before the data were seen: “greater” means the second category of the first variable has higher odds of the second category of the second variable than the first category does. The two-by-two measures stay two-sided even then, because they describe the table rather than test the hypothesis, and the Methods text says so. The chi-squared test is always non-directional.
Missing values. Rows with a missing value in either variable are left out (complete-case analysis). Filling a missing category with a mean or median would invent a category that was never observed, so these tests do not offer it. If the rows left over hold only one category of a variable, there is no association to test and the run says so.
Sources: Pearson K (1900), Philosophical Magazine 50(302):157-175, doi:10.1080/14786440009463897. Cochran WG (1954), Biometrics 10(4):417-451, doi:10.2307/3001616. Cramér H (1946), Mathematical Methods of Statistics, Princeton University Press.
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.472697 | 0.472697 | - | Yes |
| Odds ratio | 2 | 2 | - | Yes |
Fisher's tea-tasting experiment (8 cups)
- Source: Illustrative or synthetic example data
- Licence: illustrative textbook table (8 cups of tea), not patient data
- Reference: Fisher (1935), The Design of Experiments
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.485714 | 0.485714 | - | Yes |
| Odds ratio | 6.40831 | 6.40831 | - | Yes |
| Odds ratio confidence interval | 0.211733, 621.934000 | 0.211733, 621.934000 | - | Yes |
datasets::mtcars (cyl 4/6/8 x am 0/1, numeric codes)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981), Biometrics 37(2)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.0091047 | 0.0091047 | - | Yes |
MASS::survey (Smoke x Exer, 1 answer missing)
- Source: Published dataset distributed with an R package
- Licence: MASS (GPL)
- Reference: Venables & Ripley (2002), Modern Applied Statistics with S, 4th ed.
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.413845 | 0.413845 | - | Yes |
| Number of observations | 236 | 236 | - | Yes |
MASS::survey (Sex x W.Hnd, answers missing)
- Source: Published dataset distributed with an R package
- Licence: MASS (GPL)
- Reference: Venables & Ripley (2002), Modern Applied Statistics with S, 4th ed.
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.615836 | 0.615836 | - | Yes |
| Estimate | 0.688366 | 0.688366 | - | Yes |
| Lower confidence limit | 0.214024 | 0.214024 | - | Yes |
| Upper confidence limit | 2.08767 | 2.08767 | - | Yes |
| Number of observations | 235 | 235 | - | Yes |
MASS::survey (Sex x W.Hnd, answers missing)
- Source: Published dataset distributed with an R package
- Licence: MASS (GPL)
- Reference: Venables & Ripley (2002), Modern Applied Statistics with S, 4th ed.
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.615836 | 0.615836 | - | Yes |
| Estimate | 0.688366 | 0.688366 | - | Yes |
| Lower confidence limit | 0.149315 | 0.149315 | - | Yes |
| Upper confidence limit | 2.86022 | 2.86022 | - | Yes |
| Effect size lower confidence limit | 0 | 0 | - | Yes |
Fisher's tea-tasting experiment (8 cups)
- Source: Illustrative or synthetic example data
- Licence: illustrative textbook table (8 cups of tea), not patient data
- Reference: Fisher (1935), The Design of Experiments; the one-sided p as printed in R's ?fisher.test
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.242857 | 0.242857 | 0.2429 | Yes |
| Estimate | 6.40831 | 6.40831 | - | Yes |
| Lower confidence limit | 0.313569 | 0.313569 | - | Yes |
Fisher (1962) like-sex twins convictions, as printed in R's ?fisher.test
- Source: Values entered from a table in the cited publication
- Licence: base R (GPL)
- Reference: Fisher (1962), as printed in R's ?fisher.test
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.000465181 | 0.000465181 | - | Yes |
| Estimate | 0.0469366 | 0.0469366 | - | Yes |
| Lower confidence limit | 0 | 0 | - | Yes |
| Upper confidence limit | 0.28496 | 0.28496 | - | Yes |
Fisher (1962) like-sex twins convictions, as printed in R's ?fisher.test
- Source: Values entered from a table in the cited publication
- Licence: base R (GPL)
- Reference: Fisher (1962), as printed in R's ?fisher.test
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.000536724 | 0.000536724 | - | Yes |
| Estimate | 0.0469366 | 0.0469366 | - | Yes |
| Lower confidence limit | 0.00138633 | 0.00138633 | - | Yes |
| Upper confidence limit | 0.578852 | 0.578852 | - | Yes |
Agresti (2002) job satisfaction by income, as printed in R's ?fisher.test
- Source: Values entered from a table in the cited publication
- Licence: base R (GPL)
- Reference: Agresti (2002), Categorical Data Analysis, 2nd ed., p. 57, as printed in R's ?fisher.test
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value | 0.782685 | 0.782685 | 0.7827 | Yes |
| Number of observations | 96 | 96 | - | Yes |
References
- Fisher, R. A. (1922). On the interpretation of chi-squared from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 85: 87-94. doi:10.1111/j.2397-2335.1922.tb00768.x doi:10.1111/j.2397-2335.1922.tb00768.x