Student's t-test
Shown in the app as student_t. Part of the method reference, under Comparing groups.
When to use
Two independent groups, continuous outcome, equal variances assumed.
What you provide
- Outcome: one numeric column.
- Group: one categorical column with exactly two groups.
Options
Missing data
- Complete case (listwise deletion)
- Mean imputation (single)
- Median imputation (single)
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Alternative hypothesis
Two-sided unless you choose otherwise. Choose a one-sided test only when the direction was specified before the data were seen; the result then says so.
- Two-sided: the first differs from the second in either direction
- One-sided: the first is lower than the second
- One-sided: the first is higher than the second
Equivalence test
Equivalence test (two one-sided tests): available.
Related methods
Alternatives that take the same inputs:
Other methods under Comparing groups:
- Paired t-test
- Wilcoxon signed-rank test
- One-way ANOVA
- Welch's ANOVA
- Kruskal-Wallis test
- Repeated-measures ANOVA (Greenhouse-Geisser)
- ANCOVA (analysis of covariance)
- Two-way ANOVA (factorial)
Guides that discuss this method
How to read the output
These methods compare one outcome between two independent groups, for example reading scores under two teaching methods, or plant height under two fertilisers.
Welch’s and Student’s t-tests. The estimate is the difference in group means, with the direction stated (first group minus second). Read it with its 95% confidence interval: the interval is the range of differences compatible with your data, and an interval that excludes 0 corresponds to p below 0.05. The p-value is the probability of a difference at least this large if the two population means were equal; it does not measure how large or important the difference is. The effect size is Cohen’s d, the mean difference in standard-deviation units. Cohen’s benchmarks of 0.2, 0.5 and 0.8 for small, medium and large are a rough guide, not a rule (Cohen, 1988).
Welch’s test does not assume equal variances, which is why it is the default. When the variances really are equal it loses very little compared with Student’s test (Delacre, Lakens and Leys, 2017).
Equivalence. A non-significant difference is not evidence that two groups are the same: it may only mean the sample was small. To show that the difference is small enough to ignore, the t-tests can run an equivalence test (two one-sided tests). You enter a margin, one positive number in the outcome’s units, and equivalence is concluded when the whole 90% confidence interval for the difference lies inside minus the margin to plus the margin (Schuirmann, 1987). Choose and justify the margin before you see the data, from what difference would matter in your field (Lakens, 2017). Celeus suggests no default margin, and the margin needs a two-sided test.
Mann-Whitney U test. This rank test asks whether values in one group tend to be larger than values in the other. It is a test of medians only when the two distributions have the same shape. The estimate is the Hodges-Lehmann shift, the median of all differences between a value from one group and a value from the other, with its confidence interval. The effect size is the rank-biserial correlation, from -1 to 1, where 0 means neither group tends to be larger (Kerby, 2014).
Sources: Cohen J (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd ed., Lawrence Erlbaum. Delacre M, Lakens D, Leys C (2017), International Review of Social Psychology 30(1):92-101, doi:10.5334/irsp.82. Lakens D (2017), Social Psychological and Personality Science 8(4):355-362, doi:10.1177/1948550617697177. Schuirmann DJ (1987), Journal of Pharmacokinetics and Biopharmaceutics 15(6):657-680, doi:10.1007/BF01068419. Kerby DS (2014), Comprehensive Psychology 3, doi:10.2466/11.IT.3.1.
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
datasets::ToothGrowth
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Crampton (1947), J. Nutrition 33(5)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | 1.91527 | 1.91527 | 1.91527 | Yes |
| p-value | 0.0603934 | 0.0603934 | 0.0603934 | Yes |
| Estimate | 3.7 | 3.7 | 3.7 | Yes |
datasets::ToothGrowth
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Crampton (1947), J. Nutrition 33(5)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | 1.91527 | 1.91527 | - | Yes |
| p-value | 0.0603934 | 0.0603934 | - | Yes |
| Estimate | 3.7 | 3.7 | - | Yes |
| Degrees of freedom | 58 | 58 | - | Yes |
| Confidence interval | 0.47082, 6.92918 | 0.47082, 6.92918 | - | Yes |
| Effect size | 0.49452 | 0.49452 | - | Yes |
| Effect size confidence interval | 0.0610971, 0.9237640 | 0.0610971, 0.9237640 | - | Yes |
datasets::ToothGrowth
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Crampton (1947), J. Nutrition 33(5)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | 1.91527 | 1.91527 | - | Yes |
| p-value | 0.969803 | 0.969803 | - | Yes |
| Estimate | 3.7 | 3.7 | - | Yes |
| Degrees of freedom | 58 | 58 | - | Yes |
| Upper confidence limit | 6.92918 | 6.92918 | - | Yes |
Lakens (2017) worked example: two samples reconstructed to the printed n, mean and SD
- Source: Values entered from a table in the cited publication
- Licence: summary statistics from the publication; the rows are generated from them
- Reference: Lakens (2017), Social Psychological and Personality Science 8(4):355-362, pp. 357-358
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Equivalence margin (the largest difference you would still call negligible) | 0.384 | 0.384 | - | Yes |
| p-value of the lower one-sided test | 0.000988565 | 0.000988565 | 0.001 | Yes |
| p-value of the upper one-sided test | 0.00397048 | 0.00397048 | 0.004 | Yes |
| p-value of the equivalence test (the larger of the two one-sided p-values) | 0.00397048 | 0.00397048 | - | Yes |
| Test statistic of the lower one-sided test | 3.13917 | 3.13917 | 3.14 | Yes |
| Test statistic of the upper one-sided test | -2.68422 | -2.68422 | - | Yes |
| Degrees of freedom of the equivalence test | 182 | 182 | 182 | Yes |
| Lower limit of the equivalence confidence interval | -0.188036 | -0.188036 | - | Yes |
| Upper limit of the equivalence confidence interval | 0.248036 | 0.248036 | - | Yes |
| Whether the difference is shown to lie within the margin | 1 | 1 | - | Yes |
References
- Student (1908). The probable error of a mean. Biometrika 6: 1-25. doi:10.2307/2331554 doi:10.2307/2331554