Paired t-test
Shown in the app as paired_t. Part of the method reference, under Comparing groups.
When to use
Two paired/repeated continuous measurements, normal differences.
What you provide
- The two paired measurements: two numeric columns.
Options
Missing data
- Complete case (listwise deletion)
- Mean imputation (single)
- Median imputation (single)
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Alternative hypothesis
Two-sided unless you choose otherwise. Choose a one-sided test only when the direction was specified before the data were seen; the result then says so.
- Two-sided: the first differs from the second in either direction
- One-sided: the first is lower than the second
- One-sided: the first is higher than the second
Equivalence test
Equivalence test (two one-sided tests): available.
Related methods
Alternatives that take the same inputs:
Other methods under Comparing groups:
- Welch's t-test
- Student's t-test
- Mann-Whitney U test
- One-way ANOVA
- Welch's ANOVA
- Kruskal-Wallis test
- Repeated-measures ANOVA (Greenhouse-Geisser)
- ANCOVA (analysis of covariance)
- Two-way ANOVA (factorial)
Guides that discuss this method
How to read the output
These methods compare two measurements taken on the same units, for example the same students before and after a course, or the same field plots in two seasons. The analysis works on the difference within each pair, so variation between units does not blur the comparison.
Paired t-test. The estimate is the mean of the paired differences (first measurement minus second), with its 95% confidence interval. An interval that excludes 0 corresponds to p below 0.05. The effect size is Cohen’s d for paired data. The test assumes the differences, not the raw measurements, are roughly normally distributed, which is what its normality check looks at.
Equivalence. A non-significant change is not evidence of no change. To show that the mean paired difference is small enough to ignore, the paired t-test can run an equivalence test (two one-sided tests). You enter a margin, one positive number in the measurement’s units, and equivalence is concluded when the whole 90% confidence interval for the mean difference lies inside minus the margin to plus the margin (Schuirmann, 1987). Choose and justify the margin before you see the data (Lakens, 2017). Celeus suggests no default margin, and the margin needs a two-sided test.
Wilcoxon signed-rank test. This rank test makes no normality assumption about the differences. The estimate is the Hodges-Lehmann estimate of the typical paired difference, with its confidence interval. The effect size is the rank-biserial correlation, from -1 to 1, where 0 means increases and decreases are balanced.
In both, the p-value is the probability of differences at least this far from zero if there were truly no change. It says nothing about how large the change is, so read the estimate and its interval first.
Sources: Lakens D (2017), Social Psychological and Personality Science 8(4):355-362, doi:10.1177/1948550617697177. Schuirmann DJ (1987), Journal of Pharmacokinetics and Biopharmaceutics 15(6):657-680, doi:10.1007/BF01068419. Student (1908), Biometrika 6(1):1-25, doi:10.2307/2331554. Wilcoxon F (1945), Biometrics Bulletin 1(6):80-83, doi:10.2307/3001968.
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
datasets::sleep
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.06213 | -4.06213 | - | Yes |
| p-value | 0.00283289 | 0.00283289 | 0.00283289 | Yes |
| Estimate | -1.58 | -1.58 | - | Yes |
| Degrees of freedom | 9 | 9 | - | Yes |
| Confidence interval | -2.459890, -0.700114 | -2.459890, -0.700114 | - | Yes |
| Effect size | -1.28456 | -1.28456 | - | Yes |
| Effect size confidence interval | -2.118020, -0.414628 | -2.118020, -0.414628 | - | Yes |
datasets::sleep
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.06213 | -4.06213 | - | Yes |
| p-value | 0.00141645 | 0.00141645 | - | Yes |
| Estimate | -1.58 | -1.58 | - | Yes |
| Degrees of freedom | 9 | 9 | - | Yes |
| Upper confidence limit | -0.866995 | -0.866995 | - | Yes |
| Effect size | -1.28456 | -1.28456 | - | Yes |
| Effect size upper confidence limit | -0.54464 | -0.54464 | - | Yes |
datasets::sleep
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.06213 | -4.06213 | - | Yes |
| p-value | 0.00283289 | 0.00283289 | - | Yes |
| Estimate | -1.58 | -1.58 | - | Yes |
| Degrees of freedom | 9 | 9 | - | Yes |
| Confidence interval | -2.844050, -0.315948 | -2.844050, -0.315948 | - | Yes |
| Effect size | -1.28456 | -1.28456 | - | Yes |
| Effect size confidence interval | -2.401070, -0.164067 | -2.401070, -0.164067 | - | Yes |
datasets::sleep (one value removed)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.27218 | -4.27218 | - | Yes |
| p-value | 0.00207365 | 0.00207365 | - | Yes |
| Estimate | -1.71667 | -1.71667 | - | Yes |
| Degrees of freedom | 9 | 9 | - | Yes |
| Confidence interval | -2.625660, -0.807676 | -2.625660, -0.807676 | - | Yes |
| Number of observations | 10 | 10 | - | Yes |
datasets::sleep (one value removed)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -3.71673 | -3.71673 | - | Yes |
| p-value | 0.00589906 | 0.00589906 | - | Yes |
| Estimate | -1.61111 | -1.61111 | - | Yes |
| Degrees of freedom | 8 | 8 | - | Yes |
| Confidence interval | -2.610710, -0.611514 | -2.610710, -0.611514 | - | Yes |
| Number of observations | 9 | 9 | - | Yes |
datasets::sleep
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Student (1908), Biometrika 6(1):1-25; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.06213 | -4.06213 | - | Yes |
| p-value | 0.00283289 | 0.00283289 | - | Yes |
| Estimate | -1.58 | -1.58 | - | Yes |
| Degrees of freedom | 9 | 9 | - | Yes |
| Confidence interval | -2.459890, -0.700114 | -2.459890, -0.700114 | - | Yes |
| Equivalence margin (the largest difference you would still call negligible) | 3 | 3 | - | Yes |
| p-value of the lower one-sided test | 0.00265557 | 0.00265557 | - | Yes |
| p-value of the upper one-sided test | 4.52269e-07 | 4.52269e-07 | - | Yes |
| p-value of the equivalence test (the larger of the two one-sided p-values) | 0.00265557 | 0.00265557 | - | Yes |
| Test statistic of the lower one-sided test | 3.65077 | 3.65077 | - | Yes |
| Test statistic of the upper one-sided test | -11.775 | -11.775 | - | Yes |
| Degrees of freedom of the equivalence test | 9 | 9 | - | Yes |
| Lower limit of the equivalence confidence interval | -2.29301 | -2.29301 | - | Yes |
| Upper limit of the equivalence confidence interval | -0.866995 | -0.866995 | - | Yes |
| Whether the difference is shown to lie within the margin | 1 | 1 | - | Yes |
MASS::anorexia (family therapy arm)
- Source: Published dataset distributed with an R package
- Licence: MASS (GPL)
- Reference: Hand et al. (1994), A Handbook of Small Data Sets
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic | -4.18491 | -4.18491 | - | Yes |
| p-value | 0.000700253 | 0.000700253 | - | Yes |
| Estimate | -7.26471 | -7.26471 | - | Yes |
| Degrees of freedom | 16 | 16 | - | Yes |
| Confidence interval | -10.9447, -3.5847 | -10.9447, -3.5847 | - | Yes |
References
- Student (1908). The probable error of a mean. Biometrika 6: 1-25. doi:10.2307/2331554 doi:10.2307/2331554