Compare two ROC curves (DeLong)
Shown in the app as roc_compare. Part of the method reference, under Diagnostic accuracy and agreement.
When to use
Which of TWO continuous scores/markers better discriminates the SAME binary gold-standard outcome: each marker's ROC AUC with a DeLong confidence interval, and DeLong's test for the difference between the two correlated (paired) AUCs.
What you provide
- Reference standard (the true condition): one two-level column.
- The two scores being compared: two numeric columns.
Options
Missing data
- Complete case (listwise deletion)
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Related methods
Other methods under Diagnostic accuracy and agreement:
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
pROC::aSAH (s100b vs ndka)
- Source: Published dataset distributed with an R package
- Licence: pROC (GPL >= 3)
- Reference: Turck N et al. (2010), Intensive Care Medicine 36(1):107-115; the aSAH dataset as distributed with the pROC package (Robin et al. 2011, BMC Bioinformatics 12:77), comparing its two markers
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Area under the ROC curve, first score | 0.731369 | 0.731369 | - | Yes |
| Area under the ROC curve, second score | 0.611958 | 0.611958 | - | Yes |
| Difference in area under the curve | 0.119411 | 0.119411 | - | Yes |
| p-value for the difference | 0.164295 | 0.164295 | - | Yes |
| Test statistic for the difference | 1.39077 | 1.39077 | - | Yes |
pROC::aSAH (s100b vs WFNS grade)
- Source: Published dataset distributed with an R package
- Licence: pROC (GPL >= 3)
- Reference: Turck N et al. (2010), Intensive Care Medicine 36(1):107-115; the aSAH dataset as distributed with the pROC package
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic for the difference | -2.20898 | -2.20898 | - | Yes |
| p-value for the difference | 0.0271758 | 0.0271758 | - | Yes |
| Confidence interval | -0.1742140, -0.0104062 | -0.1742140, -0.0104062 | - | Yes |
pROC::aSAH (s100b vs ndka with its sign reversed, a constructed opposite-direction marker)
- Source: Published dataset distributed with an R package
- Licence: pROC (GPL >= 3)
- Reference: Turck N et al. (2010), Intensive Care Medicine 36(1):107-115; the aSAH dataset as distributed with the pROC package, ndka multiplied by -1 for this check
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| p-value for the difference | 0.164295 | 0.164295 | - | Yes |
carData::GSSvocab (vocabulary score vs years of education, against native birth)
- Source: Published dataset distributed with an R package
- Licence: carData (GPL)
- Reference: General Social Survey extract, from Fox & Weisberg, An R Companion to Applied Regression (carData)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Test statistic for the difference | 15.1649 | 15.1649 | - | Yes |
| Confidence interval | 0.100073, 0.129780 | 0.100073, 0.129780 | - | Yes |
| Number of observations | 27425 | 27425 | - | Yes |
References
- DeLong, E. R., DeLong, D. M., Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44: 837-845. doi:10.2307/2531595 doi:10.2307/2531595