Sensitivity vs specificity in a diagnostic study

Sensitivity and specificity are the two numbers a diagnostic-accuracy study reports first, and the two that are most often mixed up with each other and with the predictive values. This post covers how a study estimates them from a 2x2 table, why the predictive values behave differently, and how to report them with intervals. It says nothing about any individual result.

The example is a seeded, synthetic comparison of a rapid assay against a reference assay on 400 samples, run through a deterministic engine. A second seeded sample is described later. If your question is about two numeric measurements of the same quantity rather than two yes/no results, the Bland-Altman guide is the right starting point.

The 2x2 table behind every accuracy study

A diagnostic-accuracy study compares an index test with a reference standard on the same samples. Both give a yes/no result, so each sample lands in one of four cells.

• True positive (TP): the test is positive and the reference is positive.
• False positive (FP): the test is positive and the reference is negative.
• False negative (FN): the test is negative and the reference is positive.
• True negative (TN): the test is negative and the reference is negative.

In the example, the counts are 81 true positives, 34 false positives, 9 false negatives and 276 true negatives, 400 samples in total. The reference standard is treated as the truth for this calculation, which is a strong assumption. Everything below is accuracy against that reference, and it is only as good as the reference is.

The unit of analysis is the sample, and each sample contributes exactly once. If one participant contributes several samples, the cells are not independent and the intervals below are too narrow, so the table needs a method that accounts for the clustering.

Sensitivity and specificity

Sensitivity is TP / (TP + FN): of the reference-positive samples, the share the test calls positive. Specificity is TN / (TN + FP): of the reference-negative samples, the share the test calls negative.

Both are proportions within a column of the table, the reference status. They are properties of the test measured against a reference, and they do not depend on how many reference-positive samples the study happened to include, apart from the precision of the estimate and the spectrum issue discussed below.

In the example, the 90 reference-positive samples (81 + 9) give a sensitivity of 0.9000, and the 310 reference-negative samples (276 + 34) give a specificity of 0.8903.

The two are linked by a threshold when the test comes from dichotomising a numeric reading. Moving the cut-off to call more samples positive raises sensitivity and lowers specificity, and the reverse. That is why a single pair of values describes one cut-off only, and why a study that chose its cut-off from the same data tends to look better than it will on new samples. For the whole range of cut-offs at once, a receiver operating characteristic curve is the usual tool.

Predictive values answer a different question

Positive predictive value (PPV) is TP / (TP + FP): of the samples the test calls positive, the share that are reference-positive. Negative predictive value (NPV) is TN / (TN + FN): of the samples called negative, the share that are reference-negative. These are proportions within a row of the table, the test result. In machine-learning work, PPV is called precision and sensitivity recall (Saito and Rehmsmeier 2015).

Because rows mix the two reference groups in the proportions the study happened to sample, the predictive values depend on the proportion of reference-positive samples, the prevalence in the sample. Sensitivity and specificity do not carry that dependence in the same way.

In the example, the PPV is 0.7043 and the NPV is 0.9684. The prevalence of reference-positive samples is 0.225.

The same assay in a sample with fewer reference-positive samples

To show the effect, a second seeded sample uses the same assay characteristics but draws far fewer reference-positive samples, 1,200 samples in all, of which 48 are reference-positive. Its counts are 41 true positives, 172 false positives, 7 false negatives and 980 true negatives.

	Main example	Fewer reference-positive samples
Samples	400	1,200
Prevalence in the sample	0.225	0.04
Sensitivity	0.9000	0.8542
Specificity	0.8903	0.8507
PPV	0.7043	0.1925
NPV	0.9684	0.9929

Sensitivity and specificity are in the same region in both samples. The differences between the two samples come from sampling error, which is large for sensitivity here because it rests on only 48 samples. The PPV falls from 0.7043 to 0.1925, and the NPV rises. Only the mix of reference groups changed.

That is the reason a predictive value from one study should not be carried over to a sample with a different proportion of reference-positive samples without saying so. A study can report sensitivity and specificity as characteristics of the test, but a PPV and NPV are only descriptions of the sample they came from, unless the study sampled in proportion to the target population.

To describe another setting, a report can recompute the predictive values from sensitivity, specificity and a stated prevalence p, for example PPV = sensitivity × p / (sensitivity × p + (1 - specificity) × (1 - p)) (Altman and Bland 1994). The results inherit the uncertainty of both estimates, depend on how well the stated prevalence is known, and assume the test behaves the same in the new setting (no spectrum change).

Intervals for each estimate

Each of the four proportions is a binomial proportion with its own denominator, so each needs its own interval. The engine uses the exact (Clopper-Pearson) interval, which is the interval from the binomial test and guarantees at least the stated coverage, at the cost of being a little conservative.

In the main example, with a 95% confidence level:

Estimate	Value	95% confidence interval
Sensitivity	0.9000	0.8186 to 0.9532
Specificity	0.8903	0.8501 to 0.9228
PPV	0.7043	0.6121 to 0.7858
NPV	0.9684	0.9409 to 0.9855

The sensitivity interval is wider than the specificity interval because it rests on 90 samples where specificity rests on 310. That is the general pattern: the precision of each estimate is set by the size of its own group, not by the total sample size. A study with 400 samples sounds large, but if only 90 are reference-positive, sensitivity is estimated from 90.

The same effect is stronger in the second sample. Sensitivity is 0.8542 with an interval of 0.7224 to 0.9393, a wide range, even though 1,200 samples were run in total. Specificity, from 1,152 reference-negative samples, has an interval of 0.8288 to 0.8708. Planning works the same way: size the reference-positive group for the precision wanted on sensitivity and divide by the expected prevalence, do the same for specificity with 1 minus the prevalence, and recruit the larger of the two totals (Buderer 1996).

An interval is a statement about the procedure. Over repeated studies of the same kind, intervals built this way contain the true proportion at least 95% of the time. It is not a 95% probability that the true value lies in this one interval. See p-values and confidence intervals for the wording.

Likelihood ratios and Youden’s J

Two further summaries combine sensitivity and specificity, and neither depends on the proportion of reference-positive samples. The positive likelihood ratio is sensitivity / (1 - specificity), and the negative likelihood ratio is (1 - sensitivity) / specificity. In the example the positive ratio is 8.2059 (95% interval 5.9316 to 11.3521) and the negative ratio is 0.1123 (0.0604 to 0.2090), using log-method intervals (Simel et al. 1991).

Youden’s J is sensitivity + specificity - 1, here 0.7903. It summarises the pair at one cut-off and is 0 for a test no better than chance. It treats a missed reference-positive sample and a wrongly flagged reference-negative sample as equally costly, which is rarely true of a real study, so report the two proportions and use J only as a convenience.

Computing it in R

The intervals come from base R. With the main example’s counts:

tp <- 81; fn <- 9; tn <- 276; fp <- 34

binom.test(tp, tp + fn)$conf.int   # sensitivity
binom.test(tn, tn + fp)$conf.int   # specificity
binom.test(tp, tp + fp)$conf.int   # positive predictive value
binom.test(tn, tn + fn)$conf.int   # negative predictive value

The worked example’s own script is shown below this post, exactly as the engine generated it, with the seed and the output it printed. It rebuilds the table from the 400 synthetic samples and recomputes the four proportions with their intervals, the prevalence and both likelihood ratios; the second sample’s numbers come from a separate seeded run. In the summary the engine prints, the interval on the “95% CI” line belongs to sensitivity, the headline estimate for this method, and the full set is in the script’s printed list.

One setup detail matters in any software. A 2x2 calculation needs to know which value of each column is “positive”. Swapping that choice for both columns exchanges sensitivity with specificity and PPV with NPV, so the example sets the positive value explicitly for the test and the reference, which also documents the choice in the record of the run.

Pitfalls that change the numbers

Several problems in study design move these estimates more than the choice of interval does.

• Spectrum. Sensitivity is higher in samples with more clear-cut reference-positive cases and lower in borderline ones, so a study that sampled only clear cases overstates it. The same applies to specificity and easy negatives (Ransohoff and Feinstein 1978).
• Verification. If only some samples receive the reference standard, and the choice depends on the test result, the estimates are biased. The QUADAS-2 tool lists this and related risks of bias (Whiting et al. 2011).
• An imperfect reference. Errors in the reference standard are counted as errors of the test. When the two err independently, the estimates are biased downwards; errors on the same samples can inflate them (Reitsma et al. 2009).
• A cut-off chosen on the same data. Reporting the best cut-off from the data used to find it is optimistic. Fix the cut-off in advance or validate it on separate samples.
• Several tests compared. When two tests are applied to the same samples, the comparison is paired. The McNemar test guide covers the paired comparison of two yes/no results.

Reporting a diagnostic-accuracy study

The STARD 2015 checklist (Bossuyt et al. 2015) lists what a report should contain. For the numbers in this post, a results paragraph would give:

• the four cell counts as a 2x2 table, with the number of samples excluded and why;
• sensitivity and specificity, each with its confidence interval and the interval method;
• the predictive values with intervals, together with the proportion of reference-positive samples in the study;
• how the reference standard was applied, and whether the test was read without knowledge of it.

For example: “Against the reference assay, the rapid assay had a sensitivity of 0.9000 (95% CI 0.8186 to 0.9532, 81 of 90) and a specificity of 0.8903 (95% CI 0.8501 to 0.9228, 276 of 310), using exact intervals.” Give the counts beside each proportion, so a reader can recompute it.

Only one family of estimates is computed per table here, so there is no multiplicity adjustment. If you report accuracy for many cut-offs or many subgroups, say how many comparisons you made, and treat the subgroup results as exploratory.

What these numbers do and do not say

A sensitivity and specificity from a study describe how a test performed against a reference in the samples studied. They are research results with sampling uncertainty, shown by the intervals. They are not a statement about any single sample’s true status, and this post is not guidance on testing or diagnosing any person.

For other questions, the guide to choosing a statistical test sets out the options. To compare the odds of a positive result between groups, see odds ratios and relative risk. The method itself is described on the diagnostic accuracy reference page.

# Results

- Test: Diagnostic accuracy (2x2)
- n: 400
- statistic: NA
- p-value: NA
- effect size: Youden's J = 0.7903
- 95% CI: [0.8186, 0.9532]
- seed: 20261022

## Assumption checks
- sample size: passed=TRUE (adequate n)

## Power
- Power is not calculated for this test.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Diagnostic accuracy (2x2)  |  engine 0.31.1  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/sensitivity-vs-specificity.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("rapid_assay", "reference_assay")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- { .t <- table(factor(as.character(data[["rapid_assay"]]), levels = c("positive", "negative")), factor(as.character(data[["reference_assay"]]), levels = c("positive", "negative"))); .tp <- .t[1, 1]; .fp <- .t[1, 2]; .fn <- .t[2, 1]; .tn <- .t[2, 2]; .ci <- function(x, n) if (n > 0) as.numeric(stats::binom.test(x, n)$conf.int) else c(NA_real_, NA_real_); .p <- function(x, n) if (n > 0) x / n else NA_real_; .z <- stats::qnorm(1 - (1 - 0.95) / 2); .dis <- .tp + .fn; .nd <- .fp + .tn; .lr <- function(ok, v, se) if (ok) list(value = v, ci = exp(log(v) + c(-1, 1) * .z * se)) else list(value = NA_real_, ci = c(NA_real_, NA_real_)); .lp <- .lr(.tp > 0 && .fp > 0, (.tp / .dis) / (.fp / .nd), sqrt(1 / .tp - 1 / .dis + 1 / .fp - 1 / .nd)); .ln <- .lr(.fn > 0 && .tn > 0, (.fn / .dis) / (.tn / .nd), sqrt(1 / .fn - 1 / .dis + 1 / .tn - 1 / .nd)); .do <- .lr(all(c(.tp, .fp, .fn, .tn) > 0), (.tp * .tn) / (.fp * .fn), sqrt(1 / .tp + 1 / .fp + 1 / .fn + 1 / .tn)); list(counts = list(tp = .tp, fp = .fp, fn = .fn, tn = .tn), sensitivity = .p(.tp, .tp + .fn), specificity = .p(.tn, .tn + .fp), ppv = .p(.tp, .tp + .fp), npv = .p(.tn, .tn + .fn), sensitivity_ci = .ci(.tp, .tp + .fn), specificity_ci = .ci(.tn, .tn + .fp), ppv_ci = .ci(.tp, .tp + .fp), npv_ci = .ci(.tn, .tn + .fn), prevalence = (.tp + .fn) / (.tp + .fp + .fn + .tn), lr_pos = .lp$value, lr_pos_ci = .lp$ci, lr_neg = .ln$value, lr_neg_ci = .ln$ci, dor = .do$value, dor_ci = .do$ci) }
print(result)