Correlation vs causation in research reports

Two measures can move together very closely while neither has any effect on the other. A correlation describes your data; a causal claim describes what would happen if something were changed, and the data alone cannot supply that second part. This post shows the gap on a small seeded simulation, then lists what a design needs before a report can use causal wording.

What a correlation tells you, and what it leaves out

Pearson’s r summarises how closely two continuous measures follow a straight line, from -1 to 1. It is a statement about the sample: when one measure is high, the other tends to be high too. It says nothing about why. At least four stories produce the same r:

• One drives the other. The causal reading, and the only one that matters if you want to intervene.
• The arrow runs the other way. Reverse causation, common in cross-sectional surveys where both measures were taken at once.
• A third variable drives both. Confounding, the case worked below.
• The sample links them. Selection: if only some units enter the data, and entry depends on both measures, they can correlate in the sample and not in the population.

Chance is a fifth. Look at enough pairs and some will correlate; a pair picked out afterwards for being striking is not evidence about cause. Two measures that both trend over time, such as yearly series, tend to correlate whatever their connection; Yule (1926) called these nonsense correlations, and a correlation between two trends says little until the trend is removed or modelled. For a general introduction, see Altman and Krzywinski (2015). The choosing a statistical test guide covers when a correlation is the right summary at all.

A simulation where the answer is known

A simulation lets us build the truth in, so we can see whether the analysis recovers it. We simulated 150 summer days at one seaside town. Each day has an afternoon temperature, the number of ice cream cones sold at a kiosk, and the number of visits to the municipal pool. Ice cream sales and pool visits were each built from temperature plus independent random noise. Neither was used to build the other: by construction, selling more ice cream does not send anyone to the pool, and a busy pool does not sell ice cream. The days are treated as independent, one row per day, with no carry-over from one day to the next built in.

The engine ran a Pearson correlation of ice cream sales against pool visits on the 150 days, with the method fixed in advance:

• Test: Pearson correlation, n = 150, no rows dropped.
• Result: r = 0.696, 95% CI 0.603 to 0.7703, p < 0.001.
• Sample-size check: passed.

That is a strong correlation by any usual convention, with a narrow interval and a tiny p-value. Read as an association it is correct: on days when more ice cream was sold, more people used the pool. Read as “ice cream sales drive pool visits”, or the reverse, it is wrong, and we know it is wrong because we wrote the data.

Putting the common cause into the model

The fix is to include the third variable. We ran a second analysis on the same 150 days (same seed, same generator): a linear regression of pool visits on both ice cream sales and temperature, so each coefficient is adjusted for the other.

• Ice cream sales: coefficient -0.0933 visits per extra cone, 95% CI -0.371 to 0.1844, p = 0.508.
• Temperature: 15.48 extra visits per degree Celsius, 95% CI 12.79 to 18.17.
• R-squared for the model: 0.7259. The residual normality check (Shapiro-Wilk) found no evidence against normality.

Once temperature is in the model, the ice cream coefficient is small and its interval comfortably includes zero, while the temperature coefficient is large and well away from it. The data were built with 15 extra visits per degree, and the estimate of 15.48 lands close. The interval for ice cream spans values on both sides of zero, so the result is compatible with no effect; it is not a demonstration that the true effect is exactly zero, and with 150 days a small effect could remain undetected. Our regression guide explains how to read coefficients like these, and the linear regression assumptions post lists the checks to run before trusting them.

The reason this worked is that we knew temperature was the common cause, measured it without error, and entered it in the right form. Real research rarely offers all three.

Why adjustment is not a cure

Adjusting for measured confounders is the standard tool for observational data, and it has limits that the simulation hides.

• Unmeasured confounders. A variable you did not record cannot be adjusted for. The estimate can look tidy and still be biased.
• Mismeasured confounders. A noisy proxy for the real confounder removes only part of its influence.
• Bad controls. Adjusting for a consequence of the exposure can remove part of the effect you are trying to estimate, and adjusting for a variable that both the exposure and the outcome influence can create an association that was not there. Rohrer (2018) and Cinelli, Forney and Pearl (2024) walk through these cases with diagrams.
• Wrong form. A curved relationship fitted with a straight line leaves some confounding in the residuals.

Which variables belong in the model is a question about how the data arose, and the data cannot answer it for you. That is why a written causal diagram, drawn before the analysis, is worth the effort.

Designs that can support a causal claim

The strongest route to a causal answer removes the common causes by design rather than by adjustment. In a randomised controlled trial, chance decides who is exposed, so measured and unmeasured confounders are balanced between the groups on average, and a difference larger than chance would explain can be attributed to the assignment, provided the groups stay comparable through follow-up (little loss to follow-up, outcomes measured the same way in both). Where randomisation is impossible or unethical, a natural experiment looks for an event or rule that assigned the exposure in a way unrelated to the outcome. Craig and colleagues (2017) review the main approaches: difference-in-differences, which compares the change over time in an exposed group with the change in an unexposed one; regression discontinuity, which compares units just either side of an eligibility threshold; and instrumental variables, which use a factor that affects the exposure but reaches the outcome only through it. Each replaces the assumption of no unmeasured confounding with a different one, such as parallel trends or a valid instrument, and a report should state and defend that assumption in the same way.

What a design needs before a causal claim

No checklist converts an association into a cause, but a report that wants to use causal language should be able to answer these, in the methods section:

1. A defined comparison. What would the outcome have been under the other exposure, and which group stands in for that? A randomised assignment gives the cleanest answer, because chance, not circumstance, decides who is exposed.
2. Time order. Was the exposure measured before the outcome? A cross-sectional snapshot cannot establish this.
3. Confounders named and measured. List the variables that could influence both exposure and outcome, and say how each was measured and used.
4. Adjustment justified. Say why each adjustment variable belongs, and state which ones you chose not to adjust for and why.
5. Sensitivity to what you missed. Say how strong an unmeasured confounder would have to be to explain the result, or at least admit that you cannot rule one out.
6. Consistency beyond one dataset. Has the association appeared in other samples, designs or settings? Hill (1965) lists this among nine viewpoints that make a causal reading more or less plausible: strength, consistency, specificity, temporality, biological gradient, plausibility, coherence, experiment and analogy. None of them is a test.
7. Plausible mechanism, and a dose-response pattern where one is expected. These support a causal reading; they do not establish it.

Hernan (2018) argues that reports should say plainly when a causal question is being asked, and then discuss the assumptions needed, rather than hide it behind the word association. That is a reasonable standard: state the question, state the assumptions, and let the reader judge them.

How to word the result

Write what you measured. “Higher ice cream sales were associated with higher pool visits (r = 0.696, 95% CI 0.603 to 0.7703, n = 150)” is accurate. “Ice cream sales increased pool visits” is a causal claim the design does not support. If you adjusted, say so and name the variables: “After adjusting for temperature, the association was small (coefficient -0.0933 visits per cone, 95% CI -0.371 to 0.1844).”

Avoid “proves”, “shows that X causes Y” and “because of” for an estimate the design cannot support. Describe the estimate itself as “associated with”, “higher among” or “consistent with”. If the study’s aim is causal, say so in the introduction, as Hernan (2018) recommends, and set out in the discussion the assumptions a causal reading needs and the confounders you could not measure. Report the interval, not only the p-value, so a reader can see how large and how uncertain the association is. This is research statistics, not guidance for any individual decision.

Where Celeus fits

Celeus, the tool behind this blog, ran both analyses above. It runs the method with a fixed seed, reports the checks it ran, and gives you a package with the R script that reproduces the result. It does not decide which variables are confounders, and it cannot tell you whether your design supports a causal claim: those are judgements about your study that stay with you. See the validation page.

Rerunning the example

The R script attached below, exactly as the engine generated it, reads the simulated days from a data/ folder and reruns the Pearson correlation and the sensitivity calculation. The seed is fixed and package versions are pinned. It does not regenerate the data from the seed and it does not run the adjusted regression. To check that one, add the line summary(lm(pool_visits ~ ice_cream_sales + temperature_c, data = data)) after the script’s own lines; the data file carries all four columns. The reproducible statistical analysis guide explains what a reviewer needs to rerun a result like this.

The method reference pages describe each method used here, its inputs and its references: Pearson correlation and linear regression. To adjust a group comparison for a continuous covariate, see the ANCOVA guide.

# Results

- Test: Pearson correlation
- n: 150
- statistic: 11.79
- p-value: < 0.001
- effect size: Pearson's r = 0.696
- 95% CI: [0.603, 0.7703]
- seed: 20261022

## Assumption checks
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=150: correlation r = 0.2264
- to detect a conventional medium effect: 85 total to detect r=0.30
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Pearson correlation  |  engine 0.31.1  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/correlation-vs-causation.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("ice_cream_sales", "pool_visits")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- cor.test(data[["ice_cream_sales"]], data[["pool_visits"]], method = "pearson")
print(result)

# Sensitivity: minimum detectable effect at n=150 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.r.test(n = 150, sig.level = 0.05, power = 0.8)
print(power)