Missing data: MCAR, MAR and MNAR explained

MCAR, MAR and MNAR are three assumptions about why values are missing, and each one decides which analyses stay trustworthy. Missing completely at random (MCAR) means the gaps have nothing to do with the data. Missing at random (MAR) means they depend only on values you did observe. Missing not at random (MNAR) means they depend on the missing values themselves. None of the three can be proven from the data you have, so the useful skill is knowing what each one allows and what evidence argues against it.

The sections below define the three mechanisms, set out what each lets you conclude, and then compare the missing ozone readings in R’s airquality data are compared across months, with a test run through a deterministic engine.

What the three mechanisms say

Rubin (1976) framed missing data by asking what a missingness indicator depends on. Take a variable with some values missing, and for each row record whether the value was observed. The mechanism is the answer to one question: what does the chance of being missing depend on?

MCAR. The chance of being missing does not depend on any value, observed or missing. A sample tube dropped at random, or a survey page lost in the post, is the textbook picture. The incomplete rows are a random subset of all rows.

MAR. The chance of being missing depends only on values you observed. If older participants skip a question more often and age is recorded, the gaps are explained by a variable you have. Given age, who skips is unrelated to the skipped answer itself.

MNAR. The chance of being missing still depends on the missing value after you account for everything observed. An instrument that fails at the highest readings, or people with the most income declining to report it, are the usual examples.

The names are misleading: “missing at random” does not mean the gaps are random in the everyday sense. It means that, once you condition on the observed variables, the leftover missingness carries no information about the missing value. That makes MAR a statement about a model, and which variables you include changes whether it holds.

What each mechanism allows

The practical question is which analyses give valid answers. The table summarises what the standard references say (Little and Rubin, 2019; Schafer and Graham, 2002; Sterne and colleagues, 2009).

Two cautions sit under that table. First, “valid” for MAR carries a condition: the analysis model has to include the variables that make the data missing at random. Imputing without the outcome, or without the variable that explains the gaps, can leave the bias in place. Second, Rubin (1976) showed that ignoring the missingness process is valid for likelihood-based inference when data are MAR and the parameters of the missingness process are distinct from the parameters you are estimating. That is a condition on the process, not a free pass.

Sterne and colleagues list two situations where complete-case analysis remains valid even when data are MAR rather than MCAR. One is when only the outcome is missing and every variable related to its missingness is a covariate in the model. The other is when predictors are missing for reasons unrelated to the outcome. Outside those, a complete-case result under MAR is a result about a selected set of rows.

MNAR is the hard case because no analysis of the observed data alone removes the problem. The honest response is a sensitivity analysis: change the assumption about how the missing values differ from the observed ones, and report how far the conclusion moves. If it barely moves, the conclusion is robust to that assumption. If it moves a lot, the data cannot settle the question.

What you can and cannot test

You can gather evidence against MCAR. Compare the observed variables between rows that have a value and rows that do not. If the groups differ, missingness is related to something you measured, and MCAR is hard to defend. Little (1988) built a formal multivariate test of MCAR for this purpose, and it applies under the normal-model assumptions it was derived for.

You cannot confirm MCAR. A comparison that finds no difference only says the missingness looks unrelated to the variables you checked, with the power your sample gives you. And you cannot separate MAR from MNAR with the data at hand, because the difference between them lies in values that are not there. Sterne and colleagues state this plainly: observed data cannot distinguish MAR from MNAR, so the choice rests on knowledge of how the data were collected.

A workable routine has three parts. Count what is missing per variable and per analysis. Compare incomplete rows with complete ones on the variables you have. Then decide on subject-matter grounds which mechanism is plausible, and choose the method that fits it. The data cleaning checklist puts this as step 8, among the other checks that come before a test. The how-to on handling missing data shows the app’s side of it.

A worked example: ozone in airquality

R’s airquality data holds 153 daily readings from New York, May to September 1973 (Chambers and colleagues, 1983). Ozone is missing on 37 days and solar radiation on 7, and two days are missing both. Month, wind and temperature are complete. That makes it a good place to ask the first question: is the missing ozone related to anything we observed?

Counting missing ozone days by month gives a sharp pattern.

The counts above come from base R and are not in the engine output. June has 21 missing days out of 30, against about 16% in May, July and August and 3% in September. A chi-square test of missing against month in base R gives p below 0.001, so these gaps are not evenly spread. The engine reached the same reading in its own checks: its run raised a note, not part of the summary output below the article, that missingness in Ozone appears to relate to observed values of Month, and it labels this an indicative comparison, not a formal MCAR test.

# Ozone missing or not, by month (base R)
tab <- table(airquality$Month, is.na(airquality$Ozone))
tab
round(100 * prop.table(tab, 1)[, "TRUE"], 1)   # share missing per month
chisq.test(tab)                                # is missingness spread evenly across months?

The share-missing line prints one value per month, May to September, in base R 4.6.1:

   5    6    7    8    9 
16.1 70.0 16.1 16.1  3.3 

A second check compares temperature between days with and without an ozone reading:

tapply(airquality$Temp, is.na(airquality$Ozone), mean)

   FALSE     TRUE 
77.87069 77.91892 

The mean temperature, in degrees F, is the same to the first decimal on days with an ozone reading (FALSE) and on days without one (TRUE), so that comparison shows nothing. That is worth knowing, and it also shows why a single check is not a verdict. Missingness tracks month and not temperature, and only the variables you check can speak.

Does the month matter for the values too?

Evidence against MCAR matters most when the variable that explains the gaps also explains the values. If ozone were the same in every month, the uneven gaps would cost little. So the second question is whether recorded ozone differs by month. The engine ran a Kruskal-Wallis test (Kruskal and Wallis, 1952) of Ozone by Month, a rank-based test chosen because ozone is right-skewed and some months are clearly non-normal. It used the engine’s default for missing values, complete-case analysis, so it used the 116 days with a reading and dropped 37.

The recorded ozone levels differ between months. H = 29.27 on 4 degrees of freedom, p < 0.001, with an effect size of epsilon-squared = 0.2545 (95% CI 0.1416 to 0.4343). About a quarter of the variation in the ranks lies between months: epsilon-squared here is H divided by n - 1, the between-month share of the rank variance. That does not say which months differ, so the engine also ran Dunn’s test (Dunn, 1964) with Holm’s correction across ten comparisons. July differs from May (adjusted p = 0.0001) and from September (adjusted p = 0.010), and August differs from May (adjusted p = 0.0003) and from September (adjusted p = 0.024). The engine groups July and August together as the high months, and May and September as the low months, with June overlapping both.

In base R the recorded medians are 18 ppb in May, 23 in June, 60 in July, 52 in August and 23 in September. The Kruskal-Wallis test guide and the post on Dunn’s test explain how to read this output.

What this example allows us to conclude

The two results, side by side, point one way. Missingness in Ozone depends on month, which is observed, and the values of Ozone depend on month too. So these gaps are not MCAR, and an analysis that drops the incomplete days is analysing a selected set. June contributes only 9 of its 30 days to the complete-case sample, while May, July and August contribute 26 each, so a season-wide average built from the recorded days weights June far below its share of the season.

Bias in the season-wide mean depends on what the 21 unrecorded June days looked like, and the data cannot say. If, within June, the missing days resemble the recorded ones, the gaps are MAR given month, and an analysis that conditions on month, by stratifying or by multiple imputation with month in the model, handles them. If June’s unrecorded days were different in a way tied to ozone itself, for instance an instrument that tends to fail at high readings, the gaps are MNAR and no month adjustment removes the problem.

The observed data cannot choose between those two readings. The decision comes from outside the file: was there a known monitoring outage in June, and does it plausibly relate to how high ozone was? That is a question for whoever collected the data. Whatever the answer, report that 37 of 153 days had no ozone reading, that 21 of them fall in June, and which assumption the analysis made.

Serial dependence is one more limit on the tests in this example. These rows are consecutive days, and weather persists from one day to the next, so the days are not independent observations. The rank test and the chi-square check treat them as if they were, so the p-values here are probably too optimistic. Treat this example as showing the shape of the argument, not as a model of ozone.

Choosing a method after you have judged the mechanism

The choice of method follows from the judgement about the mechanism.

If you judge the data close to MCAR, complete-case analysis is defensible and simple, as long as you report the number of rows used and accept the lost power. If the evidence is against MCAR and you believe MAR given variables you have, use a method that conditions on them. Multiple imputation fills each gap several times from a model of the observed data, analyses every completed dataset and pools the results, so the uncertainty about the missing values reaches the standard errors. Sterne and colleagues stress that the imputation model should use the information in the outcome and include the variables that predict the missing values.

Single imputation with the mean or median is not a middle path: it treats invented values as measured and shrinks the standard errors. If MNAR is plausible, run the planned analysis, then change the assumption about how missing values differ from observed ones and see whether the conclusion survives. The prepare data for statistics guide covers the file-level decisions that come before any of this, and the guide to one-way ANOVA, Welch and Kruskal-Wallis covers the group-comparison step used in the example.

Reporting missing data

Whichever route you take, a reader needs four facts: how many values are missing for each variable, how many rows each analysis used, what you compared between complete and incomplete rows, and which mechanism you assumed and why. All four are in the reporting guidance of Sterne and colleagues, and the last two are what let a reader judge whether your assumption was reasonable.

Do not write that the data were missing at random as if it were a finding. Write that the analysis assumed it, state the evidence you looked at, and say what would change if the assumption were wrong. For airquality, a sentence of that kind would name the 37 missing days, the concentration in June, and the check against temperature, and it would say plainly that the data cannot rule out MNAR. This is a research-statistics framing: it describes how to analyse a dataset, not a decision about any individual.

Reproduce the engine result

The numbers in the Kruskal-Wallis section are from one seeded, deterministic engine run: the script, the output and the data are in the files below the article, and running the script on the same data reproduces them. The missingness counts by month, the chi-square check and the group medians are plain base R, shown in the code block above, so you can run them without the engine.

# Results

- Test: Kruskal-Wallis test
- n: 116
- statistic: 29.27
- p-value: < 0.001
- effect size: epsilon-squared = 0.2545 (95% CI 0.1416 to 0.4343)
  (two-sided percentile-bootstrap CI at the run's confidence level)
- rows used: 116 of 153 (37 dropped)
- seed: 20261110

## Assumption checks
- normality by group (Shapiro-Wilk): context only, passed=FALSE (evidence of non-normality - normality is not assumed by this rank-based test)
- sample size: passed=TRUE (adequate n)

## Post-hoc (Dunn's test; correction: Holm)
- August vs July: adj p = 1 | rank-biserial r -0.02959 [-0.3332, 0.2796]
- August vs June: adj p = 0.2074 | rank-biserial r 0.5085 [0.1164, 0.7635]
- July vs June: adj p = 0.1489 | rank-biserial r 0.5598 [0.1865, 0.7919]
- August vs May: adj p = 0.0003225 | rank-biserial r 0.6228 [0.3908, 0.7804]
- July vs May: adj p = 9.894e-05 | rank-biserial r 0.676 [0.466, 0.814]
- June vs May: adj p = 1 | rank-biserial r 0.2991 [-0.1344, 0.6366]
- August vs September: adj p = 0.02428 | rank-biserial r 0.4642 [0.1918, 0.6702]
- July vs September: adj p = 0.01036 | rank-biserial r 0.5318 [0.2769, 0.7168]
- June vs September: adj p = 1 | rank-biserial r 0.01533 [-0.3987, 0.4242]
- May vs September: adj p = 0.7457 | rank-biserial r -0.2467 [-0.5082, 0.05645]

groups: August (a), July (a), June (ab), May (b), September (b) - groups sharing a letter are not significantly different at the adjusted threshold.

## Power
- Power is not calculated for this test. The matching parametric test gives a rough guide.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Kruskal-Wallis test  |  engine 0.53.2  |  catalog c40-dc08
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261110)
.data_path <- "data/missing-data-mechanisms.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("Ozone", "Month")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- kruskal.test(`Ozone` ~ `Month`, data = data)
print(result)

# Effect size (epsilon_squared) and its confidence interval.
effect <- { set.seed(20261110); .es_d <- data; .es_d[["Month"]] <- as.factor(.es_d[["Month"]]); effectsize::rank_epsilon_squared(.es_d[["Ozone"]], .es_d[["Month"]], ci = 0.95, alternative = "two.sided", iterations = 1000L) }
print(effect)

# Post-hoc pairwise comparisons (Dunn's test; correction: Holm).
data[["Month"]] <- as.factor(data[["Month"]])
posthoc <- { .ph_d <- data; .ph_d[["Month"]] <- as.factor(.ph_d[["Month"]]); PMCMRplus::kwAllPairsDunnTest(`Ozone` ~ `Month`, data = .ph_d, p.adjust.method = "holm") }
print(posthoc)