Data cleaning checklist before a statistical test

Data cleaning for statistical analysis is the set of checks you run between receiving a file and running the first test: is the structure right, are the codes what they claim to be, are there duplicates, impossible values, mixed units, gaps or extreme values, and what does each finding change about the analysis you planned. Done well, it is short, rule-based and written down. Done badly, it is a series of quiet edits that change the answer without anyone, including you, being able to say how.

This guide gives an ordered checklist, says what each finding should change, and runs the checklist on R’s built-in airquality data, ending in a regression run through a deterministic engine.

Why cleaning is part of the analysis

Cleaning is not housekeeping before the real work. Every cleaning decision is an analysis decision. Dropping rows with a missing value changes the sample and, depending on why the values are missing, can bias the estimate. Correcting an out-of-range value changes the data the test sees. Deleting an extreme observation can move a p-value across a threshold. Recoding a category can merge two groups the study meant to compare.

Van den Broeck and colleagues describe cleaning as three repeated steps: screening for suspect values, diagnosing whether each one is an error, a true extreme or something in between, and then editing, which means correcting, deleting or leaving the value as it is. They recommend a data-cleaning plan in the study protocol, including the expectations used to screen suspect data and the decision rules applied in the editing step, and they ask for archived, documented data with a justification for every change.

The reason for writing rules down first is that cleaning decisions made after seeing results give the analyst room to steer. Simmons, Nelson and Simonsohn showed by simulation that undisclosed flexibility in how data are collected, analysed and reported can push the real false-positive rate far above the nominal 5%. A cleaning plan written first, a log of what was done, and any result that excluded observations reported beside the result with them included are the cheapest protection against fooling yourself.

The checklist at a glance

The order matters. Structural problems hide value problems, and value problems distort the missing-data and outlier checks that come last. Work down the list, and for each item record what you looked at, what you found and what you did.

#	Check	What you look for	What a finding changes
0	Keep the raw file and start a log	An untouched original and a written record	Every later change becomes reversible and reportable
1	Structure	One row per unit of analysis, one column per variable, a key that identifies each row	The unit the test treats as independent
2	Types and codes	Numbers stored as text, missing-value codes such as 999, values below a detection limit	Whether a column is analysed as what it is
3	Duplicates	Exact duplicate rows, and rows that repeat the same key	The sample size and the independence of observations
4	Categories	Spelling variants of one label, a 0/1 column with a third value	Which groups the test compares
5	Ranges	Impossible values and implausible ones	Corrections, or values set to missing with a reason
6	Units and scales	Mixed units, unit errors, scales scored in the wrong direction	Whether the values in a column are comparable
7	Logic across columns	Events in the wrong order, answers that skip patterns should have prevented	Corrections, and missingness that is structural rather than random
8	Missing data	How much, where, and why	The missing-data method and the assumption it rests on
9	Outliers and influence	Extreme values and observations that drive the fit	A diagnosis per value and a sensitivity analysis
10	Distribution shape	The shape the planned method actually assumes	The model’s scale or the choice of method

The first seven checks are mostly about errors. The last three are about decisions that remain even when the data are perfectly entered. When the script has run, repeat every check on the cleaned file and plot each variable: a histogram or scatter plot shows problems a summary hides.

Before you touch anything: keep the raw file and start a log

Save the file exactly as you received it and never edit it again. Do all cleaning in a script, or in a tool that records each change, and write the result to a new file. If a correction turns out to be wrong, you can go back. If a reviewer asks what you did, you can show them.

Broman and Woo’s recommendations for organising data in spreadsheets are a good standard for the cleaned file: be consistent, write dates as YYYY-MM-DD, put one thing in each cell, organise the data as a single rectangle with one header row, create a data dictionary, keep calculations out of the raw data file, never use colour or highlighting as data, back up, and save in a plain text format. They also recommend never leaving a cell empty, so that a blank cannot be confused with a value that was forgotten; whatever convention you use for missing, use one.

The log needs one line per change: the column, the rule that triggered it, how many values it affected, what was done and why. It becomes a sentence in the methods section, and it is what lets someone else rerun the analysis and get the same numbers.

If the file holds direct identifiers such as names, record numbers or full dates of birth, move them to a separate, access-controlled file linked by a study ID before cleaning starts. The analysis rarely needs them.

1. Structure: one row per what?

Before checking any value, decide what one row means. In a two-group comparison it is usually one participant, animal or sample. In a repeated-measures study it might be one visit, and the participant appears on several rows. The answer determines which observations a test may treat as independent, so a structural mistake here produces a wrong test, not just a messy file.

Three questions settle it. What is the unit of analysis? Which column, or combination of columns, identifies each row uniquely? Are repeated measurements stored as separate rows (long format) or as separate columns (wide format), and does the planned method expect that layout? If the file was merged from several tables, check that the key is unique in each table where it should be and that unmatched keys are ones you expected, so the merge neither duplicates nor silently drops rows.

# airquality: one row per day, May to September 1973.
str(airquality)          # 6 columns: Ozone, Solar.R, Wind, Temp, Month, Day
nrow(airquality)
# Month and Day together should identify each row.
anyDuplicated(airquality[, c("Month", "Day")])   # 0 means no repeated key

For airquality the unit is a day and the key is Month plus Day. That has a consequence worth noting now: consecutive days are not independent in the way separate animals are. Weather persists from one day to the next, and a regression on daily data assumes the model’s errors are independent. The checklist does not fix that, but it should make you write it down as a limitation or as a reason to model time explicitly.

2. Types and codes

Read every column’s type and compare it with what the data dictionary says it should be. The usual problems are numbers stored as text (often because one cell contains a stray character, a comma decimal or a trailing space), dates read as text or as numbers, categorical codes read as numbers, so that a group coded 1, 2 and 3 is treated as a quantity, and identifiers with the wrong number of characters (often leading zeros lost).

Missing-value codes need special care. Survey and clinical software often store “not asked”, “refused” or “not applicable” as 99, 999 or -9. Read as numbers, they become real values. Convert every declared code to a proper missing value before any summary, and note which code meant what, because “not applicable” and “refused” are different kinds of missing.

A related case is a laboratory value reported as below a detection limit, such as “<5”. It is not missing: the measurement says the value is somewhere below 5. Substituting half the limit, or zero, and then running an ordinary test biases the result. The guide to values below the detection limit explains how to keep those values as censored and which methods use them correctly.

sapply(airquality, class)        # all numeric here
summary(airquality)              # min, max and NA count for every column

In airquality every column is numeric and missing values are already stored as NA, so there is no code to convert. In a real export, this is the step where you would find them.

3. Duplicates

There are two kinds. Exact duplicates are rows identical in every column, usually from a file being appended twice or an export that repeated a page. Key duplicates are rows that share the identifying key but differ elsewhere, which is often more serious: the same participant entered twice with one value corrected, or two visits that should have had different dates.

Exact duplicates can usually be removed, once you are sure they are not two genuine observations that happen to match. Key duplicates must be diagnosed one by one, because one of the two rows is likely to be wrong and you need the source to know which. Never resolve key duplicates by keeping the first row: that is a rule about file order, not about the data.

sum(duplicated(airquality))                          # exact duplicate rows
key <- paste(airquality$Month, airquality$Day)
airquality[key %in% key[duplicated(key)], ]          # every row sharing a key

Duplicates matter for the analysis because they inflate the sample size and, worse, they are perfectly correlated with each other. Two copies of one observation make the data look more certain than they are, and the standard errors shrink accordingly.

4. Categories

Categorical columns collect spelling variants: “Female”, “female”, “F”, “female “ with a trailing space. Each variant becomes its own group, so a two-group comparison quietly becomes a five-group one, or a small group disappears into “other”. Tabulate every categorical column and fold the variants onto one label, with the mapping written in the log.

Two other problems are worth a specific check. A column that should be 0/1, such as an event indicator, sometimes has a third value, a 2 or a 9, which may be a missing code or an entry error. And a category with very few observations may need to be combined with another before analysis; do that by a rule set in advance (for example, combine any group below a stated size with a named neighbour), not by looking at which combination gives the clearest result.

5. Ranges and impossible values

Compare every numeric column’s minimum and maximum with what is possible and what is plausible. These are different. An impossible value cannot occur: a negative age, a proportion above 1, a temperature reading on a scale that does not go that high. An implausible value could occur but would be surprising enough to check: an adult height of 230 centimetres, a reaction time of 30 milliseconds.

Van den Broeck and colleagues give a firm rule for the first kind: impossible values are never left unchanged. They are corrected if the correct value can be found, and otherwise deleted, which in practice means set to missing with the reason recorded. Implausible values go to diagnosis. Check the source document, the instrument log or the person who entered them. If the value is confirmed, it stays.

# Plausible ranges written down BEFORE looking at the data.
limits <- list(Ozone = c(0, 500), Solar.R = c(0, 1000), Wind = c(0, 100),
               Temp = c(-20, 120), Month = c(5, 9), Day = c(1, 31))
for (v in names(limits)) {
  x <- airquality[[v]]
  out <- which(x < limits[[v]][1] | x > limits[[v]][2])
  cat(v, ": ", length(out), " value(s) outside ", limits[[v]][1], "-", limits[[v]][2], "\n", sep = "")
}

The limits in that code are illustrative, set from the units in the dataset’s documentation. The habit is the important part: write the limits first, then run the check, so that the range rule is not quietly adjusted to fit whatever the data contain.

6. Units and scales

A column is only a measurement if every value in it is on the same scale. Mixed units are common when data come from several sites, several instruments or several years: weight in kilograms at one site and pounds at another, glucose in mg/dL and mmol/L, temperature in Celsius and Fahrenheit. The range check sometimes catches them, because a cluster of values sits at the wrong magnitude, but not always, since some unit pairs overlap.

Check the data dictionary for the unit of every column, ask how each site recorded it, and look at the distribution split by site or by data source. A unit error is corrected by converting the affected values, with the rule that identified them in the log. Dates carry the same trap: 03/04 is 3 April when the day comes first and 4 March when the month does, so confirm each source’s date format before converting.

For questionnaire scales the equivalent check is direction: items worded the opposite way must be reverse-scored before items are summed.

In airquality the documentation gives the units: ozone in parts per billion, solar radiation in langleys, wind speed in miles per hour and maximum daily temperature in degrees Fahrenheit. They are what the regression coefficients below are expressed in.

7. Logic across columns

Some errors are only visible when two columns are read together. An event date before the date it depends on, a follow-up visit before the baseline visit, a pregnancy item answered for a participant recorded as male, an age at diagnosis greater than the current age. Write the logical rules your data should satisfy and count the rows that break each one.

The same reading reveals structural missingness: values that are missing because a question did not apply. An “age at event” column is empty for everyone without the event, by design. Those gaps are not a missing-data problem to be imputed, and treating them as one (for example, imputing an age at death for people who are alive) produces nonsense. Mark them as not applicable and keep them out of the missing-data counts in the next step.

8. Missing data: how much, where and why

This is where the analysis plan most often has to change. Three questions, in order.

How much is missing, per variable and per analysis? The number that matters for a test is how many rows have every variable the analysis needs. Two variables each missing in 10% of rows can leave anywhere from 80% to 90% of rows complete, depending on whether the gaps overlap.

Where is it missing? Look for clustering in one site, period or group, or in rows with high or low values of another variable. Comparing other variables between rows with and without a missing value cannot tell you about the values you do not have, but it can show that incomplete rows differ from complete ones.

Why is it missing? Rubin’s framework names the cases. Data are missing completely at random (MCAR) when missingness has nothing to do with the data, missing at random (MAR) when it depends only on observed values, and missing not at random (MNAR) when it depends on the missing value itself, as when an instrument fails at the highest readings. Rubin showed that ignoring the missing-data process gives valid likelihood-based inference when data are missing at random and that process’s parameters are distinct from the ones being estimated.

The difficulty, as Sterne and colleagues put it, is that missing at random is an assumption that justifies the analysis, not a property of the data, and observed data cannot distinguish it from missing not at random. The choice of method rests on what you know about how the data were collected.

What the answers change:

• Complete-case analysis, the default in most software, may be biased when data are missing at random but not completely at random. Sterne and colleagues note two exceptions: only the outcome is missing and every variable related to its missingness is a covariate, or predictors are missing for reasons unrelated to the outcome.
• Single imputation with the mean or median treats invented values as measured, so standard errors are too small.
• Multiple imputation fills each gap several times from a model of the observed data, analyses each completed dataset and pools the results, carrying the uncertainty about the missing values into the standard errors. It is usually safer under missing at random, provided the imputation model includes the outcome and the variables related to missingness.

Whatever you choose, report the number missing for each variable or the number of complete cases for each analysis, as Sterne and colleagues recommend, and compare key variables between complete and incomplete rows.

9. Outliers and influence

An outlier is a value far from the others. Finding it is the easy part; what matters is why it is there.

Find them with a rule chosen in advance. Tukey’s fences flag values more than one and a half times the interquartile range beyond the first or third quartile, a convention for drawing attention, not a definition of error. Flagging values several standard deviations from the mean has a weakness: the mean and standard deviation are pulled by the extremes themselves. Leys and colleagues recommend the median and the median absolute deviation instead.

Diagnose each one. An extreme value is an error (a misplaced decimal point, a unit error), a legitimate value from a population you did not mean to study, or a genuine extreme of the one you did. The first is corrected or set to missing with a reason. The second may justify exclusion if the criterion could have been stated before the data were seen. The third stays.

Ask about influence, not only extremity. A moderate value can drive a model if it sits far out on a predictor. Cook’s distance measures each observation’s contribution to the estimated coefficients; the common 4/n threshold is, like Tukey’s fences, a screening convention.

Report a sensitivity analysis. If observations look influential and you have no evidence they are wrong, run the analysis with and without them and report both. Removing them silently because the result looks better is exactly the undisclosed flexibility Simmons and colleagues warned against.

10. Distribution shape: check what the method assumes

The last check is not about errors, and it is often done in the wrong place. Check only what the planned method assumes. A two-sample t-test concerns the outcome within each group. A linear regression’s normality assumption concerns the residuals, not the outcome or the predictors. A chi-squared test concerns expected cell counts.

Read the result carefully. A significant normality test says non-normality was detected. A non-significant one says it was not detected, which in a small sample is weak reassurance and never evidence of normality. In large samples, tests of means and regression coefficients are fairly robust to non-normal residuals, provided the errors are independent, the skew or tails are not extreme, and a few influential points do not dominate.

When the shape is a real problem, the fix is a change of scale or model chosen for a substantive reason (a log scale for a strictly positive, right-skewed concentration, say), not deleting the values that make the distribution look non-normal. If the scale was not settled in advance, report the planned one and the other as a sensitivity analysis. The guide to choosing a statistical test starts from the outcome type and the design if the shape suggests a different family of method altogether.

Worked example: the checklist on airquality

R’s airquality data hold daily air quality measurements in New York from May to September 1973: ozone, solar radiation, wind speed and maximum temperature, plus the month and day. The question for this example is how daily ozone relates to solar radiation, wind and temperature, which calls for a linear regression of ozone on the three weather variables.

Seeded and run in R; the method and the variables were chosen before the run. Missing data were handled by the engine’s default, complete-case analysis, so the example shows what the default does. The full output and the R script that reproduces the regression appear below the article.

Checks 1 to 7 are the code shown in each section above: the unit is a day, the key is month plus day, the units are documented, and missing values are already stored as missing. The engine’s output covers checks 8 to 10, and that is where the findings are.

Missing data: 42 of 153 rows dropped

The data have 153 days. The engine reports that 111 were used and 42 were dropped because at least one of the four analysis variables (ozone, solar radiation, wind or temperature) was missing on that day. Its flag reads that 27% of rows had missing values in the analysis variables and were dropped listwise, and it suggests multiple imputation as the alternative.

That is a large loss, and the plan has to respond in three ways. First, everything below is about 111 days, not 153, and the report must say so.

Second, for each analysis column with at least five missing and five observed values, the engine compares the other analysis columns between rows where it is missing and rows where it is not (a rank-sum test at the 5% level). That screen raised no note. Read that correctly: no relationship was detected between missingness and the observed weather variables. It is not evidence that the data are missing completely at random, it says nothing about variables outside the analysis (the month, for instance), and it cannot detect values missing because of their own size.

Third, which variable is missing matters. The summary() output in check 2 shows gaps in both the outcome (ozone) and a predictor (solar radiation), so neither of Sterne and colleagues’ exceptions can be assumed, and the data cannot show whether either holds. The plan therefore makes no claim either way about the mechanism: it reports the complete-case model with that assumption stated and runs multiple imputation, which rests on its own missing-at-random assumption, as a sensitivity analysis.

Outliers: a few values beyond the fence, none removed

The engine’s descriptive outlier check counts values beyond Tukey’s fences in each numeric analysis column, computed over each column’s observed values. It found 2 in ozone and 3 in wind, and it removed nothing. Its note says exactly that: descriptive only, nothing changed, check whether these are genuine extreme cases or entry errors.

What this changes is a diagnosis step, not a deletion: is each flagged value possible for that variable on a summer day in New York, and is it consistent with the source? Unless there is evidence of an error, those values stay.

Influence: six observations above the screening threshold

Extremity on one variable is not the same as influence on the model. The engine’s influence check fits the regression and counts observations with a Cook’s distance above 4/n. Its note:

6 of 111 observations used in this analysis have a Cook's distance above the
conventional 4/n screening threshold (largest observed: 0.261); a small number of
rows may be disproportionately driving the fit.

That is a reason to look, not a finding of error. The plan adds a sensitivity analysis: refit without those six days and report whether the coefficients and intervals change materially. If they do, the conclusion depends on a handful of days, and the report says so.

Distribution of the residuals

For a linear regression the normality assumption concerns the residuals, and the engine’s check is a Shapiro-Wilk test on the model’s residuals: W = 0.917, p < 0.001. Non-normality was detected. (The engine’s flag text says the outcome appears non-normal; the test behind it uses the residuals.)

With 107 residual degrees of freedom, coefficient inference is fairly robust to non-normal residuals under the conditions given earlier: independent errors, no extreme tails, no small set of points dominating the fit. Two of those are in question here. The influence check found six observations worth examining, and consecutive days may have correlated errors. The engine did not test for that, so the independence assumption is untested, not satisfied.

Ozone is a strictly positive concentration, and right skew is common for such variables. A log-scale model is a reasonable alternative, decided in the plan with its reason, not by comparing which scale gives the more convenient p-values.

The model, read with the checks in mind

With those caveats stated, the complete-case model on 111 days gives these estimates, each holding the other two predictors fixed:

Predictor (unit)	Estimate, ppb per unit	95% CI	p
Solar radiation (langley)	0.0598	0.0139 to 0.106	0.0112
Wind (mph)	-3.33	-4.63 to -2.04	< 0.001
Temperature (°F)	1.65	1.15 to 2.15	< 0.001

Each additional mile per hour of wind is associated with mean ozone 3.33 parts per billion lower, and each additional degree Fahrenheit with mean ozone 1.65 parts per billion higher. The solar radiation coefficient is positive (t(107) = 2.58), and its interval, 0.0139 to 0.106 parts per billion per langley, is compatible with associations from very small to nearly twice the estimate. The model’s R-squared is 0.606. These are associations in observational data from one city in one summer, not causal effects.

The engine’s sensitivity calculation gives the smallest effect the model had 80% power to detect at the 5% level with 111 observations: Cohen’s f-squared of 0.102 for the three predictors together. That is a property of the design and sample size, computed without the observed effect, not the power of this result.

What the checklist changed

Without the checks, the output above is a regression table. With them, the plan reads differently:

• the analysed sample is 111 of 153 days, and the report says why 42 were dropped;
• complete-case analysis is kept as the primary model with its assumption stated, and multiple imputation is planned as a sensitivity analysis;
• the outliers flagged by the fence are diagnosed against the source rather than removed;
• the six influential days get a with-and-without refit, reported either way;
• residual non-normality and possible correlation between consecutive days are named as limitations, and a log-scale model is considered for a stated reason.

None of those changes a single value in the data. All of them change what the result can be trusted to say.

What Celeus does during cleaning, and what it leaves to you

Celeus runs several of these checks automatically. It does not make the decisions for you, and it never changes a value without a record.

On upload, zero-width characters are deleted from text cells, line breaks, tabs and non-breaking spaces become ordinary spaces, surrounding whitespace is trimmed, and a text column whose every value is a plain number is read as numeric. In SPSS files, values the file declares as user-missing codes become missing, so a 999 cannot enter an analysis as a real value. The reproduce script repeats the text clean-up.

Before any test is chosen, a data-quality report lists exact duplicate rows and rows differing only in an identifier column, spelling variants of a category label, a 0/1 column with a third value, values outside a codebook range, negative ages or durations, event columns in the wrong order, a column left empty where a related yes/no flag is 0, and missingness that appears related to another column’s values. Every item is a caution: the report never blocks an analysis or changes the data.

Fixes are yours to make, in Prepare, and each is recorded in the dataset’s history: recoding categories, changing a column’s type, banding a numeric column, or correcting one cell, which requires a reason (transcription error, unit error, known correction or other). An analysis restricted to a subpopulation has its filter written into its plan and reproduce script. Values you confirm as below a detection limit are kept as censored, not replaced by a substitute number.

At analysis time, rows missing an analysis variable are dropped by default and the result reports how many. Above 20% dropped, a warning says so and, where the method supports it, offers multiple imputation. Mean and median imputation are available and flagged as understating uncertainty. The outlier, influence and missingness-pattern notes from the worked example run on every applicable analysis and never alter anything.

What remains yours: deciding what a row is, writing ranges and cleaning rules before seeing results, diagnosing flagged values against their source, choosing the missing-data method, and reporting what you changed.

Common mistakes

Cleaning by looking at the results. Running the test, seeing an awkward p-value, then going back to “check the outliers” is the most common way cleaning turns into steering. Write the rules first.

Reporting the complete-case result as if no rows were lost. The sample size in the abstract must be the sample size in the model, with the number dropped and the reason.

Reading a non-significant check as a clean bill of health. No missingness pattern detected, no normality violation detected and no outliers flagged are all statements about what a check could see. None of them shows that the assumption holds.

Rerunning the example

Below the article you will find the R script exactly as the engine generated it, with the seed and the bundled data path, and the output it printed. On the same package versions the script repeats the text clean-up applied at upload, drops rows with a missing value in the four analysis variables, fits the regression with lm() and prints its summary, and computes the sensitivity figure with the pwr package.

The script does not reproduce the outlier, influence and missingness notes or the residual normality test, which come from the engine’s own checks. By hand, boxplot.stats() gives Tukey’s fences, cooks.distance() the influence values and shapiro.test(residuals(fit)) the normality check; the mice package refits the model on imputed datasets and pools the results. The guide to reproducible statistical analysis explains what a reviewer needs in order to rerun a result like this one, and the linear regression method reference describes how the engine fits and reports the model. If your outcome is binary rather than continuous, the same checklist leads into the logistic regression guide instead.

# Results

- Test: Linear regression
- n: 111
- statistic: 2.58
- p-value: 0.0112
- effect size: R-squared = 0.6059
- 95% CI: [0.01386, 0.1058]
- rows used: 111 of 153 (42 dropped)
- seed: 20261021

## Assumption checks
- normality (Shapiro-Wilk): passed=FALSE (evidence of non-normality)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=111: Cohen's f2 = 0.1019
- to detect a conventional medium effect: 77 total to detect f2=0.15
- alpha = 0.05, target power = 0.8

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Linear regression  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/data-cleaning-checklist.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("Ozone", "Solar.R", "Wind", "Temp")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- summary(lm(Ozone ~ Solar.R + Wind + Temp, data = data))
print(result)

# Sensitivity: minimum detectable effect at n=111 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.f2.test(u = 3, v = 107, sig.level = 0.05, power = 0.8)
print(power)