Reproducible statistical analysis: the rerun test

Here is a familiar scene, made up for illustration (the numbers are not from any real analysis, or from the worked example below). A reviewer asks why your adjusted estimate is 1.84 when the table in the supplement says 1.87. You open the project folder from eighteen months ago and find analysis_final.R, analysis_final_v2.R, and a spreadsheet with a column you do not remember recoding. You rerun the script that looks most recent and get 1.81.

Nothing fraudulent happened. But you can no longer show how the published number was produced, and that is the problem reproducibility is meant to solve.

This guide treats reproducibility as a test a reviewer can actually run: take what you handed over, rerun it, and compare. It covers what reproducibility means and what it does not, the seven things the package has to contain for the test to be possible, a worked example in which a seeded analysis is rerun from its own package, how much agreement to expect between machines, the common ways reruns fail, the tools that help, what to do when the data cannot be shared, and a checklist to use before you submit.

Reproducible is not the same as replicable

The two words are used loosely and sometimes in opposite senses across fields, which causes real confusion. The National Academies (2019) settled on definitions that are now widely used:

• Reproducibility means getting consistent results from the same data with the same code, methods and conditions of analysis.
• Replicability means getting consistent results in a new study that asks the same question and collects its own data.

Goodman, Fanelli and Ioannidis (2016) split the idea further into three parts. Methods reproducibility is whether enough detail is given for someone else to repeat the procedures exactly. Results reproducibility is what most people mean by replication: an independent study reaching the same result. Inferential reproducibility is whether others would draw qualitatively similar conclusions, from a reanalysis of the original study or from an independent replication, which is where disagreements about priors, multiplicity and interpretation live.

This guide is about the first kind, the computational one. It is the minimum. An analysis that cannot be reproduced from its own data cannot be meaningfully checked, and Peng (2011) makes the point that when full independent replication is not possible, reproducibility can serve as a minimum standard for judging a claim.

Reproducibility does not make a result correct. A reproducible analysis with the wrong model gives the wrong answer every time. What it does is make the analysis inspectable, so that errors can be found.

The reviewer’s rerun test

Stated as a procedure, computational reproducibility is a test with three steps:

1. Receive a self-contained package: no follow-up emails, no “you also need the file on my desktop”.
2. Rerun it in a fresh environment set up from what the package records.
3. Compare every reported number with what the rerun produced.

If all three steps go through without asking the original analyst a question, the analysis passes. If the reviewer has to ask which script, which version of the data, which package version or why a model changed, it fails, however good the science behind it.

How often analyses fail this test is not a matter of opinion. Stodden, Seiler and Ma (2018) took a random sample of 204 papers published in Science after the journal began requiring authors to provide data and code on request. They obtained the materials for 44% of the papers and could reproduce the findings for 26%. Hardwicke and colleagues (2018) looked at the journal Cognition after it made data sharing mandatory.

Even among articles whose data were judged reusable, some reported values could not be reproduced, half of the articles that were fully reproduced (11 of 22) needed help from the authors to get there, and the reasons they name are ordinary ones: suboptimal data curation, analyses that were not specified clearly enough, and reporting errors. They found no clear indication that the original conclusions were seriously affected, but in many cases the authors had to be contacted before anyone could know that.

Complex models make it harder. Seibold and colleagues (2021) tried to reproduce the longitudinal data analyses of 11 PLOS ONE articles whose data were available and whose authors agreed. Only one article had published even part of its analysis code. They could not reproduce the results of three articles, could reproduce only parts of two more, and had to contact the authors for all but two. Their conclusion is the practical one: open data alone is not enough, and the code has to be shared too.

Those are the failure modes the rest of this guide is built to prevent. None of them needs specialist software to fix. They need the right things to be written down at the time, and kept together.

The seven things a reviewer needs to rerun an analysis

Think of it as a recipe that someone else, or you in two years, can follow without asking any questions. Marwick, Boettiger and Mullen (2018) describe the same idea as a research compendium: a standard, recognisable way of organising the data, code and documentation of a project so that others can inspect, reproduce and extend it. Whatever you call it, it has to contain the following.

1. The exact data that went into the model

Not the raw export, and not “the cleaned file”. The specific dataset that the final model saw, after exclusions, recoding and derived variables. If cleaning happened in a spreadsheet by hand, that step is not reproducible, and Sandve and colleagues (2013) list avoiding manual data manipulation as one of their ten rules for exactly this reason. Put the cleaning in code.

If the analysis dataset is derived from a larger raw file, keep both: the raw file, and the code that turns it into the analysis file. A reviewer who can only see the end product cannot check the exclusions.

2. A fingerprint of that data

A file name is not an identity. Two files called trial_clean.csv can differ by one recoded value, and nobody will notice by looking. A checksum is a short string computed from every byte of the file. If one byte changes, the checksum changes. Record it next to the results, and the reviewer can confirm in seconds that the data they hold is the data you analysed.

Base R can compute one without any extra package:

tools::md5sum("data/analysis.csv")

MD5 is fine for catching accidental changes. If the package may need to stand up to a deliberate dispute, use SHA-256 instead, for example with digest::digest(file = "data/analysis.csv", algo = "sha256"). Either way, the checksum goes in a text file in the package, and in the methods section or supplement if the data are shared separately.

3. The script, and only one of it

The script that produced the reported numbers, run top to bottom in a fresh session. Not a script plus some commands typed into the console afterwards. A good test: restart the session, run the file, and compare every number to the manuscript.

The script should read the data with a path relative to the package folder, never an absolute path from your own machine. read.csv("D:/projects/trial/final.csv") fails on every computer but yours.

4. The random seed, and the generator it applies to

Bootstrap confidence intervals, permutation tests, multiple imputation, cross-validation and many optimisers use random numbers. Without a fixed seed, the reported estimate can change in its later decimal places on every run (how many depends on the number of resamples), and occasionally a p-value will cross 0.05 in one run and not the next. Setting a seed does not make the analysis less valid; it makes the specific draw you reported recoverable.

One subtlety: R’s random number generation has changed between versions. In R 3.6.0, for example, the default method for drawing from a discrete uniform distribution, which sample() uses, was changed, and the R release notes say the previous method has to be requested with RNGkind() or RNGversion() to reproduce old results (R Core Team, 2019). The same seed on a different R version is therefore not guaranteed to give the same draws, which is one reason the next two items matter. Record what RNGkind() prints alongside the seed, so a reviewer on another version can set the same generator explicitly. The post on setting seeds and pinning package versions in R shows the effect on one bootstrap interval.

Set the seed once, at the top of the script, or immediately before each random step if the steps might be run separately. A seed set halfway through a script after an unseeded random step reproduces nothing.

5. The software versions

Statistical packages change their defaults, fix bugs, and alter numerical algorithms between releases. An analysis run with one version of a mixed-model package can give slightly different estimates, or a convergence warning, under another. Sandve and colleagues put it plainly: archive the exact versions of all external programs used.

Two records cover this. Session information (sessionInfo() in R) captures the R version, operating system and every attached package with its version. A lockfile pins those versions so someone else can install them: renv::snapshot() writes one, and renv::restore() installs exactly the versions it lists. Session information tells the reviewer what you had. The lockfile lets them get it.

6. A record of the decisions

This is the part most often missing. Code shows what was run but not why. Why was the Welch test used instead of Student’s? Why were three outliers excluded? Why was the model adjusted for age but not site? Were other models tried first?

An audit trail records each analytic decision, the assumption checks that informed it, and any changes from the pre-specified plan. It is also what separates a reproducible analysis from a reproducible fishing expedition: if twenty models were fitted and one was reported, rerunning that one script will faithfully reproduce a misleading result. Reproducibility of the computation and honesty about the analysis are different properties, and the decision record is where the second one becomes checkable.

7. The output to compare against

The rerun test ends with a comparison, so the package needs the original results in a form that can be compared: the full output, not only the rounded table in the manuscript. A machine-readable results file (CSV or JSON) lets a reviewer compare every number at full precision instead of eyeballing two tables. Without it, “the numbers match” means “the numbers match after rounding”, which hides exactly the small discrepancies that point to a changed package or a changed dataset.

Worked example: rerunning a seeded analysis from its package

To make the test concrete, here is one analysis whose result depends on random numbers, run twice from the same seed and once from a different one.

The data and the question

R’s built-in InsectSprays data, from Beall (1942), records insect counts in 72 agricultural field units, each treated with one of six insecticides labelled A to F, 12 units per spray. The question is whether counts differ between sprays. Each unit received one spray and contributed one count, so the six groups are independent. The outcome is a count, and Beall’s paper was itself about how to transform data of this kind so that analysis of variance could be applied.

A rank-based alternative that avoids the transformation is the Kruskal-Wallis test (Kruskal and Wallis, 1952), which replaces every count with its rank among all 72 units and asks whether the ranks tend to be higher in some groups than in others. The guide to one-way ANOVA, Welch and Kruskal-Wallis covers how to choose between the three; the method reference for Kruskal-Wallis lists what the engine reports for it.

The test is not the point of the example. The effect size is. Alongside the test, the engine reports epsilon-squared, a rank-based effect size, with a confidence interval from a percentile bootstrap: 1000 resamples of the data, drawn at random. Change the random draws and the interval changes. That makes it a good stress test for reproducibility.

The result

The analysis was run through a deterministic engine with the seed 20261022. For all 72 units, the Kruskal-Wallis statistic was 54.69 on 5 degrees of freedom, p < 0.001. Epsilon-squared was 0.7703, with a 95% bootstrap confidence interval from 0.7327 to 0.837.

Read with the usual care: the test indicates that insect counts are not distributed the same way under all six sprays, and the effect size says spray accounts for a large share of the variation in the ranks of the counts. It does not say which sprays differ from which; that needs pairwise comparisons with a correction for multiplicity, which the package includes and which the guide to multiple comparisons explains. The only assumption check the engine reports for this test is a minimum sample size, which passed. That is a floor, not an assessment of whether the test suits the data: the independence of the 72 units comes from the design, not from any check.

Rerun with the same seed

The same analysis was then run a second time, from scratch, in the same software image, with networking switched off so nothing could be fetched or changed during the run. The machine-readable results file and the generated R script from the two runs were compared byte for byte, and they were identical: the same statistic, the same p-value, the same effect size, and the same bootstrap interval to every recorded digit.

Then the package’s own script was run again, separately, from the package folder. It read the bundled data, set the recorded seed, and recomputed the test. Its test statistic and p-value matched the recorded ones to within one part in a billion, the tolerance that check uses. Be clear about what that check covered: the statistic and the p-value. The script also recomputes the bootstrap interval with the same seed and prints it, but the automated comparison did not check the interval.

Rerun with a different seed

Finally, the identical analysis was run with one change: the seed 20261023 instead of 20261022.

The Kruskal-Wallis statistic was again 54.69, the p-value again below 0.001, and epsilon-squared again 0.7703. Those are computed from the ranks of the data, with no random numbers involved, so the seed cannot touch them. The bootstrap interval moved: it now ran from 0.7308 to 0.8339 instead of 0.7327 to 0.837.

The scientific conclusion is the same under either seed. Both intervals describe a large effect. But a reviewer comparing your reported interval with their rerun would see different numbers, and without the seed they would have no way to tell whether the difference came from random resampling, a different package version, or a different dataset. Recording the seed turns that ambiguity into a one-line check. The difference between the two intervals is also a rough indication of how much resampling noise sits in the reported bounds, which is worth knowing before anyone reads meaning into the third decimal place.

What this example does and does not show

It shows that a seeded analysis, rerun from its package in the same software environment, gives the same output, and that the seed is what holds the random part of the output still. It does not show that the same output would appear on a different operating system or under different package versions; that is a separate question, covered next. And it says nothing about whether the Kruskal-Wallis test was the right choice for these data. Reproducibility makes that question answerable by someone else. It does not answer it.

How much agreement to expect

“Same numbers” means different things at different distances from the original environment.

Same environment. The same data, script, seed, R version, package versions, operating system and maths libraries. Here a deterministic analysis should give byte-identical output, as in the example above. If it does not, something in the analysis depends on state outside the package: the clock, the order of files in a folder, a global option, or an unseeded random step. Treat any difference as a bug to find.

Same versions, different machine. Even with identical package versions, results can differ in the last few digits across operating systems or maths libraries, because floating-point arithmetic is not always performed in the same order. Goldberg (1991) gives the classic illustration: due to rounding, the associative laws of algebra do not necessarily hold for floating-point numbers, so (x + y) + z need not equal x + (y + z). A maths library that sums in a different order can change the last bits of a result. Agreement to many significant figures is what a reproducible analysis delivers in practice; exact bit-level identity across machines is a stronger claim that has to be checked, not assumed.

Different versions. A newer R or a newer package can change defaults, fix bugs or replace an algorithm. Differences can then be larger, and occasionally they matter. This is why the lockfile exists: it lets a reviewer move back to the same versions rather than argue about whether a discrepancy is meaningful.

A useful working rule for the comparison step: numbers that agree to the precision you reported, and to several more digits in the machine-readable file, are reproduced. Numbers that differ in a digit you reported need an explanation before anyone relies on them.

How a reviewer actually runs the test

If you are on the receiving end of a package, or want to check your own before you send it, the steps are short.

1. Unpack it into an empty folder on a machine where it has never been run. A folder on the original analyst’s computer will find files the package forgot to include.
2. Check the data fingerprint. Compute the checksum of the bundled data file and compare it with the recorded one. A mismatch ends the test: whatever you rerun next, it is not on the data that was analysed.
3. Recreate the environment. Install the R version named in the session information, then restore the package versions from the lockfile. If you cannot get the exact versions, note which ones differ before you start, so a later discrepancy has a candidate explanation.
4. Run the script in a fresh session, from the package folder, top to bottom, with nothing else loaded. Do not fix errors silently: an error is a finding.
5. Compare. Put the rerun’s numbers next to the recorded results file, number by number, at full precision. Then compare the recorded results with the manuscript, which is where transcription errors live.
6. Read the decision record against the methods section. A reproducible analysis whose decisions do not match the paper’s description has passed the computational test and failed the honest-reporting one.

The whole check usually takes less time than reading the paper. It is the questions it raises that take time, and those are the ones worth asking.

Common ways reruns fail

Most failures are mundane. These come up again and again.

• Objects from the workspace. The script works for its author because a data frame or function created earlier in the session is still in memory. In a fresh session the script stops on line 40. Running from a clean session before sharing catches this every time.
• Absolute paths. The script reads from a folder that exists only on the author’s computer. Use paths relative to the package folder.
• Unseeded randomness. One step, often a multiple imputation or a cross-validation buried in a helper function, draws random numbers without a seed. The results move on every run, usually only slightly, which makes the problem easy to miss.
• Seeds set in the wrong place. A seed set before one random step and not before another means the second step’s draws depend on how many random numbers the first one used. Change the first step and the second step changes too.
• Package updates. The script ran fine last year and now gives a different estimate, or an error, because a package was updated. The lockfile is the fix.
• Hidden ordering. Reading every file in a folder, or splitting data by a grouping variable, can produce a different order on a different operating system or locale. If order matters to the result (it does for some resampling and some tie-breaking), sort explicitly.
• Rounding in the wrong place. Rounding an intermediate value before it is used in a later calculation changes the final result. Round only for display.
• Manual steps. A value was corrected by hand in a spreadsheet, a figure was edited in a drawing program, or a table was typed into the manuscript. None of it is in the script, so none of it reproduces.

Tools that make the package easier to keep

None of the seven items needs special software, but two habits make them much cheaper to maintain.

Generate the report from the code. Literate programming tools such as R Markdown and Quarto mix text and code in one document, so tables and the numbers quoted in the text can be computed by the code each time the document is rendered. A table produced that way cannot drift from the analysis, because nobody typed it, and the transcription errors that the reviewer’s comparison step looks for have no way into it. Only the numbers you actually generate this way are protected; a value typed into the prose is still a transcription. The document itself is then part of the package: rerender it and the whole report is rebuilt from the data.

Keep the project under version control. A version control system such as Git records every change to the scripts with a date and a message, so analysis_final_v2.R becomes one file with a history instead of several files with guesses for names. Tag or note the commit that produced the submitted results, and the question “which version of the code made Table 2?” has a one-line answer. Keep identifiable data out of the repository; version the code that reads it, not the records.

Neither tool replaces any of the seven items. A rendered document with an unseeded bootstrap still moves on every render, and a perfectly versioned script still needs the package versions it ran under.

What does not count

• “Available on request.” Requests often go unanswered years later, when the analyst has moved on. The Science study above recovered materials for fewer than half of the papers it asked about.
• A description in the methods section alone. “Data were analysed using mixed models in R” does not specify the random-effects structure, the optimiser, the handling of missing data or the software version.
• Point-and-click output with no log. Many menu-driven tools can export the syntax they ran. If yours can, save it; if not, the analysis cannot be rerun without re-clicking from memory.
• A script that depends on objects in your workspace. If it only works after you have run something else first, it is not the script that produced the result.
• Code without the data, or data without the code. Each is half a package. If the data cannot be shared, say why, describe how to obtain them, and still share the code and the decision record, so the analysis can be checked by anyone granted access.

When the data cannot be shared

Patient records, confidential surveys and licensed registries often cannot leave the institution that holds them. That limits who can run the rerun test, but it does not remove the need for a package. It changes who receives it.

• Share everything except the records. The script, the seed, the session information, the lockfile, the decision record and the full results can usually be published. None of them needs to contain a row of data (check that the results do not list individual values or very small groups before you release them), and together they let a reader check that the code does what the methods section says.
• Publish the fingerprint. A checksum of a whole data file does not let anyone read the records in it, but it lets someone who is later granted access confirm they hold the same dataset you analysed.
• Describe the access route. Name the data custodian, the approval needed and roughly how long it takes, so a reviewer or a future researcher knows the test is possible in principle and how to start it.
• Offer a stand-in for the code check. A small synthetic or fully de-identified dataset with the same columns and types lets anyone run the script end to end and confirm it executes. It will not reproduce the published numbers, and the package should say so plainly, but it separates “the code runs” from “the numbers match”, so the first can be checked by anyone and only the second has to wait for access.

The full package, data included, still has to exist somewhere. If only people inside the institution can ever rerun the analysis, it has to be complete for them.

Why reviewers care

Peer review increasingly asks for code and data, and for good reason. A reviewer with the script can check that the model in the code matches the model in the text, that exclusions were applied as described, and that the reported interval is the one the code computes. These checks catch real errors: mislabelled groups, a reversed reference level, a filter applied in the wrong order.

It also protects you. When a question arrives after publication, a complete record lets you answer it in an afternoon instead of reconstructing an analysis from fragments. And if a result does turn out to be wrong, being able to show exactly where the error entered is the difference between a correction and a credibility problem.

There is a smaller, everyday benefit too. An analysis built to be rerun is an analysis you can change. When a reviewer asks for a sensitivity analysis or a different covariate set, a reproducible project makes it a one-line edit and a rerun, and the rest of the numbers stay where they were.

A checklist before you submit

For the reviewer’s side of the same test, see the reproducible research checklist for reviewers.

• Restart the session and run the analysis script from start to finish.
• Confirm every number in the manuscript matches the output, at the precision reported (or, better, generate the tables from the code).
• Save the seed, the output of RNGkind(), the session information and a lockfile alongside the results.
• Record a checksum of every data file the script reads.
• Use relative paths only, and check the script runs from a copy of the folder on another machine.
• Write down every decision that was not pre-specified, and why it was made.
• Store the analysis dataset, the script, the decision record and the machine-readable output together, under version control if possible.
• If the data cannot be shared, say why and how they can be obtained, and share everything else.

What to write in the methods section

The package does the work, but the paper has to point to it. A short paragraph is enough, and it saves the reviewer from guessing. Name the software and its version, say where the full list of package versions can be found, give the seed for any step that used random numbers, and say where the code, the data (or the route to them) and the checksums are kept. For an analysis like the worked example above, it would read something like this:

Analyses were performed in R 4.6.1. Exact package versions are listed in the session information and lockfile in the supplementary analysis package. The confidence interval for epsilon-squared was computed by a percentile bootstrap with 1000 resamples and the random seed 20261022. The analysis script, the analysis dataset, a text file with the dataset’s SHA-256 checksum, and a log of every analytic decision are included in the package.

Avoid the two weak versions of this paragraph: a bare “code available on request”, and a list of software names without versions. Neither lets anyone rerun anything.

Many journals also ask for a separate data availability statement. It answers a different question (where the data are, and on what terms) and should agree with the package. A version for data that cannot be shared openly might read:

The analysis dataset cannot be shared publicly because it contains confidential participant information. It is available from the data custodian named in the supplement to researchers who meet the access criteria. The analysis script, session information, lockfile, decision log and a checksum of the dataset are available in the supplementary analysis package.

Statistical method choices belong in the same record. The guide to choosing a statistical test covers the design questions that decide which test fits, so the reason for each choice can be written down at the time it is made.

How Celeus handles this

Celeus fixes the seed for every analysis and gives you a reproducible package for every finished run: the analysis plan, a script that reruns the analysis from the package folder, a copy of the analysis data (columns flagged as identifiers that the analysis does not use are de-identified in that copy), the results in readable and machine-readable form, the seed and engine version in a run specification, the session information with every package version, the lockfile those versions come from, and an audit log of each step, including the assumption checks. The package does not record a checksum of the data file, so record one yourself as described above, especially if you share the data separately from the package.

Rerunning the package under the recorded versions is designed to give the same numbers, and the engine’s automated checks test that before a change ships. The explanation of determinism and reproducibility says what that promise covers and what it does not, and the description of the reproducible package lists every file and how to handle the data copy safely. Reproducible is not the same as correct: the validation report shows how Celeus results compare with independent computations.

# Results

- Test: Kruskal-Wallis test
- n: 72
- statistic: 54.69
- p-value: < 0.001
- effect size: epsilon-squared = 0.7703 (95% CI 0.7327 to 0.837)
  (two-sided percentile-bootstrap CI at the run's confidence level)
- seed: 20261022

## Assumption checks
- sample size: passed=TRUE (adequate n)

## Post-hoc (Dunn's test; correction: Holm)
- A vs B: adj p = 1 | rank-biserial r -0.1389 [-0.545, 0.32]
- A vs C: adj p = 2.347e-05 | rank-biserial r 0.9931 [0.9823, 0.9973]
- B vs C: adj p = 5.098e-06 | rank-biserial r 0.9931 [0.9823, 0.9973]
- A vs D: adj p = 0.01276 | rank-biserial r 0.9514 [0.8798, 0.9808]
- B vs D: adj p = 0.004823 | rank-biserial r 0.9583 [0.8964, 0.9836]
- C vs D: adj p = 0.5857 | rank-biserial r -0.7222 [-0.8818, -0.4143]
- A vs E: adj p = 0.001179 | rank-biserial r 1 [1, 1]
- B vs E: adj p = 0.0003451 | rank-biserial r 1 [1, 1]
- C vs E: adj p = 1 | rank-biserial r -0.4653 [-0.7511, -0.03252]
- D vs E: adj p = 1 | rank-biserial r 0.3264 [-0.1319, 0.6697]
- A vs F: adj p = 1 | rank-biserial r -0.1944 [-0.584, 0.2678]
- B vs F: adj p = 1 | rank-biserial r -0.03472 [-0.467, 0.411]
- C vs F: adj p = 3.334e-06 | rank-biserial r -1 [-1, -1]
- D vs F: adj p = 0.003838 | rank-biserial r -0.9583 [-0.9836, -0.8964]
- E vs F: adj p = 0.0002496 | rank-biserial r -1 [-1, -1]

groups: A (a), B (a), C (b), D (b), E (b), F (a) - groups sharing a letter are not significantly different at the adjusted threshold.

## Power
- Power is not calculated for this test. The matching parametric test gives a rough guide.

# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Kruskal-Wallis test  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261022)
.data_path <- "data/reproducible-statistical-analysis.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("count", "spray")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- kruskal.test(`count` ~ `spray`, data = data)
print(result)

# Effect size (epsilon_squared) and its confidence interval.
effect <- { set.seed(20261022); .es_d <- data; .es_d[["spray"]] <- as.factor(.es_d[["spray"]]); effectsize::rank_epsilon_squared(.es_d[["count"]], .es_d[["spray"]], ci = 0.95, alternative = "two.sided", iterations = 1000L) }
print(effect)

# Post-hoc pairwise comparisons (Dunn's test; correction: Holm).
data[["spray"]] <- as.factor(data[["spray"]])
posthoc <- { .ph_d <- data; .ph_d[["spray"]] <- as.factor(.ph_d[["spray"]]); PMCMRplus::kwAllPairsDunnTest(`count` ~ `spray`, data = .ph_d, p.adjust.method = "holm") }
print(posthoc)