Celeus

Dissertation statistics: a design-first guide

Choose the statistics for a thesis from the study design: five common designs, the test, the assumption checks and the effect size to report for each.

Updated . Written by Matt Gencer.

On this page

Most thesis students meet the statistics question backwards. The data are collected, the software is open, and the question becomes “which button do I press?” Supervisors and examiners ask a different question: does the analysis match the way the study was designed? Answer that one first and the choice of test mostly follows from it.

This guide is built around that order. It starts with four questions about your design, maps the five designs that turn up most often in theses to the test, the assumption checks and the effect size you should report for each, and then works one design through in full: an association between two survey measures, run on seeded synthetic data. It closes with the analysis plan, the sample size justification and the mistakes examiners most often flag.

It is written for students in any field whose thesis has a quantitative part. It is not a substitute for your supervisor or your department’s statistician; it is meant to help you arrive at that conversation with a plan they can check.

Start from the design, not from the software

A statistical test is a model of how your data came to be. A two-sample t-test assumes two separate groups of units, each measured once. A paired test assumes the same units measured twice. A correlation assumes one group of units, each measured on two variables. If the model does not match how the data were collected, the p-value and the interval the software prints are answers to a question your study did not ask.

That is why the order matters. The design is fixed when you plan the study, before any data values exist. A choice that follows from the design can be written into your methods chapter before you collect anything, and it can be defended without reference to the results. A choice made by trying several tests and keeping the one that looks best cannot, and an experienced examiner will notice.

The general version of this reasoning, for any research question, is in our guide to choosing a statistical test. This page narrows it to the designs a thesis usually has.

Four questions that pick the method

Answer these about your study, in this order, before you look at a single outcome value.

1. What is the outcome, and what type is it? The outcome is the variable your research question is about: a test score, a reaction time, a satisfaction rating, whether a participant completed a task. Its type narrows the options at once. A continuous measurement (a score, a time, a concentration) points to methods built on means. A binary outcome (yes or no) points to proportions, odds ratios and logistic regression. An ordered category (a single Likert item, a severity grade) points to rank methods or ordinal regression. A count points to Poisson-type models. A time until an event, where some participants have not had the event by the end of the study, points to survival methods.

2. What comparison or relationship does the question ask about? Comparing two groups, comparing three or more groups, comparing the same units at two time points, or describing how two variables move together are four different questions, and each has its own family of methods.

3. How are the observations linked? This is the question students most often skip, and it matters more than any other. If every value comes from a different, unrelated unit, the observations are independent. If the same participant was measured before and after, or the units were deliberately matched in pairs, the observations are paired. If participants sit inside classes, schools, clinics or families, observations from the same cluster tend to resemble each other, and a method that ignores this will report more certainty than the data hold.

4. What is the unit of analysis? The unit is the thing that was independently sampled or assigned. If you assigned whole classrooms to a teaching method and measured every pupil, the classroom is the unit of assignment, and treating each pupil as an independent observation overstates your sample size. Count units, not rows.

Write the four answers down in one paragraph. That paragraph is the start of your analysis plan, and it is usually enough to choose the method from the table below.

The decision table: five common thesis designs

These five designs cover a large share of the quantitative theses a supervisor reads. Each row gives a sensible default, the alternative to name in your plan, the checks to report and the effect size to put next to the p-value.

Design Typical thesis question Default method Alternative to name in the plan Checks to report Effect size to report
Two independent groups Do students taught with method A score higher than those taught with method B? Welch’s t-test Mann-Whitney U, when the question is about ranks or the outcome is ordinal Independence from the design; shape of each group in a plot Mean difference with its 95% interval, plus a standardised difference
Pre-test post-test Did scores change after the programme, or change more than in a control group? Paired t-test (one group); ANCOVA on the post-test adjusted for the pre-test (two groups) Wilcoxon signed-rank (one group); a linear mixed model when there are more than two time points or missing follow-ups Pairing from the design; for ANCOVA, a straight-line relation with the pre-test and similar slopes in each group Mean change, or the adjusted group difference, with its 95% interval
Three or more groups Do satisfaction scores differ between three departments? Welch’s ANOVA, then Games-Howell pairwise comparisons Kruskal-Wallis with Dunn’s comparisons Shape and spread in each group; how many pairwise comparisons, and the correction Omega squared (less biased than eta squared), plus each pairwise difference with an adjusted interval
Survey scales Do two groups differ on a wellbeing scale, or on one item of it? A summed scale: the continuous methods above. A single item: ordinal logistic regression Mann-Whitney U for a single item in two groups Internal consistency of the scale; floor and ceiling effects; proportional odds for the ordinal model As for the continuous design, or an odds ratio with its interval for the ordinal model
Association Is weekly practice associated with lower anxiety scores? Pearson correlation for two continuous measures; chi-square test for two categorical ones Spearman correlation; Fisher’s exact test when expected counts are small; regression when other variables need adjusting for A scatterplot for straight-line shape and outliers; expected counts for chi-square r with its 95% interval; Cramer’s V or an odds ratio for a table

The defaults lean towards the versions that make fewer assumptions you cannot check, such as Welch’s tests, which do not assume equal variances; this is a deliberate choice explained below. And the “alternative” column is not a menu to try after the default disappoints. It is what you name in the plan, with the condition under which you would use it, before you see the data.

Design 1: two independent groups

Two groups of different participants, each measured once on a continuous outcome, is the most common comparison in a thesis. The usual default is Welch’s t-test. It compares the two means without assuming the groups have the same variance, and simulation work shows it loses little when the variances happen to be equal and protects the error rate when they are not, which is why Delacre, Lakens and Leys (2017) recommend it as the default rather than a fallback. The same authors point out that running a test of equal variances first, and choosing between Student’s and Welch’s tests on its result, often fails to give the right answer, so the simpler rule of using Welch’s test from the start is also the safer one.

When the outcome is an ordered category rather than a measurement, or the question really is “does a value from one group tend to exceed a value from the other?”, the Mann-Whitney U test answers it. It is not a drop-in replacement for the t-test: it compares ranks, not means, so it answers a different question. Our t-test versus Mann-Whitney guide goes through when each fits.

Report the difference in means with its 95% confidence interval, which keeps the result on the outcome’s own scale, and a standardised effect size such as Cohen’s d. Say which standard deviation the d was divided by (the pooled one, or one group’s), because different choices give different numbers, and note that a d built on the pooled standard deviation, and its usual interval, assume equal variances even when the test itself is Welch’s. The t-tests explained post works a two-sample example in full, including d with its interval.

# Welch's t-test is the default in base R
t.test(score ~ group, data = df)
# Mann-Whitney U (called the Wilcoxon rank-sum test in R)
wilcox.test(score ~ group, data = df)

Design 2: pre-test post-test

For participants measured before and after an intervention, the right analysis depends on one design fact: is there a comparison group?

One group, measured twice. Each participant’s two scores are linked, so the analysis works on the within-person change. The paired t-test asks whether the mean change differs from zero; it is a one-sample t-test on the differences. The Wilcoxon signed-rank test is the rank-based alternative. Report the mean change with its interval. If you also report a standardised effect, say whether it was divided by the standard deviation of the differences or of the raw scores, because the two give different values. The paired t-test versus Wilcoxon guide covers the choice between them.

Be careful about what this design can conclude. Without a comparison group, a change from pre-test to post-test cannot be separated from what would have happened anyway: practice on the test, natural improvement over time, or regression to the mean when participants were selected for low pre-test scores. The analysis can show that scores changed. It cannot, on its own, show that the programme caused the change, and your discussion chapter should say so.

Two groups, each measured before and after. Here the question is whether the groups differ at follow-up, allowing for where they started. There are three common ways to analyse this: compare the post-test scores only, compare the change scores, or use analysis of covariance (ANCOVA) with the post-test as the outcome and the pre-test as a covariate. Vickers and Altman (2001) set out the trade-offs for controlled trials and conclude that ANCOVA is the preferred general approach; change scores can be a reasonable alternative when the pre-test and post-test are very highly correlated and the groups were balanced at baseline. Our ANCOVA guide covers its assumptions, including the one students most often forget: that the relationship between pre-test and post-test has a similar slope in both groups.

If you measured more than twice, or some participants missed the follow-up, a linear mixed model uses every available measurement instead of dropping incomplete participants; the repeated-measures ANOVA versus mixed model guide compares the options.

# One group, before and after: a paired t-test on the two columns
t.test(df$post, df$pre, paired = TRUE)
# Two groups: ANCOVA, post-test adjusted for pre-test
summary(lm(post ~ pre + group, data = df))

Design 3: three or more groups

Comparing three or more independent groups calls for one overall analysis rather than a string of pairwise t-tests. Running every pairwise t-test inflates the chance of at least one false positive: with three groups there are three comparisons and with five groups ten.

The default here is Welch’s ANOVA, for the same reason Welch’s test is the default for two groups. Simulations show that the classical F-test can give badly misleading error rates when the group variances differ, a situation that is common in real data, and that Welch’s version holds up across a wide range of realistic conditions (Delacre, Leys, Mora and Lakens, 2019). When the overall test points to a difference, the follow-up question is which groups differ. The usual follow-up to Welch’s ANOVA in practice is Games-Howell, which also does not assume equal variances: in Games and Howell’s (1976) simulations with unequal sizes and variances, it kept error rates under control once each group had six or more observations. The rank-based route is Kruskal-Wallis followed by Dunn’s comparisons.

Decide in the plan which comparisons you care about. If you only care about each treatment against a control, say so and correct for those comparisons only; correcting for every possible pair when you only need some of them costs you power for nothing. The multiple comparisons guide explains the corrections, and the one-way ANOVA, Welch and Kruskal-Wallis guide compares the three overall tests. If your groups are defined by two factors at once, for example teaching method and year group, you have a factorial design, and a two-way ANOVA answers questions a one-way analysis cannot, including whether the effect of one factor depends on the other.

Report an overall effect size and each pairwise difference with its adjusted confidence interval. Eta squared, the share of total variation lying between groups, overestimates the population value; omega squared is a less biased correction of it (Lakens, 2013). Both come from the classical ANOVA table and its pooled error variance, not from Welch’s statistic, so say so when group variances differ clearly. The pairwise intervals are usually what a reader actually wants to know.

# Welch's ANOVA in base R
oneway.test(score ~ group, data = df, var.equal = FALSE)
# Rank-based alternative
kruskal.test(score ~ group, data = df)

Design 4: survey scales and Likert data

A frequent statistics argument in a viva is whether Likert data may be analysed with t-tests and ANOVA. The answer depends on what you are analysing.

A summed or averaged scale combines several items that are meant to measure one construct, such as a ten-item anxiety scale. The total takes many values, behaves much like a continuous measurement, and is routinely analysed with t-tests, ANOVA, correlation and regression. Norman (2010) reviews the evidence going back decades that these parametric methods are robust to the departures such scales usually show, and argues that the objection that “Likert data are ordinal, so parametric tests are forbidden” does not hold for this use. Before treating a scale this way, report its internal consistency, check that the items do measure one thing, and look for floor or ceiling effects, where many participants sit at the minimum or the maximum score. A scale squashed against its ceiling cannot show change in one direction.

A single item is different. A five-point item has five possible values, and the distance between “agree” and “strongly agree” is not obviously the same as between “neutral” and “agree”. For a single item, a rank test such as Mann-Whitney U compares two groups, and an ordinal logistic regression models the item directly and can adjust for other variables. The ordinal versus multinomial logistic regression guide explains the ordinal model and its proportional odds assumption.

Decide in the plan, before data collection, whether your primary outcome is the scale or an item. Reporting the scale and then adding significant items one at a time is a multiple comparisons problem that a careful examiner will spot.

Design 5: association between two measures

Many theses ask whether two things go together, such as sleep and exam performance. When both variables are measured once on each participant, the design is one group of independent units with two measurements each, and there is no grouping variable at all.

For two continuous measures, the Pearson correlation describes how closely the points lie around a straight line, on a scale from -1 to 1. The Spearman correlation works on ranks, so it describes any consistently increasing or decreasing relationship and is less influenced by a few extreme values. For two categorical variables, the chi-square test of independence asks whether the distribution of one differs across the levels of the other, and Fisher’s exact test replaces it when expected counts are small; the chi-square versus Fisher’s exact guide gives the rule.

Two warnings belong in every association analysis. First, Pearson’s r measures a straight-line relationship only. A strong curved relationship can produce an r near zero, and a single extreme point can produce a large r on its own, so a scatterplot is not optional. Second, the p-value and interval for Pearson’s r rely on the two variables being roughly normal together (bivariate normal), not just on each one looking normal. Simulation work shows that with clearly non-normal data the usual test can give too many false positives and lose power, and that alternatives such as rank-based approaches can do better (Bishara and Hittner, 2012). If your plot shows a strongly skewed variable, name the alternative in the plan.

When the question is whether one variable is associated with another after allowing for a third, a correlation is no longer enough; a regression model with the third variable as a covariate is the tool. Mediation, moderation and structural equation models extend that idea and are beyond this guide; each rests on a stated model of how the variables relate, which belongs in the plan before any data are seen.

Worked example: an association design, run end to end

To show what a complete analysis of one design looks like, here is design 5 worked through on data built for the purpose. The data are synthetic: 120 simulated respondents to a single survey, generated by a seeded script, with no real person behind any row. Each respondent has two values: the hours of statistics practice they report in a typical week, and a statistics anxiety score summed from eight items scored 0 to 4, so the score can run from 0 to 32. The simulation was written so that the two are negatively related. That means the example demonstrates the procedure, not a finding about students: the association is there because the script put it there.

The design check. Every row is a different respondent, measured once, so the pairs of values are independent of each other. There are no groups, no repeated measurements and no clusters. Both variables are numeric: practice hours is a measurement, and the anxiety score is a summed multi-item scale, which design 4 says can be treated as continuous. The question is about association, so the plan names the Pearson correlation as the primary analysis, with Spearman as the alternative if a plot shows a clearly non-linear pattern or extreme values.

The analysis was run through the engine with the method fixed in advance, not chosen by looking at the result. Here is the headline output:

Quantity Value
Respondents analysed 120 (none missing)
Pearson’s r -0.3604
95% confidence interval for r -0.5069 to -0.1937
Test statistic t = -4.198 on 118 degrees of freedom
p-value < 0.001
Shared variation (r squared) about 12.99%

Reading the result. The correlation is negative: in this sample, respondents who reported more practice tended to have lower anxiety scores. The estimate, about -0.36, is a moderate correlation by common rules of thumb, but the interval matters more than the label. It runs from about -0.51 to about -0.19, so the data are compatible with a fairly weak negative association and with a fairly strong one, while zero and every positive value lie outside the 95% interval. Squaring r gives about 12.99%: the share of the variation in one variable that the straight-line relationship accounts for. Most of the variation in anxiety scores is not tracked by practice hours at all, which is a useful reminder that a clearly non-zero correlation can still be a modest one.

The p-value, below 0.001, says that a correlation this far from zero would be very unusual if the true correlation were exactly zero and the assumptions held. It is not the probability that there is no association, and a small p-value is not a sign that the association is large or important; the interval does that job. Greenland and colleagues (2016) list both of those readings among the common misinterpretations of p-values.

What the result does not say. The data are cross-sectional: both values were recorded at one time. Nothing in the analysis tells us whether practice reduces anxiety, whether less anxious students choose to practise more, or whether some third factor, such as earlier success in mathematics, drives both. A thesis with this design should describe the result as an association and leave causal language out.

Reading the assumption checks honestly

The engine records which checks it ran, and for this method the record is short:

  • Sample size: passed (adequate n).

That line is a floor. It confirms there are enough observations for the method to be computed sensibly; it is not an assessment of whether the analysis is valid. The engine run did not check whether the relationship is a straight line, did not look for outliers, did not examine the joint shape of the two variables, and produced no scatterplot. Nothing in this example says those conditions hold. Those checks are yours to do and to report:

  • A straight-line relationship. Plot one variable against the other. If the points follow a curve, Pearson’s r understates the relationship and Spearman’s correlation, or a model with a curved term, is the better description.
  • Extreme points. Look for a few points far from the rest. If one or two of them change r substantially when removed, report that, rather than quietly deleting them.
  • The shape of each variable, and of the pair. A summed scale with many respondents at 0 or at 32 has floor or ceiling effects that compress the relationship. Strong skew in either variable, or a cloud of points that fans out or bunches in one corner, is a reason to name a rank-based or transformed analysis as the alternative.

Notice what is missing from that list: a formal normality test as a gate. On 120 values it can flag departures too small to matter, on 15 it can miss ones that do, and a non-significant result means no departure was detected, not that the data are normal. Plots, and a plan that says in advance what you will do if they look wrong, serve you better.

# The checks this example leaves to you: draw them, then describe them in your results
plot(df$practice_hours, df$anxiety_score,
     xlab = "Practice hours per week", ylab = "Anxiety score (0 to 32)")
hist(df$anxiety_score)
# The named alternative, if the plot shows a non-linear or outlier-driven pattern
cor.test(df$practice_hours, df$anxiety_score, method = "spearman")

Sample size: justify it before, not after

The same engine run reports a sensitivity figure. It is not observed power, which is a function of the p-value you already have and adds nothing to it (Hoenig and Heisey, 2001). A sensitivity analysis asks a different question: given the sample size you have, what is the smallest effect the study could detect with reasonable reliability?

Sensitivity (two-sided, alpha = 0.05, target power = 0.8) Value
Smallest correlation detectable with 120 respondents r = 0.2525 (in either direction)
Total sample needed to detect r = 0.30 85

Read the first line as a statement about the design: with 120 respondents, correlations of about 0.25 or larger, positive or negative, would be detected with 80% probability at the 0.05 level, and smaller ones could easily be missed. That matters most when a result is not significant, because it tells the reader which effects the study could have found and which it could not.

The second line is a planning reference, not a recommendation. The value 0.30 is a conventional “medium” correlation, used here only to show the calculation. In your own plan, the target effect should come from your field: the smallest association that would matter for your question, or a value supported by earlier studies with similar measures, and the plan should say where it came from. Do not use the effect size you observed in your own data, or in a small pilot, to justify the sample size after the fact. Lakens (2022) sets out the legitimate ways to justify a sample size, including resource constraints, accuracy of estimation and a sensitivity analysis, and a thesis can honestly use any of them as long as it says which.

# Planning: total sample for a target correlation, two-sided, alpha 0.05, power 0.8
pwr::pwr.r.test(r = 0.30, sig.level = 0.05, power = 0.8)
# Sensitivity: the smallest detectable correlation for the sample you have
pwr::pwr.r.test(n = 120, sig.level = 0.05, power = 0.8)

How to write it up

A results sentence for the worked example, in APA style, looks like this:

Weekly practice hours were negatively correlated with statistics anxiety scores, r(118) = -.36, 95% CI [-.51, -.19], p < .001. With 120 respondents, the study had 80% power to detect correlations of |r| = .25 or larger at alpha = .05.

The estimate and its interval carry the size and the precision. The p-value is reported exactly or as an inequality, never as “significant” alone. The sensitivity sentence is optional for a significant result and important for a non-significant one. In the discussion, the sentence about cause goes next to it: the design shows an association, and you name the explanations you cannot rule out.

The same pattern works for every design in the table: name the test, give the estimate with its interval, give the effect size and say how it was standardised, then the test statistic, its degrees of freedom and the p-value. If an assumption check changed the analysis, the results chapter says what was found and what was done about it.

Before any test, the results chapter usually describes the sample: how many units were analysed, how many were excluded or had missing values and why, and the mean and standard deviation (or median and interquartile range) of each main variable, by group where there are groups. A table does this best.

Writing the statistical analysis plan for your methods chapter

A statistical analysis plan is a short document, often a section of the methods chapter, that fixes the analysis before the outcome data are seen. Clinical trials have formal guidance on what one should contain (Gamble and colleagues, 2017). A thesis does not need that level of detail, but the same headings make a good skeleton:

  1. The research question and hypotheses, in one or two sentences each, stating the direction you expect if you expect one.
  2. The primary outcome, how it is measured or scored, and its type. If there are secondary outcomes, list them and label them as secondary.
  3. The design and unit of analysis: independent groups, repeated measures, matched pairs or clusters, and what counts as one observation.
  4. The primary analysis: the test or model, the effect size you will report, and the confidence level.
  5. Assumption checks and the fallback: which plots you will draw and which alternative you will use if a named problem appears.
  6. Multiple comparisons: which comparisons you will make and how you will correct for them, or why no correction is needed.
  7. Missing data: how much you expect, how you will describe it, and how you will handle it. Dropping incomplete cases is a decision, not a default.
  8. Sample size justification: the method and its inputs, with the source of the target effect.
  9. Software and reproducibility: the software and version, and how a reader could rerun the analysis.

Write the plan, show it to your supervisor, and date it. If the analysis later has to change, because the data turned out to be clustered or an outcome had a ceiling effect, the change is fine as long as you report it and say why. What a plan prevents is not change; it is undisclosed change.

Common thesis mistakes

Choosing the test after seeing the results. Running a t-test, a Mann-Whitney test and a regression, then reporting the one with the smallest p-value, makes every p-value in the thesis uninterpretable. Choose from the design, write it down, and report any departures.

Ignoring pairing or clustering. Analysing before and after scores as two independent groups throws away the pairing and usually the precision that came with it. Treating pupils in the same classes as independent does the reverse, reporting more certainty than the data hold.

Reading a non-significant result as “no effect”. A p-value above 0.05 means the data are compatible with no effect, not that the effect is zero. The interval shows which effects remain compatible, and the sensitivity figure shows which effects the study could have detected.

Using causal language for an association. “Practice reduces anxiety” is a claim the cross-sectional design in the example cannot support. “Practice was associated with lower anxiety” is the claim it can.

Where Celeus fits

Celeus, the tool behind this blog, ran the worked example above. When you upload a dataset, it proposes suitable analyses from the structure of your columns, grades the alternatives against checks run on your data, runs the method you choose with a fixed seed and gives you a package with the R script that reproduces the result. It will not decide what your research question is, it will not write your analysis plan for you, and it does not replace your supervisor or a statistician. See the validation page.

Rerunning the example

Below the article you will find the R script exactly as the engine generated it, with the seed and the bundled data path, and the output it printed. The script reads the simulated data from a data/ folder and reruns two things: the Pearson correlation test and the sensitivity calculation. It reproduces only those two analyses: it does not recompute the planning figure of 85 respondents (the planning code above does), it does not regenerate the synthetic data from the seed, and it does not draw the plots described above. Running it needs R with the readr and pwr packages. Our guide to reproducible statistical analysis explains what a reviewer needs in order to rerun a result like this one.

The method reference pages describe each method this guide names, its inputs and its references: Pearson correlation, Spearman correlation, Welch’s t-test, the paired t-test, ANCOVA and Welch’s ANOVA.

Worked example

Data: synthetic, generated in the script below. Seed: 20261021.

Run on Celeus engine 0.30.0, seed 20261021, synthetic data.

Output

# Results

- Test: Pearson correlation
- n: 120
- statistic: -4.198
- p-value: < 0.001
- effect size: Pearson's r = -0.3604
- 95% CI: [-0.5069, -0.1937]
- seed: 20261021

## Assumption checks
- sample size: passed=TRUE (adequate n)

## Power (sensitivity, not observed power)
- minimum detectable effect at n=120: correlation r = 0.2525
- to detect a conventional medium effect: 85 total to detect r=0.30
- alpha = 0.05, target power = 0.8
R script
# Celeus - reproduce.R  (run from this package folder; bundled data in data/)
# Test: Pearson correlation  |  engine 0.30.0  |  catalog c39-d3cd
# Research context: not provided for this run.
# Reproduces the recorded result deterministically.

set.seed(20261021)
.data_path <- "data/dissertation-statistics.csv"
if (!file.exists(.data_path)) stop("Run reproduce.R from the package folder (the bundled data is in data/).")
data <- readr::read_csv(.data_path, show_col_types = FALSE)  # needs the readr package
clean_cells <- function (df) 
{
    ctrl_collapse <- "[\r\n\t\v\f]+"
    zero_width_chars <- intToUtf8(c(8203L, 65279L), multiple = TRUE)
    uni_space_chars <- intToUtf8(c(160L, 8239L, 8199L), multiple = TRUE)
    clean_text <- function(x) {
        out <- x
        for (ch in zero_width_chars) out <- gsub(ch, "", out, 
            fixed = TRUE)
        out <- gsub(ctrl_collapse, " ", out)
        for (ch in uni_space_chars) out <- gsub(ch, " ", out, 
            fixed = TRUE)
        trimws(gsub(" +", " ", out))
    }
    changed_cols <- character(0)
    cells_cleaned <- 0L
    retyped <- character(0)
    for (j in seq_along(df)) {
        col <- df[[j]]
        if (!is.character(col)) 
            next
        cleaned <- clean_text(col)
        n_changed <- sum(!is.na(col) & !is.na(cleaned) & col != 
            cleaned, na.rm = TRUE)
        if (n_changed > 0L) {
            changed_cols <- c(changed_cols, names(df)[j])
            cells_cleaned <- cells_cleaned + n_changed
        }
        df[[j]] <- cleaned
        non_blank <- cleaned[!is.na(cleaned) & nzchar(cleaned)]
        if (length(non_blank) && all(grepl("^[+-]?(0|[1-9][0-9]*)([.][0-9]+)?$", 
            non_blank))) {
            parsed <- suppressWarnings(as.numeric(cleaned))
            was_blank <- is.na(cleaned) | !nzchar(cleaned)
            lost <- (is.na(parsed) | is.infinite(parsed)) & !was_blank
            if (!any(lost)) {
                finite <- parsed[!is.na(parsed)]
                all_whole <- all(finite == floor(finite))
                in_range <- all(abs(finite) <= .Machine$integer.max)
                df[[j]] <- if (all_whole && in_range) 
                  as.integer(parsed)
                else parsed
                retyped <- c(retyped, names(df)[j])
            }
        }
    }
    names(df) <- clean_text(names(df))
    if (length(changed_cols)) {
        attr(df, "ingest_clean_columns") <- changed_cols
        attr(df, "ingest_clean_cells_cleaned") <- cells_cleaned
    }
    if (length(retyped)) 
        attr(df, "ingest_clean_retyped") <- retyped
    df
}
data <- clean_cells(data)
# Complete-case analysis: drop rows missing any analysis value.
.acols <- c("practice_hours", "anxiety_score")
data <- data[stats::complete.cases(data[, .acols, drop = FALSE]), , drop = FALSE]
result <- cor.test(data[["practice_hours"]], data[["anxiety_score"]], method = "pearson")
print(result)

# Sensitivity: minimum detectable effect at n=120 (power=0.8, alpha=0.05). NOT observed power.
power <- pwr::pwr.r.test(n = 120, sig.level = 0.05, power = 0.8)
print(power)

Common questions

Which analysis should I use for a pre-test post-test design?

It depends on whether there is a comparison group. With one group measured twice, use a paired t-test on the change, or the Wilcoxon signed-rank test when the question is about ranks. With a treated group and a control group, analysis of covariance with the pre-test as a covariate is the usual first choice, because it compares the groups at follow-up after adjusting for where each person started.

Which tests should I use for Likert data?

First decide whether you are analysing single items or a summed scale. A scale built from several items that measure one construct is usually treated as continuous and analysed with t-tests, ANOVA or regression. A single item is an ordered category, so a rank test such as Mann-Whitney or an ordinal logistic regression fits it better.

How do I write a statistical analysis plan for my thesis?

Write it before you see the outcome data. Name the research question, the primary outcome and how it is scored, the design and unit of analysis, the main test and its effect size, how you will check assumptions and what you will do if one fails, how you will handle missing data, the significance level, and how the sample size was justified.

Do I need to test for normality before choosing a test?

Not as a gate. Normality tests miss real departures in small samples and flag trivial ones in large samples, so they answer the wrong question. Look at plots of the residuals instead, prefer methods that tolerate the likely problem, such as Welch's tests for unequal spread, and state in the plan what you will do if the plots show a serious problem.

Is a correlation enough to show that one variable affects another?

No. A correlation from a survey measured at one time describes how two variables move together in that sample. It cannot tell you which one drives the other, or whether a third variable drives both. Causal language needs a design that supports it, such as random assignment, or a clearly argued adjustment for confounders.

What should I do if my thesis data are messy or incomplete?

Keep the raw file untouched and do every cleaning step in a script or a written log, so each change can be traced. Check ranges and impossible values, recode categories consistently, and fix how each scale is scored before you look at the outcome. Report how many values were missing in each variable and how you handled them; dropping incomplete cases is a decision to justify, not a default.

Which statistical software should I use for my thesis?

Use what your department supports or your supervisor knows, as long as it runs the method your design needs. Most methods in this guide are available in the mainstream packages, such as R, SPSS, Stata and SAS, though a few, such as Games-Howell comparisons, need an add-on package in some of them. What matters more is recording the software and its version and keeping a script or syntax file, so that someone else can rerun the analysis.

Where can I get help with dissertation statistics?

Start with your supervisor, then your university's statistics or research methods support if it has one, because both know what your department expects. An independent statistician can help with design, analysis or a review of your work. Software, including AI tools, can run an analysis, but checking that the method fits your design remains your job.

Is it allowed to get help with my thesis statistics?

That depends on your university's rules, which differ, so read them and ask your supervisor before you start. Whatever help you get, acknowledge it as the rules require and make sure you can explain and defend every analysis yourself, because the examiners will ask you, not your helper. Before sharing participant data with anyone, check what your ethics approval allows and remove identifiers.

Sources

  1. Vickers AJ, Altman DG (2001). Analysing controlled trials with baseline and follow up measurements. BMJ 323(7321):1123-1124. doi:10.1136/bmj.323.7321.1123
  2. Delacre M, Lakens D, Leys C (2017). Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology 30(1):92-101. Correction published 2022, International Review of Social Psychology 35(1):21; it leaves the main recommendation unchanged. doi:10.5334/irsp.82
  3. Games PA, Howell JF (1976). Pairwise multiple comparison procedures with unequal N's and/or variances: a Monte Carlo study. Journal of Educational Statistics 1(2):113-125. doi:10.3102/10769986001002113
  4. Lakens D (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. doi:10.3389/fpsyg.2013.00863
  5. Delacre M, Leys C, Mora YL, Lakens D (2019). Taking parametric assumptions seriously: arguments for the use of Welch's F-test instead of the classical F-test in one-way ANOVA. International Review of Social Psychology 32(1):13. doi:10.5334/irsp.198
  6. Norman G (2010). Likert scales, levels of measurement and the 'laws' of statistics. Advances in Health Sciences Education 15(5):625-632. doi:10.1007/s10459-010-9222-y
  7. Bishara AJ, Hittner JB (2012). Testing the significance of a correlation with nonnormal data: comparison of Pearson, Spearman, transformation, and resampling approaches. Psychological Methods 17(3):399-417. doi:10.1037/a0028087
  8. Lakens D (2022). Sample size justification. Collabra: Psychology 8(1):33267. doi:10.1525/collabra.33267
  9. Hoenig JM, Heisey DM (2001). The abuse of power. The American Statistician 55(1):19-24. doi:10.1198/000313001300339897
  10. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology 31(4):337-350. doi:10.1007/s10654-016-0149-3
  11. Gamble C, Krishan A, Stocken D, et al. (2017). Guidelines for the content of statistical analysis plans in clinical trials. JAMA 318(23):2337-2343. doi:10.1001/jama.2017.18556

Methods discussed

Try it on your own data

Upload a dataset and Celeus suggests a method, checks its assumptions, runs it in R, and gives you the script to rerun it.

Try it in Celeus