Linear regression
Shown in the app as linear_regression. Part of the method reference, under Regression.
When to use
Model a continuous outcome from predictors, adjusting for covariates.
What you provide
- Outcome: one numeric column.
- Predictors: one or more columns.
Options
Missing data
- Complete case (listwise deletion)
- Mean imputation (single)
- Median imputation (single)
- Multiple imputation
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Related methods
Other methods under Regression:
- Logistic regression
- Firth penalised logistic regression
- Poisson regression
- Negative-binomial regression
- Ordinal logistic regression
- Multinomial logistic regression
- Factors associated with outcome (continuous outcome)
- Factors associated with outcome (binary outcome)
Guides that discuss this method
How to read the output
Linear regression models a numeric outcome from one or more predictors, for example house price from floor area and neighbourhood, or plant height from hours of light and amount of water. Celeus fits the model by ordinary least squares.
Coefficients. For a numeric predictor, the coefficient is the estimated change in the outcome per one-unit increase; for a categorical predictor, it is the difference from the reference category. Each is adjusted for the other predictors in the model. The reference category is the first one, which is alphabetical unless you reordered the categories. Every term is listed with its t-test and confidence interval, at 95% unless you choose 90% or 99%. The headline result is the coefficient for the first predictor you list (for a categorical predictor with more than two categories, its first comparison with the reference), so the order of predictors matters. A categorical predictor stored in a category order (an ordinal column) is still compared level by level with its first category.
Testing a predictor as a whole. A categorical predictor with three or more categories has one coefficient per category, and none of them tests the predictor itself. Celeus reports a Type II F test for every term, with its partial eta-squared and interval, so a predictor such as a treatment arm with three groups gets one overall test. A predictor that is an exact combination of the others cannot be estimated; Celeus leaves it out and says so.
R-squared. The effect size is R-squared, the proportion of the outcome’s variance that the model accounts for, from 0 to 1. It is the plain R-squared, so it never falls when a predictor is added, even one with no real relationship to the outcome; its interval comes from the overall F test. The model-fit summary beside it gives the adjusted R-squared, the residual standard error, the overall F test, AIC and BIC, which are the numbers to compare models with different numbers of predictors.
Checks to read alongside. The only assumption check on the fit itself is a Shapiro-Wilk test of normality on the residuals, which is not assessed below 3 or above 5000 residuals. It is most informative in small samples. With moderate to large samples the coefficient tests depend on the sampling distribution of the estimates rather than on the residuals being normal, so a significant Shapiro-Wilk result there is not on its own a reason to abandon the model (Lumley and colleagues, 2002).
The standard errors are the classical ones, not robust (sandwich) ones, and Celeus does not check for unequal residual spread (heteroscedasticity). It does report, for the model it fitted, how many observations have a Cook’s distance above 4/n, the largest leverage and each predictor’s variance inflation factor, with notes when those pass the screening values (4/n, and a variance inflation factor above 5). These are screening aids, so look at the residual plots yourself. A numeric predictor that holds only a few whole-number codes is fitted as one straight line across them; if the codes are categories, declare the column categorical.
Missing values. Rows with a missing value can be dropped, filled with the column mean or median, or handled by multiple imputation. With multiple imputation the coefficients and their intervals are pooled across the imputed datasets, R-squared is the average over them, and the residual normality check is replaced by a small-sample warning below 10 observations.
Survey designs. When the data come from a complex survey (sampling weights, and optionally PSUs, strata and a finite population correction), the model is fitted to the design rather than to the rows: estimates are weighted to the surveyed population and standard errors use Taylor-series linearisation, with t tests on the design’s degrees of freedom (PSUs minus strata, minus the model’s terms). Each predictor also gets a design-based Wald F test.
A subpopulation is analysed by restricting the design, not the data, so every sampled row still informs the variance. Under a design, rows with a missing value are left out of the analysed subpopulation rather than imputed, and R-squared, the residual normality check and the power estimate are not computed. Sampling weights are not the same as frequency or precision weights, and the three cannot be combined in one analysis.
Sources: Lumley T, Diehr P, Emerson S, Chen L (2002), Annual Review of Public Health 23:151-169, doi:10.1146/annurev.publhealth.23.100901.140546. Shapiro SS, Wilk MB (1965), Biometrika 52(3-4):591-611, doi:10.1093/biomet/52.3-4.591. Lumley T (2004), “Analysis of complex survey samples”, Journal of Statistical Software 9(1):1-19.
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981)
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -3.87783 | -3.87783 | - | Yes |
| p-value | 1.11965e-06 | 1.11965e-06 | - | Yes |
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -4.25558 | -4.25558 | - | Yes |
| p-value | 0.00471783 | 0.00471783 | - | Yes |
| Confidence interval | -7.09482, -1.41634 | -7.09482, -1.41634 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 2.55691 | 2.55691 | - | Yes |
| Effect size | 0.837433 | 0.837433 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 48.0787 | 48.0787 | - | Yes |
| Test statistic for each model term | 7.28557, 18.08010 | 7.28557, 18.08010 | - | Yes |
| Number of observations | 32 | 32 | - | Yes |
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -2.87579 | -2.87579 | - | Yes |
| p-value | 6.11269e-10 | 6.11269e-10 | - | Yes |
| Confidence interval | -3.53424, -2.21734 | -3.53424, -2.21734 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 3.2059 | 3.2059 | - | Yes |
| Effect size | 0.72618 | 0.72618 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 79.561 | 79.561 | - | Yes |
| Test statistic for each model term | 79.561 | 79.561 | - | Yes |
| Number of observations | 32 | 32 | - | Yes |
datasets::cars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Ezekiel (1930), Methods of Correlation Analysis; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 3.93241 | 3.93241 | - | Yes |
| p-value | 1.48984e-12 | 1.48984e-12 | - | Yes |
| Confidence interval | 3.09696, 4.76785 | 3.09696, 4.76785 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 15.3796 | 15.3796 | - | Yes |
| Effect size | 0.651079 | 0.651079 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 89.5671 | 89.5671 | - | Yes |
| Test statistic for each model term | 89.5671 | 89.5671 | - | Yes |
| Number of observations | 50 | 50 | - | Yes |
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -3.87783 | -3.87783 | - | Yes |
| p-value | 1.11965e-06 | 1.11965e-06 | - | Yes |
| Confidence interval | -4.95293, -2.80274 | -4.95293, -2.80274 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 2.59341 | 2.59341 | - | Yes |
| Effect size | 0.826785 | 0.826785 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 69.2112 | 69.2112 | - | Yes |
| Test statistic for each model term | 37.5609, 12.3813 | 37.5609, 12.3813 | - | Yes |
| Number of observations | 32 | 32 | - | Yes |
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -3.87783 | -3.87783 | - | Yes |
| p-value | 1.11965e-06 | 1.11965e-06 | - | Yes |
| Confidence interval | -5.62189, -2.13377 | -5.62189, -2.13377 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 2.59341 | 2.59341 | - | Yes |
| Effect size | 0.826785 | 0.826785 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 69.2112 | 69.2112 | - | Yes |
| Test statistic for each model term | 37.5609, 12.3813 | 37.5609, 12.3813 | - | Yes |
| Number of observations | 32 | 32 | - | Yes |
carData::Salaries
- Source: Published dataset distributed with an R package
- Licence: carData (GPL)
- Reference: Fox & Weisberg (2019), An R Companion to Applied Regression, 3rd ed.; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 12831.5 | 12831.5 | - | Yes |
| p-value | 0.00211898 | 0.00211898 | - | Yes |
| Confidence interval | 4677.02, 20986.10 | 4677.02, 20986.10 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 22554.1 | 22554.1 | - | Yes |
| Effect size | 0.452528 | 0.452528 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 64.6383 | 64.6383 | - | Yes |
| Test statistic for each model term | 69.47160, 38.31290, 4.91474, 5.06457 | 69.47160, 38.31290, 4.91474, 5.06457 | - | Yes |
| Number of observations | 397 | 397 | - | Yes |
carData::Salaries
- Source: Published dataset distributed with an R package
- Licence: carData (GPL)
- Reference: Fox & Weisberg (2019), An R Companion to Applied Regression, 3rd ed.; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -45287.7 | -45287.7 | - | Yes |
| p-value | 1.43605e-23 | 1.43605e-23 | - | Yes |
| Confidence interval | -53617.2, -36958.2 | -53617.2, -36958.2 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 22554.1 | 22554.1 | - | Yes |
| Effect size | 0.452528 | 0.452528 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 64.6383 | 64.6383 | - | Yes |
| Test statistic for each model term | 69.47160, 38.31290, 4.91474, 5.06457 | 69.47160, 38.31290, 4.91474, 5.06457 | - | Yes |
| Number of observations | 397 | 397 | - | Yes |
carData::Prestige
- Source: Published dataset distributed with an R package
- Licence: carData (GPL)
- Reference: Fox & Weisberg (2019), An R Companion to Applied Regression, 3rd ed.; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 6.03897 | 6.03897 | - | Yes |
| p-value | 0.121749 | 0.121749 | - | Yes |
| Confidence interval | -1.63984, 13.71780 | -1.63984, 13.71780 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 7.09481 | 7.09481 | - | Yes |
| Effect size | 0.834857 | 0.834857 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 117.537 | 117.537 | - | Yes |
| Test statistic for each model term | 5.87213, 32.88820, 21.03390 | 5.87213, 32.88820, 21.03390 | - | Yes |
| Number of observations | 98 | 98 | - | Yes |
datasets::anscombe
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Anscombe (1973), The American Statistician 27(1):17-21; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 0.500091 | 0.500091 | - | Yes |
| p-value | 0.00216963 | 0.00216963 | - | Yes |
| Confidence interval | 0.233370, 0.766812 | 0.233370, 0.766812 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 1.2366 | 1.2366 | - | Yes |
| Effect size | 0.666542 | 0.666542 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 17.9899 | 17.9899 | - | Yes |
| Test statistic for each model term | 17.9899 | 17.9899 | - | Yes |
| Number of observations | 11 | 11 | - | Yes |
datasets::longley
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Longley (1967), Journal of the American Statistical Association 62:819-841; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 0.0150619 | 0.0150619 | - | Yes |
| p-value | 0.863141 | 0.863141 | - | Yes |
| Confidence interval | -0.177029, 0.207153 | -0.177029, 0.207153 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 0.304854 | 0.304854 | - | Yes |
| Effect size | 0.995479 | 0.995479 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 330.285 | 330.285 | - | Yes |
| Test statistic for each model term | 0.0314623, 1.1438700, 17.1100000, 23.2515000, 0.0510991, 16.1274000 | 0.0314623, 1.1438700, 17.1100000, 23.2515000, 0.0510991, 16.1274000 | - | Yes |
| Number of observations | 16 | 16 | - | Yes |
datasets::swiss
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Mosteller & Tukey (1977), Data Analysis and Regression; independent computation
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -0.172114 | -0.172114 | - | Yes |
| p-value | 0.0187272 | 0.0187272 | - | Yes |
| Confidence interval | -0.3140960, -0.0301323 | -0.3140960, -0.0301323 | - | Yes |
| Residual standard error (the typical size of a prediction error, in the outcome's units) | 7.16537 | 7.16537 | - | Yes |
| Effect size | 0.706735 | 0.706735 | - | Yes |
| F statistic for the whole model (all predictors against an intercept-only model) | 19.7611 | 19.7611 | - | Yes |
| Test statistic for each model term | 5.99340, 1.03280, 22.64320, 8.72002, 7.96125 | 5.99340, 1.03280, 22.64320, 8.72002, 7.96125 | - | Yes |
| Number of observations | 47 | 47 | - | Yes |
| Generalized variance inflation factor (GVIF) for each model term, a check for overlap (collinearity) among the predictors | 2.28413, 3.67542, 2.77494, 1.93716, 1.10754 | 2.28413, 3.67542, 2.77494, 1.93716, 1.10754 | - | Yes |
survey::apiclus1 (one-stage cluster sample of school districts)
- Source: Published dataset distributed with an R package
- Licence: survey (GPL-2 | GPL-3)
- Reference: Lumley T, "A survey analysis example" (survey package vignette) for the published coefficients; independent computation with the survey package
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -0.508797 | -0.508797 | -0.5088 | Yes |
| Test statistic | -1.5611 | -1.5611 | -1.561 | Yes |
| p-value | 0.144474 | 0.144474 | 0.144 | Yes |
| Degrees of freedom | 12 | 12 | - | Yes |
| Confidence interval | -1.218920, 0.201328 | -1.218920, 0.201328 | - | Yes |
| Test statistic for the term, predictor 1 | 2.43702 | 2.43702 | - | Yes |
| p-value for the term, predictor 1 | 0.144474 | 0.144474 | - | Yes |
| Test statistic for the term, predictor 2 | 108.632 | 108.632 | - | Yes |
| p-value for the term, predictor 2 | 2.28562e-07 | 2.28562e-07 | - | Yes |
| Design degrees of freedom (PSUs minus strata) | 14 | 14 | - | Yes |
survey::apistrat (stratified by school type, unequal strata)
- Source: Published dataset distributed with an R package
- Licence: survey (GPL-2 | GPL-3)
- Reference: independent computation with the survey package
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -0.480587 | -0.480587 | - | Yes |
| Test statistic | -1.22607 | -1.22607 | - | Yes |
| p-value | 0.221659 | 0.221659 | - | Yes |
| Degrees of freedom | 194 | 194 | - | Yes |
| Confidence interval | -1.25366, 0.29249 | -1.25366, 0.29249 | - | Yes |
| Test statistic for the term, predictor 1 | 1.50325 | 1.50325 | - | Yes |
| p-value for the term, predictor 1 | 0.221659 | 0.221659 | - | Yes |
| Test statistic for the term, predictor 2 | 122.408 | 122.408 | - | Yes |
| p-value for the term, predictor 2 | 2.25413e-22 | 2.25413e-22 | - | Yes |
| Test statistic for the term, predictor 3 | 0.329493 | 0.329493 | - | Yes |
| p-value for the term, predictor 3 | 0.566623 | 0.566623 | - | Yes |
survey::apiclus1 (26 rows missing parental education: a complete-case domain)
- Source: Published dataset distributed with an R package
- Licence: survey (GPL-2 | GPL-3)
- Reference: independent computation with the survey package
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 17.6361 | 17.6361 | - | Yes |
| Test statistic | 1.19161 | 1.19161 | - | Yes |
| p-value | 0.25645 | 0.25645 | - | Yes |
| Degrees of freedom | 12 | 12 | - | Yes |
| Confidence interval | -14.6110, 49.8832 | -14.6110, 49.8832 | - | Yes |
| Test statistic for the term, predictor 1 | 1.41992 | 1.41992 | - | Yes |
| p-value for the term, predictor 1 | 0.25645 | 0.25645 | - | Yes |
| Test statistic for the term, predictor 2 | 84.5343 | 84.5343 | - | Yes |
| p-value for the term, predictor 2 | 8.8069e-07 | 8.8069e-07 | - | Yes |
| Number of observations | 157 | 157 | - | Yes |
survey::apistrat (sampling weights only)
- Source: Published dataset distributed with an R package
- Licence: survey (GPL-2 | GPL-3)
- Reference: independent computation with the survey package
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -0.505726 | -0.505726 | - | Yes |
| Test statistic | -1.28624 | -1.28624 | - | Yes |
| p-value | 0.199867 | 0.199867 | - | Yes |
| Degrees of freedom | 197 | 197 | - | Yes |
| Confidence interval | -1.281110, 0.269656 | -1.281110, 0.269656 | - | Yes |
| Test statistic for the term, predictor 1 | 1.65443 | 1.65443 | - | Yes |
| p-value for the term, predictor 1 | 0.199867 | 0.199867 | - | Yes |
| Test statistic for the term, predictor 2 | 120.191 | 120.191 | - | Yes |
| p-value for the term, predictor 2 | 3.85632e-22 | 3.85632e-22 | - | Yes |
References
- R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.