Firth penalised logistic regression
Shown in the app as firth_logistic. Part of the method reference, under Regression.
When to use
Model a binary outcome when ordinary logistic regression breaks down: a predictor that perfectly (or almost perfectly) separates the outcome, or very few events per predictor. Firth's penalised likelihood returns FINITE odds ratios with profile-likelihood confidence intervals where the ordinary fit runs off to infinity.
What you provide
- Outcome: one two-level column.
- Predictors: one or more columns.
Options
Missing data
- Complete case (listwise deletion)
- Mean imputation (single)
- Median imputation (single)
Multiple comparisons
- Not adjustable for this method. Where it makes post-hoc comparisons, the procedure applies its own correction.
Confidence level
Every interval in the result is reported at the level you choose:
- 90%
- 95%
- 99%
Related methods
Other methods under Regression:
- Linear regression
- Logistic regression
- Poisson regression
- Negative-binomial regression
- Ordinal logistic regression
- Multinomial logistic regression
- Factors associated with outcome (continuous outcome)
- Factors associated with outcome (binary outcome)
Guides that discuss this method
Worked examples
Celeus's result on a published or illustrative dataset, next to an independent computation and, where one exists, a published value. The data themselves are not reproduced here.
logistf::sex2
- Source: Published dataset distributed with an R package
- Licence: logistf (GPL)
- Reference: Heinze G, Schemper M (2002), Statistics in Medicine 21:2409-2419 (the worked example); data from Cytel Inc., LogXact 9 user manual. The dataset and the model are the paper's; the numbers below are this engine's, checked against logistf run independently - not against the paper's printed table.
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 3.09601 | 3.09601 | - | Yes |
| p-value | 0.00495187 | 0.00495187 | - | Yes |
| Confidence interval | 0.774568, 8.030290 | 0.774568, 8.030290 | - | Yes |
| Number of observations | 239 | 239 | - | Yes |
datasets::mtcars (am removed in rows 3, 7 and 11)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); the same logistf call on the 29 rows with an outcome
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -2.98716 | -2.98716 | - | Yes |
| p-value | 2.69437e-05 | 2.69437e-05 | - | Yes |
| Confidence interval | -6.09919, -1.20355 | -6.09919, -1.20355 | - | Yes |
| Effect size | 0.0504305 | 0.0504305 | - | Yes |
| Effect size confidence interval | 0.00224469, 0.30012600 | 0.00224469, 0.30012600 | - | Yes |
| Test statistic for each model term | 17.6221 | 17.6221 | - | Yes |
| Whole-model likelihood-ratio statistic | 17.6221 | 17.6221 | - | Yes |
| Number of observations | 29 | 29 | - | Yes |
datasets::mtcars (rows 1-15)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); the same logistf call
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -6.49848 | -6.49848 | - | Yes |
| p-value | 0.00341607 | 0.00341607 | - | Yes |
| Confidence interval | -25.55260, -1.45821 | -25.55260, -1.45821 | - | Yes |
| Effect size | 0.00150572 | 0.00150572 | - | Yes |
| Effect size confidence interval | 7.99219e-12, 2.32653e-01 | 7.99219e-12, 2.32653e-01 | - | Yes |
| Test statistic for each model term | 8.57075 | 8.57075 | - | Yes |
| Whole-model likelihood-ratio statistic | 8.57075 | 8.57075 | - | Yes |
| Number of observations | 15 | 15 | - | Yes |
datasets::mtcars (rows 1-15)
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); the same logistf call
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | -6.49848 | -6.49848 | - | Yes |
| p-value | 0.00341607 | 0.00341607 | - | Yes |
| Confidence interval | -21.11790, -2.01253 | -21.11790, -2.01253 | - | Yes |
| Effect size | 0.00150572 | 0.00150572 | - | Yes |
| Effect size confidence interval | 6.73945e-10, 1.33650e-01 | 6.73945e-10, 1.33650e-01 | - | Yes |
| Test statistic for each model term | 8.57075 | 8.57075 | - | Yes |
| Whole-model likelihood-ratio statistic | 8.57075 | 8.57075 | - | Yes |
| Number of observations | 15 | 15 | - | Yes |
datasets::mtcars
- Source: Published dataset distributed with an R package
- Licence: base R (GPL)
- Reference: Henderson & Velleman (1981); the same logistf call
| Quantity | Celeus | Independent computation | Published value | Agrees |
|---|---|---|---|---|
| Estimate | 4.08823 | 4.08823 | - | Yes |
| p-value | 1.06197e-06 | 1.06197e-06 | - | Yes |
| Confidence interval | 1.88229, 9.00826 | 1.88229, 9.00826 | - | Yes |
| Effect size | 59.6344 | 59.6344 | - | Yes |
| Effect size confidence interval | 6.56854, 8170.32000 | 6.56854, 8170.32000 | - | Yes |
| Test statistic for each model term | 23.8124 | 23.8124 | - | Yes |
| Whole-model likelihood-ratio statistic | 23.8124 | 23.8124 | - | Yes |
| Number of observations | 32 | 32 | - | Yes |
References
- Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika 80: 27-38. doi:10.1093/biomet/80.1.27 doi:10.1093/biomet/80.1.27
- Heinze, G., Schemper, M. (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine 21: 2409-2419. doi:10.1002/sim.1047 doi:10.1002/sim.1047