Sample Size Justification for Regression

Adding predictors to a regression model with insufficient sample size does not improve the analysis —it overfits the model to the training data, producing unreliable coefficients, inflated Rﾂｲ, and confidence intervals that are too narrow. The rule of thumb is n >= 10 observations per predictor (EPV >= 10 for logistic regression). Below this threshold, the model is likely to generalize poorly.

STEP 1 —The Pitfall

Overfitted regression models have several distinguishing signatures: regression coefficients that are implausibly large or in unexpected directions; standard errors that are very wide or very narrow; Rﾂｲ values that decrease substantially in cross-validation. These are all consequences of the model fitting the noise in the training data rather than the signal.

The problem is common in clinical research where many potential predictors exist but sample sizes are limited. A study with n=40 and 8 covariates has an effective EPV of 5 —substantially below the recommended minimum. The resulting model estimates are unreliable even if the overall F-test is significant.

[Image placeholder: Two regression models. Left: well-powered model (n=200, p=5 predictors, EPV=40). Right: overfitted model (n=40, p=8 predictors, EPV=5). Shows: coefficient estimates, 95% CIs (much wider in overfitted model), and a cross-validation Rﾂｲ comparison (good model: Rﾂｲ=0.42, cross-val Rﾂｲ=0.38; overfitted: Rﾂｲ=0.61, cross-val Rﾂｲ=0.12).]

Well-powered model (left) vs. overfitted model (right): cross-validation reveals the collapse in predictive accuracy when EPV is insufficient.

STEP 2 —Journal Requirement

Many journals require EPV to be reported for logistic regression models. The BMJ requires sample size justification for all primary regression analyses. Cochrane guidelines recommend EPV >= 10 for logistic regression. For OLS regression, the analogous recommendation is n/p >= 10 where p is the number of predictors. Studies below these thresholds should be clearly labeled as preliminary or exploratory.

STEP 3 —Licklider Solution

Input

Number of predictors in the regression model (p)
Total sample size (n) and, for logistic regression, number of events (n_events)
Model type: OLS / logistic / Cox proportional hazards / mixed model

Output

For OLS: n/p ratio calculated and displayed. n/p < 10 xWarning; n/p < 5 xFail
For logistic regression: EPV = n_events / p. EPV < 10 xWarning; EPV < 5 xFail
For Cox regression: events-per-variable (EPV) using event count (not total n)
Overfitting risk assessment: simulation-based optimism estimate (bias in Rﾂｲ or C-statistic from overfitting)
Recommendations: reduce predictors, use regularization (LASSO / ridge regression), collect more data, or label as preliminary
Required n for current model: displayed to show how far below the threshold the study falls

Guard

For confirmatory regression analyses with EPV or n/p < 5, the guard issues a Model Complexity Error —a blocking error for publication-ready export. Resolution requires either: reducing the number of predictors to meet the EPV threshold, applying regularization (with appropriate disclosure), or declaring the analysis as exploratory/preliminary. EPV between 5—0 triggers a Warning with mandatory disclosure.

[Image placeholder: Licklider sample size justification panel for logistic regression showing: n=120, events=34, predictors=8, EPV=4.3 (FAIL —below threshold of 5), required events for EPV=10: 80, shortfall: 46 events. Options: "Reduce to 3 predictors (EPV=11.3)" or "Apply LASSO regularization" or "Declare as exploratory."]

Sample size justification panel: EPV calculation, Fail badge, required events, and resolution options.

STEP 4 —Draft Output (Draft / Needs review)

The logistic regression model included 4 predictors (age, sex,
baseline score, treatment group) in a dataset with 34 events
(events-per-variable = 8.5). This is below the commonly recommended
EPV >= 10 threshold (Harrell et al., 1996). The model should therefore
be interpreted as exploratory. Internal validation using bootstrap
resampling (1,000 iterations) indicated a calibration optimism of 0.08
in the C-statistic, suggesting moderate overfitting. An optimism-corrected
C-statistic of 0.71 is reported alongside the apparent C-statistic of 0.79.