Logistic Regression and AUC/ROC

Logistic regression models the probability of a binary outcome — one that takes one of two values, such as responder/non-responder, alive/dead, or positive/negative. Where linear regression predicts a continuous value, logistic regression predicts a probability between 0 and 1.

When to use logistic regression

Logistic regression is appropriate when:

• The outcome variable is binary
• You want to estimate the probability of the outcome given one or more predictors
• You want to quantify the association between each predictor and the outcome as an odds ratio

If the outcome is continuous, use Linear Regression (OLS) instead.

How to request it

Describe the analysis in the Chat. For example:

• "Run a logistic regression predicting response from dose and age"
• "Model survival (yes/no) as a function of treatment group"
• "Show the ROC curve for this classifier"

What the results include

In the current logistic-regression page, Licklider returns three output groups: a coefficient table, model-level fit statistics, and a ROC visualization. These outputs answer different questions and should be read together rather than as interchangeable scores.

Coefficient table

One row per predictor, including the intercept. Each row shows:

• Coefficient — the log-odds associated with a one-unit increase in the predictor
• Odds ratio — the exponentiated coefficient. An odds ratio greater than 1 indicates that the predictor is associated with higher odds of the outcome.
• Standard error
• z-statistic
• p-value
• 95% confidence interval on the coefficient

Model fit statistics

• Predicted probabilities are summarised at the model level through discrimination metrics and the ROC curve rather than as a single yes/no threshold decision.
• AUC (area under the ROC curve) — the probability that the model assigns a higher predicted probability to a randomly chosen positive case than to a randomly chosen negative case. AUC = 0.5 indicates no better than chance; AUC = 1.0 indicates perfect discrimination.
• Pseudo R² — a measure of overall model fit analogous to R² in linear regression
• AIC — the Akaike information criterion, useful for comparing models

Licklider reports AUC because it gives a threshold-free summary of discrimination, while Pseudo R² and AIC describe different aspects of model fit and model comparison rather than classification performance [2, 3].

Note on AUC confidence intervals: The AUC is reported as a point estimate. Confidence intervals for AUC are not currently available in the current output, so the displayed AUC should not be read as a complete statement of uncertainty.

ROC curve

The receiver operating characteristic (ROC) curve plots the true positive rate against the false positive rate across all possible classification thresholds. A curve closer to the upper left corner indicates better discrimination. The diagonal line represents a classifier with no discriminating ability (AUC = 0.5).

Licklider generates the ROC curve automatically when a logistic regression is run.

That automation helps readers inspect model discrimination without having to choose a single operating threshold first. It does not, by itself, show calibration, clinical utility, or whether the apparent performance is inflated by overfitting or data leakage [2, 3].

Handling perfect separation

When one or more predictors perfectly separate the two classes — that is, the predictor alone is sufficient to classify every observation correctly — standard logistic regression cannot converge and maximum-likelihood estimates can diverge toward infinite coefficients. In this case, Licklider uses a regularized fitting method that allows the model to complete and keeps the estimates finite, then adds a warning to the output [1].

The reason for this fallback is not to "rescue" a perfect classifier claim. It is to prevent a mathematically unstable fit from failing silently or returning unusable coefficients. When the warning appears, interpret very large odds ratios and apparently excellent discrimination cautiously: they may reflect sparse data or near-deterministic structure rather than a transportable signal.

Assumptions

Logistic regression assumes:

• The outcome is binary
• Observations are independent
• The log-odds of the outcome is a linear function of the predictors (log-odds linearity)

Licklider can recognize that the outcome is binary and it automatically generates the ROC curve when the model is fit, but it cannot reliably infer every structural risk from the uploaded table alone. In particular, Licklider cannot automatically detect whether rows are clustered within subjects, litters, plates, or sites; whether repeated measurements were entered as if they were independent; whether predictor engineering leaked outcome information into the model; or whether a continuous predictor needs transformation to satisfy log-odds linearity.

Those limits matter because they can make the model look more convincing than it really is. Independence violations and clustered data can produce standard errors and p-values that are too optimistic. Data leakage can inflate AUC. Unmodelled nonlinearity can distort odds ratios and make the fitted probabilities systematically misleading even when the headline AUC looks acceptable. If those risks are plausible in your study, use an explicitly clustered model or review the design and feature construction before treating the logistic-regression output as claim-bearing.

What this page does not cover

• Continuous outcome regression → see Linear Regression (OLS)
• Proportion or bounded outcome regression → see Proportion Data OLS Prevention
• Clustered binary data → see GLMM: Gaussian and Binomial
• ROC curve visualization → see ROC Curve
• Calibration assessment, threshold selection, and external validation planning → treat these as separate evaluation tasks rather than as outputs implied by AUC alone

Design Rationale & References

Licklider's design choices

Licklider surfaces odds ratios alongside coefficients because non-specialist readers often need both the model-scale estimate and an interpretable association scale. It reports AUC together with the ROC curve because discrimination across thresholds is usually the first performance question readers ask, but AUC is shown as one summary of ranking performance rather than as a complete validation claim [2, 3]. When perfect separation occurs, Licklider switches to a regularized fit so the analysis can complete with finite estimates and an explicit warning instead of presenting a misleadingly stable maximum-likelihood result [1].

Methodological foundations

1. Heinze, G., & Schemper, M. (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine, 21(16), 2409-2419.

   → A standard reference on why separated logistic models break ordinary maximum-likelihood estimation and why penalized or regularized approaches are used instead.

2. Obuchowski, N. A. (2005). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 235(3), 769-771.

   → Clarifies what AUC does and does not summarize, which supports showing ROC/AUC as a discrimination measure rather than as a full statement of model quality.

3. Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Moons, K. G. M. (2010). Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology, 21(1), 128-138.

   → Shows why discrimination, calibration, and other evaluation dimensions should be separated rather than collapsed into a single score.

Implementation boundaries

• The current page documents the implemented output set: coefficient rows with odds ratios and confidence intervals, model-level statistics including AUC, Pseudo R², and AIC, and an automatically generated ROC curve.
• AUC confidence intervals are not currently returned in the page output.
• The page does not claim automatic calibration assessment, threshold optimization, or external validation.
• Licklider does not automatically detect every misuse of logistic regression. In particular, clustered observations, repeated measurements entered as independent rows, feature leakage, and log-odds nonlinearity can still require study-design judgment outside the current page.
• If your binary outcome is clustered by subject, site, litter, plate, or another grouping unit, this page is the wrong endpoint; use a mixed-model path such as GLMM: Gaussian and Binomial.