Sensitivity Analysis Engine

Sensitivity analysis asks: if I had made a slightly different analytical choice, would my conclusion change? The answer to this question is more informative than the primary result alone, because it reveals whether the conclusion is a general finding or an artifact of one specific decision.

STEP 1 —The Pitfall

Most published analyses represent a single analytical path through a dataset. That path was selected —sometimes consciously, sometimes by software defaults —from among many defensible alternatives. The chosen path may have been the best choice, or it may have been the choice that produced the most favorable result. Without sensitivity analysis, neither the author nor the reader can distinguish between these cases.

Sensitivity analysis is recommended by major reporting guidelines but rarely performed in practice because running parallel analyses manually is time-consuming and the results are inconvenient when the primary result is fragile.

[Image placeholder: A grid of p-values from the same dataset analyzed with 12 combinations of: 2 outlier handling methods x2 normality assumptions x3 effect size estimators. Color-coded: green (p<0.05, significant), red (p>0.05, not significant). Shows which cells are green and which are red —visualizing the fragility of the result.]

Sensitivity grid: 12 analytical variations of the same dataset. If most cells are green, the result is robust. If they vary, the result is contingent on specific choices.

STEP 2 —Journal Requirement

CONSORT requires sensitivity analyses for primary outcomes in clinical trials. STROBE recommends sensitivity analyses for observational studies. Nature Neuroscience and PLOS Biology increasingly expect "robustness checks" as supplementary figures or in-text statements. The Transparency and Openness Promotion (TOP) guidelines list sensitivity analysis as a transparency requirement.

STEP 3 —Licklider Solution

Input

Primary analysis configuration (test, outlier handling, normality assumption)
Sensitivity dimensions to explore (user-selectable or auto-recommended)
Conclusion definition: significance threshold, effect direction, minimum meaningful effect size

Output

Parallel analysis results across: outlier inclusion/exclusion, parametric test variants, bootstrap vs. analytical CI (full non-parametric branch parity is not guaranteed — see Non-Parametric Alternatives)
Sensitivity grid display: each cell shows the result under a specific combination of choices
Robustness summary: "Conclusion stable across X of Y tested configurations"
Auto-generated supplementary sensitivity table for Methods or Supplementary Information

Guard

For confirmatory analyses, if the primary conclusion changes sign or significance in more than 30% of tested configurations, a Robustness Warning is issued and must be acknowledged before export. The warning does not block export —but it is recorded in the analysis log and disclosed in the auto-generated Methods text.

[Image placeholder: Licklider sensitivity analysis panel showing: primary result (top row), sensitivity variations (below), and a summary badge showing "Robust: 9/12 configurations consistent with primary conclusion." The 3 inconsistent configurations are highlighted in orange.]

Sensitivity analysis panel: primary result with parallel variations and a robustness summary badge.

STEP 4 —Draft Output (Draft / Needs review)

Sensitivity analyses were conducted to evaluate the robustness of the
primary finding. The primary analysis (Welch's t-test, ROUT outlier
exclusion Q=1%) was replicated using: (1) no outlier exclusion,
(2) parametric t-test with Welch's correction, and (3) bootstrap CI
estimation. The primary conclusion (Group A > Group B) was consistent
across all four analytical configurations (p range: 0.009—.038).
Full sensitivity results are provided in Supplementary Table 2.