Outlier Sensitivity Report

A result that depends entirely on one data point is not a result —it is a single observation. The Outlier Sensitivity Report runs a jackknife leave-one-out analysis to identify which data points are driving the result and quantifies how much the conclusion changes when each point is removed. This is not the same as outlier exclusion; it is a diagnostic for conclusion fragility.

STEP 1 —The Pitfall

Small-sample experiments in biology are particularly vulnerable to individual-point sensitivity: a result with n=6 can be entirely driven by one extreme observation. A reviewer cannot detect this from a figure that shows only the group summary. The author may not be aware of it if they have not performed leave-one-out analysis.

The distinction between an outlier (a data point that violates a pre-declared criterion) and a high-leverage point (a data point that disproportionately influences the result) is important. A point can be high-leverage without meeting any formal outlier criterion. The Outlier Sensitivity Report detects both.

[Image placeholder: Strip chart with n=7 per group. One data point in Group A is highlighted in red. Below the chart: a sensitivity table showing the p-value when each data point is removed, one row per observation. The highlighted point's removal changes p=0.031 to p=0.18.]

Leave-one-out sensitivity: removing one data point (highlighted red) changes the primary result from significant to non-significant.

STEP 2 —Journal Requirement

ARRIVE 2.0 requires sensitivity analyses for animal studies with small n. Several biostatistics journals and statistical reviewers explicitly request leave-one-out or jackknife analysis when n < 10 per group. Cochrane Handbook recommends influence analysis as a standard sensitivity check for meta-analyses —the same principle applies to primary studies.

STEP 3 —Licklider Solution

Input

Primary analysis configuration (test, groups, outcome variable)
Influence threshold: flag if p-value changes by >ﾎ廃 (default: if any removal changes significance decision)

Output

Leave-one-out table: for each data point, the result when that point is removed —p-value, effect size, CI
High-leverage points identified: points whose removal changes significance or effect direction
Cook's distance equivalent for non-parametric tests (rank-based influence measure)
Visual overlay on figure: high-leverage points marked with sensitivity indicator
Robustness verdict: "Conclusion stable with all n>=N-1 configurations" or "Conclusion depends on [n] high-leverage points"

Guard

If any single data point's removal changes the significance decision and n < 12 per group, the guard issues an Outlier Sensitivity Warning —a non-blocking advisory that recommends disclosure and, where possible, replication. The warning is logged and included in the auto-generated disclosure text.

[Image placeholder: Licklider Outlier Sensitivity Report panel showing: full LOO table with color-coded rows (green = conclusion stable, red = conclusion changes), a robustness verdict "2 high-leverage points detected," and a recommended disclosure text for Methods.]

Outlier Sensitivity Report: LOO table with color-coded rows and robustness verdict. High-leverage points flagged for disclosure.

STEP 4 —Draft Output (Draft / Needs review)

Leave-one-out sensitivity analysis was performed to assess the influence
of individual data points on the primary conclusion. Removing any single
observation from Group A (n = 8) or Group B (n = 7) did not change the
significance of the primary comparison (all LOO p-values: 0.011—.038).
[Or: One data point in Group A was identified as high-leverage; its removal
changed the comparison from significant (p = 0.031) to non-significant
(p = 0.18). This point met inclusion criteria and was retained in the
primary analysis. Results with and without this observation are reported
in Supplementary Table 3.]