Bootstrap Confidence Interval

When asymptotic approximation breaks down in small samples, Bootstrap CI delivers the precision that papers require. Licklider's Statistical Audit displays Bootstrap CI and parametric CI side-by-side — flagging divergence when it exceeds the acceptable threshold so the researcher can act before a reviewer requests it.

STEP 1 — The Pitfall: When Parametric CI Fails

The 95% CI reported by most statistical software depends on the Central Limit Theorem and assumes that the sampling distribution of the estimate is approximately normal. This assumption holds well when n is large and the data are roughly symmetric. In life-science research, however, these conditions frequently fail:

• Small n per group (n < 15 per arm): the CLT has not yet converged; t-distribution approximation accuracy declines.
• Right-skewed distributions: survival times, cytokine concentrations, tumor volumes — asymptotic CI underestimates the upper tail.
• Outlier influence: a single high-leverage point pulls the estimated mean and standard deviation, producing a CI that reflects the outlier rather than the bulk distribution.

A representative failure: n = 8 per group t-test run with asymptotic CI. Simulation shows that the nominal 95% CI achieves only ~88% actual coverage — meaning the reported CI misses the true parameter value 12% of the time instead of the stated 5%.

STEP 2 — Journal Requirement

Nature Methods, eLife, PLOS Biology, and Psychological Science all require that the method used to compute confidence intervals is explicitly stated in the Methods section. For small samples or non-normal distributions, reviewers at these journals increasingly request Bootstrap or permutation-based intervals rather than accepting asymptotic approximations without justification.

Specific requirements reviewers enforce:

• State whether CI is parametric (t-distribution-based), Bootstrap percentile, BCa (bias-corrected and accelerated), or another method.
• For n < 30 per group: justify the choice of parametric CI, or provide Bootstrap CI as the primary or supplementary estimate.
• Report the number of Bootstrap resamples if Bootstrap CI is used (typically B = 10,000).

STEP 3 — Licklider's Solution

Input

• The dataset already uploaded and the group variable declared in the Data Contract
• The test or estimation method selected (t-test, linear regression, etc.) — no additional setup required
• Optional: number of Bootstrap resamples (default B = 10,000; high-precision mode B = 50,000)

Output

• Side-by-side display panel: parametric CI lower/upper bounds alongside Bootstrap percentile CI (and BCa CI)
• Divergence indicator: the width difference between the two CIs displayed as a numeric value and a visual bar; indicator turns red when divergence exceeds the threshold (default: > 15% of parametric width)
• Auto-generated disclosure text ready to paste into Methods (see Step 4)

Guard

Licklider escalates through three tiers based on detected divergence:

• Advisory (divergence < threshold): Bootstrap CI displayed alongside parametric CI as supplementary information. No block. A note is auto-appended to the export package stating both values.
• Warning (divergence ≥ threshold, OR n < 30 with skewness > 1.5): Banner displayed: “Parametric CI may be inaccurate. Using Bootstrap CI as the primary estimate is strongly recommended.” Export proceeds but the divergence note is prominently included in the output.
• Block (divergence > 30% of parametric width, AND n < 15): Export is blocked. Message: “The divergence between Bootstrap CI and parametric CI is too large to report the parametric CI without qualification. Please switch to Bootstrap CI as the primary estimate or increase sample size.”

STEP 4 — Draft Output (Draft / Needs review)

Pattern A — Parametric and Bootstrap CI in agreement:
Confidence intervals were computed using a two-sample t-test (parametric,
t-distribution-based). Bootstrap percentile CIs (B = 10,000 resamples) were
calculated in parallel and showed agreement with parametric CIs (divergence < 8%
of parametric width), confirming that asymptotic approximation is appropriate
for this dataset.

Pattern B — Bootstrap CI used as primary estimate (small n / skewed data):
Due to small sample size (n = 9 per group) and right-skewed distribution
(skewness = 2.1), Bootstrap percentile CIs (B = 10,000 resamples) were used
as the primary estimate. Parametric CI (t-distribution-based) diverged by 22%
from the Bootstrap CI, exceeding the acceptable threshold. All reported CIs
are Bootstrap percentile CIs unless otherwise noted.

Related Features