Bayes Factor Supplement

A p-value above 0.05 does not mean there is no effect. It means you failed to detect an effect at your chosen threshold. A Bayes Factor quantifies the evidence for and against the null hypothesis separately — making the distinction between “absence of evidence” and “evidence of absence” explicit and communicable.

STEP 1 — The Pitfall

The interpretation “p = 0.12, therefore there is no effect” is statistically invalid. With small n, p = 0.12 may reflect 50% power — the result is simply uninformative, not evidence of no effect. Similarly, p = 0.04 with n = 500 may reflect a trivially small effect that happens to clear the significance threshold.

The Bayes Factor (BF₁₀) directly addresses this: BF₁₀ = 3 means the data are 3 times more likely under H₁ than H₀. BF₁₀ = 0.1 means the data are 10 times more likely under H₀ than H₁ — actual evidence for the null. This distinction cannot be made from a p-value alone.

[Image placeholder: Bayes Factor evidence scale from BF=1/30 (strong evidence for null) to BF=30 (strong evidence for alternative), with common interpretation labels at 1/10, 1/3, 3, 10, 30. Current study's BF shown as a marker on the scale.]

Bayes Factor evidence scale: BF₁₀ > 3 = moderate evidence for H₁; BF₁₀ < 1/3 = moderate evidence for H₀; BF₁₀ ≈ 1 = uninformative data.

STEP 2 — Journal Requirement

Psychological Science, Nature Human Behaviour, and Cognition actively encourage or require Bayes Factors for null results. The JASP software team has published guidelines on BF reporting that are widely cited. eLife and Royal Society Open Science recommend BF for any study claiming a null result or comparing two positive models.

STEP 3 — Licklider Solution

Input

Primary analysis type (t-test, correlation, ANOVA) and its result
Prior selection: Cauchy (default, scale = 0.707), user-defined, or matched to a previous study's effect size
Prior sensitivity range: BF calculated across a range of prior scales to check robustness

Output

BF₁₀ and BF₀₁ (reciprocal) for the primary comparison
Evidence interpretation label from the Jeffreys scale
Prior sensitivity plot: BF across a range of Cauchy prior scales (0.3 to 2.0)
Comparison displayed alongside frequentist result in the figure legend panel
Interpretation guidance: “substantial evidence for H₁”, “anecdotal”, “uninformative”, etc.

Guard

The Bayes Factor Supplement is an advisory feature — it does not gate export. However, if the frequentist p-value is between 0.05 and 0.15 (borderline significant), the system issues an advisory suggesting BF calculation to determine whether the result is anecdotal evidence for H₁ or uninformative data. For declared null results (p > 0.1 in a confirmatory analysis), BF calculation is strongly recommended and the export package will note its absence.

[Image placeholder: Licklider BF panel showing: BF10 = 8.4 (moderate evidence for H1), prior sensitivity curve (BF stable across Cauchy scales 0.3-1.5, dropping below 3 only at very diffuse priors > 1.8), and combined frequentist+Bayesian legend: "t(24) = 4.11, p = 0.0004; BF10 = 8.4."]

Bayes Factor panel: BF₁₀ with interpretation label, prior sensitivity curve, and combined legend output.

STEP 4 — Draft Output (Draft / Needs review)

A Bayes Factor analysis was conducted to quantify the evidence for
the group difference (BayesFactor R package, Cauchy prior scale = 0.707).
BF10 = 8.4, indicating moderate evidence for H1 (Group A != Group B).
Prior sensitivity analysis confirmed that BF10 exceeded 3 across Cauchy
prior scales of 0.3-1.5 (BF10 range: 3.1-12.7), indicating robustness
to prior specification.