Power & Sample Size Justification

Underpowered studies are not just a waste of resources —they are a source of false negatives that accumulate in the literature, mislead meta-analyses, and fail to translate to clinical practice. Most journals now require explicit power and sample size justification. Most researchers still provide it only when forced to.

STEP 1 —The Pitfall

Sample sizes in many biology experiments are determined by convention (n=3, n=6, n=10) rather than by power calculation. When power justification is required, researchers often work backwards: they run the experiment, observe the result, then calculate what power the study had at the observed effect size —a circular and misleading practice called post-hoc power calculation.

The problem with post-hoc power is that it always equals approximately 50% when the result is non-significant at p~.05, regardless of the true effect size. It provides no meaningful information and actively misleads interpretation of null results.

[Image placeholder: Two panels. Left: a priori power curve showing required n vs. expected effect size for alpha=0.05 and 80% power. Right: post-hoc power paradox graph showing that observed power ~50% when p ~0.05 regardless of effect size.]

A priori power calculation (left) is meaningful. Post-hoc power (right) is a mathematical tautology and should not be interpreted.

STEP 2 —Journal Requirement

ARRIVE 2.0 (animal research), CONSORT (clinical trials), and Nature's reporting checklist all require a priori sample size justification or, when not feasible, explicit acknowledgment of the study's limitations with respect to power. Journal of Physiology and similar journals require the power calculation in Methods or as a statement that the study is preliminary/exploratory.

STEP 3 —Licklider Solution

Input

Observed effect size (from Effect Size Reporting dimension)
Declared alpha level (default 0.05)
Target power (default 0.80)
Test type (from analysis configuration)
Number of groups and tails (from One-Sided/Two-Sided Disclosure)

Output

A priori required n: calculated from declared expected effect size (can be entered separately from observed)
Achieved power: calculated from observed n and observed effect size (clearly labeled as post-hoc / sensitivity estimate)
Required n displayed in the Validity Score panel alongside observed n
Shortfall flagged if observed n < required n for the declared effect size
Auto-generated Methods text for power justification

Guard

For confirmatory / publication-ready analyses, the absence of power justification text sets this Validity Score dimension to Warning. If observed n is less than 50% of the required n for a medium effect (Cohen's d = 0.5, eta² = 0.06), the dimension is set to Fail. Exploratory analyses declared via Outcome Type Lock receive advisory text suggesting power analysis for future confirmatory studies.

[Image placeholder: Licklider power panel showing: Expected effect size input, alpha input, target power slider, Required n output, and Achieved power with a clear label that this is a sensitivity estimate —not a validation of the study design.]

Power justification panel: a priori required n alongside the observed n, with clear labeling of post-hoc estimates.

STEP 4 —Draft Output (Draft / Needs review)

Sample size was determined a priori using a power analysis (G*Power equivalent)
for a two-sample t-test with alpha = 0.05, power = 0.80, and an expected effect
size of Cohen's d = 0.8 (large), yielding a required n = 26 per group.
The study enrolled n = 14 per group; this sample is sufficient to detect
effects of d >= 1.1 at 80% power.