Paired vs. Unpaired Guard

Using an unpaired test on paired data is not a conservative choice —it is an incorrect choice. It ignores the correlation between paired measurements, loses statistical power, and produces confidence intervals that do not reflect the actual within-subject variability. The result may be non-significant not because the effect is absent, but because the wrong test was used.

STEP 1 —The Pitfall

Paired data structures are common: before/after measurements, matched controls, crossover designs, within-subject repeated measurements. When the dataset arrives as a flat table, the pairing structure is encoded in the subject ID column —not always visible to a researcher who has not explicitly declared the structure.

The consequence of ignoring pairing is a loss of power proportional to the within-subject correlation. For a correlation of 0.7 (typical for within-subject measurements), the unpaired test may require twice as many subjects as the paired test to reach the same power.

[Image placeholder: Paired data example —before/after scatter with individual subject trajectories drawn as connecting lines. Left: unpaired t-test ignores the lines (high variance, p=0.18). Right: paired t-test uses the differences (low variance, p=0.003). Same data, radically different results.]

Same data, two tests: unpaired (p=0.18) vs. paired (p=0.003). The pairing structure was present in the data but was not used.

STEP 2 —Journal Requirement

ARRIVE 2.0 and most experimental design guidelines require that the test type (paired / unpaired) be stated explicitly. Paired designs must use paired tests —this is not a preference but a statistical requirement. Journals with statistical review will flag an unpaired test on a crossover or within-subject design as a mandatory revision.

STEP 3 — What Licklider Currently Provides

What the current product does

Card 1 asks the user to explicitly choose whether the current comparison should be treated as paired or independent (paired_mode).
The product treats this as a figure-safety decision. It is not an all-knowing detector that automatically infers pairing from subject IDs.
The pairing decision feeds into downstream comparison logic, method validation, and export-facing guidance.
Unresolved pairing state can affect whether claim-bearing comparison output is treated as ready to export.
Manual method overrides are validated against the chosen pairing structure.

What the product does not do

The product does not automatically detect pairing structure from subject ID columns.
The product does not compute or display power comparisons between paired and unpaired approaches.
The product does not raise a "Validity Score Fail" based on pairing mismatch alone.
Subject and hierarchy fields provide context, but pairing requires an explicit user decision.

Known limitations

The current system asks for confirmation and then uses that answer in later logic. It does not prove pairing from raw data alone. For a fuller discussion of the pairing guard behavior, see Paired vs Unpaired Guard in the quality-checks docs.

[Image placeholder — not current product UI. Illustrative concept of a paired/unpaired guard dialog.]

Illustrative concept only. The current product asks for pairing confirmation via Card 1, not via this specific dialog layout.

STEP 4 — Draft Output (Draft / Needs review)

The following is an example of how a researcher might write a paired-design methods section, not output that Licklider generates automatically.

Pre- and post-treatment measurements were obtained from the same
subjects (n = 8 paired measurements per group). A paired t-test
was used to account for within-subject correlation.