Outlier Exclusion Log

"We excluded obvious outliers" is not enough for a reviewer. Licklider's Outlier Exclusion Log keeps the exclusion criterion, the exclusion action, and any justification-sensitive review context attached to the figure so that exclusions are disclosed deliberately rather than reconstructed from memory.

STEP 1 - The Pitfall: Undocumented Exclusions

Outlier handling is one of the highest-risk decision points in analysis. The choice of whether to keep, winsorize, or exclude an observation can materially change the reported result, yet exclusion decisions are often described only vaguely in manuscripts.

• No criterion, no reproducibility. "Obvious outliers were removed" does not tell the reader what rule was actually used.
• Post-hoc criteria raise risk. Changing the exclusion rule after seeing the data can reshape both p-values and effect sizes.
• Overrides need explanation. Manual-review decisions and explicit manual overrides are especially important to surface because they are easy to forget and hard for reviewers to infer.

STEP 2 - Journal Requirement

Journal reporting standards consistently expect criteria for data inclusion and exclusion to be disclosed. Reviewers asking whether exclusions were pre-specified are asking a standard reproducibility question.

Licklider helps by keeping the exclusion record available during export review and by surfacing when the resulting figure still needs explicit disclosure.

STEP 3 - Licklider's Solution

Input

• Any outlier exclusion action applied through the Licklider interface: rule-based exclusion (IQR-based threshold with a declared multiplier), manual-review mode (IQR-based detection with a required justification statement), or manual override for selected rows with a recorded decision and justification.
• The declared exclusion criterion, when one is required for the selected action.
• Current figure state, so exclusion-sensitive results can be reviewed before export.

Output

• Exclusion record. The figure review context shows the active exclusion criterion, affected rows or observations, and whether overrides were involved.
• Exclusion summary. Export-facing text summarizes the active criterion, manual justification, and any available group-level impact derived for the current figure. Group-level excluded or winsorized counts are included when they can be reconstructed reliably from the persisted figure context.
• Disclosure support. The exclusion record feeds related Methods, export disclosure, and response-letter drafts so the same exclusion context is reused consistently.

Guard

STEP 4 - Draft Output (Draft / Needs review)

All generated text is marked as Draft and requires researcher review before submission because statistical language carries responsibility.

Example draft: Rule-based exclusion

[Draft / Needs review] Outliers were defined before the primary analysis
using the declared exclusion criterion for this figure. Outlier handling
used IQR-based detection with the recorded multiplier for this figure.
Review whether the exclusion rule, any available group-level impact, and any
related sensitivity analysis should be described more explicitly.

Example draft: Manual-review mode (IQR with justification)

[Draft / Needs review] Outlier detection used an IQR-based threshold.
Manual-review mode was selected, and a justification for the exclusion
decision was recorded. Confirm that the exclusion criterion, affected
observation counts, and the stated justification should appear in the
manuscript-facing Methods text or supplementary reporting.

Example draft: Manual override

[Draft / Needs review] Manual override was applied to selected rows after
outlier review. Confirm that the row-level override decision, the recorded
justification, and any resulting preprocessing disclosure are described
consistently in the Methods text, export disclosure, and reviewer-facing
response.

Related Features