Outlier Exclusion Log

What the Outlier Exclusion Log records for each figure, where to find it, and what the entries mean.

When outlier exclusions are applied to a dataset, the details of every exclusion action are recorded in the Outlier Exclusion Log. Unlike the Preprocessing Audit Log — which shows the current state of the dataset — the Outlier Exclusion Log is captured at the time each figure is generated and remains fixed to that generation.

This page is about the figure-specific record of outlier handling. It does not introduce a new test result by itself. Instead, it shows which exclusion or winsorization decisions stood behind the figure so you can judge whether the reported result is credible for your study.

What the log contains

For each outlier action applied before a figure was generated, the log records:

  • Action type — whether rows were excluded, capped (winsorized), kept, or overridden manually
  • Criterion — the method used to identify outliers (IQR-based with the multiplier used, or manual with the written justification)
  • Columns — which columns the action applied to
  • Flagged row samples — the row IDs of observations that were flagged, up to a maximum of 10 rows per action
  • Manual overrides — if individual rows were manually included or excluded, each override is listed with the row ID, the decision made, and the written justification

When group information is available, the log also shows the number of excluded rows and winsorized cells broken down by group.

The log records row IDs and column names. It does not store the original values of the flagged observations.

This design keeps the log traceable to specific observations without turning the log into a second copy of the dataset. The row sample is meant to support review and disclosure, not replace the underlying data table.

Where to find it

Figure Inspector — Audit Trail tab Select a figure and open the Audit Trail tab in the Inspector. The Outlier exclusion log section shows all exclusion actions recorded at the time the figure was generated.

The log is specific to each figure generation. If the dataset's outlier settings are changed and the figure is regenerated, the new figure will have a different log reflecting the updated exclusions.

That figure-specific design is intentional. A claim about a figure should stay tied to the exact outlier handling used when that figure was generated, even if the dataset is reviewed or changed later.

Reading the log entries

Each entry in the log corresponds to one outlier action. The entry shows when the action was applied, the action type, the criterion, and the affected columns.

IQR-based criterion The entry shows the multiplier used. For example, a multiplier of 1.5 means observations were flagged if they fell below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR.

This is a common exploratory rule for flagging potential outliers. In Licklider, the log records which rule was used for the figure. It does not claim that the rule was automatically the right scientific choice for the study.

Manual criterion The entry shows the written justification provided when the criterion was declared.

Manual overrides Each individual override is listed separately with its row ID, the decision (exclude or keep), and the justification text entered at the time of the override.

This is recorded separately so later readers can distinguish a declared criterion from a case-by-case departure from that criterion.

Group counts When group information is available and exclusions were applied, the number of excluded rows and winsorized cells is shown for each group. This helps identify whether exclusions were distributed evenly across groups or concentrated in one.

Flagged row samples are capped so the log remains a readable audit surface rather than a full raw-data dump. The sample is there to make the exclusion decision inspectable, while the full dataset remains the source for deeper review.

Disclosure and export

When the Outlier Exclusion Log contains exclusions, the figure's disclosure includes a summary of what was excluded and why. This disclosure must be acknowledged before the figure can be used in a claim-bearing export.

The disclosure status is visible in the Inspector. If outlier exclusions are recorded but the disclosure is unresolved, the Inspector will indicate that confirmation is required.

Licklider records these actions automatically when the figure is generated and uses the resulting log to support disclosure before claim-bearing export.

This boundary is important: the log can show what was excluded, when, and under which declared rule, but it does not prove that the exclusion rule was scientifically justified. Licklider does not use this page to determine whether an extreme value is a measurement error, a valid rare observation, a signal from an important subgroup, or the result of an undisclosed researcher degree of freedom.

That limitation matters because outlier exclusion can change group balance, effect sizes, confidence intervals, and p-values. A recorded exclusion is therefore more transparent than an undisclosed one, but transparency is not the same as justification. Use the log as evidence for review and reporting, not as proof that the exclusion decision was correct.

If the reason an observation should be kept or excluded exists only in lab notes, assay history, instrument records, or metadata outside the uploaded table, the current log cannot detect that context automatically.

References

  1. Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
  2. Aguinis, H., Gottfredson, R. K., & Joo, H. (2013). Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16(2), 270-301. https://doi.org/10.1177/1094428112470848

These references support the interpretation of outlier flagging and handling as methodological choices that require justification. In Licklider, the Outlier Exclusion Log documents which choice was used for a given figure; it does not automatically validate that choice for you.

What this page does not cover