Assumption and Robustness Guard

Statistical analyses rest on assumptions about the data. When those assumptions are violated or unverified, the conclusions drawn from the analysis may not be reliable. Licklider evaluates a set of assumption checks automatically for every analysis and surfaces the results in the Inspector.

The assumption guard is not a single check. It is the combined result of several independent checks, each targeting a different assumption. When any check is unresolved, it appears in the Inspector as a requirement that must be addressed before claim-bearing output can be exported.

The goal is not to promise that software can certify every study design automatically. The goal is to make common assumption failures visible, require explicit acknowledgment when they matter, and prevent strong claim-bearing output from bypassing unresolved warnings.

Checks that run automatically

Assumption gate by chart type (LAYER1)

Normality and homoscedasticity checks are tied to chart type in the product. The table below lists every chart type in the assumption registry and what runs for it.

Out of scope chart types are not covered by the LAYER1 automatic Shapiro/Levene or residual normality gate in this release; specialized workflows may attach their own checks.

Automatic fallback when normality fails

When a normality check fails, Licklider automatically selects a non-parametric alternative:

• Independent 2 groups: Welch's t → Mann-Whitney U
• Paired 2 groups: Paired t → Wilcoxon signed-rank
• Independent 3+ groups: One-way ANOVA → Kruskal-Wallis
• Paired 3+ groups: RM-ANOVA → Friedman
• Mixed design: Mixed ANOVA runs without a standard non-parametric mixed-design fallback
• Regression residuals: Pearson primary → Spearman primary

This selection is recorded in the analysis record, including the normality test result, the rule applied, and the rationale for the switch.

The selection rationale is surfaced in the figure record: a compact method summary appears in the Dataset section of the Inspector, while the Assurance section preserves the applied-statistics explanation in more detail.

All automatic test switches are also monitored by the Project Audit. If figures with comparable assumption outcomes handle the same situation differently, the audit flags that difference for review. See Project Statistical Policy and Consistency Audit.

Other guard checks (design and independence)

These checks complement the assumption gate and run when the analysis path requires them:

Pairing structure Whether the analysis correctly reflects a paired or independent design. Licklider checks whether subject IDs appear across multiple conditions and flags cases where the pairing structure is ambiguous. See Paired vs Unpaired Guard for detail.

Pseudoreplication Whether observations that appear independent actually share a common biological source. This check runs when repeated rows from the same subject ID are detected. See Pseudoreplication Detection for detail.

Sampling independence Whether the independence assumption underlying the statistical test is supported by the data structure. See Statistical Independence Check for detail.

Equal variance (homoscedasticity) For group-comparison figures, Levene's test (median-centered) runs alongside Shapiro-Wilk. Welch's t-test is the default for independent two-group parametric paths and does not require equal variances. See Normality and Homoscedasticity for detail.

Important: these checks are powerful only when the dataset actually exposes the relevant structure. Licklider does not automatically detect hidden clustering, repeated measurements, or pseudoreplication if subject IDs, block IDs, cage IDs, plate IDs, or other grouping variables are missing from the data. If the observation unit is not declared correctly, the guard can miss a real design problem while the analysis still looks technically clean.

What happens when an assumption is unresolved

When a check is unresolved — that is, when the assumption has not been confirmed or the issue has not been acknowledged — the Inspector shows it as an unresolved requirement.

Unresolved assumptions do not block the figure from being generated or viewed. They do affect what can be exported:

• For exploratory analyses, unresolved assumptions are disclosed automatically. The figure can be exported with the disclosure included.
• For confirmatory and publication-ready analyses, unresolved assumptions must be resolved before claim-bearing export is allowed.

This split is deliberate. Exploratory work often needs to continue even while assumptions are still being investigated, but confirmatory or publication-ready claims should not move forward as if unresolved design risks were harmless.

How to resolve an assumption check

Each check has a specific resolution path, shown in the Inspector when the check is flagged:

• Normality — the test result is shown. If the selected test is appropriate for your data, no further action is needed. If not, redirect Licklider in the Chat.
• Equal variance — confirm whether the variance result is consistent with your analysis plan.
• Pairing — confirm whether the design is paired or independent.
• Pseudoreplication — confirm whether repeated rows are biological or technical replicates.
• Sampling independence — confirm the independence structure of the observations.

Manual overrides are recorded. If you override an assumption check, the override and its justification appear in the figure's disclosure.

The reason is traceability: a manual override may be scientifically justified, but it should never be invisible to later readers of the analysis record.

Robustness to outliers

One specific form of robustness evaluation is whether the conclusions change when outliers are included or excluded. Licklider compares results from three variants — the raw data, the processed data after exclusions, and a robust analysis method — and reports whether the conclusions are stable across these comparisons.

This is covered in detail on the Outlier Sensitivity Report page.

Design rationale and references

Licklider groups these checks into a single guard because assumption failures are cumulative in their effect on scientific claims. A reassuring normality result does not rescue pseudoreplication; a clean pairing structure does not rescue uncontrolled outlier sensitivity. The guard therefore treats assumption review as a combined decision point rather than as a set of isolated diagnostics.

The distinction between exploratory export and claim-bearing export is intentional. Exploratory analysis should remain inspectable even when the data are messy, but confirmatory statements need a higher bar because unresolved assumptions can turn a visually plausible result into a statistically invalid claim [1, 2].

Licklider records overrides because researcher degrees of freedom do not disappear when a manual decision is reasonable. Disclosure keeps those judgment calls visible to collaborators, reviewers, and your future self [2, 3].

Methodological foundations

1. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129-133. -> Supports the need to interpret statistical results in the full context of assumptions and analysis process rather than by p-values alone.

2. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. -> Direct support for making unresolved assumptions and manual overrides visible instead of letting them silently affect claim-bearing output.

3. Lazic, S. E. (2010). The problem of pseudoreplication in neuroscientific studies: Is it affecting your analysis? BMC Neuroscience, 11, 5. -> Illustrates why observation-unit mistakes and hidden dependence can invalidate otherwise standard statistical workflows.

Current support boundary

• The assumption guard can only evaluate structures that are represented in the data or explicitly declared; it does not automatically infer missing observation-unit metadata.
• Licklider does not automatically detect hidden clustering, batch structure, cage effects, litter effects, or repeated measures when those relationships are absent from the uploaded table.
• Licklider does not automatically know whether a statistically acceptable fallback is also the best scientific choice for the study question.
• A resolved or acknowledged guard state does not mean the analysis is universally correct; it means the currently exposed assumption checks have been reviewed within the available metadata.
• This page describes the guard layer, not every underlying diagnostic in detail. Specific test mechanics still belong to the linked method and quality-check pages.

What this page does not cover

• How normality and equal variance are checked → see Normality and Homoscedasticity
• How pairing is verified → see Paired vs Unpaired Guard
• How pseudoreplication is detected → see Pseudoreplication Detection
• How outlier sensitivity is evaluated → see Outlier Sensitivity Report
• How observation units should be declared in the first place → see Observation Unit Declaration