# Should you test for normality before a t-test or ANOVA?

> No — not as a gate for choosing the test. Choose the test from your design and measurement in advance, then look at the residuals. Why, with simulations and a worked example in R and Python.

Canonical: https://www.licklider.ai/guides/should-you-test-for-normality/
Section: Guides › Choosing a test
Author: Licklider · Published 2026-11-10 · Last reviewed 2026-11-10 (UTC)

> **Quick answer.** No — do not run a normality test to decide between a t-test and a rank test. Choose the analysis in advance from your design and the kind of measurement — **Welch's t-test** (or Welch's ANOVA) on a sensible scale — R's `t.test` default; in SciPy pass `equal_var=False` — **logs** for concentrations and other proportional measures, a rank test for ordinal scores — and then **look at the residuals** in a QQ plot to catch gross problems. A normality test is weak exactly when groups are small, flags harmless departures when they are large, and says nothing on equal spreads, which matter more for small groups of unequal size. Shapiro–Wilk is still fine as a description; it is a poor switch.

**Terms used below.** *Residuals* are each value minus its group mean; normality refers to them. A *QQ plot* (quantile–quantile plot) puts the sorted residuals against the values a normal distribution would give; normal-looking residuals fall close to a straight line. *Shapiro–Wilk* is a test whose null hypothesis is "these values come from a normal distribution"; a small p-value means it detected a departure. *False-positive rate* (Type I error rate) is the share of experiments in which a test reports p < 0.05 when there is no real difference. *Power* is the chance of detecting a departure or difference that is really there. *Two-stage procedure* means running one test and choosing the next test from its result. *Welch's t-test* compares two means without assuming equal spreads (also called the separate-variances t-test). *Log scale*: analysing the logarithm of each value; results come back as ratios of *geometric means* (the back-transformed mean of the logs). *Degrees of freedom (df)*: the number that sets which t distribution the p-value uses; Welch's test gives fractional df. *Log-normal*: values whose logarithms are normally distributed, typical of concentrations. *Ordinal score*: an ordered category such as a blinded 0–4 histology grade.

## What has to be roughly normal

Normality refers to the scatter around each group mean, rather than to all values pooled together: two normal groups with different means can look bimodal when pooled. Lumley and colleagues put it plainly: "It is widely but incorrectly believed that the t-test and linear regression are valid only for Normally distributed outcomes", and in large samples "they are valid for any distribution" ([Lumley et al. 2002](https://doi.org/10.1146/annurev.publhealth.23.100901.140546)).

| Design | What should be roughly normal | How to look |
| --- | --- | --- |
| Two independent groups (Welch's t-test) | Values within each group, on the scale you analyse | QQ plot of the residuals (both groups together) |
| Paired measurements (paired t-test) | The within-pair differences | QQ plot of the differences |
| Three or more groups (Welch's ANOVA) | Values within each group | QQ plot of the model residuals |
| Two factors (two-way ANOVA) | Residuals of the full model | QQ plot of the residuals, plus residuals against fitted values |

Equal spreads are a separate question, and for small groups of unequal size they matter more than the shape. Use Welch's tests, which handle unequal spreads without a pre-test ([Welch vs Student](/guides/welch-vs-student-t-test/), [Welch's ANOVA](/guides/welch-anova/)).

## Why a normality test is a poor switch

**1. It is weak when groups are small — where the choice matters most.** We simulated 20,000 samples per cell and recorded the share that Shapiro–Wilk flagged (p < 0.05):

| True distribution | n = 5 | n = 10 | n = 30 | n = 100 | n = 1,000 |
| --- | --- | --- | --- | --- | --- |
| Normal (flags should be 5%) | 5% | 5% | 5% | 5% | 5% |
| Lognormal, clearly skewed (log SD 0.5) | 10% | 24% | 72% | > 99% | 100% |
| Heavier tails than normal (t, 5 df) | 7% | 11% | 25% | 56% | 100% |
| Nearly normal (lognormal, log SD 0.1) | 5% | 6% | 9% | 18% | 93% |

With 5 or 10 mice per group, a clearly skewed sample passes the test most of the time: at n = 10 it is flagged in only 24% of samples. Ghasemi and Zahediasl summarise the pattern: "small samples most often pass normality tests", while at large sizes significant results appear "even in the case of a small deviation from normality", a deviation that "will not affect the results of a parametric test" ([Ghasemi & Zahediasl 2012](https://doi.org/10.5812/ijem.3505)). The last row shows that side: a nearly normal distribution is flagged in 93% of samples of 1,000.

**2. Choosing the test from the pre-test changes what the chosen test means.** Rochon and colleagues ran the usual two-stage procedure — Student's t-test if Shapiro–Wilk passes, Mann–Whitney otherwise — on equal-sized groups. The t-tests chosen this way had higher false-positive rates than intended: with exponential data and 30 per group, 10.8% to 17.0% false positives depending on the pre-test threshold, against 4.7% for the t-test used without a pre-test. The overall procedure, though, "seemed to satisfactorily maintain the nominal significance level" in their scenarios, and they concluded that pre-testing "is incorrect and should therefore be avoided" from a formal perspective while it "does not seem to cause much harm" in practice for the cases they studied ([Rochon, Gondan & Kieser 2012](https://doi.org/10.1186/1471-2288-12-81)).

**3. It checks the shape, while the bigger risk for small unequal groups is unequal spreads.** Rochon's groups had equal spreads. We repeated the two-stage procedure with SciPy's defaults — Shapiro–Wilk on each group, Student's t-test if both pass, Mann–Whitney otherwise — in five settings with no true difference in means, 20,000 experiments each, and compared it with using one test every time:

| Setting (no true difference) | Sent to Student's t | Two-stage false positives | Always Student | Always Welch |
| --- | --- | --- | --- | --- |
| Normal, equal SDs, 5 vs 5 | 91% | 4.7% | 4.8% | 4.3% |
| Normal, equal SDs, 10 vs 10 | 90% | 5.2% | 5.1% | 5.0% |
| Skewed (lognormal), same distribution, 10 vs 10 | 56% | 5.1% (6.1% among the t-tests) | 4.5% | 4.0% |
| Normal, SD 2 vs 1, 10 vs 10 | 91% | 5.7% | 5.5% | 5.0% |
| Normal, SD 2 vs 1, 5 vs 15 (smaller group more variable) | 90% | **15.0%** | **15.6%** | 5.7% |

R, with its own random numbers, gives the same picture (15.2% two-stage, 15.8% always Student, 5.5% always Welch in the last row). In the last row the data are perfectly normal, so the normality test passes 90% of experiments and sends them to Student's t-test, which ignores the unequal spreads: the 15% comes from Student's test itself, and the gate neither causes nor catches it. Welch's test, used every time, keeps the rate near 5%. Zimmerman found a related problem with variance pre-tests: the two-stage procedure "fails to protect the significance level and usually makes the situation worse", and "Optimum protection is assured by using a separate-variances test unconditionally whenever sample sizes are unequal" ([Zimmerman 2004](https://doi.org/10.1348/000711004849222)).

**What to do:** write the test into your plan before you see the data — Welch's t-test or Welch's ANOVA, on the log scale for proportional measures — and use the residual plot as a sanity check rather than a switch. Code and output: [simulation.py](/guides/should-you-test-for-normality/simulation.py) and [simulation.R](/guides/should-you-test-for-normality/simulation.R) (seed 20261110).

## Worked example: the test picks the wrong scale

Constructed ELISA data: IL-6 (pg/mL) in 8 control and 8 treated mice, one value per mouse, drawn once from lognormal distributions with a seed fixed in advance ([il6.csv](/guides/should-you-test-for-normality/il6.csv)).

| Group | Values (pg/mL) | SD |
| --- | --- | --- |
| Control | 19.8, 23.6, 22.8, 18.6, 35.4, 4.4, 19.5, 19.7 | 8.5 |
| Treated | 32.4, 49.0, 70.1, 28.2, 28.7, 47.3, 94.3, 99.8 | 28.7 |

What a normality gate would do here, and what the planned analysis gives:

| Step | Result |
| --- | --- |
| Shapiro–Wilk on raw values, per group | p = 0.16 and 0.14: both "pass", so the gate picks Student's t-test on raw values |
| Student's t-test, raw values | p = 0.0045 — but the treated SD is 3.4 times the control SD |
| Shapiro–Wilk on log values, per group (if you tested these instead) | control p = 0.0035: the gate would switch to Mann–Whitney |
| **Planned: Welch's t-test on log values** | **treated/control ratio 2.76 (95% CI 1.51 to 5.05), t(13.58) = 3.62, p = 0.0029** |

<img src="/guides/should-you-test-for-normality/qq-raw-vs-log.svg" alt="Two QQ plots of the IL-6 residuals. Left, raw values in pg/mL: the points form a bowl-shaped curve, with both ends above the reference line and the two largest treated values furthest above it. Right, log values: the points lie close to the line except one low point at the far left, the control mouse with 4.4 pg/mL." width="576" height="259" style="max-width:100%;height:auto;border-radius:6px" />

The raw-scale QQ plot is bowl-shaped, with both ends above the line and the two largest treated values furthest from it: right skew, together with a treated spread 3.4 times the control spread — the pattern of a measurement that changes by ratios. The test passed it anyway. On the log scale the points follow the line apart from one low control value, 4.4 pg/mL. The scale came from the measurement — concentrations change by ratios — rather than from a test. The low value is a question for the lab notebook (was it below the assay's standard curve?); check it there and keep the planned test, as explained in [when you can exclude an outlier](/guides/when-can-you-exclude-an-outlier/). Report the ratio with its interval.

## How to look instead

Plot the residuals on the scale you will analyse; for paired data, plot the differences. Python (SciPy 1.18.1, with [il6.csv](/guides/should-you-test-for-normality/il6.csv) in the working folder):

```python
import numpy as np
import pandas as pd
from scipy import stats

d = pd.read_csv("il6.csv")                      # columns: group, il6 (pg/mL)
ctl = d.loc[d.group == "control", "il6"].to_numpy()
trt = d.loc[d.group == "treated", "il6"].to_numpy()

# Look at the residuals (each value minus its group mean) on the scale you will analyse
res = np.concatenate([np.log(ctl) - np.log(ctl).mean(), np.log(trt) - np.log(trt).mean()])
import matplotlib
matplotlib.use("Agg")                           # save to a file; remove this line in a notebook
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(4, 4))
stats.probplot(res, dist="norm", plot=ax)        # normal QQ plot of the residuals
fig.savefig("qq_log_residuals.png", dpi=150)

# Planned analysis: Welch's t-test on log values, reported as a ratio
w = stats.ttest_ind(np.log(trt), np.log(ctl), equal_var=False)
ci = w.confidence_interval()
print(f"ratio {np.exp(np.log(trt).mean() - np.log(ctl).mean()):.2f}, "
      f"95% CI {np.exp(ci.low):.2f} to {np.exp(ci.high):.2f}, "
      f"t({w.df:.2f}) = {w.statistic:.2f}, p = {w.pvalue:.4f}")

# Paired design: look at the within-pair differences instead
before = np.array([5.1, 4.8, 6.0, 5.5, 4.9, 5.7])
after = np.array([5.9, 5.2, 6.4, 6.3, 5.0, 6.6])
diffs = after - before
fig2, ax2 = plt.subplots(figsize=(4, 4))
stats.probplot(diffs, dist="norm", plot=ax2)
fig2.savefig("qq_paired_differences.png", dpi=150)

# Three or more groups: residuals of the one-way model
g = pd.DataFrame({"y": [10.2, 11.5, 9.8, 12.1, 13.0, 12.4, 14.8, 15.9, 13.7],
                  "dose": ["0"] * 3 + ["10"] * 3 + ["30"] * 3})
res3 = g.y - g.groupby("dose").y.transform("mean")
fig3, ax3 = plt.subplots(figsize=(4, 4))
stats.probplot(res3, dist="norm", plot=ax3)
fig3.savefig("qq_three_groups.png", dpi=150)
```

It prints `ratio 2.76, 95% CI 1.51 to 5.05, t(13.58) = 3.62, p = 0.0029`. In R 4.3.3:

```r
d <- read.csv("il6.csv")                       # columns: group, il6 (pg/mL)
d$group <- factor(d$group, levels = c("control", "treated"))

# Look at the residuals on the scale you will analyse
fit <- lm(log(il6) ~ group, data = d)
qqnorm(residuals(fit)); qqline(residuals(fit))  # opens a plot
plot(fit, which = 1)                            # residuals vs fitted: similar spread?

# Planned analysis: Welch's t-test on log values, reported as a ratio
w <- t.test(log(il6) ~ group, data = d)         # Welch is R's default
print(w)
cat("ratio treated/control:", round(exp(diff(w$estimate)), 2),
    " 95% CI:", round(rev(exp(-w$conf.int)), 2), "\n")

# Paired design: look at the within-pair differences instead
before <- c(5.1, 4.8, 6.0, 5.5, 4.9, 5.7)
after  <- c(5.9, 5.2, 6.4, 6.3, 5.0, 6.6)
qqnorm(after - before); qqline(after - before)

# Three or more groups: residuals of the one-way model
g <- data.frame(y = c(10.2, 11.5, 9.8, 12.1, 13.0, 12.4, 14.8, 15.9, 13.7),
                dose = rep(c("0", "10", "30"), each = 3))
qqnorm(residuals(lm(y ~ dose, data = g)))
```

**Reading the plot.** Points close to the reference line: fine. A bowl-shaped curve, with both ends above the line: right skew — if the measurement is proportional, analyse logs. An S-shape, with the lowest points below the line and the highest above it: heavier tails than normal — check for data errors; keep the planned test as the main result, and if you add a rank or robust analysis, report it as a labelled sensitivity analysis and plan that method from the start next time. One point far off on its own: check that observation in your records. As the NIST handbook puts it, "The further the points vary from this line, the greater the indication of departures from normality" ([NIST/SEMATECH](https://www.itl.nist.gov/div898/handbook/eda/section3/normprpl.htm)). With unequal group spreads a pooled QQ plot mixes shape with spread, so also compare the group SDs. Decide from the measurement instead when you have five or fewer values per group, because any plot is noisy then.

## When the default needs more thought

- **Five or fewer per group.** Neither a test nor a plot can tell much; decide the scale from the measurement (logs for concentrations and fold changes) and say so in the plan. See [t-test with n = 3](/guides/t-test-with-n-3/).
- **Ordinal scores, or values piled at a floor or ceiling.** A rank test fits the question better ([Mann–Whitney vs t-test](/guides/mann-whitney-vs-t-test/); for three or more groups, [Kruskal–Wallis vs ANOVA](/guides/kruskal-wallis-vs-anova/)).
- **Counts and proportions per animal.** Use a model built for them rather than a normal-theory test on transformed values; see [next steps for non-normal ANOVA data](/guides/anova-assumptions-not-met/).
- **Large groups.** For studies with a large sample size (Fagerland's simulation went up to 1,000 per group), t-tests and their confidence intervals "can and should be used even for heavily skewed data" ([Fagerland 2012](https://doi.org/10.1186/1471-2288-12-78)); this permission does not extend to small lab groups, where the scale still has to be chosen from the measurement.
- **Tools that pre-test for you.** Some software runs a normality test and switches automatically. Report the test you planned, and turn off the automatic switch or note the software's step.

## If a reviewer asks "did you test for normality?"

> Thank you for raising this. We chose the analysis before seeing the data: [Welch's t-test on log-transformed IL-6], because [concentrations vary proportionally and Welch's test allows unequal variances]. We chose the test without a preliminary normality test, because such tests have little power at our group size ([n = 8 per group]) and selecting the test from a preliminary test alters the error rate of the chosen test (Rochon et al. 2012; Zimmerman 2004). We inspected QQ plots of the residuals on the analysed scale; they showed no marked departure from normality [apart from one low control value, which was retained after checking the laboratory record]. We have added the residual plot as Supplementary Figure [X].

Replace every bracketed part with your own details; keep the sentences that are true for your study. **Methods sentence:**

> Groups were compared with Welch's t-test on log-transformed concentrations, chosen before the analysis; results are reported as ratios of geometric means with 95% confidence intervals. Normality of the residuals was assessed graphically (QQ plots); no preliminary normality test was used to select the test.

Keep the residual plot in your supplementary material so the reviewer can see it.

## Check what an AI assistant ran

Search the code for `shapiro`, `normaltest`, `kstest` or `levene` followed by an `if` that picks the test. If you find one:

1. Ask what the analysis would be if the pre-test were removed, and decide that in advance.
2. In Python, check `ttest_ind`: SciPy runs Student's t-test by default; pass `equal_var=False` for Welch. R's `t.test` runs Welch by default.
3. Check the scale: concentrations and fold changes belong on the log scale; ask for the ratio and its interval.
4. Check what was tested for normality: the residuals or differences, rather than all values pooled.

Software notes: R's `shapiro.test` stops with an error outside 3 to 5,000 values; SciPy's `shapiro` needs at least three observations and warns that for more than 5,000 "the W test statistic is accurate, but the p-value may not be" (R 4.3.3 and SciPy 1.18.1 documentation, checked 10 November 2026).

Then run the analysis you planned and report it.

## References

- Fagerland MW (2012). t-tests, non-parametric tests, and large studies—a paradox of statistical practice? *BMC Medical Research Methodology* 12:78. [doi:10.1186/1471-2288-12-78](https://doi.org/10.1186/1471-2288-12-78)
- Ghasemi A, Zahediasl S (2012). Normality tests for statistical analysis: a guide for non-statisticians. *International Journal of Endocrinology and Metabolism* 10(2):486–489. [doi:10.5812/ijem.3505](https://doi.org/10.5812/ijem.3505)
- Lumley T, Diehr P, Emerson S, Chen L (2002). The importance of the normality assumption in large public health data sets. *Annual Review of Public Health* 23:151–169. [doi:10.1146/annurev.publhealth.23.100901.140546](https://doi.org/10.1146/annurev.publhealth.23.100901.140546)
- NIST/SEMATECH e-Handbook of Statistical Methods, §1.3.3.21 Normal Probability Plot. [itl.nist.gov](https://www.itl.nist.gov/div898/handbook/eda/section3/normprpl.htm) (accessed 10 November 2026)
- Rochon J, Gondan M, Kieser M (2012). To test or not to test: preliminary assessment of normality when comparing two independent samples. *BMC Medical Research Methodology* 12:81. [doi:10.1186/1471-2288-12-81](https://doi.org/10.1186/1471-2288-12-81)
- Zimmerman DW (2004). A note on preliminary tests of equality of variances. *British Journal of Mathematical and Statistical Psychology* 57(1):173–181. [doi:10.1348/000711004849222](https://doi.org/10.1348/000711004849222)
- Software documentation accessed 10 November 2026: R `stats::shapiro.test` and `t.test`; SciPy 1.18 `scipy.stats.shapiro`, `probplot` and `ttest_ind`.

## Related

- [Welch's t-test vs Student's t-test](https://www.licklider.ai/guides/welch-vs-student-t-test/)
- [Mann–Whitney vs t-test](https://www.licklider.ai/guides/mann-whitney-vs-t-test/)
- [ANOVA assumptions not met: what to do](https://www.licklider.ai/guides/anova-assumptions-not-met/)
- [Statistical test chooser](https://www.licklider.ai/tools/statistical-test-chooser/)

## How this page was made

- Written and published by Licklider. Every claim from the literature is cited to a peer-reviewed study, an official handbook or official software documentation listed under References; cited passages were checked against the source text.
- The simulations and the worked example (constructed data, seed fixed in advance) were produced by stored code (Python with SciPy 1.18.1; R 4.3.3). The worked example agrees to every printed digit in both languages; the simulations agree within simulation error.
- Separate review passes checked the statistics, the citations and readability before publication.