Comparing three or more groups
ANOVA and its alternatives, including two-way designs.
- Interpret a resultANOVA is significant but no post hoc test is: what it means and what to doA significant F-test with no significant pairwise comparison is not an error. The two tests ask different questions. Here is how to decide what to report, with a worked example in R and Python.
- Choose a methodWelch's ANOVA vs one-way ANOVA: which to use for three or more groupsUse Welch's ANOVA by default to compare the means of three or more independent groups, then Games–Howell for pairs. Why, with a worked example in R and Python, a simulation and the software defaults that differ.
- Choose a methodTwo-way ANOVA vs separate t-tests: does the effect differ between groups?Significant in wild type, not in knockout? That is not evidence the treatment works differently. Test the interaction in a two-way ANOVA, with a worked example in R and Python.
- Choose a methodANOVA assumptions not met: what to do for each oneWhich ANOVA assumption failed decides the fix: fix the unit, use Welch's ANOVA, analyse logs, fit a count or proportion model, or use a robust or rank test. Checks and code in R and Python.
- Choose a methodKruskal–Wallis vs one-way ANOVA: which to use, and what each tells youKruskal–Wallis asks whether some groups rank higher than others; ANOVA compares means. Choose by your question and scale, follow up with Dunn's test, and know where each breaks.
- Choose a methodTest for trend across doses: when one trend test beats comparing each dose with controlIf you expect the response to change steadily with dose, test the trend once with a linear contrast or a regression on dose, decided in advance. Comparing each dose with control has less power. Python and R code.