ANOVA is significant but no post hoc test is: what it means and what to do

Quick answer. Nothing went wrong. The ANOVA F-test asks whether the group means differ in any way; a post hoc test asks whether one particular pair differs, after paying for testing several pairs. The F-test can detect a pattern — for example, all treatments above control — that no single pair is strong enough to show. Decide by the comparisons you planned before seeing the data: for all pairs, report Tukey’s results as they are (a significant F is not required first); for treatments versus one control, Dunnett’s test — if that comparison was the plan from the start, not because Tukey came out non-significant; for a specific pattern you planned, report that contrast. Switching tests after seeing the results is not an option.

Terms used below. F(3, 16) is the F statistic with its two degrees of freedom (groups − 1, animals − groups). A post hoc (pairwise) test compares two groups at a time after the ANOVA. A contrast is one weighted comparison of means, such as the average of three treatments minus control. The familywise error rate is the chance of at least one false positive across all the comparisons you make. Adjusted p-values and simultaneous 95% CIs are widened so that this chance stays at 5% for the whole set.

Why the two tests disagree

The F-test and the pairwise tests answer different questions, so neither result contradicts the other.

• The F-test looks for any difference among the means. A significant F implies that some contrast among the means is significant after Scheffé’s adjustment, and that contrast “may be very much unlike a pairwise comparison” (Lenth, emmeans FAQ). Typical examples: control against the average of all treatments, or groups A and B together against C and D.
• Each post hoc test looks at one pair and pays for the others. Tukey’s test controls the chance of any false positive across all pairs, so each pair needs stronger evidence than a single t-test would. With four groups there are six pairs.

GraphPad’s documentation puts it directly: “it is quite possible for the overall ANOVA to reject the null hypothesis that all group means are the same yet for the post test to find no significant difference among group means” (GraphPad FAQ 1081).

It is not rare. In a simulation of four groups of 20 where one group was shifted by 0.5 standard deviations, 9.1% of significant F-tests at α = 0.05 had no significant Tukey pair, and 3.5% of non-significant F-tests had at least one (Chen et al. 2018, Table 1).

Read a significant F with no significant pair as “there is a pattern among the means, and these data cannot pin it to one pair”.

A worked example

Constructed data: a marker level (ng/mL) in five mice per group — a vehicle control and three compounds, A, B and C.

Group	Values	Mean	SD
Control	8.2, 8.5, 11.2, 8.9, 8.9	9.14	1.19
A	11.6, 10.1, 10.7, 10.9, 12.4	11.14	0.89
B	11.4, 12.2, 10.7, 11.2, 10.5	11.20	0.67
C	9.5, 10.9, 13.6, 12.1, 9.3	11.08	1.81

All three compounds sit about 2 ng/mL above control. Here is what each analysis says:

Analysis	What it tests	Result
One-way ANOVA	Any difference among the four means	F(3, 16) = 3.39, p = 0.044
Tukey HSD, all six pairs	Each pair, adjusted for six comparisons	Smallest adjusted p = 0.071 (B − control); A − control 0.082, C − control 0.094; pairs among compounds ≥ 0.99
Dunnett, three pairs vs control	Each compound against control, adjusted for three comparisons	A 0.049, B 0.042, C 0.057
Planned contrast	Mean of the three compounds minus control	2.00 ng/mL, 95% CI 0.67 to 3.33, t(16) = 3.19, p = 0.006
Unadjusted pairwise t-tests (not a valid choice here)	Each pair, no adjustment	Each compound vs control p = 0.016 to 0.023

The F-test is picking up the shared shift of all three compounds — the contrast in the fourth row. Tukey spreads the evidence over six pairs, three of which compare compounds with almost identical means, so no single pair clears the bar. Dunnett asks only three questions, so its bar is lower.

The table shows why the choice must come first: depending on which test you pick after seeing it, you could report zero, two or three “significant” compounds from the same data. The unadjusted row looks best and is the least trustworthy — it is what you get by running separate t-tests.

from scipy import stats

control = [8.2, 8.5, 11.2, 8.9, 8.9]
A = [11.6, 10.1, 10.7, 10.9, 12.4]
B = [11.4, 12.2, 10.7, 11.2, 10.5]
C = [9.5, 10.9, 13.6, 12.1, 9.3]

print(stats.f_oneway(control, A, B, C))          # F = 3.39, p = 0.044
res = stats.tukey_hsd(control, A, B, C)           # all pairs
print(res)                                        # adjusted p-values
print(res.confidence_interval().low[2, 0], res.confidence_interval().high[2, 0])  # B − control CI
print(stats.dunnett(A, B, C, control=control, rng=1).pvalue)   # vs control

d <- data.frame(group = factor(rep(c("control", "A", "B", "C"), each = 5),
                               levels = c("control", "A", "B", "C")),
                y = c(8.2, 8.5, 11.2, 8.9, 8.9,  11.6, 10.1, 10.7, 10.9, 12.4,
                      11.4, 12.2, 10.7, 11.2, 10.5,  9.5, 10.9, 13.6, 12.1, 9.3))
fit <- aov(y ~ group, data = d)
summary(fit)                                     # F(3, 16) = 3.39, p = 0.044
TukeyHSD(fit)$group                              # all pairs: diff, lwr, upr, p adj
library(multcomp)
set.seed(1)                                     # Dunnett p-values use random integration
summary(glht(fit, linfct = mcp(group = "Dunnett")))           # vs control
summary(glht(fit, linfct = mcp(group = rbind("compounds - control" = c(-1, 1/3, 1/3, 1/3)))))

Python and R agree on every number in the table. Use a fixed seed (rng=1 in SciPy, set.seed() in R) for Dunnett’s test: its p-values come from numerical integration with a random component and can differ in the fourth decimal between runs.

What to do: follow the comparisons you planned

What you planned before the experiment	What to report
Every pair of groups	Tukey’s results as they are, with adjusted p-values and simultaneous 95% CIs. In the example: no pair reached p < 0.05; the compounds were about 2 ng/mL above control (Tukey CIs about −0.2 to 4.2).
Each treatment against one control	Dunnett’s test — this was your test from the start, not a rescue. In the example: A and B differed from control; C did not reach p < 0.05.
A specific pattern (“any active compound vs vehicle”, a dose trend)	That contrast, with its CI. In the example: 2.00 ng/mL above control (95% CI 0.67 to 3.33).
Nothing specific	The F-test and Tukey’s results, the pattern described as exploratory, and a follow-up experiment designed to test it.

If nothing was written down, report all pairs (Tukey). Use Dunnett only if a document dated before the results — a protocol, grant or figure plan — names a single reference control. Say in Methods which applies. A reviewer will accept a test that matches the design; one that matches only the result invites the question “why this test?”. Write the planned comparisons into your analysis plan or Methods before you look at the results.

Do you need a significant ANOVA before a post hoc test?

No for Tukey and Dunnett; yes for Fisher’s protected LSD, which is safe only with exactly three groups. Tukey and Dunnett control the familywise error rate on their own. GraphPad notes that “with one exception, post tests are valid even if the overall ANOVA did not find a significant difference among means”, and quotes Hsu: “An unfortunate common practice is to pursue multiple comparisons only when the null hypothesis of homogeneity is rejected” (GraphPad FAQ 1081). Chen et al. reached the same conclusion from their simulation: “always perform pairwise group comparisons, regardless of the significance status of the omnibus test” (Chen et al. 2018).

The exception is Fisher’s protected LSD, which runs unadjusted t-tests only after a significant F. It has good error control with exactly three groups (Levin, Serlin & Seaman 1994); with more groups its maximum familywise error rate “may greatly exceed the nominal size α” (Hayter 1986). Use Tukey or Dunnett instead when you have four or more groups.

Procedure	Needs a significant F first?	Controls false positives across the comparisons?
Tukey HSD (all pairs)	No	Yes
Dunnett (each group vs one control)	No	Yes
Fisher’s protected LSD	Yes	With three groups; not with four or more
Separate t-tests, no adjustment	—	No

Check what your code — or your AI assistant — ran

Software names and defaults differ, and code from an AI assistant can loop over separate t-tests. Find the line that produced your p-values and match it here:

Code you see	What it does	Default to check
R TukeyHSD(aov(y ~ group))	Tukey HSD (Tukey–Kramer for unequal n)	Assumes equal variances; conf.level = 0.95
R pairwise.t.test(y, group)	Pairwise t-tests with a pooled SD	Adjusts with Holm unless p.adjust.method is set
R glht(fit, mcp(group = "Dunnett")) (multcomp)	Dunnett, each group vs the first level	The control must be the first factor level
SciPy stats.tukey_hsd(*groups)	Tukey HSD	equal_var=True; equal_var=False runs Games–Howell
SciPy stats.dunnett(*treated, control=c)	Dunnett vs control	Random integration: set rng for reproducible p
statsmodels pairwise_tukeyhsd(y, groups)	Tukey HSD	use_var="equal"; "unequal" gives Games–Howell
A loop of ttest_ind or t.test calls	Unadjusted t-tests (unprotected LSD)	No adjustment at all; Student’s test in SciPy by default

For a new analysis, decide before you see the data whether you will assume equal variances; the default we recommend is Welch’s ANOVA with Games–Howell pairs (Welch’s ANOVA vs one-way ANOVA). On this page’s data Welch’s ANOVA gives F(3, 8.50) = 3.56, p = 0.064, and no Games–Howell pair reaches 0.05 (smallest p = 0.052, B − control): one more reason to read the overall F here as weak evidence, not as a finding about particular compounds.

Report it

Marker levels differed among groups (one-way ANOVA, F(3, 16) = 3.39, p = 0.044). In Tukey’s comparisons of all pairs, no pair differed at p < 0.05; each compound was about 2 ng/mL above control (B − control 2.06 ng/mL, simultaneous 95% CI −0.14 to 4.26, adjusted p = 0.071). The pattern is consistent with a similar increase in all three compounds (an unplanned observation).

Only if a dated plan (protocol, grant or dated notebook) named comparisons with vehicle before the experiment, write instead — and fill in the document, or use the all-pairs sentence above:

Each compound was compared with vehicle by Dunnett’s test, as specified before the experiment in [document, date]: A +2.00 ng/mL (adjusted p = 0.049), B +2.06 (p = 0.042), C +1.94 (p = 0.057).

Give the effect sizes and intervals in either version, so readers can judge the size of the shift as well as its p-value; for choosing between corrections, see Which multiple-comparison correction to use.

References

• Chen T, Xu M, Tu J, Wang H, Niu X (2018). Relationship between omnibus and post-hoc tests: an investigation of performance of the F test in ANOVA. Shanghai Archives of Psychiatry 30(1):60–64. PMC5925602
• Dunnett CW (1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association 50(272):1096–1121. doi:10.1080/01621459.1955.10501294
• GraphPad Software. Do the multiple comparisons tests following one-way ANOVA provide useful information even if the overall ANOVA results are not statistically significant? FAQ 1081. graphpad.com (accessed 10 November 2026)
• Hayter AJ (1986). The maximum familywise error rate of Fisher’s least significant difference test. Journal of the American Statistical Association 81(396):1000–1004. doi:10.1080/01621459.1986.10478364
• Lenth RV. FAQs for emmeans: “My ANOVA F is significant, but no pairwise comparisons are”. rvlenth.github.io (accessed 10 November 2026)
• Levin JR, Serlin RC, Seaman MA (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin 115(1):153–159. doi:10.1037/0033-2909.115.1.153
• SciPy 1.18.1 documentation: scipy.stats.tukey_hsd, scipy.stats.dunnett. docs.scipy.org