# p = 0.06 or “not significant”: what it means and what to write

> A p-value above 0.05 is not evidence of no effect. Read the confidence interval, write the estimate with it, and use an equivalence test if you need to claim no meaningful difference.

Canonical: https://www.licklider.ai/guides/p-value-not-significant/
Section: Guides › Interpreting results
Author: Licklider · Published 2026-11-10 · Last reviewed 2026-11-10 (UTC)

> **Quick answer.** A p-value above 0.05 means your data are compatible with no effect; it does not show that there is no effect, so read the 95% confidence interval to see which effects your data rule out, and report the estimate with that interval. If you need to claim that there is no meaningful difference, set the smallest difference that matters before the analysis and run an equivalence test (TOST) against it. Use a sensitivity analysis to say what your study could have detected; "observed power" adds nothing to the p-value.

**Terms used below.** The *estimate* is the observed difference between groups (here knockout minus wild type). A *95% confidence interval (CI)* is the range of differences compatible with the data under the test's assumptions. An *equivalence test* (TOST, "two one-sided tests") checks whether the difference is smaller than a bound you set in advance, in both directions. *Observed power* (post hoc power) is power recalculated by assuming the true effect equals the one you observed. A *sensitivity analysis* reports the smallest effect your sample size could detect with a stated power, such as 80%. *Power* is the chance of detecting an effect of a given size if it is real; *α* is the false-positive rate you accept, here 0.05; *df* (degrees of freedom) is printed with the t-test. An *equivalence bound* is the smallest difference you would care about, set before the analysis; differences inside the bounds count as negligible. The *standardised effect d* is a difference divided by the *pooled SD*, the SD of both groups combined.

## Same p, different message

Two constructed mouse studies of fasting blood glucose give nearly the same p-value but say very different things. Both compare knockout (KO) with wild-type (WT) mice with Welch's t-test, the default we use for two independent groups ([Welch vs Student](/guides/welch-vs-student-t-test/)).

| | Study 1: 40 mice per group | Study 2: 5 mice per group |
| --- | --- | --- |
| Mean ± SD, WT | 7.00 ± 1.00 mmol/L | 7.00 ± 2.00 mmol/L |
| Mean ± SD, KO | 7.30 ± 1.00 mmol/L | 8.80 ± 2.20 mmol/L |
| Difference, KO − WT | 0.30 mmol/L | 1.80 mmol/L |
| Welch's t-test | t(78) = 1.34, p = 0.185 | t(7.93) = 1.35, p = 0.214 |
| 95% CI of the difference | −0.15 to 0.74 mmol/L | −1.27 to 4.87 mmol/L |
| Differences the data rule out | Increases above 0.74 and decreases beyond 0.15 | Little: from a small decrease to an increase of almost 5 mmol/L |

Study 1 is informative: at the 95% level, an increase of more than 0.74 mmol/L or a decrease of more than 0.15 mmol/L is incompatible with these data. Study 2 is uninformative: its interval includes zero (no change) and also an increase of almost 5 mmol/L. The p-values are close; the evidence differs.

The data are constructed (values drawn with a fixed seed, then rescaled to the means and SDs in the table) so that both p-values land near 0.2. The code and data are in the page record; download [glucose.csv](/guides/p-value-not-significant/glucose.csv) to rerun the analyses below.

**Do this:** report the estimate and its 95% CI for every non-significant comparison, and read the interval's ends before writing a conclusion.

## Read the interval, not the threshold

The American Statistical Association's 2016 statement lists six principles; three apply directly to a non-significant result ([ASA 2016](https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf); [Wasserstein & Lazar 2016](https://doi.org/10.1080/00031305.2016.1154108)):

| ASA principle | Wording |
| --- | --- |
| 2 | "P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone." |
| 3 | "Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold." |
| 5 | "A p-value, or statistical significance, does not measure the size of an effect or the importance of a result." |

The title of Altman and Bland's 1995 BMJ statistics note puts the core idea in six words: "Absence of evidence is not evidence of absence" ([Altman & Bland 1995](https://doi.org/10.1136/bmj.311.7003.485)). The misreading is common: in the analysis summarised by Amrhein, Greenland and McShane, around half of 791 articles across five journals "mistakenly assume non-significance means no effect" ([Amrhein et al. 2019](https://doi.org/10.1038/d41586-019-00857-9)). Their remedy is the interval: they recommend that authors "describe the practical implications of all values inside the interval, especially the observed effect (or point estimate) and the limits".

Use the interval's two ends to decide what you can write:

| What the 95% CI shows | What you can say |
| --- | --- |
| Excludes zero | The data indicate a difference in the direction of the estimate; describe the size with the CI. |
| Includes zero and lies entirely inside your equivalence bounds (the negligible range you set in advance; next section) | Large effects are unlikely; to claim the difference is negligible, confirm with an equivalence test (next section). |
| Includes zero and extends beyond your equivalence bounds | The result is inconclusive: the data are compatible with no effect and with an effect that would matter. |

The same reading applies to other designs: for a paired test, an ANOVA contrast or a regression coefficient, read the estimate and 95% CI that your test reports.

**Do this:** use the row that matches your interval; with p = 0.06 the 95% CI includes zero, so one of the last two rows applies.

## To claim "no meaningful difference", test equivalence

An equivalence test turns "we found no difference" into a claim you can support. Lakens (2017) describes the procedure: set an upper and lower equivalence bound "based on the smallest effect size of interest", test whether the difference is above the lower bound and below the upper bound, and conclude equivalence when both one-sided tests are significant ([Lakens 2017](https://doi.org/10.1177/1948550617697177)). Because each of the two one-sided tests uses α = 0.05, the matching interval is the 90% CI (1 − 2 × 0.05), which must lie entirely inside the bounds; it is narrower than the 95% CI you read above.

Set the bounds before you see the data and report them with the result; Lakens notes that prespecified bounds are expected in randomised controlled trials and recommends the same elsewhere ([Lakens 2017](https://doi.org/10.1177/1948550617697177)). For the example we set ±1.0 mmol/L in advance as the smallest change in fasting glucose that would matter; base your own bound on biology, a clinical threshold or earlier work, and cite where it comes from. If your field has no agreed threshold, state the smallest change you would act on and why (for example, a change your assay can reproduce between experiments); if you did not set a bound before the analysis, report the result as inconclusive rather than choosing one afterwards.

| | Study 1 (40 per group) | Study 2 (5 per group) |
| --- | --- | --- |
| 90% CI of the difference | −0.07 to 0.67 mmol/L | −0.68 to 4.27 mmol/L |
| One-sided test against −1.0 mmol/L (one half of TOST, not a result on its own) | t = 5.81, p < 0.001 | t = 2.10, p = 0.034 |
| One-sided test against +1.0 mmol/L (the other half) | t = −3.14, p = 0.0012 | t = 0.60, p = 0.72 |
| TOST result (larger p) | **p = 0.0012: equivalent within ±1.0 mmol/L** | p = 0.72: equivalence not shown (not equivalent within ±1.0 mmol/L) |

Study 1 supports "no change of 1 mmol/L or more"; Study 2 supports neither "a difference" nor "no difference".

Python (SciPy 1.18.1, statsmodels 0.15.0):

```python
import pandas as pd
from scipy import stats
from statsmodels.stats.weightstats import ttost_ind

d = pd.read_csv("glucose.csv")                  # replace study, genotype, glucose with your column names
s = d[d.study == "precise"]                     # Study 1 in the table
ko = s.loc[s.genotype == "KO", "glucose"]
wt = s.loc[s.genotype == "WT", "glucose"]

res = stats.ttest_ind(ko, wt, equal_var=False)   # Welch; SciPy's default is Student
print(res.statistic, res.df, res.pvalue)         # t = 1.34, df = 78, p = 0.185
print(res.confidence_interval(0.95))             # -0.15 to 0.74 mmol/L
print(res.confidence_interval(0.90))             # -0.07 to 0.67, for the equivalence test

p, lower, upper = ttost_ind(ko, wt, -1.0, 1.0, usevar="unequal")   # statsmodels default: "pooled"
print(p)                                         # TOST p = 0.0012
```

R 4.3.3 (base R; `t.test()` runs Welch's test by default):

```r
d <- read.csv("glucose.csv")
s <- subset(d, study == "precise")
ko <- s$glucose[s$genotype == "KO"]
wt <- s$glucose[s$genotype == "WT"]

t.test(ko, wt)                                    # Welch by default: t = 1.34, df = 78, p = 0.185
t.test(ko, wt, conf.level = 0.90)$conf.int        # -0.07 to 0.67, for the equivalence test

lo <- t.test(ko, wt, mu = -1, alternative = "greater")$p.value
hi <- t.test(ko, wt, mu =  1, alternative = "less")$p.value
max(lo, hi)                                       # TOST p = 0.0012
```

The two languages agree to every printed digit (stored outputs in the page record). An equivalence test needs the same conditions as the t-test: independent animals (one value per animal; see [what counts as n](/guides/what-counts-as-n/)) and roughly symmetric data; for strongly skewed values such as concentrations, set the bounds and run the test on the log scale.

**Do this:** state the bound you set in advance, the 90% CI and the TOST p whenever your conclusion is "no meaningful difference"; without a prespecified bound, write that the result is inconclusive instead.

## Use a sensitivity analysis instead of observed power

Observed power adds nothing to the p-value. For a two-sided z-test (counting both tails) it is a fixed function of p, so every result with p = 0.05 has observed power 0.50 and every non-significant result has less:

| p-value | 0.01 | 0.05 | 0.10 | 0.20 | 0.30 | 0.50 | 0.80 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Observed power | 0.73 | 0.50 | 0.38 | 0.25 | 0.18 | 0.10 | 0.06 |

The two studies above illustrate the problem: their observed power (computed for the Welch t-test, so slightly different from the z-test table) is 0.26 and 0.22, nearly identical, although one is informative and the other uninformative. Hoenig and Heisey (2001) call the practice of interpreting non-significant results with post-experiment power "fundamentally flawed" ([Hoenig & Heisey 2001](https://doi.org/10.1198/000313001300339897)).

A sensitivity analysis answers the question a reader or reviewer is asking — what could this design have detected? — and it is independent of the result. At 80% power and two-sided α = 0.05 (standard two-sample calculation):

| | Study 1 (40 per group) | Study 2 (5 per group) |
| --- | --- | --- |
| Smallest detectable standardised effect | d = 0.63 | d = 2.02 |
| In glucose units (× pooled SD) | 0.63 mmol/L | 4.26 mmol/L |

Study 2 can reliably detect only a change of 4.3 mmol/L or more, so its non-significant result says nothing about smaller changes. The calculation uses the standard (Student) formula; for Welch's test with equal SDs, power at these effects is 80% with 40 per group and about 78% with 5 per group (simulation in the page record). An SD estimated from 5 + 5 mice is itself imprecise, so treat the 4.3 mmol/L as approximate. Run the same calculation for your design with the [sample size calculator](/tools/sample-size-calculator/) (choose "Smallest detectable effect"), or in code:

```python
from statsmodels.stats.power import TTestIndPower

for n in (40, 5):   # animals per group
    d = TTestIndPower().solve_power(nobs1=n, alpha=0.05, power=0.80, ratio=1.0, alternative="two-sided")
    print(n, round(d, 2))   # 40 0.63, then 5 2.02; multiply d by your SD for units
```

```r
library(pwr)
for (n in c(40, 5)) {   # animals per group
  d <- pwr.t.test(n = n, sig.level = 0.05, power = 0.80, type = "two.sample")$d
  cat(n, round(d, 2), "\n")   # 40 0.63, then 5 2.02
}
```

**Do this:** replace any observed-power sentence with the smallest effect your sample size can detect at 80% power, with the SD you used.

## What to write

Results sentence for an inconclusive result:

> Fasting glucose did not differ significantly between KO and WT mice (Welch's t-test, t(7.93) = 1.35, p = 0.21; difference KO − WT 1.80 mmol/L, 95% CI −1.27 to 4.87). The data are compatible with no difference and with an increase of up to about 5 mmol/L. With 5 mice per group, the study can detect differences of 4.3 mmol/L or more with 80% power.

Results sentence when equivalence was tested against a bound set in advance:

> Fasting glucose was similar in KO and WT mice (difference 0.30 mmol/L, 95% CI −0.15 to 0.74; Welch's t-test p = 0.19). An equivalence test against the prespecified bounds of ±1.0 mmol/L was significant (TOST p = 0.001; 90% CI −0.07 to 0.67), indicating that any difference is smaller than 1 mmol/L.

Replace common phrases with ones the data support:

| Do not write | Write instead |
| --- | --- |
| Do not write: "There was no difference between groups" | "The difference was not statistically significant (estimate, 95% CI)" |
| Do not write: "A trend towards significance (p = 0.06)" | "p = 0.06; the 95% CI ([lower] to [upper]) includes no effect, and the data are compatible with differences from [lower] to [upper]" |
| Do not write: "The knockout has no effect on glucose" | If you set equivalence bounds in advance and the 90% CI lies inside them: "We found no evidence of an effect larger than [bound] (TOST p = …)"; otherwise: "The result is inconclusive" |
| Do not write: "The study was underpowered (observed power 0.22)" | "The study could detect differences of about X or more with 80% power" |

Wood and colleagues' BMJ analysis concludes that describing a near-significant p as a "trend towards statistical significance", or blaming the result on insufficient data, gives "a misleading impression" ([Wood et al. 2014](https://doi.org/10.1136/bmj.g2215)). A p of 0.06 is a single result: with more data it can fall or rise.

In the discussion, say what the interval means for the hypothesis: name the smallest effect the data rule out (or say that none of interest is ruled out), and say what study would settle the question — for Study 2, a sample size chosen to detect the smallest change that matters.

**Do this:** copy the template that matches your result and fill in your own estimate, CI, bound and detectable effect.

## Check what an AI assistant wrote

AI assistants produce both the code and the wording, so check both:

1. **Which t-test ran?** SciPy's `ttest_ind` runs Student's test by default, and Welch's test needs `equal_var=False`; R's `t.test` runs Welch's test by default, and Student's needs `var.equal = TRUE`.
2. **Which equivalence test ran?** statsmodels' `ttost_ind` uses `usevar="pooled"` (Student) by default; pass `usevar="unequal"` for the Welch version. Check that the bounds were set before the analysis and appear in the text.
3. **Is there a confidence interval?** If the summary reports only p, ask for the estimate and 95% CI.
4. **Does the text say "no effect" or "no difference"?** Rewrite it with the table above, or support it with an equivalence test.
5. **Did it compute observed or post hoc power?** Replace it with a sensitivity analysis.

**Do this:** ask the assistant, with the code and summary pasted back: "Show the estimate with its 95% CI, state which t-test was run, and call a non-significant result 'no effect' only with an equivalence test against prespecified bounds."

## References

- Altman DG, Bland JM (1995). Absence of evidence is not evidence of absence. *BMJ* 311:485. [doi:10.1136/bmj.311.7003.485](https://doi.org/10.1136/bmj.311.7003.485)
- American Statistical Association (2016). ASA releases statement on statistical significance and p-values (press release listing the six principles). [amstat.org](https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf)
- Amrhein V, Greenland S, McShane B (2019). Scientists rise up against statistical significance. *Nature* 567:305–307. [doi:10.1038/d41586-019-00857-9](https://doi.org/10.1038/d41586-019-00857-9)
- Hoenig JM, Heisey DM (2001). The abuse of power: the pervasive fallacy of power calculations for data analysis. *The American Statistician* 55:19–24. [doi:10.1198/000313001300339897](https://doi.org/10.1198/000313001300339897)
- Lakens D (2017). Equivalence tests: a practical primer for t tests, correlations, and meta-analyses. *Social Psychological and Personality Science* 8:355–362. [doi:10.1177/1948550617697177](https://doi.org/10.1177/1948550617697177)
- Wasserstein RL, Lazar NA (2016). The ASA's statement on p-values: context, process, and purpose. *The American Statistician* 70:129–133. [doi:10.1080/00031305.2016.1154108](https://doi.org/10.1080/00031305.2016.1154108)
- Wood J, Freemantle N, King M, Nazareth I (2014). Trap of trends to statistical significance: likelihood of near significant P value becoming more significant with extra data. *BMJ* 348:g2215. [doi:10.1136/bmj.g2215](https://doi.org/10.1136/bmj.g2215)
- SciPy 1.18.1 `scipy.stats.ttest_ind`; statsmodels 0.15.0 `statsmodels.stats.weightstats.ttost_ind`; R 4.3.3 `t.test` (installed documentation and function signatures).

## Related

- [Sample size calculator: what your n can detect](https://www.licklider.ai/tools/sample-size-calculator/)
- [How to report a t-test](https://www.licklider.ai/guides/how-to-report-a-t-test/)
- [Reviewer asks you to justify the sample size](https://www.licklider.ai/guides/reviewer-asks-to-justify-sample-size/)
- [Interpreting confidence intervals](https://www.licklider.ai/guides/how-to-interpret-confidence-intervals/)
- [Adding animals after seeing results](https://www.licklider.ai/guides/adding-animals-after-seeing-results/)

## How this page was made

- Written and published by Licklider. Statements about interpretation cite the American Statistical Association statement, peer-reviewed methods papers and official software documentation listed under References; quoted wording was checked against the source text, and sources we could read only in part are cited at that level.
- The worked example uses constructed data. Its numbers were produced by stored code (Python 3.13 with SciPy 1.18.1 and statsmodels 0.15.0; R 4.3.3), and the two implementations agree to every printed digit.
- Separate review passes checked the statistics, the citations and readability before publication.