Chi-square vs Fisher's exact test: which to use for a 2 × 2 table

Quick answer. For a 2 × 2 table of counts from independent animals (each animal counted once), work out the expected counts first. If every expected count is 5 or more, use Pearson’s chi-square test without Yates’ correction — R and SciPy switch the correction on by default, so turn it off. If any expected count is below 5 in a 2 × 2 table of independent animals, use an exact test and pick it before you see the result: Boschloo’s test gives more power and stays at or below α; Fisher’s exact test is equally valid but more cautious, and is the usual choice for tables larger than 2 × 2. The cut-off of 5 is a cautious convention: Campbell (2007), comparing seven tests by computer, recommended the “N − 1” chi-square whenever every expected count is at least 1, a choice that buys power by accepting false-positive rates slightly above α (up to 6.1% at 5 mice per group in our table below), so keep the cut-off of 5 when your protocol requires α to be respected. If the same animals were scored twice, use McNemar’s test; if you counted cells from a few animals, no 2 × 2 test applies, because the animal is n, not the cell. Already have a p-value from an AI assistant? Jump to check what an AI assistant ran.

Terms used below. A 2 × 2 table has two groups (rows) and two outcomes (columns). The expected count of a cell is what you would see if the outcome did not depend on the group: row total × column total ÷ grand total. A false positive is a significant result when there is no true difference; at α = 0.05 a test should produce them in at most 5% of experiments. An exact test computes its p-value from every possible table rather than from an approximation. Yates’ correction shrinks the chi-square statistic to make it more cautious. Fisher’s exact test fixes both the row and the column totals; Boschloo’s test is an exact test that fixes only the group sizes, which gives it more power (Lydersen et al. 2009). The “N − 1” chi-square multiplies the usual statistic by (N − 1)/N. A risk difference is the difference between the two percentages; its confidence interval (CI) is the range of differences compatible with the data.

Decide in four questions

1. What is one observation? Each count in the table must come from a different, independently treated animal (or subject). Counts of cells, wells or fields from a few animals break this: go to counts of cells from a few animals.
2. Were the same animals scored twice (before and after, left and right)? Use McNemar’s test on the pairs.
3. Is the table 2 × 2? For larger tables use the chi-square test, or Fisher’s exact test when expected counts are small; see larger tables for a Python pitfall.
4. Are any expected counts below 5? No: chi-square without Yates’ correction. Yes, or not sure: an exact test (Boschloo’s or Fisher’s). This is the usual rule — the chi-square result “is relevant only when no more than 20% of cells with expected frequencies < 5 and no cell have expected frequency < 1” (Kim 2017), and in a 2 × 2 table one cell is already 25%.

If you reached the end with a 2 × 2 table of independent animals, compute the expected counts and choose with question 4. Use the statistical test chooser to apply the same rule to other designs.

A worked example

Ten mice per group; at day 60, 8 of 10 vehicle mice and 3 of 10 drug-treated mice had a tumour (constructed data).

Group	Tumour	No tumour	Total
Vehicle	8	2	10
Drug	3	7	10
Expected counts	5.5 in each “tumour” cell	4.5 in each “no tumour” cell

Two expected counts are 4.5, below 5, so the rule above points to an exact test. Here is what each common test gives on the same table:

Test	Statistic	Two-sided p
Chi-square with Yates’ correction (R and SciPy default)	χ² = 3.23	0.072
Chi-square without correction	χ² = 5.05	0.025
“N − 1” chi-square (Campbell 2007)	χ² = 4.80	0.028
Fisher’s exact test	—	0.070
Boschloo’s exact test, SciPy boschloo_exact (groups as columns)	—	0.041
Boschloo’s exact test, R exact2x2::boschloo 1.7.0 (default tsmethod = "central")	—	0.033, too small: a tied table is dropped (see below)
Boschloo’s exact test, R with tied tables kept (boschloo_tie_fix.R)	—	0.041

The tests split around 0.05, which is exactly why the choice must be made before you see the result. Choosing the test that gives the smallest p after looking is a hidden multiple comparison.

Whatever the test, report the size of the effect: 80% vs 30% of mice developed a tumour, a risk difference of 50 percentage points (95% CI 7 to 74, Newcombe’s method). Fisher’s test in R reports the conditional odds ratio, 8.15 (95% CI 0.88 to 127): wide, because 20 mice carry little information about a ratio of odds.

tab <- matrix(c(8, 3, 2, 7), nrow = 2,
              dimnames = list(group = c("vehicle", "drug"), outcome = c("tumour", "none")))
chisq.test(tab)$expected            # 5.5 and 4.5: below 5, so use an exact test
fisher.test(tab)                    # p = 0.070, conditional odds ratio 8.15 (0.88 to 127)
# install.packages("exact2x2")     # once
library(exact2x2)
boschloo(8, 10, 3, 10)              # p = 0.033 in exact2x2 1.7.0: too small, a tied table is dropped; see boschloo_tie_fix.R (0.041)
chisq.test(tab, correct = FALSE)    # p = 0.025, only if every expected count were 5 or more

import numpy as np
from scipy import stats
from scipy.stats.contingency import expected_freq, odds_ratio
tab = [[8, 2], [3, 7]]                      # rows: vehicle, drug; columns: tumour, none
print(expected_freq(tab))                   # 5.5 and 4.5: below 5, so use an exact test
print(stats.fisher_exact(tab).pvalue)       # 0.070
print(stats.boschloo_exact(np.transpose(tab)).pvalue)  # 0.041; SciPy wants groups as columns
res = odds_ratio(tab, kind="conditional")   # matches R's fisher.test estimate
print(res.statistic, res.confidence_interval())
print(stats.chi2_contingency(tab, correction=False).pvalue)  # 0.025, only with all expected counts >= 5

Then compute the effect size and its interval:

from statsmodels.stats.proportion import confint_proportions_2indep

# Risk difference with Newcombe's 95% CI: tumours / mice in each group
low, high = confint_proportions_2indep(8, 10, 3, 10, method="newcomb", compare="diff")
print(round(8 / 10 - 3 / 10, 2), round(low, 2), round(high, 2))   # 0.5, 0.07, 0.74

# Risk difference with Newcombe's 95% CI (hybrid score method), built from Wilson intervals
wilson <- function(x, n, z = qnorm(0.975)) {
  p <- x / n; centre <- (p + z^2 / (2 * n)) / (1 + z^2 / n)
  half <- z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2)) / (1 + z^2 / n)
  c(centre - half, centre + half)
}
newcombe <- function(x1, n1, x2, n2) {
  p1 <- x1 / n1; p2 <- x2 / n2; w1 <- wilson(x1, n1); w2 <- wilson(x2, n2)
  d <- p1 - p2
  c(d, d - sqrt((p1 - w1[1])^2 + (w2[2] - p2)^2), d + sqrt((w1[2] - p1)^2 + (p2 - w2[1])^2))
}
round(newcombe(8, 10, 3, 10), 2)   # 0.50 0.07 0.74

What to do: check the expected counts, choose the test from question 4, and report the counts, the two percentages and their difference with its confidence interval.

How often each test gives a false positive

We computed the exact false-positive rate of each test at α = 0.05 for two groups of equal size with no true difference. Every possible table was weighted by its probability, so the numbers are exact, not simulated. The table shows the largest rate across common outcome risks of 10%, 20%, 30% and 50%.

Mice per group	Yates’ chi-square	Fisher’s exact	Boschloo (SciPy)	Chi-square without correction
5	0.2%	2.2%	2.2%	6.1%
10	1.3%	1.3%	4.2%	4.2%
20	2.1%	2.5%	4.5%	5.3%

Three things follow. Yates’ correction makes the chi-square test as cautious as Fisher’s test or more, so with the correction on you have paid for an exact test without getting one. With no true effect, Fisher’s test stayed at or below 5% but its false-positive rate peaked at only 1.3% at 10 mice per group (2.5% at 20 per group, across outcome risks of 10–50%), and that unused margin is lost power. Boschloo’s test also stayed at or below 5% while using more of it. The uncorrected chi-square (and the “N − 1” version, which had identical rates for these equal group sizes) went slightly above 5% in some cases, up to 6.1% with 5 mice per group, which is why small tables go to an exact test.

Lydersen et al. (2009) reached the same conclusion from a review of the tests: unconditional exact tests such as Boschloo’s “preserve the significance level and generally are more powerful than Fisher’s exact test for moderate to small samples”. Campbell (2007) compared seven tests by computer and recommended the “N − 1” chi-square when every expected count is at least 1, and Fisher’s test otherwise, as more powerful than the expected-count-of-5 rule.

The rule as a whole — uncorrected chi-square when every expected count is at least 5, an exact test otherwise — can still run slightly above 5%, because the chi-square branch does even where the rule allows it:

Mice per group	Rule with Fisher for small tables	Rule with Boschloo for small tables	Boschloo for every table
10	1.3%	4.2%	4.2%
20	5.0%	5.3%	4.5%
30	5.2%	5.2%	4.7%

(Largest exact rate across the same four risks; 5.04% and 5.30% at 20 per group with a 30% risk, 5.19% at 30 per group with a 50% risk.)

What to do: keep Yates’ correction off; if your protocol needs a strict guarantee, use Boschloo’s test for every 2 × 2 table regardless of size (Fisher’s test also stays at or below α, with less power).

Software defaults that change the answer

Software	Default	What to do
R chisq.test()	correct = TRUE: Yates’ correction “for 2 by 2 tables”; warns “Chi-squared approximation may be incorrect” when expected counts are small	Pass correct = FALSE, or switch to an exact test when the warning appears
SciPy chi2_contingency()	correction=True: Yates’ correction when the table has one degree of freedom (a 2 × 2 table); no warning	Pass correction=False
R fisher.test()	Odds ratio is the conditional maximum-likelihood estimate (8.15 in the example)	Report it with its confidence interval
SciPy fisher_exact()	Statistic for a 2 × 2 table is the sample odds ratio (9.33 in the example); same p-value as R	Use scipy.stats.contingency.odds_ratio(tab, kind="conditional") to match R and get a CI
SciPy boschloo_exact()	Treats each column as one group; two-sided p = twice the smaller one-sided p	Pass the table with groups as columns (np.transpose(tab) for a groups-as-rows table); SciPy’s 0.041 is the correct value for this definition
R exact2x2::boschloo() 1.7.0	Orders tables by Fisher’s p and compares those p-values with exact equality; a table that ties the observed one can differ in the last digits and is dropped, so here it reports 0.033 (one-sided 0.016) instead of 0.041 (one-sided 0.021), and the result can change with the order of the groups (reported to the maintainer)	Use SciPy, or order by the Fisher p rounded to 12 significant figures (boschloo_tie_fix.R, which reproduces 0.041)
SciPy fisher_exact() on tables larger than 2 × 2	p-value estimated from 9,999 random tables, without a fixed seed	Pass method=stats.MonteCarloMethod(rng=np.random.default_rng(1), n_resamples=1_000_000), or use R’s exact fisher.test()

For the larger-table case: on a 2 × 3 table, three seeded SciPy runs at the default size gave 0.035, 0.033 and 0.032; one million random tables gave 0.0336, and R’s exact fisher.test gave 0.0333. Unseeded runs give a different p each time.

What to do: write the test, the correction (on or off) and the software version in Methods, and set a seed for any resampled p-value.

Paired yes/no data: use McNemar

If the same 38 mice were scored before and after treatment, the four cells are pairs, not independent groups. McNemar’s test uses only the animals that changed category.

	After: positive	After: negative
Before: positive	10	15
Before: negative	5	8

Fifteen mice changed from positive to negative and five the other way. The exact McNemar test gives p = 0.041 (the continuity-corrected chi-square version, R’s mcnemar.test, gives 0.044). Treating the same four numbers as independent groups in a chi-square test would give p = 0.93, the wrong analysis for paired before/after scores.

paired <- matrix(c(10, 5, 15, 8), nrow = 2)   # rows: before +/-, columns: after +/-
binom.test(paired[1, 2], paired[1, 2] + paired[2, 1])   # exact McNemar: p = 0.041

from statsmodels.stats.contingency_tables import mcnemar
paired = [[10, 15], [5, 8]]                   # rows: before +/-, columns: after +/-
print(mcnemar(paired, exact=True).pvalue)     # 0.041

What to do: report the number of animals that changed in each direction and the exact McNemar p.

Counts of cells from a few animals

A common table: 3 mice per group, 100 cells scored per mouse, apoptotic or not. Pooled, that is 55 of 300 vehicle cells and 85 of 300 drug cells, and a chi-square test on the pooled table gives p = 0.004. But the mice differ: their apoptotic fractions were 12%, 25% and 18% (vehicle) and 22%, 35% and 28% (drug). Treating each mouse’s fraction as one value, Welch’s t-test gives t(4.0) = 1.88, p = 0.13, difference 10 percentage points (95% CI −5 to 25).

The pooled chi-square treats 600 cells as 600 independent animals (what counts as n). We simulated 20,000 such experiments with no true effect (3 mice per group, 100 cells per mouse, baseline 20%, seed 20261110):

Mouse-to-mouse variation (SD on the logit scale)	False positives, pooled chi-square on cells	False positives, Welch’s t-test on mouse fractions
None	5.0%	3.7%
0.3	20.9%	3.4%
0.5	37.3%	3.3%

When mice really differ, the pooled test reports a “significant” difference in about one experiment in five or more. Lazic (2010) found pseudoreplication in 12% of papers in one issue of Nature Neuroscience, with a further 36% suspected.

import numpy as np
from scipy import stats

vehicle = np.array([12, 25, 18]) / 100   # apoptotic cells / cells scored, per mouse
drug = np.array([22, 35, 28]) / 100
res = stats.ttest_ind(drug, vehicle, equal_var=False)
print(round(res.statistic, 2), round(res.df, 1), round(res.pvalue, 2))   # 1.88, 4.0, 0.13

vehicle <- c(12, 25, 18) / 100   # apoptotic cells / cells scored, per mouse
drug <- c(22, 35, 28) / 100
t.test(drug, vehicle)            # Welch: t = 1.88, df = 4.0, p = 0.13

What to do: use one proportion per animal and compare the animals with Welch’s t-test, as above; report n as the number of animals. A binomial mixed model with animal as a random effect (R lme4::glmer) also uses every cell, but its z-test p-values are too optimistic with only a few animals per group (on these data p = 0.023 against 0.13 from the mouse fractions), so keep it for studies with many animals and see nested data. When cell totals differ between animals, or you want an odds ratio, see comparing percentages between groups.

Check what an AI assistant ran

1. Yates on or off? chisq.test(tab) and chi2_contingency(tab) apply it silently for 2 × 2 tables. Look for correct = FALSE or correction=False.
2. Expected counts checked? Ask for them; if any is below 5, ask for Fisher’s or Boschloo’s test instead.
3. What is n? If the table holds cells, wells or images rather than animals, the test is on the wrong unit.
4. Paired data? Before/after scores on the same animals need McNemar’s test, not a chi-square test.
5. Larger table in Python? fisher_exact on a 2 × 3 or bigger table returns a random estimate unless a seed is set.

Ask the assistant to rerun with the fix if any answer is wrong, and keep the expected counts in your notes.

Report it

Pick the exact test before the analysis and name it in the report; the template below uses Fisher’s test, and the note after it shows the change for Boschloo’s.

Tumour incidence at day 60 was 8 of 10 vehicle-treated and 3 of 10 drug-treated mice (80% vs 30%; difference 50 percentage points, 95% CI 7 to 74, Newcombe’s method). Because two expected counts were below 5, incidence was compared with Fisher’s exact test (two-sided, p = 0.070; conditional odds ratio 8.15, 95% CI 0.88 to 127). Analyses used R 4.3.3.

If you used Boschloo’s test, name the package and version and say that the two-sided p is twice the smaller one-sided p (for example “Boschloo’s test, SciPy 1.18.1 boschloo_exact”). A p of 0.070 does not show that the drug has no effect; the interval for the difference runs from 7 to 74 percentage points. Copy the template, replace the numbers, and keep the effect size and interval in the sentence.

Corrections

10 November 2026: the Python code for Boschloo’s test passed the table with groups as rows, but SciPy treats each column as one group. The code now transposes the table; the SciPy p-value changed from 0.032 to 0.041, and the Boschloo error rates at 20 and 30 mice per group were recomputed (4.5% and 4.7%, previously 4.9%). The chi-square and Fisher results were not affected. The R function exact2x2::boschloo() 1.7.0 gives 0.033 on the same table because it drops a tied table (exact comparison of floating-point p-values); the page previously presented that value as an equally valid alternative definition. A workaround that keeps ties (boschloo_tie_fix.R) reproduces SciPy’s 0.041.

References

• Campbell I (2007). Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations. Statistics in Medicine 26:3661–3675. doi:10.1002/sim.2832
• Lydersen S, Fagerland MW, Laake P (2009). Recommended tests for association in 2×2 tables. Statistics in Medicine 28:1159–1175. doi:10.1002/sim.3531
• Kim HY (2017). Statistical notes for clinical researchers: Chi-squared test and Fisher’s exact test. Restorative Dentistry & Endodontics 42:152–155. doi:10.5395/rde.2017.42.2.152
• Lazic SE (2010). The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? BMC Neuroscience 11:5. doi:10.1186/1471-2202-11-5
• R Core Team. chisq.test and fisher.test documentation, R 4.3.3. stat.ethz.ch/R-manual
• Fay MP. exact2x2 package documentation, version 1.7.0. cran.r-project.org/package=exact2x2
• SciPy 1.18.1 documentation: chi2_contingency, fisher_exact, boschloo_exact.