Multiple comparison correction calculator (Holm, Bonferroni, Benjamini–Hochberg and more)

Quick answer. Paste the raw p-values of every test in the family — every test you ran for the question, including the unremarkable ones — keep Holm unless your plan named another method, and read off the adjusted p-values: a test is rejected when its adjusted p is at or below α. Already analysed without a plan? Use Holm across every test you ran for the question, not only the ones you reported as findings, and say in Methods that the family was defined after the analysis. Tukey, Games–Howell and Dunnett output is already adjusted: report those adjusted p-values as they are. The decision guide explains which correction fits which question.

Terms used below. Raw p-value: the p-value of one test before any correction. Family: the set of tests whose errors you count together, usually every test you ran to answer one question. Adjusted p-value: a p-value corrected for the family; compare it with α as usual. α (alpha): the significance level, usually 0.05. Familywise error rate: the chance of one or more false positives across the family. False discovery rate: the expected share of false positives among the results you call significant. Positively associated: tests whose results tend to move together, for example several outcomes measured in the same animals; we use this one phrase throughout.

How to use it

1. Paste the raw p-values, one per line, optionally with a label (IL6 0.00162), at full precision.
2. If you are entering only part of the family, put the full count in Total number of tests in the family.
3. Keep Holm unless your plan named another method.
4. Set the Significance level (α): 0.05 unless your plan named another level (the field accepts 0.001 to 0.2).
5. Read the adjusted p-values and which tests are rejected at that α.
6. Copy the Methods sentence, the R or Python code, or the CSV.

Copying from a spreadsheet: copy the label and raw p columns. The calculator reads one number per line, which keeps already-adjusted values from being adjusted twice, and it asks which separator you use when values have decimal commas (0,05).

What the adjusted p-value means

An adjusted p-value is the smallest significance level at which a test would be rejected once the whole family is taken into account (Wright 1992). Compare it with your α as you would a raw p-value.

• Familywise error rate (Holm, Hochberg, Hommel, Bonferroni, Šidák, Holm–Šidák). If the family is the one you planned, the chance of one or more false positives among all the tests is at most α. Use it when one false claim would matter, such as a few outcomes each reported as a finding.
• False discovery rate (Benjamini–Hochberg, Benjamini–Yekutieli). On average, at most a proportion α of the rejected results are false positives; which ones stays open. Use it for a screen whose output is a list of candidates to confirm.

Hochberg and Hommel need tests that are independent or positively associated (R p.adjust documentation); Šidák and Holm–Šidák are exact for independent tests, valid for two-sided tests whose statistics are jointly normal with any correlation (Šidák 1967), and otherwise need positively associated tests; Holm and Benjamini–Yekutieli keep their error rate under any dependence (they guarantee the rate; they do not model the correlation). When unsure, keep Holm.

A worked example

A constructed example: five outcomes measured in the same mice, treated versus vehicle, each with a Welch t-test (the example on the reviewer-response guide). The calculator opens with these five values; if you cleared it, re-enter them from the table:

Outcome	Raw p	Holm	Benjamini–Hochberg
IL-6	0.00162	0.0081	0.0081
TNF-α	0.00443	0.0177	0.0111
Paw thickness	0.01705	0.0512	0.0284
CRP	0.02631	0.0526	0.0329
Body weight	0.5674	0.567	0.567

With Holm’s method, 2 of 5 tests are rejected at α = 0.05; with Benjamini–Hochberg, 4 of 5. The two methods answer different questions, so the choice belongs in the plan: if these five outcomes were all reported as findings, Holm is the right familywise correction, and paw thickness and CRP did not pass it (adjusted p just above 0.05); report them with their estimates and confidence intervals.

Write it as: “IL-6 and TNF-α differed between groups after Holm’s correction across the five outcomes (adjusted p = 0.0081 and 0.0177); the differences in paw thickness and CRP had adjusted p = 0.051 and 0.053.”

The same numbers in R and Python:

p <- c(IL6 = 0.00162, TNFa = 0.00443, paw = 0.01705, CRP = 0.02631, weight = 0.5674)
round(p.adjust(p, method = "holm"), 4)   # Holm
round(p.adjust(p, method = "BH"), 4)     # Benjamini-Hochberg
round(p.adjust(p, method = "holm", n = 12), 4)   # Holm when these 5 are part of a family of 12

# needs statsmodels (pip install statsmodels)
from statsmodels.stats.multitest import multipletests

p = [0.00162, 0.00443, 0.01705, 0.02631, 0.5674]   # IL-6, TNF-a, paw, CRP, weight
print(multipletests(p, method="holm")[1].round(4))     # Holm (pass method="holm": the default is "hs")
print(multipletests(p, method="fdr_bh")[1].round(4))   # Benjamini-Hochberg

Before you use it

• Choose the family and the method before the data. The family is every test you ran for the question, including the unremarkable ones; which correction to use gives the decision table. If you entered only some of the tests, put the full count in Total number of tests in the family (R’s n argument).
• Comparing groups after an ANOVA? Tukey, Games–Howell and Dunnett use the joint distribution of the comparisons and need the data rather than a list of p-values; run them as shown in the correction guide and the post hoc guide.
• Enter exact p-values. Copy the full-precision value your software prints; an inequality such as “<0.001” has no adjusted value. In GraphPad Prism, change the number format to show more digits; in Excel, increase the decimal places. Plugging in the bound (0.001) would give only an upper limit for the adjusted p, not the adjusted p itself.
• Paste raw p-values only. The calculator cannot tell whether a value has already been adjusted; correcting adjusted p-values again makes them too large.
• Nothing survives correction? Report each estimate with its confidence interval and say which tests were corrected; the reviewer-response guide has templates.

How it is calculated

Sort the m raw p-values from smallest to largest, p₍₁₎ ≤ … ≤ p₍ₘ₎, apply the method’s formula, enforce the order with a running maximum or minimum, and cap at 1. Results return in your input order.

Method	Adjusted p for rank i	Controls	Valid when
Holm (1979) — default	running maximum (from the smallest) of (m − i + 1) · p₍ᵢ₎	Familywise error	Any dependence
Hochberg (1988)	running minimum (from the largest) of (m − i + 1) · p₍ᵢ₎	Familywise error (under the condition at right)	Independent or positively associated tests
Hommel (1988)	closed testing with Simes tests (R’s algorithm)	Familywise error (under the condition at right)	Independent or positively associated tests
Bonferroni	m · pᵢ	Familywise error	Any dependence; never more powerful than Holm
Holm–Šidák (statsmodels’ default method)	running maximum of 1 − (1 − p₍ᵢ₎)^(m − i + 1)	Familywise error (under the condition at right)	Exact if independent; valid if positively associated
Šidák (1967)	1 − (1 − pᵢ)^m	Familywise error (under the condition at right)	Exact if independent; valid if positively associated
Benjamini–Hochberg (1995)	running minimum (from the largest) of (m / i) · p₍ᵢ₎	False discovery rate (under the condition at right)	Independent or positively associated tests
Benjamini–Yekutieli (2001)	as Benjamini–Hochberg, multiplied by 1 + 1/2 + … + 1/m	False discovery rate	Any dependence

The six methods that R offers follow p.adjust step by step, including the order of the arithmetic. The Šidák pair follows statsmodels and is computed with log1p and expm1 so that very small p-values keep their precision. Hommel needs time proportional to m², so it is computed for up to 10,000 tests; above that, use R (this is why the Hommel rows below have one case fewer: the case of 100 p-values in a family of 20,000 is outside that limit). A test is rejected when its adjusted p is at or below α (statsmodels’ own reject flag compares with critical values and can differ exactly at the boundary).

Checked against R and statsmodels

42 test cases (1 to 10,000 p-values; ties, 0 and 1, values down to 5 × 10⁻³²⁴, and a “total number of tests” larger than the list), compared with R 4.3.3 p.adjust, statsmodels 0.15.0 multipletests, SciPy 1.18.1 false_discovery_control and high-precision mpmath 1.4.1. The tolerance is a relative difference of 10⁻¹²; a failing check blocks publication.

Reference	Method	Cases	Values	Bit-identical	Largest relative difference
R p.adjust	Holm	42	18,428	18,428	identical
R p.adjust	Hochberg	42	18,428	18,428	identical
R p.adjust	Hommel	41	18,328	18,328	identical
R p.adjust	Bonferroni	42	18,428	18,428	identical
R p.adjust	Benjamini–Hochberg	42	18,428	18,428	identical
R p.adjust	Benjamini–Yekutieli	42	18,428	18,428	identical
statsmodels multipletests	Holm	40	3,428	3,428	identical
statsmodels multipletests	Hochberg	40	3,428	3,428	identical
statsmodels multipletests	Hommel	39	3,328	3,328	identical
statsmodels multipletests	Bonferroni	40	3,428	3,428	identical
statsmodels multipletests	Holm–Šidák	40	3,428	3,389	2.5 × 10⁻¹⁶
statsmodels multipletests	Šidák	40	3,428	3,401	2.2 × 10⁻¹⁶
statsmodels multipletests	Benjamini–Hochberg	40	3,428	1,894	2.5 × 10⁻¹⁶
statsmodels multipletests	Benjamini–Yekutieli	40	3,428	3,165	4.3 × 10⁻¹⁶
SciPy false_discovery_control	Benjamini–Hochberg	29	2,652	2,652	identical
SciPy false_discovery_control	Benjamini–Yekutieli	29	2,652	2,439	4.1 × 10⁻¹⁶
mpmath (high precision)	Holm–Šidák	42	3,584	3,434	2.3 × 10⁻¹⁶
mpmath (high precision)	Šidák	42	3,584	3,443	2.2 × 10⁻¹⁶

Runs in Node.js (V8) on every change and in headless Chromium; Firefox and Safari were unavailable in the test environment. The Šidák methods use a bundled port of fdlibm’s log1p and expm1 instead of the browser’s, so every browser computes the same bits.

The cases, the scripts that produced the reference values and the tests are kept with the site’s source (scripts/tools/correction_reference.py, correction_reference.R and multiple-correction.test.mjs).

Questions a reviewer may ask

“Why Holm and not Bonferroni?” Holm controls the same familywise error rate under any dependence and always rejects at least as many tests; R’s documentation says there “seems no reason to use the unmodified Bonferroni correction because it is dominated by Holm’s method” (R p.adjust).

“Why not the false discovery rate?” Use it when the output is a list of candidates to confirm. When each result is reported as a finding, a familywise method fits the claim.

“Why did you correct for 6 tests rather than the 3 you discuss?” Every test you ran for the question belongs in the family; correcting only for the ones that looked interesting would undo the correction.

“Your statsmodels output differs from Holm.” multipletests uses Holm–Šidák ("hs") unless you pass method="holm"; choose Holm–Šidák in the calculator to reproduce it.

Report it

SAMPL asks authors to “indicate whether and how any allowance or adjustments were made for multiple comparisons” (Lang & Altman 2013). For the example, Copy methods sentence gives:

P-values for the [5] tests [name the family, e.g. the five outcomes][, fixed before the analysis — only if a dated plan says so] were adjusted with Holm’s method (Holm 1979), which controls the familywise error rate at [0.05]; adjusted p-values are reported. Calculated with the Licklider multiple-comparison correction calculator (https://www.licklider.ai/tools/multiple-comparison-correction-calculator/, accessed [date]), the same calculation as R p.adjust(p, method = “holm”).

Add which tests formed the family and, if it was, that the family was fixed before the analysis.

How SciGround helps

SciGround can test all your planned outcomes for the same two groups together with a pre-specified correction, so one lucky result can’t pass as a finding, and every outcome stays in the report (supported analyses). It covers outcome sets fixed in advance; outcomes chosen after seeing the results are outside its scope.

Cite this tool

Licklider. Multiple comparison correction calculator. https://www.licklider.ai/tools/multiple-comparison-correction-calculator/ (version 1.0, accessed [date]).

Vancouver: Licklider. Multiple comparison correction calculator [Internet]. Version 1.0. 2026 [cited YYYY Mon DD]. Available from: https://www.licklider.ai/tools/multiple-comparison-correction-calculator/

@misc{licklider_correction_calculator,
  author = {{Licklider}},
  title  = {Multiple comparison correction calculator},
  year   = {2026},
  note   = {Version 1.0},
  url    = {https://www.licklider.ai/tools/multiple-comparison-correction-calculator/}
}

Changelog: version 1.0 (10 November 2026) — eight methods, total-number-of-tests option, R and Python code, Methods sentence, CSV. Last verified 10 November 2026.

References

• Benjamini Y, Hochberg Y (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B 57(1):289–300. doi:10.1111/j.2517-6161.1995.tb02031.x
• Benjamini Y, Yekutieli D (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics 29(4):1165–1188. doi:10.1214/aos/1013699998
• Hochberg Y (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75(4):800–802. doi:10.1093/biomet/75.4.800
• Holm S (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6(2):65–70. jstor.org/stable/4615733
• Hommel G (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75(2):383–386. doi:10.1093/biomet/75.2.383
• Lang T, Altman D (2013). Statistical Analyses and Methods in the Published Literature: the SAMPL Guidelines. EQUATOR Network. equator-network.org
• R Core Team. p.adjust: Adjust P-values for Multiple Comparisons. R 4.3.3 documentation. stat.ethz.ch
• Šidák Z (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association 62(318):626–633. doi:10.1080/01621459.1967.10482935
• statsmodels developers. multipletests. statsmodels 0.15.0 documentation. statsmodels.org
• Wright SP (1992). Adjusted P-values for simultaneous inference. Biometrics 48(4):1005–1013. doi:10.2307/2532694