Why a statistical method name is not enough

Current status

The finding

A familiar procedure name is not enough to identify what a statistical result guarantees. The meaning also depends on which claims belong to the comparison family, which error quantity is controlled, the theorem's assumptions, sidedness, balance conditions, and the exact procedure variant.

This matters because closely associated names can refer to different questions. Controlling the probability of any false rejection in a family is not the same as controlling the expected proportion of false discoveries. Comparing every group with one control is not the same as comparing every pair. A later extension also cannot inherit the evidence for an earlier procedure merely by sharing its name.

What the primary sources establish

The investigation inspected the primary texts for Holm's sequentially rejective procedure, the original Benjamini–Hochberg false-discovery-rate procedure, Dunnett's many-to-one construction, and Tukey's 1949 procedure. The review kept their claims separate rather than treating the four names as interchangeable labels for “multiple comparisons.”

• Holm's main 1979 result gives strong family-wise error control using valid marginal test levels and the union bound. A separate product-form construction in the same paper uses an independence assumption.
• The 1995 Benjamini–Hochberg theorem is an independence result about false discovery rate. Later results for other dependence structures are not supplied by that paper.
• Dunnett's 1955 construction addresses many-to-one comparisons. Its numerical tables have recorded limits involving balance, sidedness, variance assumptions, and approximation.
• Tukey's 1949 paper describes a three-stage gap, straggler, and upper-tail F procedure. It does not establish the later Studentized-range all-pairs procedure or its unequal-size extension.

Separating later all-pairs sources

A follow-up investigation examined later sources commonly associated with all-pairs comparisons. It found that their evidential roles must also remain separate.

The authenticated text of Tukey's 1953 manuscript or its 1994 archival printing was not inspected. Exact historical attribution to that manuscript therefore remains open even though the later mathematical and source-role questions above have been narrowed and closed.

Limits

Independent close reviews checked the frozen reports, their source-claim boundaries, and the repository evidence. Some inspected source PDFs could not be redistributed in the public repository and were unavailable to the close-review environments. The review records preserve that access limitation instead of reconstructing or broadening the source claims.

This note does not compare the operating performance of the procedures, recommend a default, verify a software implementation, or show that one error criterion is universally preferable. Results remain limited to the cited editions, theorem conditions, and source roles recorded in the investigation.

Relation to nomue Protocol

The finding is research input for future identity and interpretation design. It does not add a Record field, vocabulary term, method identifier, schema, public check, supported procedure, or implementation behavior.

Four of eight recorded FND-1 holds are closed in narrowed form. Four holds remain open, alongside the historical-attribution residual described above. The complete FND-1 research gate is therefore still open, and adopting any result would require a separate Protocol proposal and review.

Public evidence

• Multiplicity primary-text result— inspected claims, page locations, and evidence boundaries
• Multiplicity close review— independent review result and recorded access limits
• Multiplicity steward disposition— accepted findings, remaining holds, and explicit non-decisions
• All-pairs source completion— later-source inspection and claim ledger
• All-pairs close review— independent reconciliation review
• All-pairs steward disposition— closed source roles and the remaining historical-attribution question