Published Updated

A Julia signed-rank p-value above 1, now fixed in a release

HypothesisTests.jl returned 1.25 for an exact two-sided signed-rank p-value. The matching correction shipped in v0.12.0 and remains in v0.12.2.

Current status

We reported HypothesisTests.jl issue #373 on August 25, 2026 after reproducing an exact two-sided signed-rank p-value above 1. Pull request #377 was opened byyoninazarathy on September 2 and merged into master byandreasnoack later that day.

As of September 11, 2026, the matching correction has shipped inv0.12.0 (September 3) and remains in the latest registered version,v0.12.2 (September 10). Issue #373 remains open. The public tracker does not state that #373 caused #377, so this page does not make that attribution.

What we observed

In HypothesisTests.jl 0.11.8 on Julia 1.12.7, the untied exact path returned 1.25 for this call:

pvalue(ExactSignedRankTest([1.0, 2.0, -3.0]))

A p-value is a probability, so it must remain between 0 and 1. For this three-observation example, exhaustive enumeration of the eight possible sign assignments gives two one-sided probabilities of 5/8. The standard doubled-tail value is therefore min(1, 2 × 5/8) = 1.

What changed upstream

Pull request #377 changed the no-ties exact two-sided branch so the selected tail is doubled and capped at 1.0. The merged regression test pins the example above to exactly 1.

The pull request also sweeps every untied sign pattern through sample sizen = 12 and asserts that no two-sided result exceeds 1. Its author reported that 220 of 8,190 swept patterns exceeded 1 before the patch.

How we checked it

We ran the same fixture in clean Julia 1.12.7 environments against two pinned package states. Registered HypothesisTests.jl 0.11.8returned 1.25; merge commit f758eea returned1.0. Both checks used assertions on the displayed result rather than relying only on source inspection.

On September 11, we checked the release records, General registry, and v0.12.2 source and regression test. The source retains min(2 * p, 1.0), and the test still requires the three-observation example to return exactly 1. The tagged commit 3949d8ca752d815904e24eba561e982c003b0463 has tree 0b7026cbe6b5380b8f8a5c2a7306363e2ebd2146, matching the registry entry. This confirms release inclusion by source inspection; the execution results below remain the original pinned runs.

Package stateObserved p-valueStatus
Registered version 0.11.81.25Released behavior reproduced
Merge commit f758eea1.0Matching fix reproduced

Scope

This report concerns the untied, no-zero, exact two-sided signed-rank path in the tested HypothesisTests.jl versions. It does not show that every signed-rank call is affected, and it does not estimate how often ordinary research workflows reach a statistic at the null-distribution center.

Version 0.11.8 predates the fix; v0.12.0, v0.12.1 and v0.12.2 contain it. Release inclusion does not establish correctness for every rank-test path. Our report and the separately authored patch remain distinct because the upstream record does not establish a causal link.

Relation to verification

The case shows the value of checking simple mathematical invariants alongside high-precision reference values. A range assertion identified an invalid result; exhaustive enumeration explained the expected answer; and a pinned cross-version run separated released behavior from the current upstream fix.

This is evidence of Licklider's verification engineering method. It does not mean that nomue Protocol currently supports Julia or signed-rank verification.

Public evidence