Published Updated
A Julia signed-rank p-value above 1, now fixed in a release
HypothesisTests.jl returned 1.25 for an exact two-sided signed-rank p-value. The matching correction shipped in v0.12.0 and remains in v0.12.2.
Earlier version. nomue was an earlier version of SciGround. Its verification capabilities are now part of SciGround; earlier publications, packages, and records keep the nomue name as a historical record.
Current status
We reported HypothesisTests.jl issue #373 on August 25, 2026 after reproducing an exact two-sided signed-rank p-value above 1. Pull request #377 was opened by yoninazarathy on September 2 and merged into master by andreasnoack later that day.
On November 14, 2026, the upstream project closed issue #373 as fixed and identified #377 as the resolution. The correction first shipped in v0.12.0 on September 3 and is included in v0.12.2. The upstream follow-up confirmed that the original example returns 1.0on master at e7ac59e.
What we observed
In HypothesisTests.jl 0.11.8 on Julia 1.12.7, the untied exact path returned 1.25 for this call:
pvalue(ExactSignedRankTest([1.0, 2.0, -3.0]))A p-value is a probability, so it must remain between 0 and 1. For this three-observation example, exhaustive enumeration of the eight possible sign assignments gives two one-sided probabilities of 5/8. The standard doubled-tail value is therefore min(1, 2 × 5/8) = 1.
What changed upstream
Pull request #377 changed the no-ties exact two-sided branch so the selected tail is doubled and capped at 1.0. The merged regression test pins the example above to exactly 1.
The pull request also sweeps every untied sign pattern through sample size n = 12 and asserts that no two-sided result exceeds 1. Its author reported that 220 of 8,190 swept patterns exceeded 1 before the patch.
How we checked it
We ran the same fixture in clean Julia 1.12.7 environments against two pinned package states. Registered HypothesisTests.jl 0.11.8 returned 1.25; merge commit f758eea returned 1.0. Both checks used assertions on the displayed result rather than relying only on source inspection.
On September 11, we checked the release records, General registry, and v0.12.2 source and regression test. The source retains min(2 * p, 1.0), and the test still requires the three-observation example to return exactly 1. The tagged commit 3949d8ca752d815904e24eba561e982c003b0463 has tree 0b7026cbe6b5380b8f8a5c2a7306363e2ebd2146, matching the registry entry. This confirms release inclusion by source inspection; the execution results below remain the original pinned runs.
| Package state | Observed p-value | Status |
|---|---|---|
Registered version 0.11.8 | 1.25 | Released behavior reproduced |
Merge commit f758eea | 1.0 | Matching fix reproduced |
Scope
This report concerns the untied, no-zero, exact two-sided signed-rank path in the tested HypothesisTests.jl versions. It does not show that every signed-rank call is affected, and it does not estimate how often ordinary research workflows reach a statistic at the null-distribution center.
Version 0.11.8 predates the fix; v0.12.0, v0.12.1 and v0.12.2 contain it. The regression test covers the upper bound for the untied, exact two-sided path.
Relation to verification
The case shows the value of checking simple mathematical invariants alongside high-precision reference values. A range assertion identified an invalid result; exhaustive enumeration explained the expected answer; and a pinned cross-version run separated released behavior from the current upstream fix.
This is evidence of Licklider's verification engineering method. It does not mean that nomue Protocol currently supports Julia or signed-rank verification.
Public evidence
- v0.12.0 release notes— correction #377, plus separate contributions on interval direction (#382), point estimates (#383), interval consistency and zeros (#386), computation limits (#388), and mathematical specifications (#389). These are not additional outcomes attributed to our report; callers should review the breaking changes.
- v0.12.2 release, released source and regression test— latest checked release and retained correction; the source and test links are commit-pinned
- HypothesisTests.jl issue #373— report, reproducer, exact enumeration, source diagnosis, and environment
- Upstream resolution of #373— confirms #377 as the fix, v0.12.0 as the first release with the fix, and a p-value of 1.0 for the original example
- Reporter follow-up on #373— historical September 3 master and release-status snapshot
- HypothesisTests.jl pull request #377— patch rationale, regression tests, review, and merge
- Merge commit f758eea— correction merged into
master - HypothesisTests.jl v0.11.8 release— affected release used in the original reproduction
- Julia General version registry— registered versions through 0.12.2 at the September 11 check