Publications

Follow the finding to its upstream outcome.

Reproducible checks and upstream reports on scientific software and research infrastructure, including SciPy, python-flint and the MCP Python SDK.

Upstream reports

Findings and upstream outcomes

Statuses checked November 9, 2026 (UTC).

25 reports submitted ·8 matching fixes adopted ·1 limitation acknowledged ·1 closed without a planned fix

Each report separates the behavior we reproduced, the upstream response, and the status of a repair. An adopted fix records a committed or merged correction; the report also states whether that fix has reached a release. A proposed repair or a recommended alternative has its own status.

Maintainers weigh practical impact, implementation cost, and available alternatives when deciding whether to change code. A report can therefore close without a repair while retaining its reproducible finding.

Maintainer-acknowledged limitations

These findings led to an upstream explanation of a retained limitation and an available alternative. They are counted separately from adopted fixes. A tracker label, a community reproduction or a reporter closing their own issue does not by itself qualify.

SciPy: maintainer response — the subnormal Student-t case remains outside the planned NumPy repair work; MPArray provides the demonstrated high-precision route. This is an acknowledged boundary, rather than maintainer acceptance of a required bug fix.

SciPy: Student-t: a representable subnormal tail returns zero — Closed as not planned on September 30, 2026 — MPArray recommended; no NumPy-backend fix merged. The report records the rationale and the available alternative. It remains in the submitted-report total and is excluded from the merged-fix total.

Engineering series

How the paired-t evidence was built

Five notes follow the work from a proof pipeline to a reviewed formal-decision packet. Read them in order to see what each stage established and how later review narrowed the remaining questions.

  1. 1 · Proof pipelineCertifying paired-t numerical evidence before protocol support
  2. 2 · Floating-point boundariesTesting a paired-t evaluator at floating-point boundaries
  3. 3 · Tables and per-input checksBuilding paired-t numerical tables and input-specific error checks
  4. 4 · Full p-value pathTracing a paired-t calculation from observations to p-value
  5. 5 · Formal decision readinessFrom numerical bounds to a controlled paired-t execution candidate

The paired-t candidate was reviewed for a Release 2 decision. No decision was published, so it was never adopted or offered as a supported capability.

Engineering entries

  1. November 14, 2026

    Upstream report

    Batching can change SciPy’s Wilcoxon signed-rank p-value

    An unchanged set of 16 paired differences crosses the 5% threshold when another row in the same SciPy call contains a tie.

    Closed as duplicate of SciPy #26115 · Wilcoxon behavior tracked there

  2. November 14, 2026

    Upstream report

    R’s exact Kendall test can return a negative p-value

    With 20 perfectly concordant pairs, R’s cor.test() returns an exact two-sided p-value of −4.44e−16. Reversing one variable returns the correct value of 8.22e−19.

    Reported to R Core in Bugzilla #19319 · upstream confirmation pending

  3. November 11, 2026

    Upstream report

    R’s exact2x2 drops a tied table in Boschloo’s test

    For 8 of 10 vs 3 of 10 events, exact2x2::boschloo() gives a two-sided p of 0.033 instead of 0.041, because two tables that tie by symmetry are compared with exact floating-point equality.

    Maintainer plans a package update · released fix pending

  4. November 10, 2026

    Upstream report

    SciPy’s noncentral t distribution returns NaN for small-sample power calculations

    With few observations and a large effect, scipy.special.nctdtr returns nan for a lower-tail probability near 1e-28, so a two-sided t-test power calculation returns nan instead of about 1.

    Related fix merged in Boost.Math · SciPy update PR #26702 in draft

  5. November 9, 2026

    Upstream report

    Missing pairs make bioinfokit’s confidence interval contradict its p-value

    A paired t-test reports p = 0.0756 alongside a 95% confidence interval that excludes zero because missing pairs still count toward its degrees of freedom.

    Reported as bioinfokit #74; upstream confirmation and repair pending

  6. November 9, 2026

    Upstream report

    An unused column changes bioinfokit’s Z-test significance decision

    Two missing notes change a Z-test p-value from 0.0398 to 0.0660, although all ten measurements remain identical and complete.

    Reported as bioinfokit #73; upstream confirmation and repair pending

  7. November 9, 2026

    Upstream report

    Z3 merges a repair for Python selection constraints with no candidates

    Z3 adopted the reported Python-wrapper repair and six regression examples, restoring meaningful zero-sum constraints for empty candidate lists in development source.

    Fix merged upstream in Z3 #11621; issue #11617 closed; release pending

  8. November 9, 2026

    Upstream report

    A zero-count category makes statsmodels’ G-test return NaN

    A valid three-category count table loses its G-test statistic and p-value because the calculation does not preserve the zero-count limit.

    statsmodels PR #10704 narrowed to lambd=0 · open

  9. November 9, 2026

    Upstream report

    MCP Python SDK can turn an HTTP image into corrupted text

    A valid PNG loses its original bytes when HttpResource decodes it as text, while the same image returned through BinaryResource is preserved.

    MCP Python SDK #3843 open; contributor reproduction reported; maintainer confirmation and repair pending

  10. November 9, 2026

    Upstream report

    python-flint merges a repair that preserves decimal error bounds

    python-flint merged the reported constructor repair and regression tests, preserving midpoint uncertainty when tuple radii are added to inexact decimal inputs.

    Fix merged upstream in python-flint PR #449; issue #448 closed; release pending

  11. November 9, 2026

    Upstream report

    Large integers can change SciPy’s Mann–Whitney U decision

    Eight distinct integers become artificial ties inside SciPy, changing an exact p-value from 1/35 to 2/35 without changing the original ordering.

    Fix proposed in SciPy PR #26667 · open

  12. September 27, 2026

    Upstream report

    SciPy returns zero for a representable Student-t tail

    A Welch test and two lower-level Student-t functions return zero for a positive subnormal probability that remains representable in binary64.

    Closed as not planned on September 30, 2026 — MPArray recommended; no NumPy-backend fix merged

  13. September 23, 2026

    Upstream report

    jStat returns zero for a probability near 44%

    Independent integration and a mathematical lower bound expose a noncentral-t probability collapse that remains after an earlier convergence safeguard.

    Reported in jStat issue #300 — upstream confirmation pending

  14. September 22, 2026

    Upstream report

    statsmodels returns zero for a representable chi-square tail

    A reported p-value collapse for a representable chi-square tail led to a merged statsmodels repair with regression coverage for related contingency-table paths.

    Fix merged into statsmodels main in PR #10276 — not yet released

  15. September 19, 2026

    Implementation note

    Preserving Welch confidence intervals near zero

    A public-source verifier repair retains precision before nearly equal quantities are subtracted, with independent reference checks around the boundary where the repair takes over.

    Merged source repair; not in the published npm 0.2.1-rc.1; no Protocol scope or tolerance change

  16. September 19, 2026

    Implementation note

    Keeping independent checks visible when verification fails

    An experimental Holm verifier preserves eligible Record checks when the caller’s expected context differs, and explains why dependent checks cannot run.

    Unreleased development version; research review and implementation not finished; no new supported capability

  17. September 18, 2026

    Implementation note

    Preserving Student-t probability near zero at one degree of freedom

    A closed-form calculation keeps small but representable probability differences from disappearing and gives the public nomue verifier an independent regression check for a known numerical failure.

    Implemented in @licklider/nomue-verifier 0.2.1-rc.1; SciPy issue #25667 remains open; no Protocol scope change

  18. September 15, 2026

    Implementation note

    Making verification results more useful to research agents

    nomue development improves difficult calculations, makes completed checks explicit, and preserves the meaning of historical results as versions change.

    Product development update — integrated candidates and internal milestones; no new hosted release or public verifier support

  19. September 14, 2026

    Upstream report

    statsmodels loses finite Welch results when an intermediate sum overflows

    Four exactly represented observations per group make statsmodels overflow an intermediate sum, losing finite variances and Welch t-test results.

    Fix merged into statsmodels main in PR #10255 — not yet released

  20. September 13, 2026

    Implementation note

    Separating format checks from verification results

    An experimental Holm verifier separates format and declaration checks from integrity, context and arithmetic results, preserving explicit outcomes for checks that did not run.

    Earlier unreleased version, kept for reference; a later version revises it; not adopted or supported

  21. September 12, 2026

    Upstream report

    Checking Welch results with exact rescaling

    Exact inputs and independent references expose a changed Welch p-value despite a finite result and no warning in a SciPy boundary test.

    NumPy repair PR #26209 open; not merged as of November 9, 2026; issue #26169 open

  22. September 11, 2026

    Implementation note

    When a verification call must discard its result

    An experimental Holm verifier connects Record checks to shared execution budgets, operating-system limits and cleanup evidence before deciding whether a result can be returned.

    Earlier unreleased version, kept for reference; a later version is in development; no new supported capability

  23. September 11, 2026

    Implementation note

    Binding Holm corrections to the intended comparisons

    An experiment checks exact Holm adjustments together with the expected declaration and supplied p-values, including changes that leave the displayed answer unchanged.

    Original binding experiment preserved; successor includes scoped numerical review and deterministic sort repair; no additional supported capability

  24. September 11, 2026

    Implementation note

    Checking factorial probability evidence against the raw observations

    The factorial candidate now joins Record checks, exact probability evidence, a complete report and controlled execution in an independently reviewed readiness package.

    Release 4 candidate reviewed; discussion ended without a published decision — not adopted or supported

  25. September 10, 2026

    Technical method

    Checking factorial statistics without trusting rounded intermediates

    Exact arithmetic and probability bounds offer a path beyond scaling repairs, while a review shows why matching rounded answers does not certify an interval.

    Reviewed research components now connected experimentally and archived; no additional supported capability

  26. September 10, 2026

    Upstream report

    A nonzero Studentized-range tail disappears in SciPy

    An exact special-case reference shows SciPy returning zero for a probability near 0.00000002, with no warning in the recorded runs.

    Additional reproducer reported — upstream confirmation pending

  27. September 10, 2026

    Upstream report

    Exact rescaling can reverse SciPy’s Welch ANOVA decision

    At an extreme input scale, SciPy’s Welch ANOVA changes a p-value from 0.02650 to 0.05611, crossing the 5% threshold without losing input information.

    NumPy repair PR #26209 open; not merged as of November 9, 2026; issue #26146 open

  28. September 10, 2026

    Upstream report

    Renaming treatment groups changes agricolae’s REGW result

    With observations and group membership unchanged, renaming groups changes a p-value from 0.0363 to 0.0791 and reverses a 5% decision.

    Reported by email — upstream confirmation pending

  29. September 9, 2026

    Technical method

    What power-of-two scaling can and cannot repair

    Exact references show when rescaling recovers a factorial F calculation, and when lost inputs or rounding residuals require a different numerical decision.

    Reviewed and steward-accepted bounded research; no additional product support

  30. September 9, 2026

    Technical method

    What “approximate” means for Games–Howell comparisons

    Original-paper checks separate the construction of unequal-variance comparisons from simulation evidence and a guaranteed bound on false positives.

    Reviewed and steward-accepted bounded research; no additional product support

  31. September 9, 2026

    Technical method

    Comparing with a control and comparing with the best answer different questions

    Source review separates fixed-control tests, step-up and step-down calibration, and intervals that compare each treatment with the best of the others.

    Reviewed and steward-accepted bounded research; no additional product support

  32. September 9, 2026

    Technical method

    What must stay fixed in a multiple-testing graph

    Closed testing and graphical procedures make error control inspectable, but order, weights, stopping rules, and zero-level behavior still need precise definitions.

    Reviewed and steward-accepted bounded research; no additional product support

  33. September 8, 2026

    Technical method

    What a multiple-comparison procedure actually guarantees

    Original-paper checks separated overall tests, individual comparisons, and simultaneous intervals, giving future verification rules a more precise statement of what they protect.

    Release 3 research; checked against the original sources

  34. September 8, 2026

    Technical method

    When floating-point calculations change a tiny factorial effect

    A 945-case comparison separated effects lost during input rounding from errors introduced by cell means and QR calculations, including cases where centering did not help.

    Steward-accepted, independently reviewed bounded Release 4 numerical research

  35. September 7, 2026

    Technical method

    Checking multiple-testing procedures against their original papers

    Reviewing six original papers clarified multiple-testing guarantees and exposed a numerical table entry that disagrees with its defining equation.

    Independently reviewed SR-C source evidence with bounded acceptance recorded; Release 3 preparation

  36. September 7, 2026

    Upstream report

    SciPy’s automatic Mann–Whitney U test can change a result when tests are batched

    An unchanged sample pair crosses the 5% significance threshold when another pair contains repeated values, because SciPy selects one calculation method for the batch.

    SciPy #26115 now tracks vectorized tests with method='auto' · numerical study proposed

  37. September 7, 2026

    Upstream report

    SciPy t-tests can return p=0 or p=1 after exact rescaling

    SciPy’s one-sample and paired t-tests can reverse a 5% decision after exact power-of-two rescaling because an intermediate variance underflows or overflows.

    NumPy repair PR #26135 closed without merging on November 1, 2026; issue #26113 open; MPArray alternative available

  38. September 4, 2026

    Technical method

    Cataloguing every multi-group procedure before proposing any of them

    We catalogued 49 multi-group comparison procedures, gave each an explicit disposition, and found only seven backed by primary text we had actually read.

    Reviewed catalogue research; bounded Release 3 public discussion now open

  39. September 3, 2026

    Upstream report

    A Julia signed-rank p-value above 1, now fixed in a release

    HypothesisTests.jl returned 1.25 for an exact two-sided signed-rank p-value. The matching correction shipped in v0.12.0 and remains in v0.12.2.

    Fix released in v0.12.0; issue #373 closed as fixed

  40. September 1, 2026

    Implementation note

    From numerical bounds to a controlled paired-t execution candidate

    The reviewed Release 2 formal decision packet now assembles the paired-t evidence and required decisions without adopting or issuing Protocol support.

    Release 2 candidate reviewed; no decision published — not adopted or supported

  41. September 1, 2026

    Upstream report

    R’s exact Wilcoxon test can return p-values outside the valid range

    R Core confirmed the exact Wilcoxon tail error and committed a correction in R-devel r90641, following our report and independent exact-arithmetic checks.

    Closed / FIXED by R Core; tail-accuracy correction committed in R-devel r90641; released availability not verified

  42. August 31, 2026

    Implementation note

    Tracing a paired-t calculation from observations to p-value

    We connected paired observations to a p-value in one reviewed trace; later work closed its two numerical error ledgers and added a reviewed interval trace.

    Independently reviewed p-value and confidence-interval execution traces; interval proof continues

  43. August 31, 2026

    Implementation note

    Building paired-t numerical tables and input-specific error checks

    We built two reviewed 200-value tables and input-specific error checks; later decisions selected the p-value bound and one table for candidate interval work.

    Two independently reviewed 200-value tables and input-specific error checks — candidate Release 2 work

  44. August 30, 2026

    Implementation note

    Testing a paired-t evaluator at floating-point boundaries

    We built and independently reviewed a deterministic paired-t probability evaluator and boundary evidence while leaving accuracy bounds, supported inputs, and protocol registration open.

    Independently reviewed deterministic evaluator and floating-point boundary evidence — candidate Release 2 work

  45. August 28, 2026

    Technical method

    Certifying paired-t numerical evidence before protocol support

    We built and independently reviewed a proof pipeline for paired-t p-values and critical values before deciding what nomue Protocol will support.

    Independently reviewed proof pipeline for paired-t p-values and critical values — candidate Release 2 work

  46. August 26, 2026

    Upstream report

    A SciPy exact Wilcoxon p-value error, fixed upstream

    SciPy’s exact Wilcoxon path could return zero for a positive p-value. SciPy diagnosed the cause and merged a fix the same day it was reported.

    Fix merged in SciPy — awaiting a SciPy release

  47. August 23, 2026

    Bug report

    An extreme-tail sign error in SciPy’s Student-t quantile

    A SciPy bug can return positive infinity instead of a large negative value for an extreme-tail Student-t quantile.

    Fix merged in Boost.Math — awaiting a Boost release