Published Updated

statsmodels returns zero for a representable chi-square tail

A reported p-value collapse for a representable chi-square tail led to a merged statsmodels repair with regression coverage for related contingency-table paths.

Current status

statsmodels closed issue #10274 as completed after merging pull request #10276 into main on September 23, 2026. The merged change replaces the cancellation-prone subtraction with the chi-square survival function and adds regression tests for the reported path and related contingency-table paths.

Chetan Sahney authored the repair, and Kevin Sheppard merged it as commit 9f49dcd. The fix is present on the development branch; this page does not claim that a released statsmodels version contains it yet.

A finite probability disappears in the final subtraction

For the 2-by-2 contingency table [[40, 1], [1, 40]], statsmodels’ Table.test_nominal_association returns a p-value of exactly 0.0. The Pearson statistic is finite, and the corresponding chi-square upper-tail probability is approximately 7.076 × 10^-18—a representable positive float64 value.

Tasuku Kobayashi reported the example in statsmodels issue #10274 on September 22, 2026. It was reproduced in a fresh environment with statsmodels 0.15.0 and in a source build of main commit 2cb70c2ce. The upstream issue was subsequently resolved by PR #10276.

Reproducing the zero result

import numpy as np
from scipy import stats
from statsmodels.stats.contingency_tables import Table

table = np.array([[40, 1], [1, 40]], dtype=float)
res = Table(table, shift_zeros=False).test_nominal_association()

print(res.statistic, res.df, res.pvalue)
print(stats.chi2.sf(res.statistic, res.df))
print(1 - stats.chi2.cdf(res.statistic, res.df))
74.19512195121949 1 0.0
7.07649484571079e-18
0.0

The release reproduction used Python 3.11.15, NumPy 2.4.6, SciPy 1.17.1, pandas 3.0.6, patsy 1.0.3, and statsmodels 0.15.0 on Linux x86_64. The table contains no zero cells, so the default shift_zeros=True produces the same result.

The survival function preserves the tail

The implementation forms the p-value as 1 - stats.chi2.cdf(statistic, df). At this statistic, the cumulative probability rounds to exactly 1.0, so subtracting it from one cancels the entire tail. This is not underflow of the probability itself: the positive result is roughly 306 orders of magnitude above the smallest positive float64 subnormal value.

The survival function stats.chi2.sf evaluates the upper tail directly and returns a nonzero value. statsmodels already uses that function in other paths in the same module, including McNemar's test and Cochran's Q test.

Loss of precision begins before the answer reaches zero. For[[30, 1], [1, 30]], the method returns1.758593271006248e-13, while the survival function returns1.7581411194883224e-13, a relative difference of about2.57 × 10^-4.

What statsmodels changed

The merged patch changes Table.test_nominal_association from 1 - stats.chi2.cdf(statistic, df) to stats.chi2.sf(statistic, df). Its regression test requires the reported [[40, 1], [1, 40]] case to stay positive and agree with the survival-function result within a relative tolerance of 1e-12. It also covers the earlier-loss [[30, 1], [1, 30]] case.

The pull request applies the same numerical correction to three other contingency-table calculations and to related cancellation-prone chi-square and normal upper-tail call sites identified elsewhere in statsmodels. Additional regression tests exerciseSquareTable.homogeneity and StratifiedTable.test_null_odds. This is broader than the single method established by our original reproducer; those added paths are upstream's repair scope, not additional Licklider reports.

An independent reference confirms the magnitude

Every row and column total is 41, and every expected count is 20.5. The Pearson statistic is therefore exactly 3042/41. With one degree of freedom, the upper tail is erfc(sqrt(x/2)). Evaluated at 100 decimal digits, it is 7.0764948457107158... × 10^-18; the nearest float64 is7.076494845710716 × 10^-18. This reference does not depend on SciPy or statsmodels.

What this establishes

The reject-or-do-not-reject decision is unchanged in this example because both probabilities are extremely small. The error matters when downstream work uses the p-value's magnitude—for example, -log10(p) or meta-analytic p-value combination—because a finite probability becomes indistinguishable from mathematical zero.

The independent check covers Table.test_nominal_association. Similar1 - stats.chi2.cdf(...) expressions elsewhere in the module were audit targets rather than claims established by our report; upstream chose to include related paths in its repair and regression coverage. The finding and upstream fix do not add a method to nomue's supported scope. They demonstrate the bounded, independently referenced verification work Licklider uses when reporting numerical behavior upstream.

Public evidence