Chi-Square Test

How to run a chi-square test of independence in Licklider, what the results include, and when to use a correction or an alternative test.

The chi-square test of independence evaluates whether two categorical variables are associated with each other. It compares the observed frequencies in a contingency table to the frequencies that would be expected if the two variables were independent.

When to use it

The chi-square test is appropriate when:

  • Both variables are categorical
  • You want to test whether the distribution of one variable differs across levels of the other
  • The sample is large enough that expected cell counts are adequate (see below)

Licklider cannot determine automatically whether the rows in the table are statistically independent. In particular, it cannot infer from two categorical columns alone whether the same subject appears multiple times, whether the data come from a paired design, or whether clustering within animals, plates, families, or batches makes the ordinary chi-square test inappropriate.

These limits matter because the chi-square test assumes that each counted observation contributes independent information. If repeated or clustered observations are treated as separate rows, the p-value can look more convincing than the design really supports.

How to request it

Describe the analysis in the Chat. For example:

  • "Test whether treatment group and response are independent"
  • "Run a chi-square test on these two columns"
  • "Is there an association between genotype and phenotype?"

Licklider identifies the two categorical columns and builds the contingency table from the raw observations automatically.

What the results include

Test statistic and p-value

The chi-square statistic and its p-value, based on the appropriate degrees of freedom for the table size.

Observed and expected counts

The full contingency table showing both the observed count and the expected count for each cell. Expected counts are what would be observed if the two variables were completely independent.

Cramér's V

The effect size for the chi-square test, reported as Cramér's V. This measures the strength of association between the two variables on a scale from 0 (no association) to 1 (perfect association). Cramér's V is comparable across tables of different sizes.

Note on effect size naming: For power calculations, Licklider uses Cohen's w. For 2x2 tables, Cramér's V and Cohen's w are equivalent. For larger tables they differ; if you are using the chi-square power calculator and entering an effect size from a published result, confirm which measure was used.

Yates' continuity correction

For 2x2 tables, Licklider can apply Yates' continuity correction, which adjusts the chi-square statistic to reduce the overestimation of significance that can occur with small samples.

The correction is off by default. To request it, specify in the Chat:

  • "Use Yates' correction"
  • "Apply a continuity correction"

Whether the correction was applied is shown in the results panel.

Small expected counts

The chi-square approximation becomes unreliable when expected cell counts are small. When any cell in the contingency table has an expected count below 5, Licklider displays a warning in the results panel.

The < 5 warning is used because the large-sample approximation behind the chi-square test becomes less stable when expected counts are sparse. In that setting, the test can overstate or understate evidence more easily than when cell expectations are comfortably larger.

For tables with small expected counts, Fisher's exact test is the statistically preferable alternative and is available in Licklider. The 2x2 form returns an exact p-value with an odds ratio and confidence interval; for larger tables (3x2, 2x3, 3x3, ...) the Freeman-Halton extension returns a two-sided exact p-value (or a Monte Carlo estimate when the table is too large to enumerate). When this warning appears, the Fisher / Freeman-Halton result is shown in the results panel alongside the chi-square result.

This side-by-side display is intentional: it lets you see the standard chi-square result, the sparsity warning, and the exact-test alternative in one place rather than leaving you to guess whether the approximation is still trustworthy.

For more detail → see Fisher's Exact Test.

Table size

The chi-square test is not limited to 2x2 tables. Licklider supports contingency tables of any size — for example, 3x2 or 3x3 — as long as both variables have two or more categories and the data contains sufficient observations.

What the results panel shows

The results are visible in the Inspector. The panel shows the test statistic, degrees of freedom, p-value, Cramér's V, the correction setting, and the full observed/expected table. Cells with expected counts below 5 are visually flagged.

Expected counts are shown because they help you judge whether the approximation assumptions are plausible for each cell, not just whether the overall p-value is small. Cramér's V is shown alongside the p-value so that the result reports both whether an association is detectable and how large that association appears to be.

Design Rationale & References

This page follows a simple rule: a categorical association result should show both the significance test and the table structure that makes that test interpretable. That is why Licklider reports observed and expected counts, includes Cramér's V alongside the p-value, and warns when sparse cells make the chi-square approximation less reliable.

The default remains the ordinary chi-square test because it is the standard large-sample test of independence for contingency tables [1]. Yates' correction is available but not turned on by default because it is mainly relevant to 2x2 tables with smaller counts, where users may prefer a more conservative approximation [2].

The Fisher alternative is surfaced when expected counts are small because exact inference is often preferable in sparse 2x2 tables [3]. Keeping the expected counts visible is also intentional: the validity of the chi-square approximation is partly a property of the table itself, not only of the final test statistic.

  1. Pearson, K. (1900). On the Criterion That a Given System of Deviations From the Probable in the Case of a Correlated System of Variables Is Such That It Can Be Reasonably Supposed to Have Arisen From Random Sampling. Philosophical Magazine, 50(302), 157-175.
  2. Yates, F. (1934). Contingency Tables Involving Small Numbers and the chi-square Test. Supplement to the Journal of the Royal Statistical Society, 1(2), 217-235.
  3. Fisher, R. A. (1922). On the Interpretation of Chi-Square from Contingency Tables, and the Calculation of P. Journal of the Royal Statistical Society, 85(1), 87-94. https://doi.org/10.2307/2340521

What this page does not cover