Unequal group sizes: can you still run a t-test or ANOVA?

Quick answer. Different group sizes are fine for comparing independent groups on a continuous outcome, provided you use Welch’s versions of the t-test and ANOVA. The problem is the combination of unequal sizes and unequal spread, which makes Student’s t-test and the ordinary ANOVA give too many or too few false positives. Use Welch’s t-test (two groups) or Welch’s ANOVA (three or more), which handle both, and report the actual number of animals per group (n). Keep every animal you measured: dropping some to make the groups equal throws away power. If animals were lost for reasons linked to the outcome, no test fixes the bias; report each loss. With equal spreads, equal groups give the most power for a fixed total; for two-factor designs with lost animals, say which type of sums of squares your software used.

Terms used below. Unbalanced: groups of different sizes. Spread: how much values vary within a group, measured by the standard deviation (SD) or its square, the variance. Student’s t-test: assumes both groups have the same spread. Welch’s t-test: does not assume that. Power: the chance of detecting an effect that is really there. α: the false-positive threshold, usually 0.05. Effect size in SD units: the difference between groups divided by the SD; 1.2 means the groups differ by 1.2 standard deviations. Welch’s ANOVA: the one-way ANOVA that does not assume equal spread; Games–Howell is its pairwise follow-up. False positive: a “significant” result when there is no real difference. Familywise rate: the chance of at least one false positive across a set of comparisons. Interaction: when the effect of one factor depends on the other. Simple effects: the effect of one factor within each level of the other. Nominal rate: the false-positive rate a test is meant to have, usually 5%. Behrens–Fisher statistic: the separate-variances statistic behind Welch’s test. Heteroscedasticity-robust test: a test whose standard errors allow unequal spread. 95% confidence interval (CI): the range of effects compatible with the data. df (degrees of freedom): a number printed with the test that reflects the amount of independent data. Main effect: the average effect of one factor across the levels of the other. Cell: one combination of factor levels, such as knockout mice given a drug. Sums of squares, Type I, II and III: the ways software splits up variation in a two-factor ANOVA when cells have different sizes.

When unequal sizes matter

Situation	What happens	What to do
Unequal sizes, similar spread (independent groups)	Student’s and Welch’s tests give nearly the same answer	Welch’s test anyway, decided in advance
Unequal sizes, the smaller group more variable	Student’s test gives too many false positives	Welch’s test
Unequal sizes, the larger group more variable	Student’s test gives too few and loses power	Welch’s test
Three or more groups of different sizes	Ordinary ANOVA has the same problem	Welch’s ANOVA, then Games–Howell for pairs
Animals lost unevenly, for a reason related to the outcome (animals that died or became too sick to measure)	Neither test can fix the bias from who is missing	Report the losses and their reasons; see below

The problem with Student’s t-test can be large. Ruxton summarises a classic simulation (adapted from Coombs and colleagues, 1996) with 11 and 21 per group: “the Type I error rate for the Student’s t-test was over 3 times the nominal rate when the higher variance was associated with the smaller sample size and less than a quarter the nominal rate when the higher variance was associated with the higher sample size” (Ruxton 2006). Welch’s t-test stayed close to 5% in both cases. Zimmerman adds that testing the variances first and then choosing a test “fails to protect the significance level”; “optimum protection is assured by using a separate-variances test unconditionally whenever sample sizes are unequal” (Zimmerman 2004). The Welch vs Student guide shows the same pattern in lab-sized numbers.

What imbalance costs in power

When the groups have similar spread, equal groups give the most power for the same total number of animals, but moderate imbalance costs little. The table shows the power of a two-sided t-test (α = 0.05, equal SDs) to detect a difference of 1.2 SD. Stored code: power.py, power.R and the output; both languages print the same numbers.

Group sizes	Total	Power
10 + 10	20	0.72
12 + 8	20	0.70
14 + 6	20	0.64
16 + 4	20	0.53
10 + 20	30	0.85
10 + 30	40	0.89
15 + 15	30	0.89
10 + 6	16	0.58
6 + 6 (four animals dropped to balance)	12	0.47

Three practical points follow. A 12 + 8 split loses almost nothing against 10 + 10. Adding extra animals to one group still raises power (10 + 20 beats 10 + 10), though less than adding the same animals evenly (15 + 15 gives 0.89, against 0.85 for 10 + 20). And dropping animals to balance the groups is always worse: going from 10 + 6 to 6 + 6 lowers the power to detect a 1.2-SD difference (two-sided, α = 0.05, equal SDs) from 0.58 to 0.47 and wastes animals that were already used.

from statsmodels.stats.power import TTestIndPower
print(TTestIndPower().power(effect_size=1.2, nobs1=10, ratio=6 / 10, alpha=0.05))   # 10 + 6 animals

library(pwr)
pwr.t2n.test(n1 = 10, n2 = 6, d = 1.2, sig.level = 0.05)$power   # 10 + 6 animals

These figures assume equal SDs, as the usual power formulas do. When spreads are equal, Welch’s test has about the same power as Student’s; when they are not, it keeps the false-positive rate near 5%. With unequal spreads, equal groups are no longer best: more animals in the more variable group give more power for the same total. Welch’s test also assumes roughly symmetric values; Ruxton notes it “is not necessarily any more reliable than the Student’s t-test if the assumption of normality of the underlying populations is violated”, so look at the distributions first, and consider logs for right-skewed data. With two animals in a group, a t-test has very little power and leans heavily on its assumptions (see the Welch vs Student guide); plan more animals; with three to five, report the confidence interval and treat the p-value cautiously. To plan a study, use the sample size calculator with the group sizes you expect after losses.

Why the groups are unequal matters

If groups are unequal because of how the study was planned (fewer animals of a rare genotype, more controls shared across experiments), the analysis above is all you need. If they are unequal because animals were lost, ask why. Losses unrelated to the outcome (a failed injection, a cage flood) only shrink the group; analyse the animals you have. Losses related to the outcome (the sickest treated animals died) leave a biased group, and no test corrects that. Report, for each group, every exclusion and its reason, and follow the rules in when you can exclude an outlier for any value removed after the data were seen.

Three or more groups and pairwise comparisons

With three or more groups of different sizes, use Welch’s ANOVA and the Games–Howell test for pairwise comparisons. Games and Howell found that “with unequal n’s of six and up, only the Behrens-Fisher statistic provided satisfactory control of both the familywise rate of Type I errors and Type I error rate on each contrast” among the methods they compared, and that “satisfactory control with unequal n’s of three and up is dubious even with this statistic” (Games & Howell 1976); with very small groups, report intervals and be cautious. If you assume equal spread, the Tukey test can be used with mildly unequal sizes: R’s documentation notes that the basic intervals “would only apply exactly to balanced designs” and that TukeyHSD “incorporates an adjustment for sample size that produces sensible intervals for mildly unbalanced designs” (R documentation); this adjustment is usually called Tukey–Kramer. For strongly unequal sizes or spreads, use Games–Howell. In code: r = scipy.stats.ttest_ind(a, b, equal_var=False) then r.confidence_interval() and r.df for the interval and df, or R’s t.test(a, b) for two groups; scipy.stats.f_oneway(a, b, c, equal_var=False) in Python or oneway.test(y ~ group) in R for Welch’s ANOVA (Welch’s ANOVA guide has Python and Games–Howell code).

Two-factor designs with unequal cells

In a two-way ANOVA (for example genotype × treatment) with different numbers of animals per cell, the main-effect tests depend on how the software splits the variation. Langsrud notes that “most major statistical programs perform, by default, unbalanced ANOVA based on Type III sums of squares” and argues against the usual objection that Type II assumes no interaction: that argument “is, however, also founded on unrealistic models. Furthermore, by considering the power of the two methods, it is clear that Type II is preferable” (Langsrud 2003). If the interaction matters, interpret the simple effects within each level rather than the main effects (two-way ANOVA vs separate t-tests). Whichever you use, state the type: R’s anova() gives Type I (sequential) by default, car::Anova() Type II by default, statsmodels’ anova_lm() Type I unless you pass typ=2, and SPSS defaults to Type III. Unequal cell sizes also make unequal spreads harder to ignore: check the cell SDs, and if they differ clearly use heteroscedasticity-robust tests, as the two-way ANOVA guide shows.

Check what an AI assistant ran

1. Student’s t-test or an ordinary one-way ANOVA on groups of different sizes (equal_var=True, SciPy’s default; statsmodels’ usevar="pooled"; var.equal = TRUE; aov(), which has no Welch option). Use Welch’s versions.
2. Losses not reported. Ask how many animals were excluded in each group and why.
3. A variance test used to choose the t-test. Decide on Welch’s test in advance. If Student’s and Welch’s p-values differ, report Welch’s as planned, even if Student’s p-value is smaller.
4. Animals removed to make groups equal. Put them back and report the real n.
5. An unstated sums-of-squares type in a two-way ANOVA with unequal cells. Ask which type was used and why.

Report it

Groups were compared with Welch’s t-test[, the pre-specified test], which does not assume equal variances or equal group sizes (vehicle n = [10], drug n = [6]; [two] mice were excluded before analysis because [reason]; see Supplementary Table [S1]). Means were [x] (SD [x]) and [x] (SD [x]); difference [x] (95% CI [x] to [x]); Welch’s t = [x], df = [x], p = [x] (SciPy [1.18.1] / R [4.3.3]).

Replace the bracketed values with your own.

References

• Ruxton GD (2006). The unequal variance t-test is an underused alternative to Student’s t-test and the Mann–Whitney U test. Behavioral Ecology 17:688–690. doi:10.1093/beheco/ark016
• Coombs WT, Algina J, Oltman DO (1996). Univariate and multivariate omnibus hypothesis tests selected to control Type I error rates when population variances are not necessarily equal. Review of Educational Research 66:137–179. doi:10.3102/00346543066002137
• Zimmerman DW (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology 57:173–181. doi:10.1348/000711004849222
• Games PA, Howell JF (1976). Pairwise multiple comparison procedures with unequal n’s and/or variances: a Monte Carlo study. Journal of Educational Statistics 1:113–125. doi:10.3102/10769986001002113
• Langsrud Ø (2003). ANOVA for unbalanced data: use Type II instead of Type III sums of squares. Statistics and Computing 13:163–167. doi:10.1023/a:1023260610025
• R Core Team. TukeyHSD: compute Tukey honest significant differences. R documentation (checked 11 November 2026)