Values below the detection limit: why LOD/2 is a shortcut, and how to analyse non-detects

Quick answer. A sample reported as “below the limit of detection” (LOD) has a concentration somewhere between zero and the LOD: the value is censored, not zero and not exactly LOD/2. First report how many values are below the limit in each group. Then analyse with a method that uses that information, chosen in advance: a censored-data model on log concentrations, which estimates the ratio of geometric means between groups and assumes a similar log-SD in each group (with about 12 animals per group, read its confidence interval rather than a p-value near 0.05), or, when every sample shares one detection limit, a Mann–Whitney test with the non-detects tied at the bottom. If a group is almost all non-detects, compare the proportion detected as well, and treat any ratio as rough. These methods assume independent animals; for paired or repeated samples, use a paired or mixed model. Substituting LOD/2 is a shortcut that changes the spread of the data and is discouraged by the methods literature; if you use it, say so and check that the conclusion does not depend on it.

Terms used below. Limit of detection (LOD): the lowest concentration the assay can distinguish from zero. Lower limit of quantification (LLOQ): the lowest concentration measured with acceptable accuracy and precision; values between the LOD and the LLOQ are detected but not reliably quantified. Non-detect: a sample below the LOD. Censored value: a value known only to lie below (left-censored) or above (right-censored) a limit. Substitution: replacing non-detects with a fixed number such as LOD/2 or LOD/√2. Censored-data model: a regression that uses non-detects as “below this limit” (a Tobit model, or a survival-type model fitted with left censoring). Geometric mean: the mean on the log scale, converted back. Ratio of geometric means: how many times higher the concentrations in one group are than in the other; the usual effect for log-normal data. Log-SD: the standard deviation on the log scale. Maximum likelihood: fitting a model by finding the parameter values that make the data most probable. Power: the chance of detecting an effect that is really there. Log-normal: skewed values whose logarithms follow a normal distribution, as concentrations often do. Tobit model: another name for a censored-data regression. Reporting limit: the limit below which a laboratory reports “not detected”, here the LOD. Welch’s t-test: compares two means without assuming equal spread. Fisher’s exact test: compares two proportions in small samples. Tied at the bottom: all non-detects given the same lowest rank. 95% confidence interval (CI): the range of effects compatible with the data. Independent: each value from a different animal, with no pairing. Sensitivity check: repeating the analysis another way (here with substitution) to show that the conclusion does not change.

Which analysis for your data

Your data	Analysis	Report
A few non-detects (under about 10%)	A censored-data model; substitution only as a stated sensitivity check	Number below the limit per group; ratio of geometric means with 95% CI
More non-detects, one common limit	Censored-data model on log concentrations; or Mann–Whitney with non-detects tied	As above; for the rank test, say which group tended to be higher (a median cannot be estimated for a group with more than half non-detects)
Several detection limits (different plates or dilutions)	Censored-data model, which takes each sample’s own limit	As above
Most values below the limit in a group	Compare the proportion detected (Fisher’s exact test) as well, and treat any ratio as rough	Both

Helsel puts the case against substitution directly: “any substitution of a constant fraction of reporting limits will distort estimates of the standard deviation, and therefore all (parametric) hypothesis tests using that statistic”, and “substituting values for nondetects should be used rarely” (Helsel 2006). In an epidemiological comparison, Lubin and colleagues found that substitution “can be biased unless the percentage of measurements below detection limits is small (5-10%)” (Lubin et al. 2004). For a rank test, Helsel notes that with one reporting limit, “censored observations are represented as a tied rank lower than the rank for the lowest detected observation” (Helsel 2012). The LOD/√2 rule often attributed to Hornung and Reed was proposed for estimating an average: “when the data are not highly skewed, replacement of nondetectable values by L/√2 should produce very good estimates”, while LOD/2 “appears warranted only when the data are highly skewed” (Hornung & Reed 1990); neither rule was proposed for comparing groups.

Worked example: a cytokine with 3 non-detects in the control group

Constructed data: a cytokine in 12 control and 12 treated mice, LOD 5 pg/mL. By construction concentrations are log-normal and twice as high with treatment; 3 control values and no treated values were below the LOD. The random seed (3) was the lowest, counting up from 1, that gave three or four control non-detects and at most one treated non-detect, the typical pattern for this model. Stored code and data: cytokine.csv, model.py (the concentration model), make_data.py, analysis.py, analysis.R and the analysis output; analysis.py and analysis.R print the same output.

Analysis	Result
Values below the LOD	Control 3 of 12; treated 0 of 12
LOD/2 substitution, Welch’s t-test on log values	Ratio 2.05; control log-SD 1.145; p = 0.070
Mann–Whitney test (non-detects tied at the bottom)	p = 0.012
Censored-data (left-censored log-normal) model	Ratio 1.97 (95% CI 0.98 to 3.98), log-SD 0.860, p = 0.058

The substitution and censored-model analyses estimate a similar effect, about twofold (the rank test gives a p-value, not a ratio), but substitution places three values at an invented 2.5 pg/mL, which changes the control group’s spread (log-SD 1.145 after substitution, against 1.07 for the detected control values alone; one high value, 114 pg/mL, contributes much to either). The censored model gives the ratio with a confidence interval that uses what is known about the non-detects; it assumes the same log-SD in both groups (0.860 here), so check the spread of the detected log values in each group: here 1.07 in controls and 0.59 with treatment, a difference driven partly by the one high value and within chance for 9 and 12 values (the true log-SDs are equal by construction). If the spreads clearly differ in your data (for example more than twofold), allow a separate log-SD per group (in R, add + strata(group); here that gives a ratio of 2.10, 95% CI 0.98 to 4.51) and say so. With 12 mice per group, this example cannot separate the groups: the interval is compatible with no effect and with a fourfold effect. The Mann–Whitney p-value (0.012) is smaller because it tests a different question (whether one group tends to have higher values, not the ratio); choose one primary analysis in advance and report the other as supporting. In 10,000 simulated experiments per scenario with 12 mice per group (simulation.py, the simulation output):

Scenario (control values below the LOD)	LOD/2 substitution	Mann–Whitney	Censored-data model
No effect, 28% (false positives)	5.1%	4.7%	7.0%
Twofold effect, 28% (power)	51.7%	48.8%	58.7%
No effect, 50% (false positives)	4.7%	4.3%	5.7%
Twofold effect, 50% (power)	50.2%	47.2%	54.6%

The censored model estimated the ratio without bias (median 2.00 in both effect scenarios) and had the highest power, partly because its false-positive rate was 7.0% rather than 5% with 12 mice per group and 28% of control values censored; so read its confidence interval rather than a p-value near 0.05. In these two-group scenarios substitution’s ratio estimate stayed close to the truth, but it distorted the spread: with half the control values censored, the median control log-SD after substitution was 0.73 against a true 0.80. That distortion matters most for descriptive statistics, correlations and regressions, where Helsel and Lubin found the largest problems.

# Condensed: analysis.py also runs the substitution and Mann-Whitney comparisons
import numpy as np, pandas as pd
from scipy import stats, optimize

d = pd.read_csv("cytokine.csv")   # mouse, group, conc_pg_ml, detected: for a non-detect enter the LOD in conc_pg_ml AND detected = 0
# change "treated" below to your treatment label; every other label is treated as the reference group
y, ev = np.log(d.conc_pg_ml.to_numpy()), d.detected.to_numpy()
x = (d.group == "treated").to_numpy().astype(float)
def nll(q):                                          # left-censored log-normal: a non-detect contributes P(X < LOD)
    z = (y - q[0] - q[1] * x) / np.exp(q[2])
    return -(ev * (stats.norm.logpdf(z) - q[2]) + (1 - ev) * stats.norm.logcdf(z)).sum()
fit = optimize.minimize(nll, [np.log(8), 0, np.log(0.8)], method="BFGS")
assert fit.success, "model did not converge: check that each group has some detected values"
assert d.group.nunique() == 2, "this snippet compares two groups; use a factor-coded model for more"
E = np.eye(3) * 1e-4; p0 = fit.x                     # numerical Hessian -> standard error
H = np.array([[(nll(p0 + a + b) - nll(p0 + a - b) - nll(p0 - a + b) + nll(p0 - a - b)) / 4e-8 for b in E] for a in E])
se = np.sqrt(np.linalg.inv(H)[1, 1])
assert np.isfinite(se), "standard error not available: too few detected values for this model"
print(np.exp(p0[1]), np.exp(p0[1] + np.array([-1.96, 1.96]) * se), 2 * stats.norm.sf(abs(p0[1] / se)))   # ratio, 95% CI, p

library(survival)                                    # used only as a fitting engine for the censored likelihood
d <- read.csv("cytokine.csv"); d$group <- factor(d$group, levels = c("control", "treated"))
fit <- survreg(Surv(conc_pg_ml, detected, type = "left") ~ group, data = d, dist = "lognormal")
exp(cbind(coef(fit), confint(fit)))["grouptreated", ]   # ratio of geometric means with 95% CI (Wald)
summary(fit)$table["grouptreated", "p"]               # p-value

Before you analyse

1. Record non-detects as “below the limit”, with the limit, not as 0, blank or LOD/2. Enter the limit as the value and a 0/1 detection indicator, as in the example file (columns mouse, group, conc_pg_ml, detected). Take the LOD from the kit insert or from your own blank replicates, and say which you used; in analysis.py and analysis.R the constant LOD is used only for the LOD/2 comparison, so change it to your limit. Average technical replicate wells first; n is the number of animals.
2. Know which limit applies. Values between the LOD and the LLOQ are detected but not reliably quantified; ICH M10 defines quantification limits for bioanalysis (ICH M10). Decide in advance whether you treat values below the LLOQ as censored at the LLOQ.
3. Count non-detects per group before choosing the analysis, and report the counts.
4. Diluted samples have their own limit (the LOD times the dilution factor); put each non-detect’s own limit in conc_pg_ml, and the censored-data model uses it.
5. Check the log-normal assumption and the spread on the detected values (a plot on the log scale and the log-SD in each group, printed by analysis.py); with heavy censoring, the model leans on these assumptions for the unseen values.
6. Paired or repeated samples (the same animal before and after, several time points) need a paired or mixed censored model, which this page does not cover; ask a statistician or use a package for censored mixed models.
7. If you substitute anyway, say so, give the fraction substituted, and show that LOD/√2 or the censored model gives the same conclusion.

Check what an AI assistant ran

1. Non-detects set to zero, which makes logs impossible and pulls means down. Ask for the counts and a censored analysis.
2. Non-detects deleted, which removes the lowest values and biases the group upward. Ask for them back as censored values.
3. LOD/2 substituted without saying so. Ask how many values were substituted and for a censored model as a check.
4. A t-test on raw concentrations of skewed data with many identical LOD values. Ask for log concentrations and a censored model, or a rank test.

Report it

Methods:

Cytokine concentrations below the limit of detection ([5] pg/mL) were treated as left-censored. Log concentrations were compared between groups with a censored-data (Tobit) log-normal regression fitted by maximum likelihood, assuming a common log-SD (survival package version 3.5-8, R 4.3.3; replace with your versions), giving the ratio of geometric means.

Results (bracketed values are from the constructed example):

[3] of [12] control and [0] of [12] treated values were below the limit of detection. The ratio of geometric means (treated/control) was [1.97] (95% CI [0.98] to [3.98]; p = [0.058]), so the data are compatible with no effect and with a fourfold increase (with 12 animals per group, give more weight to the interval than to the p-value).

Replace the bracketed values with your own.

References

• Helsel DR (2006). Fabricating data: how substituting values for nondetects can ruin results, and what can be done about it. Chemosphere 65:2434–2439. doi:10.1016/j.chemosphere.2006.04.051
• Lubin JH, Colt JS, Camann D, et al. (2004). Epidemiologic evaluation of measurement data in the presence of detection limits. Environmental Health Perspectives 112:1691–1696. doi:10.1289/ehp.7199
• Helsel DR (2012). Statistics for Censored Environmental Data Using Minitab and R, 2nd edition. Wiley. doi:10.1002/9781118162729
• Hornung RW, Reed LD (1990). Estimation of average concentration in the presence of nondetectable values. Applied Occupational and Environmental Hygiene 5:46–51. doi:10.1080/1047322X.1990.10389587
• ICH (2022). M10: Bioanalytical method validation and study sample analysis. database.ich.org (checked 11 November 2026)