Sholl analysis statistics: intersections per radius, neurons within animals and testing every radius

Quick answer. In Sholl analysis each traced neuron gives a profile of intersections at every radius, but the animal (or culture) is usually the unit that received the genotype or treatment, and neurons from one animal resemble each other. Choose a summary in advance, such as total intersections (the area under the profile when radii are equally spaced), average it over each animal’s neurons, and compare animals; or fit a mixed model to the per-neuron totals with a random intercept per animal and small-sample p-values. If the unit is a culture, use cultures in place of animals. To ask where along the dendrite groups differ, test each radius on animal means (or with a mixed model that has a random intercept per animal) and control the false discovery rate. In a simulation of 6 mice per group with 8 neurons each and no real difference, a t-test at each of 20 radii with neurons as n found at least one “significant” radius in 68.7% of studies; total intersections per mouse gave 4.5%.

Terms used below. Sholl analysis: counting how many times a neuron’s dendrites cross circles (or spheres) of increasing radius centred on the soma. Sholl profile: intersections plotted against radius for one neuron. Total intersections: the sum over all radii; with equal radius steps it is proportional to the area under the profile. Critical value and critical radius: the peak number of intersections and the radius where it occurs, read from each neuron’s profile or from a smooth curve fitted to it (the Fiji plugin fits a polynomial). Experimental unit: the thing that was randomised to a group, here the mouse. Mixed model: a regression that gives each animal its own baseline (a random intercept), estimated from all its neurons, so that neurons from one animal are not treated as independent; with few animals its p-values should use small-sample (Satterthwaite) degrees of freedom. Conservative: giving fewer than 5% false positives when there is no effect. False discovery rate (FDR): the expected share of “significant” results that are false; the Benjamini–Hochberg procedure controls it and gives adjusted p-values (q). Welch t-test: a two-group t-test that does not assume equal spread. 95% confidence interval (CI): the range of values compatible with the data. Power: the chance of detecting an effect that is really there. False positive (type I error): a “significant” result when there is no real effect.

Which analysis for your question

Question	Analysis	Report
Is the dendritic tree more or less complex overall?	Total intersections per neuron, averaged per animal; Welch t-test across animals; or a mixed model on per-neuron totals with a random intercept per animal (lmerTest in R)	Difference with 95% CI, n animals (and neurons per animal)
Three or more groups	Total intersections per animal; Welch’s ANOVA across animals, then planned comparisons with a Holm correction (which correction)	Differences with 95% CI and adjusted p
Where along the dendrite do groups differ?	At each radius, animal means compared across groups, Benjamini–Hochberg across radii; or one mixed model per radius with a random intercept per animal, with FDR control	Radii with q < 0.05, effect at each
Does the peak shift outwards or inwards?	Critical radius per neuron (from the Fiji plugin’s polynomial fit), averaged per animal, then the same Welch t-test as for total intersections; not simulated here	Difference in µm with 95% CI
Neurons from cultures rather than animals	The same, with the coverslip or independent culture as the unit	Difference with 95% CI, n cultures
Each animal contributes a single neuron	That neuron is the animal’s value, so the comparison is still between animals	Difference with 95% CI

Wilson and colleagues describe the method: “concentric circles at specified radii (usually in 10 micrometer (μm) increments) are centered on the neuronal soma and the number of dendritic intersections at each circle is counted”, with “the sum of all intersections within the Sholl radii, the number of intersections at individual radii, and the area under the curve” as common endpoints. They warn that, without correction, “when 10 radii are used, the probability of at least one type I error is about 40%”, that “mixed effects models correctly model the clustering that results from measurements made on multiple neurons per animal”, and that “resources should be geared towards more animals rather than more neurons per animal”; for radius-by-radius questions they suggest that “multiple mixed effects models can be fit, one for each radius, and then a procedure to control the false discovery rate can be implemented” (Wilson et al. 2017). Lazic notes that “neurons from the same brain are not independent: they are identical genetically, they have the same developmental history, and they share the same environment” (Lazic 2010). The Fiji Sholl Analysis plugin (Ferreira et al. 2014) “creates a series of concentric shells (circles or spheres) around the focus of a neuronal arbor, and counts how many times connected voxels defining the arbor intersect the sampling shells”, in the words of its documentation. Wilson and colleagues also describe “multi-level nesting (radii within neuron as well as neuron within animal) and an autoregressive covariance structure”, which a single mixed model of the whole profile can accommodate for an overall test, and warn that with many radii “there is a substantial loss of power when using an FDR of 5%”.

Worked example: a knockout with no real effect

Constructed data: intersections at radii from 10 to 200 µm (in 10 µm steps) for 8 neurons in each of 6 wild-type and 6 knockout mice (96 neurons, 1,920 counts). Each neuron’s profile rises to a peak of about 8 intersections near 60 µm and then falls. Mice differ in overall branching (standard deviation 0.15 on the log scale), neurons differ within a mouse (0.25, also on the log scale), and each count scatters around the neuron’s profile. By design, the knockout does not change branching. The data’s random seed was the first one tried and was not chosen for its result. Stored code and data: sholl.csv, model.py, make_data.py, analysis.py, analysis.R and the analysis output; analysis.py and analysis.R print the same output.

The mean profile peaked at 9.2 intersections at 60 µm in wild-type neurons and 8.7 at 70 µm in knockout neurons, one radius step apart by chance.

Analysis, knockout vs wild-type	Result
A. Welch t-test at each of 20 radii, neurons as n (48 vs 48; not valid)	no radius with p < 0.05; smallest p = 0.07
B. Total intersections per neuron, neurons as n (48 vs 48; not valid), Welch t-test	94.0 vs 92.8, p = 0.85
C. Total intersections per mouse (mean of its 8 neurons), mice as n (6 vs 6), Welch t-test	difference 1.1 (95% CI −20.9 to 23.2), p = 0.91
D. Each radius on mouse means, Welch t-tests, Benjamini–Hochberg	no radius with q < 0.05; smallest q = 0.98
E. Mixed model on per-neuron totals, random intercept per mouse (mixed_model.R, R only)	difference 1.1 (95% CI −19.8 to 22.0), t(10.0) = 0.12, p = 0.91

With this data set even the shortcuts (A and B) came out non-significant; the simulation below shows how often they do not. The interval in C, about ±22 intersections around a total of about 93, is what 6 mice per group can resolve. Across 5,000 simulated studies of this design (simulation.py, the simulation output):

Scenario	A. Any radius, neurons as n (invalid)	B. Total, neurons as n (invalid)	C. Total per mouse	D. Radii on mouse means, FDR
No effect, 6 mice × 8 neurons (false positives)	68.7%	22.8%	4.5%	3.4%
15% fewer intersections, 6 mice × 8 neurons (power)	88.2%	63.1%	27.3%	16.1%
15% fewer, 6 mice × 16 neurons (power)	92.8%	78.2%	31.7%	20.1%
15% fewer, 12 mice × 8 neurons (power)	95.3%	81.9%	53.9%	38.4%

These percentages describe this simulated design (20 radii, the variation in model.py), not every Sholl study. Testing every radius with neurons as n combines two errors: 20 tests and 48 neurons treated as independent, so it found a “difference” in most null studies, and its high power means nothing. Total intersections per neuron with neurons as n (B) still treats neurons from one mouse as independent, which gave false positives in 22.8% of studies. One value per mouse (C) kept false positives near 5%, and per-radius tests on mouse means with FDR control (D) were slightly conservative. With 6 mice per group a 15% drop was detected in only 27.3% of studies; doubling the neurons per mouse raised that to 31.7%, while doubling the mice raised it to 53.9%. The mixed model (E) gave the same estimate as C because every mouse had 8 neurons; its interval is slightly narrower because it pools the spread of both genotypes. It handles unequal numbers of neurons per mouse better than simple averaging. With only 3 or 4 mice per group, every valid analysis has wide intervals, and per-radius tests will rarely survive FDR control. The simulation only scales the whole profile up or down; a treatment that moves branches outward without changing the total would be missed by total intersections and needs the critical radius or a whole-profile model.

The code expects one row per neuron and radius with columns genotype (wild-type or knockout), mouse, neuron, radius_um and intersections; mouse IDs must be unique across genotypes, and neuron numbers may repeat across mice. If your export has one row per neuron and one column per radius, reshape it first, for example d = wide.melt( (where wide is your exported table read with pd.read_csv, id_vars=["genotype", "mouse", "neuron"], var_name="radius_um", value_name="intersections") in pandas. Rename your columns and labels to match, and run the code in the folder that holds the CSV file. Python needs NumPy, pandas and SciPy; the R code uses base R only.

import numpy as np, pandas as pd
from scipy import stats

d = pd.read_csv("sholl.csv")
tot = d.groupby(["genotype", "mouse", "neuron"]).intersections.sum()   # total intersections per neuron
m = tot.groupby(["genotype", "mouse"]).mean()                          # mean of each mouse's neurons
ko, wt = m["knockout"], m["wild-type"]; t = stats.ttest_ind(ko, wt, equal_var=False); ci = t.confidence_interval()
print(f"difference {ko.mean() - wt.mean():.1f} (95% CI {ci.low:.1f} to {ci.high:.1f}), p = {t.pvalue:.2g}")
# difference 1.1 (95% CI -20.9 to 23.2), p = 0.91

# Where along the dendrite? Each radius on mouse means, Benjamini-Hochberg across radii (analysis D)
M = d.groupby(["genotype", "mouse", "radius_um"]).intersections.mean().unstack()
p = [stats.ttest_ind(M.loc["knockout", r], M.loc["wild-type", r], equal_var=False).pvalue for r in M.columns]
print(f"radii with q < 0.05: {(stats.false_discovery_control(p) < 0.05).sum()}")   # 0

d <- read.csv("sholl.csv")
tot <- aggregate(intersections ~ genotype + mouse + neuron, data = d, FUN = sum)   # total per neuron
m <- aggregate(intersections ~ genotype + mouse, data = tot, FUN = mean)            # mean per mouse
t.test(intersections ~ genotype, data = m)    # Welch; knockout - wild-type 1.1, 95% CI -20.9 to 23.2, p = 0.91
# Mixed model on per-neuron totals (lme4 + lmerTest packages, Satterthwaite p-value): see mixed_model.R
# summary(lmerTest::lmer(intersections ~ genotype + (1 | mouse), data = tot))

Before you analyse

1. Decide the summary before tracing: total intersections (area under the profile, with equal radius steps) for overall complexity, the critical radius for a shift, or specific radius bands if you have a reason.
2. Choose neurons by a rule set in advance, blind to genotype: for example all neurons that meet fixed criteria (cell type, complete filling, soma in the section), taken in a fixed scanning order up to a set number per animal, and exclude neurons with cut or truncated dendrites the same way in every group.
3. Use the same radius step, starting radius and maximum radius for every neuron, and record whether the analysis is 2D or 3D.
4. Plan the number of animals, not just neurons: in our simulation, doubling the mice helped far more than doubling the neurons per mouse.
5. If you test radius by radius, test animal means (or one mixed model per radius with a random intercept per animal) and control the false discovery rate, and say how many radii you tested (which correction).

Check what an AI assistant ran

1. The assistant ran a t-test or ANOVA at every radius and starred the significant ones. Ask for one summary per animal, or per-radius tests on animal means with FDR control.
2. The assistant used neurons as n. Ask for animal means or a mixed model with animal as a random effect (nested data).
3. The assistant fitted a two-way ANOVA with genotype and radius as factors, on all neurons. Ask how neurons within animals and radii within neurons were handled.
4. The assistant reported only the radii that differed. Ask for the planned summary and the number of radii tested.

Report it

Dendrites of [8] neurons per mouse, chosen by [rule] blind to genotype, were traced in [6] wild-type and [6] knockout mice, and intersections were counted at radii of [10] to [200] µm in [10] µm steps (Fiji Sholl Analysis). Total intersections per neuron were averaged per mouse and compared between genotypes with a Welch t-test; radius-by-radius differences were tested on mouse means with Benjamini–Hochberg control across [20] radii (R [4.3.3]).

In the constructed example, which has no effect built in, total intersections did not clearly differ between genotypes (knockout − wild-type [1.1], 95% CI [−20.9] to [23.2]; p = [0.91]; n = [6] mice per genotype, [8] neurons per mouse), and no radius differed after FDR control.

Replace the bracketed values with your own.

References

• Wilson MD, Sethi S, Lein PJ, Keil KP (2017). Valid statistical approaches for analyzing Sholl data: mixed effects versus simple linear models. Journal of Neuroscience Methods 279:33–43. doi:10.1016/j.jneumeth.2017.01.003
• Lazic SE (2010). The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? BMC Neuroscience 11:5. doi:10.1186/1471-2202-11-5
• Ferreira TA, Blackman AV, Oyrer J, et al. (2014). Neuronal morphometry directly from bitmap images. Nature Methods 11:982–984. doi:10.1038/nmeth.3125
• Sholl Analysis (Fiji plugin) documentation. ImageJ wiki. imagej.net/imagej-wiki-static/Sholl. Accessed 12 November 2026.