Colocalisation analysis: Pearson or Manders coefficient, and why n is not the number of cells

Quick answer. Decide which question you are asking. “How much of protein A sits where protein B is?” is answered by Manders’ coefficients (M1 and M2), the fraction of each channel’s signal that overlaps the other, after a threshold removes background. “Do the two signals rise and fall together?” is answered by Pearson’s correlation coefficient (PCC). Compute them within a region of interest (a cell or a cell mask), with the threshold method and settings stated. To compare conditions, average the per-cell values within each independent experiment (or animal; Pearson coefficients on the Fisher z scale) and test those averages, paired by experiment when every experiment contains both conditions: the cells in one dish are not independent. In our simulation, treating 150 cells as the sample gave a false positive in 41% of simulated studies with no real difference.

Terms used below. Colocalisation: two fluorescent labels found in the same place in an image. Channel: the image of one label. Pearson’s correlation coefficient (PCC): from −1 to 1, how closely the two channels’ pixel intensities vary together. Manders’ coefficients (M1, M2): from 0 to 1, the share of channel 1’s signal found where channel 2 is present (M1), and the reverse (M2), computed only above a stated background threshold. Threshold: the intensity below which a pixel counts as background. Costes threshold: an automatic threshold chosen so that pixels below it show no correlation. Region of interest (ROI): the area analysed, such as one cell. Fisher z: a transformation of a correlation coefficient (arctanh) that makes averaging and testing valid. Independent experiment: a separate culture, day or animal, the unit that can differ by chance from the next. Pseudoreplication: treating repeated measurements from one unit (cells from one dish) as independent. False positive: a “significant” result when there is no real effect. Power: the chance of detecting an effect that is really there. Cell mask: an outline of each cell, drawn or segmented, inside which coefficients are computed. Paired t-test: compares two conditions within each experiment and tests the average difference. Welch’s t-test: compares two independent groups without assuming equal spread. Mixed model: a regression with a random effect for each experiment (and each dish), an alternative to averaging. Bleed-through: signal from one fluorophore detected in the other channel. Saturated pixel: a pixel at the detector’s maximum, whose true intensity is unknown. Costes randomisation test: compares the observed Pearson coefficient with values from images whose pixel blocks have been shuffled. Spearman’s rank correlation: a correlation on ranks, less affected by non-linear intensity relations. SD: standard deviation. 95% confidence interval (CI): the range of effects compatible with the data. Back-transformed: converted from the Fisher z scale back to a correlation (tanh). Random effect: a model term for variation between units such as experiments or dishes. lmerTest: an R package that adds p-values to mixed models fitted with lme4’s lmer.

Which coefficient for your question

Question	Coefficient	Note
What fraction of A is where B is (and the reverse)?	Manders’ M1 and M2 (thresholded)	Needs a background threshold; report the method (Costes or a fixed value)
Do A and B vary together in intensity?	Pearson’s correlation coefficient	Sensitive to background and to the area analysed; compute within a cell mask
Is the overlap more than chance?	Costes’ randomisation test	Shuffles blocks of pixels to compare the observed value with chance
Did a treatment change colocalisation?	Either coefficient per cell, averaged per experiment (Pearson on the Fisher z scale; Manders’ values directly), then compared: paired by experiment if every experiment has both conditions, otherwise Welch’s t-test on the means	n is the number of independent experiments or animals

In short: Manders’ coefficients answer “how much overlaps”, Pearson’s coefficient “how closely intensities track”, and both depend on how the images were acquired and thresholded. Aaron and colleagues separate the two ideas: “co-occurrence and correlation. The former describes the extent of spatial overlap between two fluorophores. The latter determines the degree to which the abundance of two spatially overlapping fluorophores are related to each other” (Aaron et al. 2018); they also recommend Spearman’s rank correlation over Pearson’s when the intensity relation may not be linear; compute it per cell and average per experiment in the same way (the Fisher z transformation is approximately right for Spearman values too). Dunn and colleagues note that Manders’ coefficients, “by providing measures of the fraction of total probe fluorescence that colocalizes with the fluorescence of a second probe”, give “an intuitive and direct metric of the quantity of interest for most biological colocalization” studies (Dunn et al. 2011). Costes and colleagues set the threshold by “estimating simultaneously the maximum threshold of intensity for each color below which pixels do not show any statistical correlation” (Costes et al. 2004). Fiji’s Coloc 2 plugin implements Pearson’s, Manders’ and Costes’ methods, “for scatterplots, analysis, automatic thresholding and statistical significance testing” (Coloc 2 documentation).

Worked example 1: what each coefficient sees

Three constructed two-channel images (64 × 64 pixels, all with the same channel 1): in A, channel 2 sits on the same spots with proportional brightness; in B, on the same spots with brightness unrelated between spots; in C, half of channel 2’s spots lie elsewhere. In this constructed example the threshold was set at the mean plus 3 SD of background pixels (pixels in a cell-free area) (16 on the images’ arbitrary intensity scale, fixed for every image). Stored code and data: images.csv, make_data.py, analysis.py, analysis.R and the analysis output; analysis.py and analysis.R print the same output.

Image	Pearson	Manders’ M1	Manders’ M2
A: same spots, proportional brightness	0.991	0.957	0.980
B: same spots, unrelated brightness	0.952	0.963	0.977
C: half the spots elsewhere	0.490	0.603	0.615

Manders’ coefficients read directly: in A and B almost all of each signal overlaps the other; in C about 60% of each signal does. Pearson’s coefficient falls a little from A to B, because brightness no longer tracks between channels, and drops to 0.49 in C, a value that has no direct meaning as a fraction. Pearson’s coefficient over a whole field is also dominated by the contrast between bright structures and background, which is why Dunn and colleagues advise measuring it “for individual cells” within a hand-drawn or segmented region of interest (Dunn et al. 2011).

Worked example 2: cells are not the sample

Constructed per-cell Pearson coefficients from 3 independent experiments × 2 conditions × 25 cells, with no true treatment effect: experiments differ from each other (SD 0.08 on the Fisher z scale), the two dishes within an experiment differ by chance (SD 0.08), and cells vary with SD 0.15. The random seed (2) was the first from 1 upward for which a per-cell test gives p < 0.05, the situation this example illustrates. Data: cells.csv.

Analysis	Result
Per-cell Welch’s t-test on Fisher z (150 cells as independent)	t = −2.72, p = 0.0073
Mean Pearson per experiment, control vs treated (back-transformed from the mean Fisher z)	0.580 vs 0.426; 0.469 vs 0.525; 0.607 vs 0.559
Paired t-test on the 3 experiment means	Difference in z −0.069 (95% CI −0.420 to 0.282), p = 0.49

The per-cell test mistakes dish-to-dish variation for a treatment effect; the experiment means show no consistent direction. Lord and colleagues recommend exactly this: “pool the cell-level data from each experiment separately and compare the subsequent sample-level” means (Lord et al. 2020; SuperPlots). In 10,000 simulated studies of this design (simulation.py, the simulation output):

Design	Per-cell test: false positives / power	Paired test on experiment means: false positives / power
3 experiments, 25 cells per dish	40.8% / 98.1%	5.0% / 37.5%
6 experiments, 25 cells per dish	40.4% / 99.9%	4.7% / 90.0%

(Power is for a difference of 0.2 on the Fisher z scale, about 0.11 in Pearson’s r around r = 0.6.) The false-positive rate of the per-cell test depends on how much dishes and experiments differ; with these variance components it was about 40% whatever the number of experiments. Adding cells makes the per-cell test even more misleading, because its standard error shrinks while the dish-to-dish variation stays; adding independent experiments raises the power of the correct one.

# Condensed: analysis.py also computes the three example images
import numpy as np, pandas as pd
from scipy import stats

d = pd.read_csv("cells.csv")                         # one row per cell: experiment, condition, cell, pearson
d["z"] = np.arctanh(d.pearson)                       # Fisher z before averaging correlation coefficients
m = d.groupby(["experiment", "condition"]).z.mean().unstack()   # one value per experiment and condition
print(np.tanh(m))                                    # back-transformed means
print(stats.ttest_rel(m.treated, m.control))         # n = number of independent experiments

d <- read.csv("cells.csv"); d$z <- atanh(d$pearson)
m <- tapply(d$z, list(d$experiment, d$condition), mean)          # one value per experiment and condition
tanh(m)
t.test(m[, "treated"], m[, "control"], paired = TRUE)
# Alternative using every cell (needs lmerTest for p-values; here p = 0.43, close to the paired test):
# lmerTest::lmer(z ~ condition + (1 | experiment/condition), data = d)
# Manders M1 or M2: average the per-cell values directly (no Fisher z) per experiment and test them the same way

Before you analyse

1. Acquire for colocalisation: no saturated pixels, matched settings across conditions, and controls for bleed-through between channels (single-labelled samples). Noise lowers Manders’ coefficients, so compare conditions only when signal-to-noise and expression levels are similar, or report that they differ. Bolte and Cordelières give practical tips for “image acquisition and treatment that should precede proper colocalization analysis” (Bolte & Cordelières 2006).
2. Define the region and build the table: draw or segment a cell mask blind to condition, run Coloc 2 (or your own code) on each cell with the same threshold rule (Coloc 2 writes its numbers to the Log window, which can be saved; a macro looping over the ROI Manager does many cells), and record one row per cell with columns experiment, condition, cell and the coefficients (as in cells.csv). In Coloc 2, the thresholded Manders values are reported as tM1 and tM2.
3. Choose and report the threshold: Costes’ automatic threshold, or a fixed rule set in advance; Manders’ coefficients depend on it.
4. Average within each independent experiment (or animal) before testing; average Pearson coefficients on the Fisher z scale. Use a paired t-test when every experiment contains both conditions, Welch’s t-test on the means otherwise. A mixed model is the alternative, but it needs random effects for experiment and for each dish within it, (1 | experiment/condition); a random intercept for experiment alone keeps much of the per-cell problem (SuperPlots; nested data). Unequal numbers of cells per dish are fine once you average; with only two experiments, report the experiment means and treat any test as descriptive.
5. Plan enough independent experiments: in our simulation, for a difference of 0.2 on the Fisher z scale, 3 experiments gave 37.5% power and 6 gave 90.0%.

Check what an AI assistant ran

1. The assistant tested hundreds of cells as independent observations. Ask for one value per experiment or animal.
2. The assistant averaged Pearson coefficients directly. Ask for the Fisher z transformation before averaging.
3. The assistant reported a coefficient without the region or threshold. Ask how the cell mask and threshold were set.
4. The assistant called a Pearson value of 0.5 “50% colocalisation”. Pearson’s coefficient is not a fraction; use Manders’ coefficients for overlap.

Report it

Colocalisation of [A] and [B] was quantified in [Fiji Coloc 2 / custom code] within cell masks drawn blind to condition. thresholded Manders’ coefficients (tM1, tM2) were computed above [Costes’ automatic thresholds / a threshold of background mean + 3 SD], and Pearson’s coefficient within each mask. Per-cell coefficients were averaged within each independent experiment (Pearson coefficients on the Fisher z scale), and conditions were compared with a [paired t-test on the experiment means, as every experiment included both conditions / Welch’s t-test on the experiment means] (n = [3] experiments, [25] cells per condition per experiment).

Mean Pearson’s coefficient (back-transformed) was [0.55] in control and [0.50] in treated cells; the difference did not differ clearly from zero (difference in z [−0.069], 95% CI [−0.420] to [0.282]; p = [0.49]; n = [3] experiments). [For Manders: mean M1 was [ ] vs [ ] (difference [ ], 95% CI [ ] to [ ]).]

Replace the bracketed values with your own.

References

• Aaron JS, Taylor AB, Chew TL (2018). Image co-localization – co-occurrence versus correlation. Journal of Cell Science 131:jcs211847. doi:10.1242/jcs.211847
• Dunn KW, Kamocka MM, McDonald JH (2011). A practical guide to evaluating colocalization in biological microscopy. American Journal of Physiology – Cell Physiology 300:C723–C742. doi:10.1152/ajpcell.00462.2010
• Costes SV, Daelemans D, Cho EH, Dobbin Z, Pavlakis G, Lockett S (2004). Automatic and quantitative measurement of protein-protein colocalization in live cells. Biophysical Journal 86:3993–4003. doi:10.1529/biophysj.103.038422
• Bolte S, Cordelières FP (2006). A guided tour into subcellular colocalization analysis in light microscopy. Journal of Microscopy 224:213–232. doi:10.1111/j.1365-2818.2006.01706.x
• Lord SJ, Velle KB, Mullins RD, Fritz-Laylin LK (2020). SuperPlots: communicating reproducibility and variability in cell biology. Journal of Cell Biology 219:e202001064. doi:10.1083/jcb.202001064
• Fiji. Coloc 2. imagej.github.io (checked 11 November 2026)