Novel object recognition and preference indices: testing against chance and comparing groups

Quick answer. A one-sample test of the discrimination index against chance (0, or 50% for a preference index) answers only “does this group remember?”, and only when the mean is above chance and objects and positions were counterbalanced. To claim that a knockout or treatment impairs memory, compare the groups directly (for example Welch’s t-test on the discrimination index) and report the difference with its 95% confidence interval. “Significantly above chance in wild-type, not in knockout” is not evidence of a difference: in our simulation with identical groups (12 mice each, true discrimination index 0.20), one group was above chance and the other was not in 45.6% of experiments. Set the exclusion rule for low exploration (for example less than 20 s exploring the two objects combined) before testing, and report total exploration for each group.

Terms used below. Novel object recognition (NOR) test: a memory test in which a mouse that remembers a familiar object spends more time exploring a new one. Sample (training) session: the mouse explores two identical objects. Test session: one object is replaced by a new one. Discrimination index (DI): (novel − familiar)/(novel + familiar), from −1 to 1, with 0 meaning no preference. Preference (recognition) index: novel ÷ total, often as a percentage, with 50% meaning no preference; preference % = 50 × (DI + 1), so DI 0.20 is a preference of 60%. True DI: the value set when constructing the example data. Chance level: the value expected without memory (DI 0, preference 50%). One-sample t-test: tests whether a group’s mean differs from a fixed value. Welch’s t-test: compares two groups’ means without assuming equal spreads. Total exploration: novel plus familiar exploration time. ANOVA: compares means across several groups or factors; an interaction tests whether the effect of one factor (genotype) differs between levels of another (sex). Exclusion criterion: a rule, set before testing, for leaving out animals that explored too little. Counterbalancing: varying which object is novel and where it sits across animals. Blinding: scoring without knowing each animal’s group. Paired test: compares two measurements from the same animal, such as novel and familiar exploration times. Random seed: the number that fixes a computer’s random draws, so constructed data can be reproduced exactly. 95% confidence interval (CI): the range of effects compatible with the data. Power: the chance of detecting an effect that is really there. False positive: a “significant” result when there is no real effect.

Which test for your question

Question	Test	Report
Does this group show recognition memory?	One-sample t-test of DI against 0 (or preference against 50%); memory only if the mean is above chance, with objects and positions counterbalanced	Mean DI with 95% CI, n, p
Does the knockout or treatment change memory?	Welch’s t-test between groups on DI (Welch’s ANOVA or ANOVA, for more groups or factors)	Difference in DI with 95% CI, p
Does the effect differ between sexes or ages?	Two-way ANOVA with the interaction, with group sizes planned for it (analysing both sexes)	Interaction estimate with 95% CI
Did groups explore differently?	Welch’s t-test on the logarithm of total exploration (which evens out its right skew), or report it descriptively	Mean total exploration per group, before and after exclusions

Denninger and colleagues state the rule directly: if memory alone is the question, “a one-sample t-test of percent time investigating an object versus the fixed value of 50% can be used”, but to compare groups “it is erroneous to conclude that these two groups are significantly different from each other” from separate tests against chance; “those groups must be statistically compared to each other, not only compared to an external standard” (Denninger et al. 2018). Nieuwenhuis and colleagues make the general point: “a comparison of two experimental effects requires a statistical test on their difference” (Nieuwenhuis et al. 2011). For exclusions, Lueptow’s protocol excludes mice that “do not reach a 20 s minimum of exploration for both objects” in the sample or test session, the 20 s being total exploration of the two objects in that session (Lueptow 2017), and Leger and colleagues use “a selection criterion based on a minimal time of exploration for both objects during each session” (Leger et al. 2013).

Worked example: wild-type and knockout, 12 mice each

Constructed data: test-session exploration times for 12 wild-type (WT) and 12 knockout (KO) mice. By construction both genotypes have the same true mean DI, 0.20; the exclusion rule, set in advance, removes mice with less than 20 s exploring the two objects combined (the example data contain only the test session). The random seed (3) was the first from 1 upward for which the wild-type group was significant against chance, the knockout was not, and at least one mouse was excluded: the situation this page is about. Stored code and data: nor.csv, model.py, make_data.py, analysis.py, analysis.R and the analysis output; analysis.py and analysis.R print the same output.

Analysis	Result
Excluded (total exploration < 20 s)	WT4 and KO1 (one mouse per group; n = 11 each)
WT, one-sample test against 0	Mean DI 0.293 (95% CI 0.162 to 0.424; preference 64.7%), p = 0.0005
KO, one-sample test against 0	Mean DI 0.105 (95% CI −0.079 to 0.290; preference 55.3%), p = 0.23
KO − WT, Welch’s t-test	−0.188 (95% CI −0.401 to 0.025), p = 0.080
Mean total exploration	All mice: WT 36.0 s, KO 34.4 s; included mice: WT 38.2 s, KO 36.9 s

Taken as separate tests, the results say the knockout “has no memory”. Compared directly, the two groups do not differ clearly: the interval for the difference runs from a large impairment to a slight improvement. In 10,000 simulated experiments of this design (simulation.py, the simulation output):

Scenario	One group “above chance”, the other not	Welch’s t-test between groups, p < 0.05
Same true DI in both groups (0.20)	45.6%	5.1% (false positives)
No memory in either group (true DI 0)	4.7%	5.0% (false positives)
KO lower (WT 0.25, KO 0.05)	77.1%	40.4% (power)

How often the groups disagree depends on the true DI and the group size: it is high even when both groups have the same moderate memory, as here, and higher still when they truly differ. With 12 mice per group, a difference of DI 0.25 vs 0.05 was detected in only 40.4% of experiments; plan group sizes for the comparison between groups (how many biological replicates).

The code expects one row per mouse with columns mouse, genotype, novel_s and familiar_s; rename your columns and group labels to match.

# Condensed: analysis.py also prints total exploration
import pandas as pd
from scipy import stats

d = pd.read_csv("nor.csv")                           # one row per mouse: mouse, genotype, novel_s, familiar_s
d["total"] = d.novel_s + d.familiar_s
k = d[d.total >= 20].copy()                          # exclusion rule set before testing
k["di"] = (k.novel_s - k.familiar_s) / k.total
for g in ("WT", "KO"):
    t = stats.ttest_1samp(k.di[k.genotype == g], 0)        # recognition memory within each group (DI above 0)
    print(g, t, t.confidence_interval())                 # for preference %, test 100 * novel_s / total against 50
t = stats.ttest_ind(k.di[k.genotype == "KO"], k.di[k.genotype == "WT"], equal_var=False)   # the group comparison, KO - WT
print(t, t.confidence_interval())

d <- read.csv("nor.csv"); d$total <- d$novel_s + d$familiar_s
k <- d[d$total >= 20, ]; k$di <- (k$novel_s - k$familiar_s) / k$total
t.test(k$di[k$genotype == "WT"], mu = 0); t.test(k$di[k$genotype == "KO"], mu = 0)   # memory within each group
t.test(di ~ factor(genotype, levels = c("KO", "WT")), data = k)                       # Welch; KO listed first, so KO - WT

Before you test

1. Write the exclusion rule in advance (for example at least 20 s exploring the two objects combined, in both the sample and the test session) and report how many mice it excluded in each group (when can you exclude).
2. Counterbalance objects and positions, and score blind to group; Lueptow’s protocol counterbalances “the use of each set of objects” and “the location of the novel object” and asks researchers to “blind the scorer to the experimental conditions” (Lueptow 2017; blinding is item 5 of the ARRIVE 2.0 animal-research reporting guidelines, Percie du Sert et al. 2020).
3. Report total exploration, before and after exclusions. Ratios adjust for differences in exploration, but Akkerman and colleagues note that “differences in exploration levels may affect the statistical evaluation of group differences” (Akkerman et al. 2012); if totals differ between groups, say so when you interpret the indices. The same paper suggests comparing each group with a constructed group that shows no discrimination, and notes that a one-sided test is acceptable “provided that place- or object biases can be ruled out”; decide on a one-sided test before seeing the data, not after a two-sided p-value misses 0.05.
4. Choose one index in advance (DI or preference) and use it throughout; they carry the same information.
5. Plan the comparison between groups, not the test against chance, when choosing the group size.
6. Look at the indices before testing. DI lies between −1 and +1; if many mice sit at the limits or one value is extreme, add a rank-based test as a check (Wilcoxon signed-rank against 0 within a group, Mann–Whitney between groups). If mice share cages or litters, add cage or litter as a random effect (nested data).

Check what an AI assistant ran

1. The assistant concluded that one genotype is impaired because one group was significant against chance and the other was not. Ask for a direct comparison between the groups.
2. The assistant excluded mice after seeing the results. Ask for the rule and when it was set.
3. The assistant reported only DI, not total exploration. Ask for both.
4. The assistant tested raw novel and familiar times with a paired test in each group and compared p-values. Ask for the DI comparison between groups.

Report it

Recognition memory was assessed as the discrimination index (novel − familiar)/(novel + familiar) in the test session. Mice with less than [20] s of total exploration were excluded, as specified in advance ([1] WT, [1] KO). Each genotype’s index was compared with 0 using one-sample t-tests, and genotypes were compared with Welch’s t-test (R [4.3.3]).

After exclusions, n = [11] wild-type and [11] knockout mice were analysed. Wild-type mice preferred the novel object (mean DI [0.29], 95% CI [0.16] to [0.42], p = [0.0005]); the knockout mean DI was [0.11] (95% CI [−0.08] to [0.29], p = [0.23]). The difference between genotypes was [−0.19] (95% CI [−0.40] to [0.03]; p = [0.080]), so the data are compatible with anything from a clear impairment to a slight improvement in the knockout. Mean total exploration was [38.2] s (WT) and [36.9] s (KO).

Replace the bracketed values with your own.

References

• Denninger JK, Smith BM, Kirby ED (2018). Novel object recognition and object location behavioral testing in mice on a budget. Journal of Visualized Experiments (141):e58593. doi:10.3791/58593
• Nieuwenhuis S, Forstmann BU, Wagenmakers EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nature Neuroscience 14:1105–1107. doi:10.1038/nn.2886
• Lueptow LM (2017). Novel object recognition test for the investigation of learning and memory in mice. Journal of Visualized Experiments (126):55718. doi:10.3791/55718
• Leger M, Quiedeville A, Bouet V, et al. (2013). Object recognition test in mice. Nature Protocols 8:2531–2537. doi:10.1038/nprot.2013.155
• Akkerman S, Prickaerts J, Steinbusch HWM, Blokland A (2012). Object recognition testing: statistical considerations. Behavioural Brain Research 232:317–322. doi:10.1016/j.bbr.2012.03.024
• Percie du Sert N, Hurst V, Ahluwalia A, et al. (2020). The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. PLOS Biology 18:e3000410. doi:10.1371/journal.pbio.3000410