A real-time, multi-animal model for automatic face detection and identification of freely moving common marmosets based on YOLOv8 algorithms

eLife Assessment

This valuable study presents a real-time system for identifying multiple unrestrained marmosets in a home cage setting using a combination of facial features and color-coded beads. While there is solid evidence that the system has a precision comparable to human experimenters in the tested scenarios, there is limited evidence that this would generalize to unconstrained multi-animal environments

https://doi.org/10.7554/eLife.110932.3.sa0

Significance of the findings:

Valuable: Findings that have theoretical or practical implications for a subfield

Landmark
Fundamental
Important
Valuable
Useful

Strength of evidence:

Solid: Methods, data and analyses broadly support the claims with only minor weaknesses

Exceptional
Compelling
Convincing
Solid
Incomplete
Inadequate

During the peer-review process the editor and reviewers write an eLife Assessment that summarises the significance of the findings reported in the article (on a scale ranging from landmark to useful) and the strength of the evidence (on a scale ranging from exceptional to inadequate). Learn more about eLife Assessments

Abstract
Introduction
Results
Discussion
Materials and methods
Appendix 1
Data availability
References
Article and author information
Metrics

Abstract

Precise and up-to-date information about animal location and identity allows us to better quantify individual behaviors in studies of neural activity, cognition, and animal health. In socially housed laboratory animals, identification is usually defined by observation or invasive markers, making the data collection time-consuming, variable across experimenters, and disruptive to animals. We established an automatic pipeline for real-time identification of common marmosets in captivity using a close-view camera. It uses the supervised deep-learning YOLOv8 model to localize individuals, detect faces, and classify identities. Moreover, we use recognition of uniquely color-coded collar beads to improve detection accuracy among visually similar individuals. Across adult and juvenile marmosets, our system automatically identifies marmosets with >82.9% precision and >91.5% recall, achieving human-level performance. This pipeline is designed to be easy to use and generalizable across non-human primate species, ages, and recording hardware, providing rapid and automatic identity recognition from real-time video.

Introduction

In behavioral neuroscience, an accurate identification of laboratory animals is crucial for welfare assessment and ethical care (National Research Council, 2011; Vidal et al., 2021). More importantly, the animal identity is also required when conducting individual-specific behavioral experiments. However, identifying specific animals in a family group can be challenging, especially if they are not marked and share similar facial features. Invasive recognition methods, including branding, tattooing, and ear tagging, may disrupt animals’ behaviors and cause potential stress (Carstens and Moberg, 2000; Lim et al., 2019; Roughan and Sevenoaks, 2019). To avoid such harms, video analysis is becoming more popular as a non-invasive method to identify the animals by their appearance, with no direct contact or handling being involved (Norouzzadeh et al., 2018; Schindler and Steinhage, 2021). However, collecting video data of animals that is subsequently labeled by human observation can be time-consuming and hard to reproduce across experimenters (Buchan et al., 2003; Marion et al., 2020). Therefore, considering replacement of human work, machine learning tools have been introduced and increasingly used, providing an accurate and automatic estimation of animal identities.

Computer vision and machine learning have made many advancements and offered the foundation to build these automatic systems (Sturman et al., 2020). Within these approaches, deep learning, a subset of machine learning, has been widely used in animal identification based on video and image data. Specifically, by utilizing deep learning with video recordings, object detection allows an accurate localization and identification of animals, even with those who live in complicated environments (Yu et al., 2018; Zhuang et al., 2025). Algorithms such as ResNet (He et al., 2015) and You Only Look Once (YOLO) models Jocher et al., 2023; Redmon et al., 2015 have demonstrated promising accuracy and computational speed in identifying animals in naturalistic environments (Bakana et al., 2024; Petso et al., 2021). In addition, unique facial features can also be used as inputs to classify and recognize animals. This approach has the advantage that it provides information regarding whether a specific animal is present or absent in the camera recording view at each time point. Although facial recognition has been demonstrated to be successful in wild and domestic animals (Bergman et al., 2024; Norouzzadeh et al., 2021; Schofield et al., 2023), its application in controlled laboratory settings is not yet extensively evaluated.

The common marmoset (Callithrix jacchus), a small non-human primate, is becoming more popular recently in neuroscience research for many reasons (Kishi et al., 2014; Okano, 2021; Sasaki et al., 2009). Marmosets are family-bonded, cooperative, and notably pro-social. Moreover, they can perform multiple series of behavioral and cognitive tasks, such as observational and reversal learning (Koski and Burkart, 2015; Miller et al., 2016). As marmosets are typically maintained in social family groups (Yoshimoto et al., 2018), accurate identification of specific individuals is therefore essential for assessing which animal is performing a given task, which also enables tracking of their behavioral performance over time. However, existing identification techniques of marmosets usually involve wearable micro-sensing devices, requiring stable device positioning to the sensor and constant adjustments to ensure the devices fit the animal. This automatic system can also become unreliable during rapid movement or if multiple marmosets are present at the sensor, which limits the consistency and stability of identity tracking in freely moving marmosets during their tasks. Thus, facial recognition offers a non-invasive alternative, without a strict need for a physical identity marker, enabling continuous, accurate, and automated identification when the marmoset enters the designated task performance space.

Here, we developed a pipeline for automatically detecting and identifying marmosets simultaneously from real-time videos, based on their faces. Previous studies have introduced the feasibility of the methods we used in this study (Dave et al., 2023; XiaoAn et al., 2024), of which we adapted them for facial-based marmoset recognition. We apply the YOLOv8 model (Jocher et al., 2023), trained on images of specific individuals (three adult and two young marmosets), to generate real-time marmoset recognition on a frame-by-frame manner. In addition to marmoset faces, we use the color-coded bead on individual collars to facilitate the identity prediction accuracy. Finally, we further evaluated our model performance on a dataset from both adult marmosets and young marmosets at different developmental stages. We show that our facial-based pipeline achieves high detection performance comparable to human expert level, with strong generalizability across recording settings and marmosets at various ages.

Results

Comparison of pre-trained YOLOv8 models

Model performance included the precision (the proportion of correct positive predictions), recall (the proportion of corrected predicted ground truth labels), and mAP@50–95 (the average detection accuracy across different intersection over union [IoU] object localization thresholds; see Materials and methods section for detailed definitions). The results of the training and evaluation of the pre-trained YOLOv8 models (YOLOv8 nano, YOLOv8 small, YOLOv8 medium) were shown in Table 1. The models reached their best performance at 180 epochs (YOLOv8 nano), 132 epochs (YOLOv8 small), and 124 epochs (YOLOv8 medium). Each of the models was trained until reaching the early stopping criteria (i.e. no improvement within the last 100 training epochs). By comparing the performance differences between the YOLOv8 pre-trained networks, we aimed to find the most suitable model for the real-time marmoset identification program.

Table 1

Performance comparison of multi-marmoset face classification models.

Comparison of recall, precision, F1 score, mean average precision (mAP) at intersection over union (IoU) = 0.5:0.95, validation distribution focal loss (DFL), and training time of the three pre-trained models, based on the performance of marmoset face detection and identification on the adult marmosets’ dataset. The highest values of each parameter were highlighted in bold.

Metrics	YOLOv8 nano	YOLOv8 small	YOLOv8 medium
Recall	0.964	0.957	0.956
Precision	0.932	0.927	0.937
F1 score	0.948	0.942	0.946
mAP@50–95	0.710	0.710	0.713
Val DFL	0.917	0.953	1.010
Training time (s)	5544	5908	10705

As presented in Table 1, the YOLOv8 nano model reached the best overall detection performance of the adult marmosets’ dataset. The YOLOv8 medium model achieved a higher precision among all three pre-trained networks. However, the recall and F1 score were higher for the YOLOv8 nano model, indicating robust model sensitivity and a balanced precision-recall value. Mean average precision (mAP@50–95) showed values ranging from 0.710 to 0.713 for the three models. The YOLOv8 medium model reached the highest detection accuracy for adult marmoset faces, though the difference was minor while comparing to the YOLOv8 nano and small models. In addition, we considered the validation distribution focal loss (DFL) and training time to support the selection of the optimal pre-trained model. Validation DFL increased with model sizes, suggesting that the detection localization became less accurate from YOLOv8 nano to YOLOv8 medium. Moreover, the training time increased from YOLOv8 nano to the medium model, while the YOLOv8 medium took about twice the training time compared to the YOLOv8 nano model. Thus, we selected the YOLOv8 nano model for training the real-time program due to its high computational speed and efficiency, while maintaining reliable marmoset face prediction compared to larger YOLOv8 models (Jocher et al., 2023).

Prediction based on adult marmoset faces shows reliable identification

We applied the YOLOv8 nano model for the adult marmoset face classification. The training curves of the overall precision, recall, and mAP@50–95 (intersection over union [IoU]=0.5:0.95) were visualized in Figure 1A, B, C, Figure 1—figure supplement 1. The face classifier weights were selected at the training epoch 183 (denoted by the red dotted line), reaching the precision at 0.932, recall at 0.964, and the mAP@50–95 (IoU = 0.5:0.95) at 0.710 (see Table 1). We evaluated the face classifier performance per label class using the adult marmoset’ validation dataset (Figure 1D, see Appendix 1—table 1). Analyzing the label classes, the adult marmoset face classifier showed stable and high precision and recall across both individual identity (i.e. Adult1, Adult2, Adult3) and bead-color (i.e. collar_Adult1, collar_Adult2, collar_Adult3) label classes (Figure 1—video 1). In addition, the marmoset face classes showed high mAP across varied IoU thresholds, suggesting a robust prediction localization for marmoset faces (mAP@50–95=0.841, 0.787, 0.868 for three adults, respectively). The lower mAP value observed for the bead-color label category (mAP@50–95=0.603, 0.665, 0.515 for collars of three adults) suggested a greater variability in bounding box localization for the marmoset collar beads. This effect was consistent with the small bounding box sizes of the collar-bead label class, which increased the model sensitivity to localization variability in the prediction (example detection shown in Figure 10C). The normalized confusion matrix comparison was shown in Figure 2. The detection accuracy shows similar performance using the adult marmosets’ training dataset (Figure 2A) and validation dataset (Figure 2B), with very low occurrence of false detection between the marmoset individuals and the background images.

Figure 1 with 2 supplements see all

Download asset Open asset

Training performance of the multi-marmoset face classification model for three adult marmosets, using the pre-trained YOLOv8 nano model.

(A) Precision of all detection classes across training epochs. The red dotted line denoted the final model with the best performance at training epoch 183. (B) Similar to (A), except for overall recall. (C) Similar to (A), except for the mean average precision (mAP) at the intersection over union (IoU) at 0.5:0.95. (D) Overall model precision, recall, and the mAP@50–95 (IoU = 0.5:0.95) for each label class.

Figure 2

Download asset Open asset

Normalized confusion matrix per-class classification across the six label classes in the adult marmoset recognition model.

The y-axis represents the predicted class, and the x-axis represents the manually labeled class. Proportion was generated by the (A) training dataset and the (B) validation dataset, showing whether certain classes were frequently mislabeled as a different class.

Automatic facial and identity extraction accurately localizes new marmoset faces and their collar beads

The labeled adult marmosets’ dataset was used for training the automatic facial and identity extraction, using the YOLOv8 nano pre-trained model. This framework aimed to extract and localize the marmoset faces and collar beads from the collected video, including new marmosets. The trend of the training was shown in Figure 3A–C and Figure 3—figure supplement 1, where the detection accuracy improved rapidly in the beginning, with the increasing rate then stabilizing and reaching a plateau. We selected the final weights at the training epoch of 124, with a prediction score at 0.940, a recall score at 0.970, and the mAP@50–95 (IoU = 0.5:0.95) at 0.716. In addition, precision, recall, and mAP@50–95 (IoU = 0.5:0.95) were calculated per label class for marmoset faces and collar beads (Figure 3D). Based on the model performance of the adult marmosets’ validation dataset, the final face extraction system accurately predicted the marmoset faces (precision = 0.919, recall = 0.968) and collar beads (precision = 0.957, recall = 0.95). The bead-color class showed lower mAP@50–95 (IoU = 0.5:0.95) values at 0.6, compared to the mAP@50–95 (IoU = 0.5:0.95) for marmoset face class at 0.95, suggesting a similar effect for the face classification of adult marmosets in Figure 1D and Appendix 1—table 2. The normalized confusion matrices were compared between the training and validation dataset (Figure 4). It was evident that the automatic face and identity extraction exhibited a high detection accuracy. The background class of the confusion matrix showed frequent predictions as marmoset faces and collar beads for the training (Figure 4A) and validation set (Figure 4B). However, it does not necessarily indicate incorrect predictions or misclassifications. Instead, these values were mostly explained by multiple detections of the same object class. For instance, additional marmoset faces were predicted when multiple animals were present within a single video frame. The long collar structure or motion blur of the marmosets could also cause multiple detections of beads that belong to the same collar. This also corresponded to the high precision and recall scores observed across prediction classes (Figure 3D), suggesting that the increased background false positives were mainly related to the object-count discrepancies, instead of poor detection performance.

Figure 3 with 2 supplements see all

Download asset Open asset

Training performance of the automatic facial and identity extraction model.

(A) Precision of all detection classes across training epochs. The red dotted line denoted the final model with the best performance at training epoch 124. (B) Similar to (A), except for overall recall. (C) Similar to (A), except for the mean average precision (mAP) at the intersection over union (IoU) at 0.5:0.95. (D) Overall model precision, recall, and the mAP@50–95 (IoU = 0.5:0.95) for each label class.

Figure 4

Download asset Open asset

Normalized confusion matrix per-class classification across the two label classes in the automatic identity extraction model.

Proportion was generated by the (A) training dataset and the (B) validation dataset, showing whether certain classes were frequently mislabeled as a different class.

To test the performance of the automatic face and identity extraction, we used different short clips of new marmosets. We showed that our automatic face and identity extraction pipeline could detect and localize new marmoset faces and collar beads at different ages. Figure 3—video 1 showed an example of the detection of faces and bead-color labels for two new younger marmosets at 7 months and 17 months, with the confidence threshold of the detection set at 0.6.

Comparison of identification accuracy in young marmosets across developmental stages

The training and validation datasets of the young marmosets were annotated automatically using the automatic facial and identity extraction. All automatically generated labels were reviewed by the experimenter, and mislabeled images were removed prior to the model training. The training performance curves were presented in Figure 5A–C and Figure 5—figure supplement 1. The final weights were selected at the training epoch of 245, where the training performance metrics achieved a plateau. This program achieved the best performance metrics at this epoch, with precision score at 0.979, recall at 0.975, and mAP@50–95 (IoU = 0.5:0.95) at 0.892. In the young marmoset faces and their collar bead label classes, our face classifier reached high precision and recall performances using the validation dataset (Figure 5D, see Appendix 1—table 3). The precision and recall scores for the label classes were: (1) face of Young1: precision = 1, recall = 0.935; (2) face of Young2: precision = 0.958, recall = 1; (3) collar of Young1: precision = 0.998, recall = 1; (4) collar of Young2: precision = 0.963, recall = 0.964. In terms of localization accuracy, we found a similar pattern of reduced mAP@50–95 (IoU = 0.5:0.95) in bead-color class (mAP@50–95=0.808, 0.855), compared to the marmoset faces (mAP@50–95=0.95, 0.947). We further computed the normalized confusion matrices of this face classifier using the training and validation dataset. We found that our face classification was relatively accurate across detection of most class labels (Figure 6). The normalized confusion matrices showed high accuracy and consistency of most marmoset faces and collars detection in training (Figure 6A) and validation (Figure 6B) tests, with some exceptions. Particularly, the background was frequently identified as the collar of Young2 marmoset. This elevated background score was likely contributed by the multi-color design of the Young2 marmoset collar, making it more difficult to distinguish compared to collars with a single bead color. In this occasion, if one bead is occluded, blurred, or outside the field of view, the other visible collar bead could affect the prediction and lead to an incorrect identification from the ground truth.

Figure 5 with 5 supplements see all

Download asset Open asset

Training performance of the multi-marmoset face recognition model for young marmosets at 7 months.

(A) Precision of all detection classes across training epochs. The red dotted line denoted the final model with the best performance at training epoch 245. (B) Similar to (A), except for overall recall. (C) Similar to (A), except for the mean average precision (mAP) at the intersection over union (IoU) at 0.5:0.95. (D) Overall model precision, recall, and the mAP (IoU = 0.5:0.95) for each label class.

Figure 6

Download asset Open asset

Normalized confusion matrix per-class classification across the four label classes in the young marmoset recognition model.

Proportion was generated by the (A) training dataset and the (B) validation dataset, showing whether certain classes were frequently mislabeled as a different class.

To test the detection performance of the face classifier trained on 7-month-old young marmosets across different developmental stages, we used short clips of the two young marmosets at the age of 7 months (Figure 5—video 1, Figure 5—video 2) and 11 months (Figure 5—video 3, Figure 5—video 4). Figure 5—videos 1–4 presented the example detections of the young twins at different ages, showing the difficulty of identifying and distinguishing between twin marmosets.

By combining the identification of marmoset faces and collar beads, the program showed valid detection of the correct identity of individual marmosets. However, the detection of the twin marmosets also posed various challenges for computer vision and resulted in mislabeling, including highly similar faces of twin marmosets, face blur or mislabeling due to complex backgrounds and dark lighting conditions (Figure 5—video 1), fur occluding collar beads, marmosets only showing faces during short time intervals (Figure 5—video 4), similar or same color of collar beads between two marmosets (Figure 5—video 3, Figure 5—video 4), and fewer training data (young marmosets’ model only included 449 training and validation images for two marmosets at 7 months of age).

We tested the prediction performance without collar and its longitudinal application, using the face-only prediction on the young marmosets at 11 months and 16 months (Figure 7—video 1, Figure 7—video 2, Figure 7—video 3). The face classifier correctly identified the young twin marmosets solely based on their facial features, indicating that facial identity classification was performed independently of collar information and that the collar beads acted only as an auxiliary cue rather than the main classifier of the system (Figure 7—video 1).

Facial similarities of marmosets affect the performance of the facial detection and identification model

We quantified the inter-individual face similarities using cosine similarity and Euclidean distance between mean normalized embeddings of marmoset pairs. We included both adult and young marmosets’ dataset and computed the face similarity based on the four relationship pairs: (1) mother-father, (2) father-son, (3) mother-son, and (4) twin-twin. To visualize across the two independently trained models, the inter-individual face similarity scores were normalized using z-score within each model (Figure 7, Figure 7—figure supplements 1 and 2).

Figure 7 with 5 supplements see all

Download asset Open asset

Across-model visualization of the face similarity between marmoset pairs.

Four types of family relationships (mother-father, father-son, mother-son, and twin1-twin2) were compared, based on the training results of adult and young marmosets. The similarity was calculated using (A) cosine similarity and the (B) Euclidean distance. Heat map of the (C) cosine similarity and (D) Euclidean distance was plotted between the three relationship pairs in the adult family. The cosine similarity score ranged from 0 (very different) to 1 (exactly the same) for marmoset faces. The Euclidean distance score ranged from 0 (exactly the same) to 1 (very different) for marmoset faces.

Within the model trained using the adult marmoset family, both face similarity measures showed a consistent pattern across the family relationships. In the adult family, we found that the father-son pair showed the highest face similarity values, with the cosine similarity at a positive mean z-score (z=0.43) and the Euclidean distance at a negative mean z-score (z=–0.47) (Figure 7C and D). This pattern suggested that the father-son pair exhibited a higher-than-average similarity relative to other family relationships. On the other hand, the mother-father relationship showed the lowest face similarity in the family (negative cosine similarity z=–0.38, positive Euclidean distance z=0.43) compared to the adult marmosets’ model mean. The similarity of the mother-son pair was calculated at cosine similarity z=–0.05 and Euclidean distance z=0.04, indicating an intermediate face similarity in the adult family.

The young marmosets’ identifier was independently trained specifically on the two twin marmosets. Thus, the normalized face similarity z-scores were centered near zero and not informative of the twin similarity in family relationship comparison (Figure 7). However, we found a narrow distribution of the image embeddings between the twin marmosets, indicating nearly invariant facial embeddings and structure between the two twins, with an extremely low raw variance for cosine similarity (std = 0.0005) and Euclidean distance (std = 0.0034).

We performed the statistical tests only on the face classifier for the adult marmoset family, as the twin marmoset model only involved two individuals and thus was not valid for within-model statistical analysis (Tables 2 and 3). Cosine similarity showed a trend of lower similarity in the mother-father pair compared to the father-son pair (t=–1.941, p=0.083>0.05, Cohen’s d=–0.868 [large effect]), though not significant. Differences between other relationship pairs were not significant (Table 2). Our analysis on the Euclidean distance indicated a significant difference between mother-father and father-son pairs (t=2.28, p=0.046<0.05, d=1.02 [large effect]), while the remaining relationship pairs appeared non-significant (Table 3).

Table 2

Statistical analysis of inter-individual face similarity (cosine similarity) between different family relationships, within the adult marmoset family.

Comparison of t-statistics, p-value, and Cohen’s d on cosine similarities were compared between the three family relationships. The relationship pairs that showed significant differences were bolded.

Relationships	t-Value	p-Value	Cohen’s d
Mother-father vs. father-son	–1.941	0.083	–0.868
Mother-father vs. mother-son	–0.613	0.548	–0.274
Father-son vs. mother-son	1.460	0.177	0.653

Table 3

Statistical analysis of inter-individual face similarity (Euclidean distance) between different family relationships, within the adult marmoset family.

Comparison of t-statistics, p-value, and Cohen’s d on Euclidean distances were compared between the three family relationships. The relationship pairs that showed significant differences were bolded.

Relationships	t-Value	p-Value	Cohen’s d
Mother-father vs. father-son	2.276	0.046	1.018
Mother-father vs. mother-son	0.747	0.465	0.334
Father-son vs. mother-son	–1.397	0.192	–0.625

A real-time marmoset identification program based on the trained networks

We developed a real-time interface for detecting and identifying the marmosets, based on the trained models of multi-marmoset classification. The real-time footages were acquired using the same camera and experimental setup as the training videos (see examples in Figure 1—video 1–Figure 5—video 4). Our real-time program processes the real-time footage like the detection of the offline collected videos, using the best model selected from each multi-marmoset face classification training model. Prior to running the real-time program, the experimenter first trains the multi-marmoset face classification model based on the specific subjects of interest. Next, the model with the best performance from training is selected and inputted into the real-time program. No programming is required from the experimenter using this real-time marmoset face identification program. The experimenter clicks a run button on the program console to initiate the real-time detection. While this program is running, the experimenter can check the frame-by-frame label detection results (30 frames per second) displayed on the screen. Potential mislabeling is mitigated by combining the marmoset faces and collar beads detection, with greater weights assigned to the collar bead detections than the marmoset face (example mislabeling in Figure 5—videos 3 and 4). The program automatically records the most frequently detected marmoset identity across 30 frames (approximately 1 s). At the end of the experiment, the program is terminated by clicking a stop button of the Python console. After the completion of the experiment, the full record of marmoset identity detection results is available for review by the experimenter.

Discussion

We developed a real-time computer vision program for automatic marmoset identity recognition, using their facial features and uniquely color-coded bead collars. By combining the automatic annotation of the real-time footage with the individual-specific classification (i.e. faces and collar beads), our program allows continuous identity tracking during behavioral experiments, with minimal human interference. Utilizing the object detection YOLOv8 model based on pre-trained image networks, we adapted it to a non-human primate (i.e. marmosets) face dataset and applied it in real-time tracking. Pose estimation tools have been widely used in characterizing animal behavior; however, these tools require extensive computational power, thus limiting the identification between visually similar individuals, especially animals housed in family units (Camilleri et al., 2023; Gill et al., 2025; Lauer et al., 2022; Mathis et al., 2018; Nath et al., 2019). Notably, the main goal of the current system is marmoset identity recognition, instead of pose estimation or behavioral analysis, such that identity characterization is not dependent on posture cues or movement tracking. Thus, we selected the YOLOv8 object detection algorithms for the development of our real-time marmoset face recognition pipeline. Among the YOLOv8 pre-trained models, we selected the lightweight YOLOv8 nano model, which provided the optimal balance between detection accuracy and computational inference speed, supporting its feasibility for the final real-time marmoset identity recognition program.

Our real-time marmoset recognition pipeline includes two face classifications for both adult and young marmosets, with an additional automatic face and identity extraction program for all marmosets. The models presented in this paper achieved reliable detection accuracy across adult marmosets’ and twin marmosets’ datasets, with detection accuracy improving with increased amounts of training images. While comparing between the adult and young marmoset datasets, we found that the adult marmosets’ face classifier, trained with a larger number of varied images, showed more reliability and efficiency in marmoset identity recognition. Moreover, we anticipate that experimenters can integrate this program into behavioral experiments, as an automated marmoset identity extraction and classification tool for real-time video monitoring. Once fully trained on recognizing subject marmosets, this pipeline operates automatically and can be applied across individuals of different ages, with minimal manual work needed. While existing marmoset identification approaches usually utilize visible markers, radio frequency identification (RFID), or observation, the manual work and human interventions involved can impact animal behaviors, especially during their behavioral task performance. The facial identification tool aims to collect data from marmosets without having experimenters check the identity continuously, instead of outperforming the experimenters’ role. Its automated pipeline substantially reduces the time and work required for traditional manual identity labeling, while maintaining an expert-level human performance and reproducibility across experimenters (95.83% average accuracy for animal health technicians, responsible for daily health checks and husbandry, while lab experiments range between 25% and 80% of accuracy depending on the amount of time spent with each animal, Figure 8—figure supplement 1). The tool’s advantages are particularly efficient for large datasets and longitudinal studies, where manual identity labeling becomes difficult, as variability and errors increase along with dataset size and experimenter number.

The motivation for this real-time marmoset identity recognition program was to develop an easy-to-use, generalizable pipeline that could be applied across different marmosets and lab environments, such as using larger fields of view or phone cameras (Figure 8). The pipeline was designed to have no specific hardware requirements and can be implemented for any standard recording device, including any commonly available cameras, primate chair systems, and computer-based devices. Based on the collected individual marmoset face data, the automatic extraction program allows consistent identity annotation of marmoset faces and collar beads, ensuring accurate and stable identity interpretation during the marmoset face classification model training. The experimenter only needs to run the Python pipeline scripts to perform the real-time marmoset identification during experiments, without the need to manually label large marmoset identity datasets. Moreover, the system operates directly based on the raw video input from the camera, with no preprocessing such as video cropping or resolution modification required. The identity recognition system recognizes each marmoset based on pre-defined labels from collected videos of individuals, without other external tools such as RFID systems (Pereira et al., 2023). We focused the marmoset identification on individual-specific facial features and uniquely colored collar beads to ensure the pipeline’s robustness across various recording conditions, regardless of lighting conditions, camera angles, or animal postures. Together, this pipeline design and setup enable fast, reliable, and accurate identity recognition for efficient real-time monitoring of multiple animals in complex experimental environments. However, with the high visual similarity between closely related marmoset family members, the facial features and the collar beads must be carefully integrated into our pipeline design, as even experienced human observers can misidentify marmoset twins.

Figure 8 with 1 supplement see all

Download asset Open asset

Examples of marmoset face detection and classifiers for in-cage and open-field marmoset images.

This image illustrated the (**A, B, D, E**) detection of marmosets’ faces and (C) their identities across different cameras, environments, and marmoset behaviors. Additionally, it included scenarios where animals were housed in-cage (**A–C**) or moving freely (**D–E**). The open-field marmoset videos (**D–E**) were obtained from the YouTube Billabong Zoo, Koala, and Wildlife Park: https://www.youtube.com/watch?v=lDJNk4rjqqQ.

Because of the high extent of natural facial similarity between related marmosets, our real-time pipeline can experience challenges in identifying closely related individuals, as the results in Figures 5—7 and Figure 5—videos 1–4 indicate. Fur textures, coloring patterns, and genetic relatedness contribute to the morphological similarities, making it challenging for human observers when identifying individual primates (Alvergne et al., 2009; Guan et al., 2023; Leopold and Rhodes, 2010). Facial features serve as the main and intrinsic biometric identifier for each marmoset, providing a unique source of individual recognition. Since collar-based confirmation could be affected by visibility limitations, we implemented the uniquely color-coded bead collars as an auxiliary cue to provide additional confirmation in identity prediction. For example, this issue can be caused by identical or similar bead colors between individuals (Figure 5—videos 3 and 4) and beads that are occluded by fur (Figure 5—videos 1–4). In addition, collar beads may change over time or not be worn by all animals. An alternate experimental solution is to improve collar visibility, including using distinct color codes across individuals within a family, increasing the size of the beads, or increasing the number of collar beads to reduce occlusion. Moreover, the performance of the system is strongly dependent on the amount and variability of the training data, with identity classification improving as more marmoset images are involved in the model training. This relation is specifically important when characterizing between young and adult marmosets, as facial features may change across marmosets’ developmental stages. Images from both the juvenile and adult periods of the same marmoset can be included in the model training, which could improve model performance and generalizability across developmental stages, and avoid repeated training of the individual recognition model at different ages.

Because our pipeline was designed specifically for marmoset identity recognition, it was optimized for characterizing individuals within a defined housing unit, which usually represents a family or pair group. With one separated model trained per family unit, our system can utilize distinct collar colors as an additional identifier when available, while facial features performed as the main biometric marker. Even though multiple marmosets with visually similar faces may present close to the camera, the additional collar information can improve confidence in identity prediction without replacing facial recognition as the primary mechanism of identification (Figure 7—video 1). Moreover, while changes in the recording environment setup (e.g. lighting conditions, camera angles, etc.) do not affect the model performance, the model needs to be adjusted when family members or composition changes. In particular, the introduction of new members (e.g. newborns) requires assignment of new colored collar beads and collection of face images; thus, it is necessary that the recognition model of this marmoset family is retrained.

This system may be particularly useful when experimenters need to know which individual is performing a specific task for cognitive and behavioral experiments (Kangas et al., 2016; Kangas and Bergman, 2017; Marshall and Ridley, 2003). In these experiments, especially those involving long-term or continuous behavioral responses, experimenters often need to be present to record animal identity or to manually review video recordings after the experiment. This may distract the animals from their task and affect their behaviors, while also demanding sustained attention and a large time commitment for the experimenters. Our pipeline resolves this limitation by automatically detecting and recording marmoset identity throughout the experiment. A typical system requires the experimenter to only collect video clips from each marmoset, verify the automatically annotated marmoset faces and collar beads, and initiate the training of a family-specific face classification. This process takes approximately 20–30 hr of time. Once trained, the system operates automatically to collect real-time identity and can work to present subject-specific behavioral or cognitive tasks based on the identity of the detected animal, with no work or presence needed on the user’s end. This tool has already been implemented in touchscreen-based marmoset cognitive tasks, including pairwise visual discrimination paradigm.

Future extensions can further improve the detection accuracy and the pipeline utility. First, the pipeline can be easily combined with additional programs to characterize the detailed facial features of the marmosets (Correia-Caeiro et al., 2022; Kawaguchi et al., 2023). Considering differences in their eye distance, mouth shape, and fur coloring pattern, in addition to the global facial structure, we can improve the identity prediction, particularly between two closely related marmosets with similar faces. Also, aside from the colored collar beads, experimenters can label marmosets’ identities using other visual markers, including color dyes on marmoset ear tufts. Using a more evident visual marker could be helpful for accurate identity prediction and avoid mislabeling. Moreover, it would be possible to integrate pose estimation tools, such as DeepLabCut and MarmoPose (Cheng et al., 2025; Lauer et al., 2022), with this marmoset recognition pipeline, thus allowing the estimation of the postures and behaviors of specific animals of interest. While we developed this pipeline and demonstrate its utility for common marmosets in laboratory captivity, there is no application restriction of this system. With appropriate training data and experimental design, our pipeline can be applied to other non-human primates in various settings such as lab housing, conservation fields, or even in the wild.

Materials and methods

Animals

Three adult common marmosets (C. jacchus, 1 father, 1 mother, 1 son, 6 years weighted at 464 g, 5 years at 566 g, and 3 years at 450 g, respectively) and two young common marmosets (two female twins, average weights 329 g and 338 g, aged 7–11 months) were involved in this study. The three adult marmosets belonged to the same family unit (parents: Adult1, Adult2, and their adult offspring: Adult3), while the young marmosets were twin siblings (Young1 and Young2) from another family. The marmosets are housed at The Neuro’s animal facility in family units in indoor enclosures. Housing cages included two sizes. The first size housed three adult marmosets with dimensions of 1.372 m length × 0.760 m width × 2.092 m height, and the second cage that housed the two young marmosets with their family has the dimensions of 1.065 m length × 1.067 m width × 2.092 m height. We placed a collar with a uniquely colored bead on each marmoset to allow consistent visual identification during the experiment. All experimental procedures were conducted under the Canadian Council on Animal Care guidelines, Standard Operating Procedures of marmosets, and Animal Use Protocol (AUP# 10000 and 10001) approved by The Neuro and McGill’s Animal Care Committee.

Video collection

Request a detailed protocol

Marmoset face dataset was collected by mounting a primate testing chair with an added camera component to the door of marmoset housing cages (Figure 9A and C). High-resolution videos were acquired using an industrial color/RGB camera (JIERUIWEITONG DF200-1080P). The industrial camera was selected specifically for its small size (camera box: 36 mm width × 36 mm height) and capability of close-distance recordings. High-resolution recordings (1920×1080 pixels, 30 frames per second) obtained with the small focal length lens (2.8 mm) enabled a wide field of camera view. This allowed the collection of full facial features and posture variability at close recording distances. The camera was adjusted for focus and placed approximately 10.50 cm from the housing cage door (Figure 9A and B). The camera was fixed in position using a transparent protective case, which was designed to prevent damage to the camera by the marmosets (Figure 9B).

Figure 9

Download asset Open asset

Experimental setup of the camera system and primate chair for face detection and identification.

(A) Side view of the camera (green box) and primate chair, attached to the housing cage. (B) Frontal view of the camera fixed on the primate chair, enclosed within a protective cover. (C) A schematic illustration of the real-time facial image recording and automatic identification of the marmoset entering the primate chair.

Prior to the experiments, marmosets were acclimated to entering the primate testing chair space. This space minimized the chair surface reflection, reducing interference between true facial features and reflections during face detection and identification. Three adult marmosets from one family were recorded for 1 hr across each of the 5 recording days, with unrestricted voluntary access to the primate chair space. For the adult marmosets, the housing cage door was opened at the beginning of the recording session, allowing them to enter and exit freely into the primate chair space for food rewards and observation (Figure 9C). Two young marmosets were briefly isolated and recorded separately for testing and improving the automatic face extraction program. We recorded them at two developmental time points, 7 months of age and 11 months of age (an additional time point at 16 months of age has been added for one marmoset to test the identification without collar). During video collection, a sliding panel and an in-cage box were positioned near the housing cage door to temporarily isolate individual marmosets from other family members. Individual isolation was kept brief (approximately 10 min) to prevent disturbance and potential stress due to family separation. Marmosets accessed the primate chair space through the housing cage door (11 cm width × 10.16 cm height).

Video preprocessing and bounding box annotation

Request a detailed protocol

The collected videos were preprocessed to identify which video clips included valid face and collar information of the marmoset. We defined valid face and collar features by whether the full-face details and colored bead appeared visible from the clips (Figure 10A, Step 1). To capture sufficient variability in postures, individuals, and lighting conditions, clips were sampled throughout the adult marmoset videos (approximately 5 hr) (Figure 10B). We used OpenCV to extract frames from the selected clips, and the extracted frames were reviewed to exclude those that were blurry (Bradski, 2000). Computer Vision Annotation Tool (CVAT) was applied to perform bounding box annotation of marmoset faces and collar colors from the extracted frames (Sekachev et al., 2020).

Figure 10

Download asset Open asset

Workflow and design of the marmoset facial detection and identification model.

(A) The architecture of the real-time marmoset facial recognition program. (B) Marmoset face images from three camera angles. (C) Bounding boxes of marmoset faces (green box) and collars (pink box) were manually labeled in the training and validation datasets to train the multi-marmoset face classification model. (D) The schematic of the automatic face (blue box) and collar bead (cyan box) extraction model.

Marmoset face and identity dataset

Request a detailed protocol

To minimize image computations and data storage, we created a dataset of 2498 annotated images from the three adult marmosets. All images were manually annotated to label marmoset faces, individual identities, and the collar bead colors (Figure 10C). The annotated images were used for training models of multi-marmoset face classification and the automatic identity extraction, which can automatically detect, localize, and identify marmoset faces (Figure 10A, Steps 1–4). We created another dataset of two young marmosets at 7 months of age (total images = 502) for testing the automatic facial and identity extraction (Figure 10A, Steps 4–5). For both adult and young marmoset datasets, images were randomly divided into a training set and a validation set at a ratio of 8:2. Moreover, we evaluated the model performance and its generalization across developmental stages using new videos: (1) from the adult marmosets and (2) from the same young marmosets at 11 months, which were not involved during initial program training.

Marmoset face recognition pipeline

Request a detailed protocol

The marmoset facial detection and identification was primarily based on the YOLOv8 algorithm (Jocher et al., 2023; Varghese and Sambath, 2024). The architecture of the program consists of two main models: a multi-marmoset face classifier and an automatic facial and identity extraction (Figure 10A). In the initial step, we deployed multiple pre-trained YOLOv8 models (YOLOv8 nano, YOLOv8 small, and YOLOv8 medium) on the adult marmosets’ dataset (total images = 2498) to train the multi-marmoset face classifier (Lin et al., 2014). Training performance and detection accuracy of these models, generated by YOLO metrics as a CSV file, were evaluated to select the optimal model for subsequent training (see Results section). These pre-trained models share the same object detection backbone but differ in the number of parameters and computing power. Larger models, such as YOLOv8 medium, provide higher detection accuracy but require greater computational resources and longer inference time. In contrast, smaller models prioritize computational efficiency. The YOLOv8 nano model was selected and used for the training of all models in the program (Figure 10C). The adult marmosets’ dataset was also applied in the training of automatic facial and identity extraction. This trained automatic face extraction was utilized on the young marmoset dataset (total images = 502). Since young marmosets were recorded individually, we assigned the automatic annotation of marmoset faces and collar beads based on their recorded identity (Figure 10D). All automatic labeling was manually reviewed to remove mislabeled annotations. We used this dataset (total images = 449) to train the multi-marmoset face classifier for the young marmosets.

The marmoset facial recognition program was trained in the Anaconda virtual environment. The model training terminated at the early stopping criteria when no improvement was observed across 100 most recent epochs (i.e. training iterations). This tool involved training of three YOLOv8-based programs: (1) multi-marmoset face classifier for three adult marmosets was trained for 183 epochs; (2) automatic facial and identity extraction was trained for 124 epochs; (3) multi-marmoset face classifier for two young marmosets was trained for 245 epochs. The final models generated 2D bounding boxes, assigning identity and collar information of individual marmosets. The detection results of input videos were exported as text files, which represented the corresponding label identity, the bounding box size, and its location (Jocher et al., 2023; Redmon et al., 2015).

Model evaluation and analysis

The marmoset facial recognition program was evaluated using a range of performance evaluation metrics. The evaluation metrics included: precision, recall, mAP, validation DFL, and training time. We also computed the F1 score to select the best setup to be applied in the final real-time detection pipeline. In addition, we calculated the inter-individual face similarity between marmosets to test whether the biological face similarity between family members can explain the potential mislabeling of the face classifier programs.

Intersection over union

Request a detailed protocol

IoU calculates the amount of spatial overlapping between a predicted bounding box and the ground truth (i.e. manually labeled bounding box) (Jocher et al., 2023; Rezatofighi et al., 2019). It measures the localization accuracy and error amount between the predicted annotation and the manually labeled ground truth. A detection was considered correct if the IoU reached specific thresholds. In this study, we reported the mAP at the IoU thresholds from 0.5 to 0.95 (increments = 0.05). The IoU is calculated by the area of overlap ( $A \cap B$ ) and area of union ( $A \cup B$ ) of the predicted bounding box (A) and ground truth bounding box (B). If IoU = 1, the predicted bounding box is perfectly aligned with the true annotation. If IoU =0, there is no overlap between the two boxes.

I o U = \frac{A \cap B}{A \cup B}

Precision, recall, and F1 score

Request a detailed protocol

Precision, recall, and F1 score are commonly used in the evaluation of classification models (Dehmer and Basak, 2012; Powers, 2008). Precision measures the overall proportion of correct face identifications over all positive results, indicating the accuracy of the detection model. Recall quantifies the proportion of the face identification that are correct compared with all actual positives, which reflects the ability of the model to predict correct marmoset face and collar beads. The F1 score is the harmonic mean of precision and recall, considering the false positives and false negatives in the model assessment.

P r e c i s i o n = \frac{T r u e p o s i t i v e}{T r u e p o s i t i v e + F a l s e p o s i t i v e}

R e c a l l = \frac{T r u e p o s i t i v e}{T r u e p o s i t i v e + F a l s e n e g a t i v e}

F 1 s c o r e = 2 \times \frac{P r e c i s i o n \times R e c a l l}{P r e c i s i o n + R e c a l l}

Mean average precision

Request a detailed protocol

mAP assesses how accurate the model detects objects and how well the model localizes the object in the image. In object detection algorithms, precision is the number of correctly detected objects, with recall being the number of true objects that are successfully detected. As the precision and recall are affected by different detection confidence thresholds, the performance is evaluated by the precision-recall curve, which plots the model precision (y-axis) against recall (x-axis) for a particular threshold. The average precision (AP) is the area under the precision-recall curve (Padilla et al., 2021). Before calculating the AP, the precision-recall pairs are interpolated to create a monotonically non-increasing precision-recall function, meaning that the interpolated precision ( $P_{i n t e r p o l a t i o n}$ ) is the maximum precision ( ${m a x}_{j \geq i} P r e c i s i o n_{j}$ ) at the recall level larger than or equal to Recall_i. This interpolation assigns a highest precision value at each recall level, which smooths the precision-recall curve and reduces the impact of measurement fluctuations and noises to the evaluation of model performance.

P_{i n t e r p o l a t i o n} ({R e c a l l}_{i}) = {m a x}_{j \geq i} P r e c i s i o n_{j}

For each label class in the object detection model, the AP is the area under the curve of the interpolated precision-recall curve, calculated by summing the interpolated precision weighted by the incremental increase in the recall (Maxwell et al., 2021; Padilla et al., 2021).

A P = \sum_{i = 1}^{N} ({R e c a l l}_{i} - {R e c a l l}_{i - 1}) P_{i n t e r p o l a t i o n} ({R e c a l l}_{i})

N = t o t a l n u m b e r o f p o i n t s o n t h e p r e c i s i o n - r e c a l l c u r v e

The AP is extended further to calculate the mAP, which is the averaged AP values of different label classes in the object detection model (total class number = n). The AP_i is the mean AP calculated per label class at index i at multiple IoU levels (IoU = {0.50, 0.55, …, 0.95}). This calculation generates a more comprehensive evaluation with multiple label classes and their predicted localization accuracy of our marmoset facial recognition program.

m A P @ 50 - 95 = \frac{1}{n} \sum_{i = 1}^{n} {A P}_{i}

Validation DFL and training time

Request a detailed protocol

The DFL and training time were also used to support the selection of the final marmoset facial recognition model. The DFL function measures the model’s performance to refine the bounding box predictions, based on the localization uncertainty of the bounding box annotations (Jocher et al., 2023). The predicted bounding box position is compared with the ground truth annotations, with a lower DFL value indicating a more precise bounding box prediction.

Training time was used the training time to assess the efficiency and the feasibility of the real-time marmoset face recognition program. The training time estimated the computational cost of the program, which provided additional information to select models with similar accuracy. The training time is calculated by the total time duration to train the model. For the number of training epochs, N_epoch, the total training time is calculated through multiplying N_epoch by the training time per epoch (t_epoch):

T r a i n i n g t i m e = N_{e p o c h} \times t_{e p o c h}

Inter-individual face similarity analysis

Request a detailed protocol

We calculated the inter-individual face similarity using customized Python scripts (see Data availability). The image feature embeddings, based on YOLOv8 algorithms, calculate the numerical representations (i.e. vectors) using a computer vision model, which encodes the semantic and visual maps of the image (Long Chai et al., 2023; Varghese and Sambath, 2024). In our marmoset facial recognition program, the intermediate layers included extensive feature maps of the visual details of the marmoset face images. By extracting and analyzing the image feature embeddings, we obtained various numerical representations of the marmoset faces, which allowed us to compare face shapes, distances between face features (eyes, mouth, nose, etc.), and the fur patterns of different individuals. We assessed the inter-individual face similarities for two family groups: (1) within the adult marmoset family (parents and adult offspring) and (2) between the two young marmosets (twin siblings). For each individual marmoset, we selected 10 images with similar head orientation, lighting conditions, and camera angles, and cropped them to only include the face regions. As the marmoset face images passed through the YOLO convolutional layers, the images were outputted into a feature map (Aghdam and Heravi, 2017; Dumoulin and Visin, 2016; Long Chai et al., 2023; Redmon et al., 2015). For one marmoset face image, the feature map (F_i) is encoded in the number of inputs (n_input) and the feature map shapes (Aghdam and Heravi, 2017; Dumoulin and Visin, 2016).

F_{i} (h, w, c) = n_{i n p u t} \times h e i g h t \times w i d t h \times c h a n n e l

We extracted face embeddings from the face classifiers of the adult marmosets’ family (parents and son) and young twin marmosets from another family. To generate a single vector (i.e. embedding, e_i) representing the marmoset face feature per channel, we averaged across the spatial dimension (i.e. height and width of the image) per channel for one marmoset face image (Lee et al., 2025). The extracted embedding (e_i) was averaged across the 10 images for each individual marmoset, which allowed us to have an averaged embedding vector (e_marmoset) and feature maps to calculate the inter-individual face similarities. We performed L2 normalization on this embedding (e_marmoset) to create a unit vector (e_norm) for inter-individual comparison, using scikit-learn preprocessing function (Pedregosa et al., 2011).

e_{i} = \frac{1}{h e i g h t} \frac{1}{w i d t h} \sum_{h = 1}^{h e i g h t} \sum_{w = 1}^{w i d r t h} F_{i} (h, w, c)

e_{m a r m o s e t} = \frac{1}{10} \sum_{i = 1}^{10} e_{i}

e_{n o r m} = \frac{e_{m a r m o s e t}}{‖ e_{m a r m o s e t} ‖}, w h e r e ‖ e_{m a r m o s e t} ‖ = \sqrt{\sum e_{m a r m o s e t}^{2}}

As such, with the normalized embedding vectors from the marmosets in this study, we calculated the inter-individual face similarities between marmoset pairs using cosine similarity (Li and Han, 2013; Nguyen and Bai, 2011; Thongtan and Phienthrakul, 2019). Moreover, we also calculated the Euclidean distances between extracted embeddings from two marmosets (Jozwik et al., 2022; Sugase-Miyamoto et al., 2014). The calculation equations were described using the normalized embedding vectors of a marmoset pair (e_{norm – 1} and e_{norm – 2}) in scikit-learn library (Merchant et al., 2023; Pedregosa et al., 2011):

C o s i n e s i m i l a r i t y = \frac{e_{n o r m - 1} \cdot e_{n o r m - 2}}{| | e_{n o r m - 1} | | | | e_{n o r m - 2} | |}

E u c l i d e a n d i s t a n c e = \sqrt{\sum_{i = 1}^{n} {(e_{n o r m - 1} - e_{n o r m - 2})}^{2}}

We plotted a violin plot of cosine similarity and Euclidean distance to interpret the inter-individual facial similarities calculated from the marmoset pairs’ face embeddings. In addition, the similarity values were standardized using within-model z-scores.

Statistical tests were performed only within the face classifier of each marmoset family, as embedding spaces may vary in scaling, learned features, and baseline metrics making cross-model comparison of inter-individual face similarity unreliable (Bollegala, 2017). To test whether the learned facial embeddings between different marmosets are related to the biological facial feature similarities, we performed the pairwise Welch’s t-test and the Cohen’s d to examine the significance and effect size. Within each family unit, facial similarity was compared between two types of family relationships (e.g. mother-son vs. father-son).

Real-time face recognition program

Request a detailed protocol

We performed the real-time marmoset face detection and identification using the customized Python scripts, based on the trained weights from the final multi-marmoset facial classification model in YOLOv8. Live videos were collected using the color (RGB) camera and simultaneously processed frame-by-frame to detect the marmoset identities. For each detected bounding box, the scripts returned a corresponding label of marmoset face and collar bead color. We assigned the detected collar beads as the corresponding marmoset identity with a higher weight, which improved the detection confidence across frames.

To ensure the program accuracy and efficiency, we temporally stored the detection results of the most recent 30 frames (approximately 1 s). The most frequently detected marmoset identity within this time window was exported as the current output. We wrote and updated the detection results continuously into an output JSON file, enabling real-time identity reading of the marmoset who entered the primate chair most recently.

Appendix 1

Appendix 1—table 1

Model performance evaluation of the validation dataset in the adult marmoset recognition model.

Comparison of precision, recall, mAP@50, and mAP@50–95 were computed for all the detected label classes in this model.

Label classes	Precision	Recall	mAP50	mAP@50–95
All	0.936	0.955	0.980	0.713
Adult1	0.829	0.945	0.957	0.841
Adult2	0.926	0.983	0.991	0.787
Adult3	0.951	0.998	0.992	0.868
collar_Adult1	0.968	0.966	0.988	0.603
collar_Adult2	0.978	0.972	0.990	0.665
collar_Adult3	0.962	0.866	0.959	0.515

Appendix 1—table 2

Model performance evaluation of the validation dataset in the automatic face and identity extraction model.

Comparison of precision, recall, mAP@50, and mAP@50–95 were computed for all the detected label classes in this model.

Label classes	Precision	Recall	mAP50	mAP@50–95
All	0.938	0.968	0.984	0.715
Face	0.919	0.986	0.990	0.831
Collar	0.957	0.950	0.978	0.600

Appendix 1—table 3

Model performance evaluation of the validation dataset in the young marmoset recognition model.

Comparison of precision, recall, mAP@50, and mAP@50–95 were computed for all the detected label classes in this model.

Label classes	Precision	Recall	mAP50	mAP@50–95
All	0.979	0.975	0.994	0.890
Young1	1.000	0.935	0.993	0.950
collar_Young1	0.998	1.000	0.995	0.808
Young2	0.958	1.000	0.995	0.947
collar_Young2	0.963	0.964	0.993	0.855

Data availability

All code in this paper is publicly available at GitHub: https://github.com/Jy-Yang-bot/real-time-marmoset-recognition (copy archived at Yang, 2026). This repository contains the scripts for pre-processing the images, the main facial recognition pipeline, and the analytic tools. The program generalizability was tested on additional video obtained from publicly available online sources, including a YouTube video "Common Marmoset", uploaded by Billabong Zoo, Koala & Wildlife Park on 2 June 2020 (https://www.youtube.com/watch?v=lDJNk4rjqqQ). As per McGill policy on sharing and use of images of animals involved in Research, Teaching and Testing, we cannot upload the raw videos recorded within our animal facilities into a domain-specific public archive or generic repositories. However, those raw videos can be made available upon request by reaching out to the corresponding authors (Jiayue Yang and Justine Cléry).

References

Book
1. Aghdam HH
2. Heravi EJ
(2017) Guide to Convolutional Neural Networks: A Practical Application to Traffic-Sign Detection and Classification
SpringerLink Bücher.

https://doi.org/10.1007/978-3-319-57550-6
- Google Scholar
1. Alvergne A
2. Huchard E
3. Caillaud D
4. Charpentier MJE
5. Setchell JM
6. Ruppli C
7. Féjan D
8. Martinez L
9. Cowlishaw G
10. Raymond M
(2009) Human ability to recognize kin visually within primates
International Journal of Primatology 30:199–210.

https://doi.org/10.1007/s10764-009-9339-0
- PubMed
- Google Scholar
(2024) WildARe-YOLO: A lightweight and efficient wild animal recognition model
Ecological Informatics 80:102541.

https://doi.org/10.1016/j.ecoinf.2024.102541
- Google Scholar
(2024) Biometric identification of dairy cows via real-time facial recognition
Animal 18:101079.

https://doi.org/10.1016/j.animal.2024.101079
- PubMed
- Google Scholar
1. Bollegala D
(2017) Dynamic feature scaling for online learning of binary classifiers
Knowledge-Based Systems 129:97–105.

https://doi.org/10.1016/j.knosys.2017.05.010
- Google Scholar
1. Bradski G
(2000)
The opencv library

Dr. Dobb’s Journal of Software Tools 25:120–125.
- Google Scholar
(2003) True paternal care in a multi-male primate society
Nature 425:179–181.

https://doi.org/10.1038/nature01866
- PubMed
- Google Scholar
(2023) Persistent animal identification leveraging non-visual markers
Machine Vision and Applications 34:68.

https://doi.org/10.1007/s00138-023-01414-1
- PubMed
- Google Scholar
1. Carstens E
2. Moberg GP
(2000) Recognizing pain and distress in laboratory animals
ILAR Journal 41:62–71.

https://doi.org/10.1093/ilar.41.2.62
- PubMed
- Google Scholar
1. Cheng C
2. Huang Z
3. Zhang R
4. Huang G
5. Wang H
6. Tang L
7. Wang X
(2025) A real-time, multi-subject three-dimensional pose tracking system for the behavioral analysis of non-human primates
Cell Reports Methods 5:100986.

https://doi.org/10.1016/j.crmeth.2025.100986
- PubMed
- Google Scholar
(2022) CalliFACS: the common marmoset facial action coding system
PLOS ONE 17:e0266442.

https://doi.org/10.1371/journal.pone.0266442
- PubMed
- Google Scholar
1. Dave B
2. Mori M
3. Bathani A
4. Goel P
(2023) Wild animal detection using YOLOv8
Procedia Computer Science 230:100–111.

https://doi.org/10.1016/j.procs.2023.12.065
- Google Scholar
Book
1. Dehmer M
2. Basak SC
(2012) Statistical and Machine Learning Approaches for Network Analysis
Wiley.

https://doi.org/10.1002/9781118346990
- Google Scholar
Preprint
1. Dumoulin V
2. Visin F
(2016) A guide to convolution arithmetic for deep learning
arXiv.

https://doi.org/10.48550/ARXIV.1603.07285
- Google Scholar
1. Gill H
2. Charles J
3. Grant R
4. Gardiner J
5. Bates K
6. Brassey C
(2025) Gait tracking in dogs using DeepLabCut: A markerless machine learning approach for controlled settings
Applied Animal Behaviour Science 292:106769.

https://doi.org/10.1016/j.applanim.2025.106769
- Google Scholar
1. Guan Y
2. Lei Y
3. Zhu Y
4. Li T
5. Xiang Y
6. Dong P
7. Jiang R
8. Luo J
9. Huang A
10. Fan Y
11. Teng P
12. Zhang Y
13. Zeng C
14. Xie M
15. Mu J
16. Ni Q
(2023) Face recognition of a Lorisidae species based on computer vision
Global Ecology and Conservation 45:e02511.

https://doi.org/10.1016/j.gecco.2023.e02511
- Google Scholar
Preprint
1. He K
2. Zhang X
3. Ren S
4. Sun J
(2015) Deep residual learning for image recognition
arXiv.

https://doi.org/10.48550/ARXIV.1512.03385
- Google Scholar
Software
(2023) Ultralytics yolov8, version 8.0.0
Ultralytics.

https://github.com/ultralytics/ultralytics
1. Jozwik KM
2. O’Keeffe J
3. Storrs KR
4. Guo W
5. Golan T
6. Kriegeskorte N
(2022) Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models
PNAS 119:e2115047119.

https://doi.org/10.1073/pnas.2115047119
- PubMed
- Google Scholar
(2016) Touchscreen assays of learning, response inhibition, and motivation in the marmoset (Callithrix jacchus)
Animal Cognition 19:673–677.

https://doi.org/10.1007/s10071-016-0959-4
- PubMed
- Google Scholar
1. Kangas BD
2. Bergman J
(2017) Touchscreen technology in the study of cognition-related behavior
Behavioural Pharmacology 28:623–629.

https://doi.org/10.1097/FBP.0000000000000356
- PubMed
- Google Scholar
(2023) Revisiting the baby schema by a geometric morphometric analysis of infant facial characteristics across great apes
Scientific Reports 13:5129.

https://doi.org/10.1038/s41598-023-31731-4
- PubMed
- Google Scholar
1. Kishi N
2. Sato K
3. Sasaki E
4. Okano H
(2014) Common marmoset as a new model animal for neuroscience research and genome editing technology
Development, Growth & Differentiation 56:53–62.

https://doi.org/10.1111/dgd.12109
- PubMed
- Google Scholar
1. Koski SE
2. Burkart JM
(2015) Common marmosets show social plasticity and group-level similarity in personality
Scientific Reports 5:8878.

https://doi.org/10.1038/srep08878
- PubMed
- Google Scholar
1. Lauer J
2. Zhou M
3. Ye S
4. Menegas W
5. Schneider S
6. Nath T
7. Rahman MM
8. Di Santo V
9. Soberanes D
10. Feng G
11. Murthy VN
12. Lauder G
13. Dulac C
14. Mathis MW
15. Mathis A
(2022) Multi-animal pose estimation, identification and tracking with DeepLabCut
Nature Methods 19:496–504.

https://doi.org/10.1038/s41592-022-01443-0
- PubMed
- Google Scholar
Preprint
1. Lee W
2. Ham B
3. Kim S
(2025) Maximizing the position embedding for vision transformers with global average pooling
arXiv.

https://doi.org/10.48550/ARXIV.2502.02919
- Google Scholar
1. Leopold DA
2. Rhodes G
(2010) A comparative view of face perception
Journal of Comparative Psychology 124:233–251.

https://doi.org/10.1037/a0019460
- PubMed
- Google Scholar
Book
1. Li B
2. Han L
(2013) Distance weighted cosine similarity measure for text classification
In: Yin H, Tang K, Gao Y, Klawonn F, Lee M, Weise T, Li Bin, Yao X, editors. Intelligent Data Engineering and Automated Learning – IDEAL 2013, Lecture Notes in Computer Science. Berlin Heidelberg: Springer. pp. 611–618.

https://doi.org/10.1007/978-3-642-41278-3_74
- Google Scholar
1. Lim MA
2. Defensor EB
3. Mechanic JA
4. Shah PP
5. Jaime EA
6. Roberts CR
7. Hutto DL
8. Schaevitz LR
(2019) Retrospective analysis of the effects of identification procedures and cage changing by using data from automated, continuous monitoring
Journal of the American Association for Laboratory Animal Science 58:126–141.

https://doi.org/10.30802/AALAS-JAALAS-18-000056
- PubMed
- Google Scholar
Book
1. Lin TY
2. Maire M
3. Belongie S
4. Bourdev L
5. Girshick R
6. Hays J
7. Perona P
8. Ramanan D
9. Zitnick CL
10. Dollár P
(2014) Microsoft COCO: common objects in context
In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer Vision – ECCV 2014. Springer. pp. 740–755.

https://doi.org/10.1007/978-3-319-10602-1_48
- Google Scholar
Conference
1. Long Chai JC
2. Ng TS
3. Low CY
4. Park J
5. Jin Teoh AB
(2023) Recognizability Embedding Enhancement for Very Low-Resolution Face Recognition and Quality Estimation
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9957–9967.

https://doi.org/10.1109/CVPR52729.2023.00960
- Google Scholar
1. Marion S
2. Davies A
3. Demšar U
4. Irvine RJ
5. Stephens PA
6. Long J
(2020) A systematic review of methods for studying the impacts of outdoor recreation on terrestrial wildlife
Global Ecology and Conservation 22:e00917.

https://doi.org/10.1016/j.gecco.2020.e00917
- Google Scholar
1. Marshall JWB
2. Ridley RM
(2003) Assessment of cognitive and motor deficits in a marmoset model of stroke
ILAR Journal 44:153–160.

https://doi.org/10.1093/ilar.44.2.153
- PubMed
- Google Scholar
1. Mathis A
2. Mamidanna P
3. Cury KM
4. Abe T
5. Murthy VN
6. Mathis MW
7. Bethge M
(2018) DeepLabCut: Markerless pose estimation of user-defined body parts with deep learning
Nature Neuroscience 21:1281–1289.

https://doi.org/10.1038/s41593-018-0209-y
- PubMed
- Google Scholar
(2021) Accuracy assessment in convolutional neural network-based deep learning remote sensing studies—Part 1: Literature Review
Remote Sensing 13:2450.

https://doi.org/10.3390/rs13132450
- Google Scholar
Book
(2023) Object measurement
In: Merchant FA, Castleman KR, editors. Microscope Image Processing. Elsevier. pp. 153–175.

https://doi.org/10.1016/B978-0-12-821049-9.00017-4
- Google Scholar
1. Miller CT
2. Freiwald WA
3. Leopold DA
4. Mitchell JF
5. Silva AC
6. Wang X
(2016) Marmosets: A neuroscientific model of human social behavior
Neuron 90:219–233.

https://doi.org/10.1016/j.neuron.2016.03.018
- PubMed
- Google Scholar
1. Nath T
2. Mathis A
3. Chen AC
4. Patel A
5. Bethge M
6. Mathis MW
(2019) Using DeepLabCut for 3D markerless pose estimation across species and behaviors
Nature Protocols 14:2152–2176.

https://doi.org/10.1038/s41596-019-0176-0
- PubMed
- Google Scholar
Book
1. National Research Council
(2011) Guide for the Care and Use of Laboratory Animals
National Academies Press.

https://doi.org/10.17226/12910
- Google Scholar
Book
1. Nguyen HV
2. Bai L
(2011) Cosine similarity metric learning for face verification
In: Kimmel R, Klette R, Sugimoto A, editors. Computer Vision – ACCV 2010, Lecture Notes in Computer Science. Springer. pp. 709–720.

https://doi.org/10.1007/978-3-642-19309-5_55
- Google Scholar
1. Norouzzadeh MS
2. Nguyen A
3. Kosmala M
4. Swanson A
5. Palmer MS
6. Packer C
7. Clune J
(2018) Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning
PNAS 115:E5716–E5725.

https://doi.org/10.1073/pnas.1719367115
- PubMed
- Google Scholar
1. Norouzzadeh MS
2. Morris D
3. Beery S
4. Joshi N
5. Jojic N
6. Clune J
(2021) A deep active learning system for species identification and counting in camera trap images
Methods in Ecology and Evolution 12:150–161.

https://doi.org/10.1111/2041-210X.13504
- Google Scholar
1. Okano H
(2021) Current status of and perspectives on the application of marmosets in neurobiology
Annual Review of Neuroscience 44:27–48.

https://doi.org/10.1146/annurev-neuro-030520-101844
- PubMed
- Google Scholar
(2021) A comparative analysis of object detection metrics with a companion open-source toolkit
Electronics 10:279.

https://doi.org/10.3390/electronics10030279
- Google Scholar
1. Pedregosa F
2. Varoquaux G
3. Gramfort A
4. Michel V
5. Thirion B
6. Grisel O
7. Blondel M
8. Prettenhofer P
9. Weiss R
10. Dubourg V
11. Vanderplas J
12. Passos A
13. Cournapeau D
14. Brucher M
15. Perrot M
16. Duchesnay E
(2011)
Scikit-learn: machine learning in python

Journal of Machine Learning Research 12:2825–2830.
- Google Scholar
1. Pereira E
2. Araújo Í
3. Silva LFV
4. Batista M
5. Júnior S
6. Barboza E
7. Santos E
8. Gomes F
9. Fraga IT
10. Davanso R
11. dos Santos D
12. de A Nascimento J
(2023) RFID technology for animal tracking: a survey
IEEE Journal of Radio Frequency Identification 7:609–620.

https://doi.org/10.1109/JRFID.2023.3334952
- Google Scholar
(2021) Automatic animal identification from drone camera based on point pattern analysis of herd behaviour
Ecological Informatics 66:101485.

https://doi.org/10.1016/j.ecoinf.2021.101485
- Google Scholar
1. Powers D
(2008) Evaluation: from precision, recall and F-Factor to ROC, informedness markedness & correlation
Journal of Machine Learning Technologies 2:37–63.

https://doi.org/10.9735/2229-3981
- Google Scholar
Preprint
(2015) You only look once: unified, real-time object detection
arXiv.

https://doi.org/10.48550/ARXIV.1506.02640
- Google Scholar
Preprint
1. Rezatofighi H
2. Tsoi N
3. Gwak J
4. Sadeghian A
5. Reid I
6. Savarese S
(2019) Generalized intersection over union: A metric and a loss for bounding box regression
arXiv.

https://doi.org/10.1109/CVPR.2019.00075
- Google Scholar
1. Roughan JV
2. Sevenoaks T
(2019) Welfare and scientific considerations of tattooing and ear tagging for mouse identification
Journal of the American Association for Laboratory Animal Science 58:142–153.

https://doi.org/10.30802/AALAS-JAALAS-18-000057
- PubMed
- Google Scholar
1. Sasaki E
2. Suemizu H
3. Shimada A
4. Hanazawa K
5. Oiwa R
6. Kamioka M
7. Tomioka I
8. Sotomaru Y
9. Hirakawa R
10. Eto T
11. Shiozawa S
12. Maeda T
13. Ito M
14. Ito R
15. Kito C
16. Yagihashi C
17. Kawai K
18. Miyoshi H
19. Tanioka Y
20. Tamaoki N
21. Habu S
22. Okano H
23. Nomura T
(2009) Generation of transgenic non-human primates with germline transmission
Nature 459:523–527.

https://doi.org/10.1038/nature08090
- PubMed
- Google Scholar
1. Schindler F
2. Steinhage V
(2021) Identification of animals and recognition of their actions in wildlife videos using deep learning techniques
Ecological Informatics 61:101215.

https://doi.org/10.1016/j.ecoinf.2021.101215
- Google Scholar
1. Schofield DP
2. Albery GF
3. Firth JA
4. Mielke A
5. Hayashi M
6. Matsuzawa T
7. Biro D
8. Carvalho S
(2023) Automated face recognition using deep neural networks produces robust primate social networks and sociality measures
Methods in Ecology and Evolution 14:1937–1951.

https://doi.org/10.1111/2041-210X.14181
- Google Scholar
Software
1. Sekachev B
2. Manovich N
3. Zhiltsov M
4. Zhavoronkov A
5. Kalinin D
6. Hoff B
7. TOsmanov K
8. Zankevich D
9. DmitriySidnev A
10. Markelov M
11. Johannes C
12. Telenachos M
13. Jijoong Kim A
14. Ilouz L
15. Glazov N
16. Priya T
17. Seungwon Jeong R
18. Skubriev V
19. Yonekura S
(2020) Opencv/cvat: v1.1.0, version v1.1.0
Zenodo.

https://doi.org/10.5281/zenodo.4009388
1. Sturman O
2. von Ziegler L
3. Schläppi C
4. Akyol F
5. Privitera M
6. Slominski D
7. Grimm C
8. Thieren L
9. Zerbi V
10. Grewe B
11. Bohacek J
(2020) Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions
Neuropsychopharmacology 45:1942–1952.

https://doi.org/10.1038/s41386-020-0776-y
- PubMed
- Google Scholar
(2014) Face inversion decreased information about facial identity and expression in face-responsive neurons in macaque area TE
The Journal of Neuroscience 34:12457–12469.

https://doi.org/10.1523/JNEUROSCI.0485-14.2014
- PubMed
- Google Scholar
Conference
1. Thongtan T
2. Phienthrakul T
(2019) Sentiment Classification Using Document Embeddings Trained with Cosine Similarity
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 407–414.

https://doi.org/10.18653/v1/P19-2057
- Google Scholar
Conference
1. Varghese R
2. Sambath M
(2024) YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness
2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS). pp. 1–6.

https://doi.org/10.1109/ADICS58448.2024.10533619
- Google Scholar
1. Vidal M
2. Wolf N
3. Rosenberg B
4. Harris BP
5. Mathis A
(2021) Perspectives on individual animal identification from biology and computer vision
Integrative and Comparative Biology 61:900–916.

https://doi.org/10.1093/icb/icab107
- PubMed
- Google Scholar
1. XiaoAn B
2. LinQing Z
3. XiaoMei T
4. Biao W
5. QingQi Z
6. YuTing J
7. Na Z
(2024) Wildlife target detection based on improved YOLOX-s network
Scientific Reports 14:23608.

https://doi.org/10.1038/s41598-024-73631-1
- PubMed
- Google Scholar
Software
1. Yang J
(2026) Real-time-marmoset-recognition, version swh:1:rev:465da2c9066eed5f575f0b5b175dd0e38c71f0d6
Software Heritage.

https://archive.softwareheritage.org/swh:1:dir:70ade0da3d3eec0606370e9c52ff34fd881b3cfc;origin=https://github.com/Jy-Yang-bot/real-time-marmoset-recognition;visit=swh:1:snp:6f254daa22f253760c06845f9fa660380a5b5d28;anchor=swh:1:rev:465da2c9066eed5f575f0b5b175dd0e38c71f0d6
(2018) Larger cages with housing unit environment enrichment improve the welfare of marmosets
Experimental Animals 67:31–39.

https://doi.org/10.1538/expanim.17-0058
- PubMed
- Google Scholar
Conference
1. Yu L
2. Chen X
3. Zhou S
(2018) Research of Image Main Objects Detection Algorithm Based on Deep Learning
2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). pp. 70–75.

https://doi.org/10.1109/ICIVC.2018.8492803
- Google Scholar
1. Zhuang Y
2. Liu J
3. Zhao H
4. Ma L
5. Fang Z
6. Li L
7. Wu C
8. Cui W
9. Liu Z
(2025) A deep learning framework based on structured space model for detecting small objects in complex underwater environments
Communications Engineering 4:24.

https://doi.org/10.1038/s44172-025-00367-9
- PubMed
- Google Scholar

Article and author information

Author details

Jiayue Yang
1. The Neuro, Department of Neurology and Neurosurgery, McGill University, Montreal, Canada
2. Integrated Program in Neuroscience, McGill University, Montreal, Canada
Contribution
Conceptualization, Resources, Data curation, Software, Formal analysis, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing, Data collection

For correspondence
jiayue.yang@mail.mcgill.ca

Competing interests
No competing interests declared

"This ORCID iD identifies the author of this article:" 0009-0003-3727-9586
James Wang
1. The Neuro, Department of Neurology and Neurosurgery, McGill University, Montreal, Canada
2. Integrated Program in Neuroscience, McGill University, Montreal, Canada
Contribution
Investigation, Methodology, Writing – review and editing, Data collection

Competing interests
No competing interests declared

"This ORCID iD identifies the author of this article:" 0009-0002-6656-7176
Justine Cléry
1. The Neuro, Department of Neurology and Neurosurgery, McGill University, Montreal, Canada
2. McConnell Brain Imaging Centre, The Neuro, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
3. Azrieli Centre for Autism Research, The Neuro, Montreal, Canada
Contribution
Conceptualization, Resources, Supervision, Funding acquisition, Validation, Investigation, Methodology, Project administration, Writing – review and editing

For correspondence
justine.clery@mcgill.ca

Competing interests
No competing interests declared

"This ORCID iD identifies the author of this article:" 0000-0003-1020-1845

Funding

New Frontiers in Research Fund (NFRFT-2022-00051)

Justine Cléry

Fonds de recherche du Québec

https://doi.org/10.69777/347426

Justine Cléry

Fonds de recherche du Québec (358082)

Justine Cléry

Fonds de recherche du Québec

https://doi.org/10.69777/2003934

Jiayue Yang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We would like to thank our animal health technicians D Hau-Aquino, V Comtois, C Hunt; and veterinarians F Chaurand, and J Hutta for animal health care and support. We are thankful to M Gacoin and T Cook for helpful discussions and feedback on the visualization and format of the manuscript. We thank J Smith and C O'Hare-Freire for the design and construction of a transparent protective case for the camera. We acknowledge the support of the Government of Canada’s New Frontiers in Research Fund (NFRF) (NFRFT-2022-00051) and by the Fonds de Recherche du Québec-Santé (FRQS) (#347426, #358082, and #2003934). Ces travaux ont bénéficié d’un octroi des fonds Nouvelles frontières en recherche du gouvernement du Canada (NFRFT-2022-00051) et du Fonds de recherche du Québec-Santé (FRQS, #347426, #358082, et #2003934).

Ethics

All experimental procedures were conducted under the Canadian Council on Animal Care guidelines, Standard Operating Procedures of marmosets, and Animal Use Protocol (AUP# 10000 and 10001) approved by The Neuro and McGill's Animal Care Committee.

Version history

Preprint posted: February 4, 2026
Sent for peer review: February 12, 2026
Reviewed Preprint version 1: April 1, 2026
Reviewed Preprint version 2: September 10, 2026
Version of Record published: September 29, 2026

Cite all versions

You can cite all versions using the DOI https://doi.org/10.7554/eLife.110932. This DOI represents all versions, and will always resolve to the latest one.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

566

views
34

downloads
0

citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Mendeley

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Jiayue Yang
James Wang
Justine Cléry

(2026)

A real-time, multi-animal model for automatic face detection and identification of freely moving common marmosets based on YOLOv8 algorithms

eLife 15:RP110932.

https://doi.org/10.7554/eLife.110932.3

Share this article

Cite this article

Performance comparison of multi-marmoset face classification models.

Training performance of the multi-marmoset face classification model for three adult marmosets, using the pre-trained YOLOv8 nano model.

Normalized confusion matrix per-class classification across the six label classes in the adult marmoset recognition model.

Training performance of the automatic facial and identity extraction model.

Normalized confusion matrix per-class classification across the two label classes in the automatic identity extraction model.

Training performance of the multi-marmoset face recognition model for young marmosets at 7 months.

Normalized confusion matrix per-class classification across the four label classes in the young marmoset recognition model.

Across-model visualization of the face similarity between marmoset pairs.

Statistical analysis of inter-individual face similarity (cosine similarity) between different family relationships, within the adult marmoset family.

Statistical analysis of inter-individual face similarity (Euclidean distance) between different family relationships, within the adult marmoset family.

Examples of marmoset face detection and classifiers for in-cage and open-field marmoset images.

Experimental setup of the camera system and primate chair for face detection and identification.

Workflow and design of the marmoset facial detection and identification model.

Model performance evaluation of the validation dataset in the adult marmoset recognition model.

Model performance evaluation of the validation dataset in the automatic face and identity extraction model.

Model performance evaluation of the validation dataset in the young marmoset recognition model.

Author details

Jiayue Yang

Contribution

For correspondence

Competing interests

James Wang

Contribution

Competing interests

Justine Cléry

Contribution

For correspondence

Competing interests

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Categories and tags