SA-FARI: Wildlife Tracking Benchmark

• SA-FARI is an open-source dataset and benchmark for multi-animal tracking, comprising 11,609 camera-trap videos collected over 10 years from diverse global sites.
• It features 16,224 masklet identities with 942,702 segmentation masks, derived via a multi-stage process combining automated pseudo-annotation and human review.
• The dataset supports tasks such as detection, segmentation, re-identification, and promptable classification, advancing wildlife conservation and ecological research.

Searching arXiv for the SA-FARI dataset paper to ground the article and citations. SA-FARI, short for Segment Anything in Footage of Animals for Recognition and Identification, is an open-source dataset and benchmark for multi-animal tracking (MAT) in camera-trap video (Wasmuht et al., 19 Nov 2025). It comprises 11,609 camera trap videos collected over approximately 10 years (2014–2024) from 741 locations across 4 continents, spanning 99 species categories. Each video is exhaustively annotated, yielding approximately 46 hours of densely annotated footage, 16,224 masklet identities, and 942,702 individual bounding boxes, segmentation masks, and species labels. The dataset is positioned as the largest open-source MAT dataset for wild animals, and as a benchmark for training general-purpose MAT models applicable across wild animal populations (Wasmuht et al., 19 Nov 2025).

1. Scope, composition, and sampling structure

SA-FARI contains 2,747 min $\simeq 45.78$ h of annotated camera-trap footage. The collection spans 741 independent sampling sites distributed across four continents / four major ecoregions: Central Africa, South America, Mesoamerica, and Southern Europe (Wasmuht et al., 19 Nov 2025). The species inventory comprises 99 common-name species categories, spanning Kingdom: Animalia $\rightarrow$ Phylum: Chordata $\rightarrow$ 4 classes, 23 orders, 53 families. The distribution is explicitly long-tailed: the dataset has an average $\simeq 117$ videos/species, 29 species account for 90 % of all videos, and several rare species appear exclusively in the test split (Wasmuht et al., 19 Nov 2025).

The train/test partition is also specified. The train split contains 10,776 videos, 2,545 min, 91 species, 15,141 masklets, 880,361 boxes + masks, 31,282 video–species pairs, and 650 sites. The test split contains 833 videos, 202 min, 83 species, 1,083 masklets, 62,341 boxes + masks, 2,322 video–species pairs, and 91 sites. Across both splits, the total remains 11,609 videos, 2,747 min, 99 species, 16,224 masklets, 942,702 boxes + masks, 33,604 video–species pairs, and 741 sites (Wasmuht et al., 19 Nov 2025).

The dataset is framed around the absence of a suitable benchmark for general-purpose MAT in wildlife footage. Existing datasets are described as being limited in scale, constrained to a few species, or lacking sufficient temporal and geographical diversity. SA-FARI is presented as a response to those deficiencies, with emphasis on high species diversity, multi-region coverage, and high-quality spatio-temporal annotations (Wasmuht et al., 19 Nov 2025).

2. Annotation protocol, masklets, and anonymization

The core annotation unit in SA-FARI is the masklet, defined as a spatio-temporal segment with per-frame masks. The dataset contains 16,224 unique identities at the masklet level and 942,702 segmentation masks. Bounding boxes are derived automatically from mask extents, giving the same total count as the masks. In addition, SA-FARI provides 33,604 video–species pairs (positive + hard/easy negatives) and 741 sampling locations (Wasmuht et al., 19 Nov 2025).

The annotation workflow is explicitly staged (Wasmuht et al., 19 Nov 2025):

1. Automatic pseudo-annotation at 6 fps with SAM 3 to generate initial masklets
2. Deduplication by mask–mask IoU to remove overlaps
3. Human review stage 1: remove unsegmentable or blurred masks
4. Human review stage 2: correct/add masks in-loop using SAM 2 promptable tools
5. Exhaustivity check: verify no animal was missed
6. Bounding boxes extracted from refined masks

This protocol places the dataset at the intersection of promptable segmentation and manual quality control. A plausible implication is that SA-FARI is intended not merely as a static benchmark, but as an example of a practical annotation pipeline for wildlife video at scale.

Anonymization is handled by dropping exact GPS information. Each site is represented by a random site-ID string plus a continent/ecoregion code, while date/time stamps are preserved when available (Wasmuht et al., 19 Nov 2025). This design preserves temporal and coarse regional context while restricting exact geospatial disclosure.

3. MAT task definition and evaluation methodology

SA-FARI supports four MAT sub-tasks (Wasmuht et al., 19 Nov 2025):

• Detection: localize animals via box/mask
• Segmentation: pixel-accurate outline per animal
• Re-identification: maintain consistent IDs across frames
• Promptable classification: species-specific vs. generic “animal” prompts

The benchmark uses standard tracking metrics. Multi-Object Tracking Accuracy (MOTA) is defined as

$$\mathrm{MOTA} = 1 \;-\; \frac{\sum_{t}\bigl(\mathrm{FP}_t + \mathrm{FN}_t + \mathrm{IDSW}_t\bigr)} {\sum_{t}\mathrm{GT}_t},$$

where $\mathrm{FP}_t$, $\mathrm{FN}_t$, and $\mathrm{IDSW}_t$ are false positives, false negatives, and ID switches at frame $t$, and $\mathrm{GT}_t$ is the number of ground-truth objects (Wasmuht et al., 19 Nov 2025).

ID F1-score (IDF1) is defined as

$\rightarrow$0

where IDTP/FP/FN count true positives, false positives, and false negatives under ID-aware matching (Wasmuht et al., 19 Nov 2025).

The evaluation suite also includes mean Average Precision (mAP) at IoU $\rightarrow$1, and the tracking metrics pHOTA and TETA. In the benchmark description, pHOTA and TETA are characterized as phrase-based and multi-category extensions of HOTA, and are reported together with their DetA and AssA components (Wasmuht et al., 19 Nov 2025). All metrics are computed per frame and aggregated over time, with species-specific metrics averaged across species categories.

4. Benchmarks, model families, and reported results

The benchmark evaluates two model families (Wasmuht et al., 19 Nov 2025). The first consists of vision–language promptable models: GLEE, LLMDet + independent tracker, and SAM 3 in three variants—SAM 3 (baseline), SAM 3 trained with SA-FARI data mix, and SAM 3 fine-tuned (FT) on SA-FARI. The second consists of vision-only species-agnostic pipelines: MegaDetector (MD) paired with ByteTrack, OCSort, or BoostSort++.

Under species-specific prompting, the reported numbers are (Wasmuht et al., 19 Nov 2025):

• GLEE: IDF1 $\rightarrow$2, pHOTA-Total 7.5, Det 1.2, Ass 49.7, TETA 22.0
• LLMDet + tracker: IDF1 2.6, pHOTA-Total 41.3, Det 21.4, Ass 80.0, TETA 30.4
• SAM 3 (baseline): IDF1 14.0, pHOTA-Total 48.5, Det 28.4, Ass 83.4, TETA 39.6
• SAM 3 + SA-FARI (mixed): IDF1 39.0, pHOTA-Total 63.1, Det 47.9, Ass 83.9, TETA 52.1
• SAM 3 FT (SA-FARI): IDF1 46.9, pHOTA-Total 68.1, Det 55.4, Ass 84.6, TETA 58.7

The accompanying interpretation in the benchmark is specific: fine-tuning on SA-FARI yields a three-fold jump in IDF1 and $\rightarrow$3 pHOTA points over the SAM 3 baseline (Wasmuht et al., 19 Nov 2025).

Under species-agnostic “animal” prompting, the reported comparison is again favorable to SA-FARI-trained SAM 3 (Wasmuht et al., 19 Nov 2025):

• MD + ByteTrack: IDF1 38.6, HOTA-Total 39.5, Det 20.8, Ass 75.6
• MD + OCSort: IDF1 33.9, HOTA-Total 45.5, Det 27.9, Ass 74.9
• MD + BoostSort++: IDF1 47.2, HOTA-Total 38.3, Det 18.4, Ass 80.8
• SAM 3 + SA-FARI (animal): IDF1 71.1, HOTA-Total 64.4, Det 50.0, Ass 83.7

The benchmark summarizes this as follows: even without retraining for the “animal” prompt, SAM 3 trained on SA-FARI outperforms all vision-only baselines by $\rightarrow$4 IDF1 and $\rightarrow$5 HOTA (Wasmuht et al., 19 Nov 2025).

A further test-subset analysis for SAM 3 FT on SA-FARI/test shows the stratification of difficulty (Wasmuht et al., 19 Nov 2025):

• Large masks: IDF1 63.4, pHOTA-Tot 81.4, Det 72.5, Ass 91.6, TETA 71.7
• Small masks: IDF1 25.3, pHOTA-Tot 52.2, Det 38.5, Ass 71.3, TETA 46.9
• Multiple animals: IDF1 45.9, pHOTA-Tot 67.8, Det 54.5, Ass 85.1, TETA 59.6
• Challenging: IDF1 36.8, pHOTA-Tot 61.7, Det 51.3, Ass 75.3, TETA 59.6
• Night: IDF1 44.1, pHOTA-Tot 66.0, Det 53.2, Ass 83.1, TETA 61.9
• All test: IDF1 46.9, pHOTA-Tot 68.1, Det 55.4, Ass 84.6, TETA 58.7

The benchmark notes that small and occluded masklets remain the hardest cases ($\rightarrow$6 pHOTA vs. large masks), while nighttime and multiple-animal clips produce moderate performance drops (Wasmuht et al., 19 Nov 2025).

5. Scientific role, applications, and current limitations

SA-FARI is positioned as a dataset for wildlife conservation and ecology. The applications named in the benchmark include automated population counts, occupancy modelling, abundance estimation, individual re-identification for long-term health and movement studies, and behavioural analysis (e.g. gait, interactions) when coupled with downstream classifiers (Wasmuht et al., 19 Nov 2025). Because the annotations are dense, spatio-temporal, and identity-aware, the resource is relevant not only for frame-level detection but also for longitudinal inference over individuals.

The benchmark also proposes several technical extensions: adding multi-modal channels such as audio, depth, and infrared; extending annotations to include keypoints (pose), detailed behavior labels, and natural-language scene captions; broadening geographic coverage and taxonomic breadth; and building semi-automated active-learning pipelines for continued expansion (Wasmuht et al., 19 Nov 2025). This suggests a roadmap in which SA-FARI functions as a seed benchmark for a broader, multimodal wildlife video ecosystem.

The limitations are also explicit (Wasmuht et al., 19 Nov 2025). The long-tail species distribution yields fewer examples for rare taxa. There is a lack of explicit behavior annotations beyond movement/occlusion flags. Anonymized site IDs preclude exact geo-spatial modelling, even though continent/ecoregion information is retained. There is also no ground-truth depth or pose data yet. These constraints matter methodologically: they bound the kinds of generalization claims that can be made from the benchmark, especially for rare-species re-identification, fine-grained behavior analysis, and exact spatial ecology.

6. Nomenclature and disambiguation

The designation SA-FARI in this context refers specifically to the wildlife MAT dataset “The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification” (Wasmuht et al., 19 Nov 2025). It is distinct from several unrelated arXiv works with similar acronyms or orthography: “Safire: Similarity Framework for Visualization Retrieval” (Nguyen et al., 18 Oct 2025), “SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation” (Mao et al., 11 Oct 2025), “SAFARI: Safe and Active Robot Imitation Learning with Imagination” (Palo et al., 2020), “Spatial-and-Frequency-aware Restoration method for Images based on Diffusion Models” (Lee et al., 2024), and “SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation” (Zhu et al., 23 Jun 2026).

That disambiguation is important because the shared naming pattern does not indicate a shared research program. In the arXiv record, SA-FARI denotes a benchmark for camera-trap wildlife video, masklets, multi-animal tracking, and promptable segmentation/classification; the similarly named works belong to visualization retrieval, referring image segmentation, imitation learning, image restoration, and long-horizon agentic fault attribution, respectively (Wasmuht et al., 19 Nov 2025).