---
title: 'SA-FARI: Wildlife Tracking Benchmark'
url: https://www.emergentmind.com/topics/sa-fari
type: topic
---

# SA-FARI: Wildlife Tracking Benchmark

Searching arXiv for the SA-FARI dataset paper to ground the article and citations.
SA-FARI, short for **Segment Anything in Footage of Animals for Recognition and Identification**, is an open-source dataset and benchmark for **multi-animal tracking (MAT)** in camera-trap video [2511.15622]. It comprises **11,609 camera trap videos** collected over approximately **10 years (2014–2024)** from **741 locations across 4 continents**, spanning **99 species categories**. Each video is exhaustively annotated, yielding approximately **46 hours of densely annotated footage**, **16,224 masklet identities**, and **942,702 individual bounding boxes, segmentation masks, and species labels**. The dataset is positioned as the **largest open-source MAT dataset for wild animals**, and as a benchmark for training general-purpose MAT models applicable across wild animal populations [2511.15622].

## 1. Scope, composition, and sampling structure

SA-FARI contains **2,747 min \(\simeq 45.78\) h** of annotated camera-trap footage. The collection spans **741 independent sampling sites** distributed across **four continents / four major ecoregions**: **Central Africa, South America, Mesoamerica,** and **Southern Europe** [2511.15622]. The species inventory comprises **99 common-name species categories**, spanning **Kingdom: Animalia \(\rightarrow\) Phylum: Chordata \(\rightarrow\) 4 classes, 23 orders, 53 families**. The distribution is explicitly long-tailed: the dataset has an **average \(\simeq 117\) videos/species**, **29 species account for 90 % of all videos**, and **several rare species appear exclusively in the test split** [2511.15622].

The train/test partition is also specified. The **train** split contains **10,776 videos**, **2,545 min**, **91 species**, **15,141 masklets**, **880,361 boxes + masks**, **31,282 video–species pairs**, and **650 sites**. The **test** split contains **833 videos**, **202 min**, **83 species**, **1,083 masklets**, **62,341 boxes + masks**, **2,322 video–species pairs**, and **91 sites**. Across both splits, the total remains **11,609 videos**, **2,747 min**, **99 species**, **16,224 masklets**, **942,702 boxes + masks**, **33,604 video–species pairs**, and **741 sites** [2511.15622].

The dataset is framed around the absence of a suitable benchmark for general-purpose MAT in wildlife footage. Existing datasets are described as being limited in scale, constrained to a few species, or lacking sufficient temporal and geographical diversity. SA-FARI is presented as a response to those deficiencies, with emphasis on **high species diversity**, **multi-region coverage**, and **high-quality spatio-temporal annotations** [2511.15622].

## 2. Annotation protocol, masklets, and anonymization

The core annotation unit in SA-FARI is the **masklet**, defined as a **spatio-temporal segment with per-frame masks**. The dataset contains **16,224 unique identities** at the masklet level and **942,702 segmentation masks**. **Bounding boxes** are **derived automatically from mask extents**, giving the **same total count** as the masks. In addition, SA-FARI provides **33,604 video–species pairs (positive + hard/easy negatives)** and **741 sampling locations** [2511.15622].

The annotation workflow is explicitly staged [2511.15622]:

1. **Automatic pseudo-annotation at 6 fps with SAM 3 to generate initial masklets**
2. **Deduplication by mask–mask IoU to remove overlaps**
3. **Human review stage 1: remove unsegmentable or blurred masks**
4. **Human review stage 2: correct/add masks in-loop using SAM 2 promptable tools**
5. **Exhaustivity check: verify no animal was missed**
6. **Bounding boxes extracted from refined masks**

This protocol places the dataset at the intersection of promptable segmentation and manual quality control. A plausible implication is that SA-FARI is intended not merely as a static benchmark, but as an example of a practical annotation pipeline for wildlife video at scale.

Anonymization is handled by dropping exact GPS information. Each site is represented by a **random site-ID string plus a continent/ecoregion code**, while **date/time stamps are preserved when available** [2511.15622]. This design preserves temporal and coarse regional context while restricting exact geospatial disclosure.

## 3. MAT task definition and evaluation methodology

SA-FARI supports four MAT sub-tasks [2511.15622]:

- **Detection**: localize animals via box/mask
- **Segmentation**: pixel-accurate outline per animal
- **Re-identification**: maintain consistent IDs across frames
- **Promptable classification**: species-specific vs. generic “animal” prompts

The benchmark uses standard tracking metrics. **Multi-Object Tracking Accuracy (MOTA)** is defined as

$$
\mathrm{MOTA}
= 1 \;-\;
\frac{\sum_{t}\bigl(\mathrm{FP}_t + \mathrm{FN}_t + \mathrm{IDSW}_t\bigr)}
     {\sum_{t}\mathrm{GT}_t},
$$

where \(\mathrm{FP}_t\), \(\mathrm{FN}_t\), and \(\mathrm{IDSW}_t\) are false positives, false negatives, and ID switches at frame \(t\), and \(\mathrm{GT}_t\) is the number of ground-truth objects [2511.15622].

**ID F1-score (IDF1)** is defined as

$$
\mathrm{IDF1}
= \frac{2\,\mathrm{IDTP}}
       {2\,\mathrm{IDTP} + \mathrm{IDFP} + \mathrm{IDFN}},
$$

where IDTP/FP/FN count true positives, false positives, and false negatives under ID-aware matching [2511.15622].

The evaluation suite also includes **mean Average Precision (mAP) at IoU \(\ge 0.5\)**, and the tracking metrics **pHOTA** and **TETA**. In the benchmark description, **pHOTA and TETA** are characterized as **phrase-based and multi-category extensions of HOTA**, and are reported together with their **DetA** and **AssA** components [2511.15622]. All metrics are computed per frame and aggregated over time, with species-specific metrics averaged across species categories.

## 4. Benchmarks, model families, and reported results

The benchmark evaluates two model families [2511.15622]. The first consists of **vision–language promptable models**: **GLEE**, **LLMDet + independent tracker**, and **SAM 3** in three variants—**SAM 3 (baseline)**, **SAM 3 trained with SA-FARI data mix**, and **SAM 3 fine-tuned (FT) on SA-FARI**. The second consists of **vision-only species-agnostic pipelines**: **MegaDetector (MD)** paired with **ByteTrack**, **OCSort**, or **BoostSort++**.

Under **species-specific prompting**, the reported numbers are [2511.15622]:

- **GLEE**: **IDF1 \(-0.2\)**, **pHOTA-Total 7.5**, **Det 1.2**, **Ass 49.7**, **TETA 22.0**
- **LLMDet + tracker**: **IDF1 2.6**, **pHOTA-Total 41.3**, **Det 21.4**, **Ass 80.0**, **TETA 30.4**
- **SAM 3 (baseline)**: **IDF1 14.0**, **pHOTA-Total 48.5**, **Det 28.4**, **Ass 83.4**, **TETA 39.6**
- **SAM 3 + SA-FARI (mixed)**: **IDF1 39.0**, **pHOTA-Total 63.1**, **Det 47.9**, **Ass 83.9**, **TETA 52.1**
- **SAM 3 FT (SA-FARI)**: **IDF1 46.9**, **pHOTA-Total 68.1**, **Det 55.4**, **Ass 84.6**, **TETA 58.7**

The accompanying interpretation in the benchmark is specific: **fine-tuning on SA-FARI yields a three-fold jump in IDF1 and \(> 19\) pHOTA points over the SAM 3 baseline** [2511.15622].

Under **species-agnostic “animal” prompting**, the reported comparison is again favorable to SA-FARI-trained SAM 3 [2511.15622]:

- **MD + ByteTrack**: **IDF1 38.6**, **HOTA-Total 39.5**, **Det 20.8**, **Ass 75.6**
- **MD + OCSort**: **IDF1 33.9**, **HOTA-Total 45.5**, **Det 27.9**, **Ass 74.9**
- **MD + BoostSort++**: **IDF1 47.2**, **HOTA-Total 38.3**, **Det 18.4**, **Ass 80.8**
- **SAM 3 + SA-FARI (animal)**: **IDF1 71.1**, **HOTA-Total 64.4**, **Det 50.0**, **Ass 83.7**

The benchmark summarizes this as follows: **even without retraining for the “animal” prompt, SAM 3 trained on SA-FARI outperforms all vision-only baselines by \(> 23\) IDF1 and \(> 18\) HOTA** [2511.15622].

A further **test-subset analysis** for **SAM 3 FT on SA-FARI/test** shows the stratification of difficulty [2511.15622]:

- **Large masks**: **IDF1 63.4**, **pHOTA-Tot 81.4**, **Det 72.5**, **Ass 91.6**, **TETA 71.7**
- **Small masks**: **IDF1 25.3**, **pHOTA-Tot 52.2**, **Det 38.5**, **Ass 71.3**, **TETA 46.9**
- **Multiple animals**: **IDF1 45.9**, **pHOTA-Tot 67.8**, **Det 54.5**, **Ass 85.1**, **TETA 59.6**
- **Challenging**: **IDF1 36.8**, **pHOTA-Tot 61.7**, **Det 51.3**, **Ass 75.3**, **TETA 59.6**
- **Night**: **IDF1 44.1**, **pHOTA-Tot 66.0**, **Det 53.2**, **Ass 83.1**, **TETA 61.9**
- **All test**: **IDF1 46.9**, **pHOTA-Tot 68.1**, **Det 55.4**, **Ass 84.6**, **TETA 58.7**

The benchmark notes that **small and occluded masklets remain the hardest cases (\(-29\) pHOTA vs. large masks)**, while **nighttime** and **multiple-animal clips** produce moderate performance drops [2511.15622].

## 5. Scientific role, applications, and current limitations

SA-FARI is positioned as a dataset for **wildlife conservation and ecology**. The applications named in the benchmark include **automated population counts**, **occupancy modelling**, **abundance estimation**, **individual re-identification for long-term health and movement studies**, and **behavioural analysis (e.g. gait, interactions) when coupled with downstream classifiers** [2511.15622]. Because the annotations are dense, spatio-temporal, and identity-aware, the resource is relevant not only for frame-level detection but also for longitudinal inference over individuals.

The benchmark also proposes several technical extensions: adding **multi-modal channels** such as **audio**, **depth**, and **infrared**; extending annotations to include **keypoints (pose)**, **detailed behavior labels**, and **natural-language scene captions**; broadening **geographic coverage** and **taxonomic breadth**; and building **semi-automated active-learning pipelines** for continued expansion [2511.15622]. This suggests a roadmap in which SA-FARI functions as a seed benchmark for a broader, multimodal wildlife video ecosystem.

The limitations are also explicit [2511.15622]. The **long-tail species distribution yields fewer examples for rare taxa**. There is a **lack of explicit behavior annotations beyond movement/occlusion flags**. **Anonymized site IDs preclude exact geo-spatial modelling**, even though continent/ecoregion information is retained. There is also **no ground-truth depth or pose data yet**. These constraints matter methodologically: they bound the kinds of generalization claims that can be made from the benchmark, especially for rare-species re-identification, fine-grained behavior analysis, and exact spatial ecology.

## 6. Nomenclature and disambiguation

The designation **SA-FARI** in this context refers specifically to the wildlife MAT dataset **“The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification”** [2511.15622]. It is distinct from several unrelated arXiv works with similar acronyms or orthography: **“Safire: Similarity Framework for Visualization Retrieval”** [2510.16662], **“SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation”** [2510.10160], **“SAFARI: Safe and Active Robot Imitation Learning with Imagination”** [2011.09586], **“Spatial-and-Frequency-aware Restoration method for Images based on Diffusion Models”** [2401.17629], and **“SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation”** [2606.24626].

That disambiguation is important because the shared naming pattern does not indicate a shared research program. In the arXiv record, SA-FARI denotes a benchmark for **camera-trap wildlife video**, **masklets**, **multi-animal tracking**, and **promptable segmentation/classification**; the similarly named works belong to visualization retrieval, referring image segmentation, imitation learning, image restoration, and long-horizon agentic fault attribution, respectively [2511.15622].

Source: https://www.emergentmind.com/topics/sa-fari