SAMIDARE: Advanced Tracking-by-Segmentation for Dense Scenarios

Abstract: Automated sports analysis demands robust multi-object tracking (MOT), yet segmentation-based methods often struggle with mask errors and ID switches in dense scenes. We propose SAMIDARE, a framework that enhances SAM2MOT for crowded scenes through three key components: (1) density-aware mask re-generation and (2) selective memory updates, both for adaptive mask control to preserve target feature integrity, and (3) state-aware association and new track initialization, which improves robustness under mutual occlusions and frequent frame-out events. Evaluated on the SportsMOT dataset, SAMIDARE achieves state-of-the-art performance, outperforming the baseline by 2.5 HOTA and 4.2 IDF1 points on the validation set. These results demonstrate that adaptive feature management using mask control and state-aware association provide a robust and efficient solution for dense sports tracking. Code is available at https://github.com/ZabuZabuZabu/SAMIDARE

• The paper presents a tracking-by-segmentation framework that leverages density-aware masking, hybrid confidence measures, and state-aware matching to mitigate ID switches in dense sports scenes.
• It combines three key modules—DA-QR, H-CoI, and SA-OA—to handle occlusion, feature contamination, and reactivation challenges effectively.
• Experimental results show significant improvements over prior methods, with gains of 2.5 HOTA and 4.2 IDF1 points on the SportsMOT benchmark.

SAMIDARE: Tracking-by-Segmentation for Crowded Sports Scenarios

Motivation and Context

Multi-object tracking (MOT) in sports domains is fundamentally challenged by high object density, frequent occlusion, rapid motion, and visually similar targets. Traditional tracking-by-detection approaches—relying heavily on bounding box (BBox) association via motion or appearance cues—are fundamentally limited in these crowded spaces due to feature contamination from neighboring objects and subsequent ID-switches. Segment-based methods, such as SAM2MOT, have demonstrated improved robustness by employing mask propagation anchored in the Segment Anything Model (SAM), yet remain susceptible to drift, mask expansion, and memory corruption when isolated feature management fails under dense overlaps.

Architecture Overview

SAMIDARE extends the SAM2MOT paradigm to address the deficiencies observed in crowded sporting scenarios. The framework integrates three principal modules:

• Density-Aware Quality Reconstruction (DA-QR): Suppresses mask regeneration when a target resides within a spatially dense region, determined via intersection ratios among detection BBoxes, thus minimizing erroneous mask expansion.
• Hybrid Cross-Object Interaction (H-CoI): Identifies unreliable masks by adaptively switching between mean and variance of mask confidence, enabling more robust memory management and mitigating ID switches induced by feature contamination.
• State-Aware Object Addition (SA-OA): Utilizes a three-stage hierarchical matching protocol to intelligently handle active, occluded, and frame-out tracks, executing matching either via mask or stored BBox, thereby ensuring stable track recovery post occlusion or out-of-frame events.

Figure 1: Baseline SAM2MOT performance, exemplifying target loss and ID corruption in high-density scenes.

Figure 2: Mask re-generation is dynamically regulated in crowded scenes to avoid expansion and feature contamination.

Figure 3: Full pipeline of SAMIDARE, showcasing adaptive mask control (DA-QR, H-CoI) and state-aware matching (SA-OA).

Methodological Contributions

Density-Aware Quality Reconstruction

DA-QR prevents reckless mask regeneration by quantifying local spatial density using summed intersection ratios of a target's BBox against other detections. Mask re-generation is conditionally triggered by both confidence thresholds and density criteria, restricting updates in environments likely to induce mask drift.

Hybrid Cross-Object Interaction

H-CoI supersedes SAM2MOT’s CoI by combining mean and variance indicators in confidence scores. When masks overlap (mIoU above threshold), both indicators are considered: mean-based discrimination is prioritized if confidence divergence is high, otherwise variance determines instability. Memory updates for unreliable masks are skipped, thereby preserving feature integrity.

State-Aware Object Addition

SA-OA introduces a hierarchical matching strategy:

• Stage 1: Hungarian assignment of detection BBoxes to active and occluded tracks via a composite cost blending spatial IoU and confidence mean.
• Stage 2: For tracks unmatched in Stage 1 (often due to mask fragmentation), matching reverts to last successful BBox, allowing QR to reset corrupted masks.
• Stage 3: Frame-out tracks (those leaving the field of view) are not propagated to avoid unnecessary computation; instead, spatial continuity is leveraged by comparing recent detection BBoxes, ensuring reliable and efficient reactivation.

Experimental Evaluation

Extensive evaluations were conducted on SportsMOT, utilizing HOTA and IDF1 metrics. SAMIDARE surpassed SAM2MOT by 2.5 HOTA and 4.2 IDF1 points on validation, and outperformed SOTA trackers—including McByte, DiffMOT, and DeepEIoU—across all sports. The integration of DA-QR and SA-OA yielded the largest accuracy boost, especially in highly dense basketball and volleyball sequences, attesting to the criticality of explicit density regulation and hierarchical track management.

Module Effectiveness and Sensitivity

Ablation studies revealed that:

• DA-QR independently provided significant gains in crowded scenes.
• H-CoI, while insufficient in isolation, delivered strong synergy when combined with DA-QR, confirming the necessity of both spatial and confidence adaptive controls.
• SA-OA’s three-stage matching structure produced the most substantial improvements, especially in handling frame-out and reactivation scenarios.

Sensitivity analysis indicated that optimal density thresholds for DA-QR (e.g., $\theta_{\text{density}=1.5$) vary by sport, necessitating dynamic threshold selection for generalizable deployment.

Figure 4: Distribution of frame-out durations, validating the rationale for spatial continuity-based reactivation in SA-OA.

Qualitative Performance in Dense Scenarios

SAMIDARE maintained ID consistency and trajectory coherence in prototypical sequences featuring occlusion and player overlap. DA-QR suppressed inappropriate mask re-generation, preserving discriminative features; H-CoI prevented ID switches by robustly managing memory updates; Stage 2 in SA-OA enabled clean recovery from mask fragmentation post-occlusion.

Figure 5: Robust tracking of a player in crowded scenes enabled by DA-QR, maintaining ID through occlusion and reappearance.

Practical and Theoretical Implications

SAMIDARE’s adaptive control modules substantially raise the baseline for tracking-by-segmentation in sports analytics. The explicit density-aware and state-aware mechanisms support robust trajectory maintenance in environments characterized by rapid appearance/disappearance and frequent occlusion. This architectural philosophy is poised for transfer to other high-density domains (e.g., surveillance, autonomous driving), where object overlap and occlusion are prevalent.

Theoretical implications point to the importance of context-aware feature management and dynamic matching strategies for temporal segmentation systems. The success of hybrid confidence modeling suggests further exploration into meta-indicator selection for mask reliability assessment.

Speculation on Future Directions

Dynamic adjustment of $\theta_{\text{density}$ based on instance-specific or domain-specific density statistics is a logical next step. Integration with domain-adaptive learning or online self-supervised threshold tuning would enhance generalizability to other sports and dense tracking scenarios. Moreover, extending adaptive mask management to account for motion cues and appearance changes could further minimize ID switches amid complex interactions.

Conclusion

SAMIDARE delivers a comprehensive, technically rigorous solution for tracking-by-segmentation in dense sporting scenes, addressing long-standing limitations in mask expansion, ID stability, and memory corruption. Through density-aware mask re-generation, robust memory management, and state-aware hierarchical matching, it achieves superior performance over both detection-based and prior segmentation-based trackers. Continued refinement in density adaptation and hybrid confidence evaluation will catalyze broader application in dense MOT contexts and advance the theory of adaptive segmentation tracking (2604.22162).