CylindTrack: Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking

Abstract: Multi-Object Tracking (MOT) is a core capability for embodied perception, and panoramic cameras are attractive for embodied systems because their 360° field of view reduces blind spots and keeps surrounding targets observable for longer durations. However, panoramic MOT is not a straightforward extension of perspective MOT. In equirectangular panoramic videos, the horizontal image domain is periodic rather than Euclidean, which breaks planar motion assumptions and makes IoU-based association unreliable near the 0°/360° seam. Meanwhile, large-FoV scenes often contain more objects, stronger scale variation, and more frequent interactions, making online association particularly sensitive to unstable frame-wise depth cues. To address these issues, we propose CylindTrack, a depth-aware cylindrical tracking-by-detection framework for panoramic MOT. CylindTrack first introduces Depth-Temporal Trajectory Modeling (DTM), which promotes instance depth from an isolated frame-wise cue to a temporally filtered trajectory-level state. To improve the reliability of depth observations, we further develop Spherical Spatio-Temporal Consistency Learning (SSTC), which combines a Temporal Mixer and Spherical Geometry-aware Attention to enhance temporal coherence and panoramic geometric alignment in depth-aware representations. Finally, we design a Topology-Aware Cylindrical Motion Model (TCMM) that lifts horizontal motion into a continuous angular state space and performs seam-consistent motion prediction and association in the periodic panoramic domain. By jointly modeling trajectory-level depth consistency and panoramic topology, CylindTrack improves identity preservation and trajectory continuity in challenging panoramic scenes. The source code will be released at https://github.com/warriordby/CylindTrack.

• The paper introduces a novel depth-temporal and cylindrical motion model for panoramic MOT that improves identity preservation by leveraging temporal smoothing and topology-aware state propagation.
• It integrates Depth-Temporal Trajectory Modeling, Spherical Spatio-Temporal Consistency, and a Topology-Aware Cylindrical Motion Model to counteract distortions, occlusions, and seam discontinuities in 360° imagery.
• Empirical evaluations on QuadTrack and JRDB benchmarks show substantial improvements in IDF1, AssA, and BCIC metrics, confirming the framework's robustness in cross-seam tracking scenarios.

Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking: CylindTrack

CylindTrack introduces a systematic methodology for motion modeling and trajectory association in panoramic multi-object tracking (MOT), integrating depth cues with an explicit cylindrical state representation. The framework directly addresses the intrinsic challenges of 360° imaging—periodic topology, scale variation, and depth ambiguity—by unifying temporal depth filtering, spatio-temporal representation learning, and topology-consistent Kalman filtering within a modular tracking-by-detection (TBD) paradigm.

---

Motivation and Problem Analysis

The proliferation of panoramic cameras in robotics and surveillance has escalated requirements for persistent target tracking in full-surround environments. However, equirectangular panoramic imagery violates planar geometric assumptions at the heart of classical MOT, producing non-uniform spatial distortions and discontinuities at the $0^\circ/360^\circ$ seam. This not only undermines the fidelity of standard image-based Kalman filtering and IoU-based association—especially under boundary crossing—but also induces substantial identity fragmentation.

Compounding these issues, panoramic scenarios exhibit more severe occlusion, denser interactions, and variable scale relative to perspective scenes, increasing the dependence of association on reliable geometric cues. Monocular depth estimation, when naively applied, supplies unstable per-frame evidence unsuited for online identity consistency. The fundamental insight of CylindTrack is to elevate both position and depth modeling from frame-level, local observations to trajectory-level, temporally filtered latent states in a topology-respecting geometric space.

---

CylindTrack Framework Overview

CylindTrack extends the modular online TBD pipeline with three key technical innovations: (1) Depth-Temporal Trajectory Modeling (DTM), (2) Spherical Spatio-Temporal Consistency learning (SSTC), and (3) a Topology-Aware Cylindrical Motion Model (TCMM).

(Figure 1)

Figure 1: Performance for depth-only trajectory association across all sequences. Adjacent-frame accuracy (Acc) and sequence-level IDF1 are measured for multiple zero-shot depth models, demonstrating the benefit of depth temporal filtering via DTM.

1. Depth-Temporal Trajectory Modeling (DTM)

DTM upgrades instance depth from a volatile, frame-wise feature to a temporally smoothed trajectory state modeled by a 1D constant-velocity Kalman filter. At each timestep, the predicted trajectory depth is informed by historical depth predictions, attenuating monocular depth noise and providing a robust geometric basis for association even under occlusion or major visual perturbations. Empirical analysis (Figure 1) demonstrates that sequence-level IDF1 scores increase markedly when depth is temporally filtered, underscoring the necessity of DTM for identity preservation.

2. Spherical Spatio-Temporal Consistency Learning (SSTC)

SSTC enhances detector-side geometric representations through two mechanisms:

• Temporal Mixer aligns and aggregates depth-query tokens within each batch, filtering out high-frequency fluctuations in depth estimates that degrade association reliability.
• Spherical Geometry-aware Attention (SGA) injects equirectangular panoramic priors into instance representations, enforcing alignment with the physical topology of 360° imagery.

The coupling of these components produces more temporally coherent and geometrically robust depth-aware detections, improving downstream trajectory discrimination—particularly when traditional 2D features are ambiguous.

(Figure 2)

Figure 2: Visualization of the SSTC-enhanced detector pipeline, combining query-based temporal refinement with spherical geometry priors for panoramic depth-aware instance representation.

3. Topology-Aware Cylindrical Motion Model (TCMM)

The cylindrical model reformulates the tracker’s horizontal motion and spatial overlap metrics:

• Trajectory states maintain horizontal position as an unwrapped longitude $\theta$, and updates are angle-wrapped to preserve continuity at the $0^\circ/360^\circ$ seam.
• Association cost fuses periodic overlap, angular consistency, depth difference (from DTM), and normalized detection confidence.
• Overlap computation uses a horizontal-periodic, pixel-vertical IoU, which more faithfully reflects physical proximity in panoramic scenes.

TCMM seamlessly integrates with existing TBD trackers, replacing Cartesian prediction and matching with angular-periodic logic in state propagation and association.

---

Experimental Evaluation

CylindTrack is evaluated on the QuadTrack and JRDB benchmarks—each presenting distinct challenges in mobile robot panoramic tracking. Results strongly validate each element of the design.

Figure 3: Visualization of boundary annotation gaps in panoramic MOT—annotation discontinuities cluster near seams and heavy occlusions, emphasizing the need for topological and depth consistency.

Quantitative comparisons across seven tracker backbones demonstrate that incorporating CylindTrack’s SSTC and DTM modules yields substantial gains in HOTA, IDF1, AssA, and MOTA, with improvements of up to $11.83\%$ (IDF1) and $8.22\%$ (AssA) averaged over all trackers on QuadTrack. The TCMM further boosts boundary-crossing identity consistency (BCIC), yielding absolute BCIC gains of up to $24.38$ points over strong baselines.

Ablation studies confirm:

• DTM is essential for leveraging depth cues; without temporal smoothing, per-frame depth remains too noisy for robust association.
• SSTC’s geometric alignment and temporal coherence are synergistic—improvements in association metrics are maximal when jointly enabled.
• The TCMM’s cylindrical representation and horizontal-periodic overlap computation are both necessary for seamless tracking through panoramic seams; neither component alone suffices.
• Spherical overlap computation is compatible but does not yield consistent improvements over horizontal-periodic models in scenarios without extreme vertical parallax.

(Figure 4)

Figure 4: Cylindrical lifting of position and depth—conventional 2D bounding boxes are mapped to a periodic angular-depth state for topology-consistent association.

Qualitative visualization confirms that CylindTrack maintains identity continuity across panoramic boundaries and under occlusion, preventing the association failures that afflict UV-centric trackers.

---

Efficiency and Practical Considerations

Parameter tuning and resource profiling reveal that the computational overhead of CylindTrack is minimal, thanks to the design choices:

• Query-level temporal mixing replaces dense memory computation for depth fusion, reducing GMACs and GFLOPs by over $12\%$ compared to DETR-derived baselines.
• FPS is preserved (and sometimes improved), and the modular match-cost formulation maintains backbone-agnostic deployment.

Hyperparameter sweeps indicate that CylindTrack exhibits stable performance across a range of association weights and thresholds, attesting to practical robustness.

---

Implications and Future Directions

Practically, CylindTrack demonstrates that panoramic MOT in robotics, surveillance, and autonomous mobility gains substantial robustness by modeling both position and geometry in a manner congruent with omnidirectional sensor topology. Enhanced identity preservation, especially under occlusion and cross-seam events, enables more accurate situational awareness for embodied agents.

Theoretically, the combination of periodic state-spaces and temporally filtered depth modeling opens new avenues for lightweight, geometric-consistent tracking in domains where full 3D reconstruction is infeasible. The modular approach, which decouples detector-side feature adaptation from tracker-side state representation, suggests that specialized panoramic and depth processing can be grafted onto general-purpose tracking frameworks without extensive redesign.

Future research should address:

• Joint estimation of pose and depth to move from relative panoramic consistency toward metric 3D consistency.
• End-to-end learning frameworks that holistically optimize detection, depth, and association in a topology-aware manner.
• Extension to more varied object categories and broader robotics sensory suites, accommodating domain shifts and non-standard projection geometries.

---

Conclusion

CylindTrack provides an effective and efficient approach for panoramic multi-object tracking by integrating temporally filtered depth, panoramic geometric consistency, and topology-aware state propagation within a modular, online tracking-by-detection paradigm. This enables robust identity preservation and trajectory continuity under challenging panoramic scenarios, delivering state-of-the-art results on established benchmarks and offering a scalable template for future panoramic tracking systems.