PS-Track: Point-Supervised Multi-Object Tracking

• PS-Track is a point-supervised multi-object tracking framework that uses sparse point annotations to generate spatially and temporally consistent object trajectories.
• The method integrates Temporal-Feedback Prompting, Point-Excited Wavelet Attention, and Uncertainty-Guided Gaussian Learning to enhance annotation efficiency and robustness against spatial ambiguity.
• Empirical results demonstrate significant improvements on benchmarks like DanceTrack, reducing annotation time and closing the gap with traditional box-supervised methods.

Searching arXiv for the primary PS-Track paper and closely related tracking supervision work to ground the article in current literature. I’m going to look up arXiv entries for PS-Track / PS-MOT and PathTrack. PS-Track most directly denotes the hierarchical pipeline introduced for point-supervised multi-object tracking (PS-MOT), where a video $I_{1:T}$ with $T$ frames is annotated not with dense boxes but with a single 2D image point $p_t^i \in \mathbb{R}^2$ for each object $i$ at time $t$. In that formulation, the task is simultaneously to maintain identity-consistent trajectories across time and to localize each instance per frame, such as by predicting a bounding box $B_t^i$ or a mask $M_t^i$, using only point annotations during training (Luo et al., 29 Jun 2026). The method reframes supervision from full spatial fitting to a topological center-driven representation, and organizes the learning problem across data, model, and loss levels through Temporal-Feedback Prompting (TFP), Point-Excited Wavelet Attention (PEWA), and Uncertainty-Guided Gaussian Learning (UGL) (Luo et al., 29 Jun 2026).

1. Formal setting and representational shift

In PS-MOT, the identity-consistent output is a set of trajectories $\{\tau^i\}$ with $\tau^i = \{(B_t^i, id=i)\}$ over time, learned from point supervision rather than dense boxes (Luo et al., 29 Jun 2026). This differs from traditional bounding-box supervision, which enforces spatial fitting by regressing geometric extent, alignment, and scale. PS-Track instead adopts a topological center-driven representation in which the annotated point functions as a semantic anchor indicating presence and approximate object center, but not explicit geometry or scale (Luo et al., 29 Jun 2026).

The motivation is primarily annotation efficiency and robustness to difficult imaging geometry. The reported annotation cost is approximately $0.7\!-\!0.9$ s per point versus approximately $T$0 s per box per instance, and point supervision avoids ambiguities arising from perspective distortions and ill-defined bounding boxes in fisheye or panoramic views (Luo et al., 29 Jun 2026). At the same time, the representation introduces three stated difficulties: spatial ambiguity, because the point does not specify boundary or scale; identity drift, because weak scale priors undermine temporal consistency; and lack of explicit geometry, because there are no direct constraints on extent, aspect ratio, or orientation (Luo et al., 29 Jun 2026).

PS-Track is designed specifically around these failure modes. Its hierarchy is explicit: TFP produces temporally consistent pseudo-labels from sparse point seeds, PEWA injects boundary-aware feature priors during training, and UGL calibrates the influence of noisy pseudo-labels during optimization (Luo et al., 29 Jun 2026).

2. Data-level mechanism: Temporal-Feedback Prompting

TFP is the data-level component that evolves sparse point seeds into temporally consistent pseudo-labels by combining negative spatial cues, motion priors, and a temporal feedback loop (Luo et al., 29 Jun 2026). The method maintains a per-track Kalman state, or any motion model $T$1, and predicts the center and motion-prior box as

$T$2

To disambiguate nearby instances, neighbors are defined by a proximity threshold $T$3, and negative prompts are constructed from nearby centers:

$T$4

A positive Gaussian heatmap is centered on the seed point,

$T$5

and a promptable segmentation backbone, reported as SAM-v3 in the experiments, is constrained by the positive point, negative prompts, and motion-prior box to generate a mask and then a pseudo-box:

$T$6

$T$7

Pseudo-label quality is not treated as binary. Instead, TFP computes a joint quality score that multiplies SAM confidence and motion consistency through IoU:

$T$8

The temporal feedback loop accepts the pseudo observation when $T$9, updates the Kalman filter with center and scale, and caches $p_t^i \in \mathbb{R}^2$0 for later loss weighting. Otherwise, the method re-runs SAM with a tighter prior or stronger negatives, or falls back to the motion prior with reduced $p_t^i \in \mathbb{R}^2$1 (Luo et al., 29 Jun 2026). The stated effect is suppression of identity merging, reduced fragmentation under occlusion, and stronger temporal consistency (Luo et al., 29 Jun 2026).

3. Model-level and loss-level design

PEWA is the model-level mechanism for boundary hallucination from point cues (Luo et al., 29 Jun 2026). Given backbone features $p_t^i \in \mathbb{R}^2$2, a one-level Haar DWT separates low-frequency semantics and high-frequency edges:

$p_t^i \in \mathbb{R}^2$3

A Gaussian centered at the point seed is downsampled to wavelet resolution,

$p_t^i \in \mathbb{R}^2$4

then passed through a lightweight modulator $p_t^i \in \mathbb{R}^2$5 and a sigmoid to form a frequency excitation mask,

$p_t^i \in \mathbb{R}^2$6

The high-frequency sub-bands are modulated near the point,

$p_t^i \in \mathbb{R}^2$7

while the low-frequency branch is left intact or processed with standard self-attention. Inverse reconstruction and residual fusion are then applied:

$p_t^i \in \mathbb{R}^2$8

PEWA is explicitly a training-time prior and is bypassed at inference, so test-time execution remains point-free (Luo et al., 29 Jun 2026).

UGL is the loss-level mechanism. It treats each pseudo-box as a noisy observation drawn from a Gaussian centered on latent ground truth:

$p_t^i \in \mathbb{R}^2$9

The regression objective is the heteroscedastic Gaussian negative log-likelihood

$i$0

Reliable pseudo-labels therefore impose strong supervision, while unreliable ones are down-weighted. The full training objective combines regression, classification, and identity prediction:

$i$1

This design is presented as a hierarchical data-model-loss solution to the specific weaknesses of point supervision: TFP addresses temporal disambiguation, PEWA boundary recovery, and UGL uncertainty calibration (Luo et al., 29 Jun 2026).

4. Architecture, optimization, and runtime profile

The reported implementation uses a ResNet-50 backbone pretrained on ImageNet, four multi-scale feature levels, and a Deformable DETR detector with a 6-layer encoder, 6-layer decoder, and 300 queries (Luo et al., 29 Jun 2026). PEWA is inserted after the backbone, while the transformer and heads are unchanged. TFP is an offline preprocessing stage that generates temporally consistent pseudo-labels and per-instance $i$2, and UGL replaces deterministic box regression losses with heteroscedastic Gaussian NLL governed by $i$3 (Luo et al., 29 Jun 2026).

For query-based tracking, the paper places PS-Track in the MOTIP setting, where decoder identity logits and the identity loss $i$4 learn re-identification embeddings and improve temporal association inside the decoder, with no external association required (Luo et al., 29 Jun 2026). For association-based track-by-detection integrations such as BYTE, the detector benefits from TFP+PEWA+UGL under point supervision, while online association remains standard and relies on motion and appearance cues (Luo et al., 29 Jun 2026). In both cases, the role of point seeds is indirect at inference time: they improve pseudo-label quality and boundary-sensitive embeddings during training, which in turn reduce merged masks and fragmented detections (Luo et al., 29 Jun 2026).

The training setup uses AdamW with base learning rate $i$5, backbone learning rate $i$6, and weight decay $i$7; a 1-epoch warmup; learning-rate decays by a factor of $i$8 at epochs 6 and 9; 10 epochs for state-of-the-art runs; 30-frame clips; and gradient clipping with max-norm $i$9 (Luo et al., 29 Jun 2026). Data augmentation consists of random horizontal flip, random crop, scale jitter with short side in $t$0 and maximum 1440, and color jitter (Luo et al., 29 Jun 2026). The reported hardware is a single NVIDIA GeForce RTX 5090, with training time approximately $t$1 h per epoch on DanceTrack, training memory $t$2 GB versus $t$3 GB for MOTIP, and parameter count increasing from $t$4M to $t$5M (Luo et al., 29 Jun 2026).

Inference remains point-free. SAM and PEWA are not used at test time, the FLOPs are unchanged relative to MOTIP, and the reported inference speed is approximately $t$6 FPS (Luo et al., 29 Jun 2026). The source code is reported at https://github.com/xifen523/PS-MOT (Luo et al., 29 Jun 2026).

5. Empirical performance and ablation structure

PS-Track is evaluated on DanceTrack, SportsMOT, JRDB, and EmboTrack using HOTA, DetA, AssA, IDF1, and MOTA, with OSPA additionally reported for embodied panoramic data (Luo et al., 29 Jun 2026).

Dataset	Reported result	Context
DanceTrack test	HOTA 52.3, DetA 68.2, AssA 40.4, MOTA 71.7, IDF1 53.4	$t$7 HOTA over CenterTrack, $t$8 over FairMOT
SportsMOT test	HOTA 45.2, DetA 50.9, AssA 41.6, MOTA 30.2, IDF1 46.9	Reported as robust under rapid motion and pose deformation
JRDB test	HOTA 20.72, OSPA 0.92, IDF1 22.21, MOTA 12.56	HOTA above TrackFormer 19.16 and DiffMOT 19.96
EmboTrack QuadTrack	HOTA 33.9, OSPA 0.94, IDF1 38.6, MOTA -1.3	Ego-centric environments with dense crowds and severe scale variations
EmboTrack BipTrack	HOTA 33.9, OSPA 0.97, IDF1 32.7, MOTA 14.7	Same evaluation setting

On DanceTrack validation, the ablation sequence isolates the contribution of each level in the hierarchy. The baseline without TFP, PEWA, or UGL yields HOTA 30.3. Adding TFP raises HOTA to 49.0, adding UGL yields 49.4, and the full TFP+PEWA+UGL configuration reaches 50.3, for a total improvement of $t$9 over the baseline (Luo et al., 29 Jun 2026). Annotation-noise robustness is reported up to perturbation $B_t^i$0 px, where HOTA remains approximately 42.8 (Luo et al., 29 Jun 2026). Sensitivity to the loss weight indicates the best result around $B_t^i$1 in the fast 2-epoch protocol, giving HOTA 44.8, while full 10-epoch training reaches 50.3 HOTA (Luo et al., 29 Jun 2026).

Cross-paradigm experiments indicate that the same framework can support association-based, autoregressive, and query-based trackers. On DanceTrack validation, BYTE obtains 47.1 HOTA with full box supervision and 42.9 with PS-Track point supervision; AR-MOT obtains 42.4 versus 36.9; and MOTIP obtains 62.2 versus 50.3 (Luo et al., 29 Jun 2026). A related comparison shows that directly coupling Point2RBox-v3 with standard associators fails at approximately 10.8 HOTA, whereas PS-Track-trained YOLOX + BYTE reaches HOTA 42.9 and IDF1 47.9, which is presented as evidence for the necessity of the unified framework (Luo et al., 29 Jun 2026).

6. Comparative position, limitations, and terminological scope

Relative to box-supervised MOT, the method is described as nearly closing the gap on several benchmarks while requiring only point labels (Luo et al., 29 Jun 2026). Relative to earlier weakly or point-supervised trackers, the stated distinction is the combination of temporal disambiguation through TFP, boundary hallucination through PEWA, and uncertainty-aware supervision through UGL, which together mitigate spatial leakage and identity drift (Luo et al., 29 Jun 2026). The reported operating regimes are also explicit: center-driven supervision works best when appearance is uniform and motions are complex, as in DanceTrack, and when panoramic or fisheye views make axis-aligned boxes ill-posed; it struggles under severe occlusion with overlapping point prompts, extreme motion blur that erodes high-frequency edges, and rapidly varying scale without strong motion priors (Luo et al., 29 Jun 2026).

The limitations section identifies several concrete directions. TFP can still fragment under severe mutual occlusion; PEWA depends on high-frequency cues and may benefit from deblurring or multi-frame fusion; the annotation protocol uses centers of ground-truth boxes for controlled benchmarking and may therefore differ from noisier real clicks; and extension to 3D MOT in LiDAR or BEV space with topological anchors is described as promising (Luo et al., 29 Jun 2026). These statements situate PS-Track as a point-supervised alternative rather than a universal replacement for dense annotation.

The term itself is not unique across arXiv. In a separate high-energy-physics context, a 4D fast track-finding system has been summarized as a PS-Track approach that parameterizes tracks in space and time, scores hypotheses with a separable Gaussian kernel in hardware, achieves sub-microsecond latency, and estimates track time at approximately 3.5 ps with 8 hits and no noise, or approximately 6.3 ps with 5% occupancy (Neri et al., 2015). In proton CT, PS-Track has been used as a label for a track-following procedure that grows tracks by minimizing accumulated angular change, with approximately 77% correct reconstruction at $B_t^i$2 after filtering and an estimated capacity of 5–25 million protons/s depending on spot size and acceptable fake rate (Pettersen et al., 2020). In annotation research, the closely related PathTrack framework converts weak path supervision and detections into dense trajectories, and the resulting dataset contains 720 sequences and 16,287 person trajectories (Manen et al., 2017). In antimatter instrumentation, a FACT-like scintillating fiber detector has likewise been presented as a PS-Track digital calorimeter for monitoring ortho-positronium, with 20 ns analysis bins and about 9.5 bits dynamic range (Rienäcker et al., 2021).

Within this broader terminological field, PS-Track in the sense of PS-MOT is the point-supervised multi-object-tracking pipeline built around TFP, PEWA, and UGL, and its central contribution is to cultivate instance awareness from sparse point seeds while preserving a standard, point-free inference pipeline (Luo et al., 29 Jun 2026).