Scale-Aware & Unscented Tracker (SU-T)

• The paper presents SU-T, a novel framework that integrates a UKF motion model, explicit scale-aware noise, and the morphology-based FishIoU metric for accurate underwater fish tracking.
• It employs a cascaded data association strategy with high- and low-confidence stages and optional Re-ID to manage frequent occlusions and similar appearances among fish.
• Empirical results on the MFT25 dataset demonstrate significant gains, achieving 34.1 HOTA and 44.6 IDF1, underscoring domain-specific challenges in underwater MOT.

Scale-aware and Unscented Tracker (SU-T) is a specialized framework for underwater multiple fish tracking introduced alongside the Multiple Fish Tracking Dataset 2025 (MFT25). It combines an Unscented Kalman Filter (UKF) motion model, explicit scale-aware noise and gating, and a morphology-aware association metric termed FishIoU to address the characteristic failure modes of underwater MOT, including frequent occlusion, near-identical appearance within species, and erratic non-linear swimming behavior. In the reported MFT25 benchmark, SU-T with re-identification achieves 34.1 HOTA and 44.6 IDF1, and the accompanying analysis argues that fish tracking differs fundamentally from terrestrial object tracking scenarios (Li et al., 8 Jul 2025).

1. Problem setting and design rationale

SU-T is motivated by the observation that underwater multiple fish tracking differs from terrestrial MOT in three ways. First, water turbidity, reflections, and complex backgrounds produce frequent partial and complete occlusions. Second, individuals within the same species often have near-identical texture and shape, which increases identity switches. Third, fish exhibit non-linear, high-variance motion with rapid direction changes and speed fluctuations; the paper reports higher angular velocity variability than pedestrians or cells. These conditions jointly degrade standard linear-motion and overlap-based association pipelines (Li et al., 8 Jul 2025).

The framework addresses these conditions through four coupled design choices. Its motion model uses a UKF rather than a linear Kalman filter, with the stated goal of handling non-linear fish movement more effectively. Its scale-aware logic adapts motion, noise, and gating to target size, which is particularly relevant for small, elongated fish. Its FishIoU metric augments overlap with morphology-, center-, aspect-, and area-consistency terms tailored to aquatic species. Its data association is cascaded across high-confidence, low-confidence, and last-chance stages, with optional Re-ID embeddings to disambiguate visually similar individuals.

A recurrent misconception in this area is that underwater MOT is a minor domain transfer of terrestrial MOT. The benchmark and baseline analysis instead indicate domain-specific dynamics, especially in association quality and motion uncertainty, and explicitly report “fundamental differences between fish tracking and terrestrial object tracking scenarios” (Li et al., 8 Jul 2025).

2. Pipeline architecture and online tracking workflow

SU-T operates as an online pipeline. YOLOX-X produces per-frame detections with bounding boxes and confidence scores. Each active track is then predicted forward with a UKF, yielding a predicted box and state covariance. Association is performed in three stages using FishIoU, optionally combined with appearance embeddings, and solved with Hungarian assignment. Matched tracks update their UKF state with the detection; unmatched tracks age; unmatched detections can initialize new tracks. Track lifecycle management includes initialization, confirmation after successive matches, aging, termination when time_since_update > max_age, duplicate removal, and occlusion recovery using cascaded matching and track history (Li et al., 8 Jul 2025).

The association cascade follows a ByteTrack-style dual-threshold structure. High-confidence detections are matched first, then remaining tracks are matched against low-confidence detections, and a final “last-chance” stage matches residual tracks against historical last observations. This ordering is intended to preserve recall without immediately accepting low-confidence detections as new identities. Optional FastReID embeddings, with SBS-S101 recommended by ablation, re-enter the cost function to reduce identity switches among similar-looking fish.

SU-T itself does not require frame batching. The implementation description characterizes the tracker as online and lightweight, with standard detector preprocessing and a detector/Re-ID stack kept consistent across compared SDE trackers for fairness.

3. UKF formulation and scale-aware dynamics

SU-T tracks axis-aligned bounding boxes with a six-dimensional state,

$x_k = [c_x, c_y, v_x, v_y, w, h]^T,$

where $c_x, c_y$ are box-center coordinates, $v_x, v_y$ are velocities, and $w, h$ are width and height. The measurement vector is

$$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$

Because MFT25 runs at 25 FPS, SU-T uses $\Delta t = 1/25\,\mathrm{s}$ in an “almost constant-velocity” model with free scale evolution:

$$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$

The state transition is intentionally simple; the reported justification for UKF is that sigma-point propagation gives better covariance shaping under erratic deviations and non-linear box-to-center/size conversions than a linear KF (Li et al., 8 Jul 2025).

SU-T uses standard UKF sigma-point definitions with

$\lambda = \alpha^2 (n+\kappa) - n,$

typical choices $\alpha \in (10^{-3},1]$, $\beta = 2$ for Gaussian priors, and $c_x, c_y$0 or small. For $c_x, c_y$1, the filter uses $c_x, c_y$2 sigma points. Prediction and update follow the standard unscented recursion: sigma-point propagation through $c_x, c_y$3, reconstruction of the predicted mean and covariance, transformation into measurement space through $c_x, c_y$4, computation of the innovation covariance and cross-covariance, and posterior correction via the Kalman gain.

The scale-aware mechanism appears in both noise design and gating. Process noise is diagonal and proportional to object size,

$c_x, c_y$5

with $c_x, c_y$6 and $c_x, c_y$7. Measurement noise is similarly diagonal and scale-aware,

$c_x, c_y$8

with $c_x, c_y$9 and $v_x, v_y$0. The stated rationale is that small fish boxes jitter more because of detector uncertainty, whereas larger boxes are more stable.

Scale also enters gating. The predicted box diagonal is

$v_x, v_y$1

and a normalized center-distance gate is defined as

$v_x, v_y$2

with acceptance if $v_x, v_y$3. This makes gates stricter for small fish and wider for large fish, consistent with the noise model.

4. FishIoU and cascaded association

FishIoU is the tracker’s principal association metric. It starts from standard IoU,

$v_x, v_y$4

and augments it with a center-distance penalty, a central-region overlap term $v_x, v_y$5, aspect-ratio consistency $v_x, v_y$6, and area consistency $v_x, v_y$7. The central-region construction is asymmetric:

$v_x, v_y$8

$v_x, v_y$9

with empirically chosen constants $w, h$0, $w, h$1, and $w, h$2. The aspect-ratio and area-consistency terms are

$w, h$3

and

$w, h$4

The scale factor controlling the center penalty is

$w, h$5

The final score is

$w, h$6

with $w, h$7, $w, h$8, $w, h$9, $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$0, and $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$1 (Li et al., 8 Jul 2025).

The stated motivation for FishIoU is morphological. Elongated fish bodies and salient head/body regions mean that center overlap alone is insufficient; the central-region term emphasizes central alignment while tolerating tail variability. Aspect and area consistency discourage mismatches implying implausible inter-frame size changes. The scale-normalized center penalty reduces over-penalization of small targets, whose pixel-level motion can appear larger because of detector noise and hydrodynamics.

Association is performed by building a cost matrix between predicted tracks and detections using FishIoU and, optionally, appearance distance. For high-confidence detections,

$$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$2

and assignment is computed on $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$3 to maximize match quality. Remaining tracks are then matched against low-confidence detections under thresholded acceptance, and a final stage matches remaining tracks against historical last observations. Pairs outside UKF covariance gates or the scale-normalized center-distance gate are removed or assigned $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$4 before Hungarian optimization.

FishIoU is therefore not a replacement for appearance modeling. The original formulation explicitly allows optional FastReID embeddings to enter the cost matrix, and the empirical study reports further identity gains when Re-ID is enabled.

5. Track management, implementation, and computational profile

Track management in SU-T follows a conventional but explicitly staged lifecycle. Unmatched high-confidence detections spawn tentative tracks, with the UKF initialized from center and size while velocities are set to $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$5 with large $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$6. Confirmation requires matching in a minimum number of consecutive frames, or within a window, to suppress false initiations. Unmatched tracks increment time_since_update and are terminated when time_since_update > max_age. Duplicate removal prunes overlapping confirmed tracks with near-identical states, retaining higher-confidence and longer-lived hypotheses. Short-term occlusion recovery is handled through the three-stage cascade and history-based last-chance matching (Li et al., 8 Jul 2025).

The implementation used in the benchmark fixes YOLOX-X as detector and FastReID as the appearance subsystem, with SBS-S101 giving the best Re-ID performance in ablation. MFT25 comprises 15 videos, 48,066 frames, and 408,578 annotated boxes at 25 FPS, drawn from multi-species and diverse aquaculture settings, with capture setups including Canon EOS R6 and Sony $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$7M3 footage. Training and evaluation are kept consistent across compared SDE trackers, and the released resources include the dataset and code.

The reported computational profile is compatible with online deployment. For the six-dimensional state, UKF complexity per track is $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$8 for the Cholesky factorization of $$z_k = [c_x^{(\mathrm{meas})}, c_y^{(\mathrm{meas})}, w^{(\mathrm{meas})}, h^{(\mathrm{meas})}]^T.$$9, with $\Delta t = 1/25\,\mathrm{s}$0 propagation and update, and constant-time behavior in practice because $\Delta t = 1/25\,\mathrm{s}$1. FishIoU is $\Delta t = 1/25\,\mathrm{s}$2 per track-detection pair. Hungarian assignment is $\Delta t = 1/25\,\mathrm{s}$3 per cascade stage in the number of candidates, but the dual confidence cascades and gating reduce the effective candidate count. Per-track memory stores the state $\Delta t = 1/25\,\mathrm{s}$4, covariance $\Delta t = 1/25\,\mathrm{s}$5, last observation, and an optional embedding. The paper therefore characterizes SU-T as lightweight and suitable for online use, with real-time aquaculture monitoring identified as a target setting when detector and Re-ID are accelerated.

6. Empirical behavior, limitations, and later extensions

On MFT25, SU-T with Re-ID achieves HOTA $\Delta t = 1/25\,\mathrm{s}$6 ($\Delta t = 1/25\,\mathrm{s}$7), IDF1 $\Delta t = 1/25\,\mathrm{s}$8 ($\Delta t = 1/25\,\mathrm{s}$9), and MOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$0 ($$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$1). Without Re-ID, it reaches HOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$2 and IDF1 $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$3. The paper reports that SU-T outperforms strong baselines in association metrics. ByteTrack obtains HOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$4, IDF1 $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$5, and MOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$6; BoT-SORT reaches HOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$7 and IDF1 $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$8; OC-SORT reaches HOTA $$f(x_{k-1}) = \begin{bmatrix} c_x + v_x \Delta t \ c_y + v_y \Delta t \ v_x \ v_y \ w \ h \end{bmatrix}, \qquad h(x_k) = [c_x, c_y, w, h]^T.$$9 and IDF1 $\lambda = \alpha^2 (n+\kappa) - n,$0. TrackFormer has the highest MOTA at $\lambda = \alpha^2 (n+\kappa) - n,$1 but lower identity metrics, underscoring that MOTA and identity preservation do not coincide in this domain (Li et al., 8 Jul 2025).

Method	HOTA	IDF1
SU-T + Re-ID	34.067	44.643
ByteTrack	31.758	40.355
BoT-SORT	26.848	36.847
OC-SORT	25.017	34.620

The ablations isolate the contribution of each component. FishIoU improves over standard IoU, GIoU, and HMIoU. With no Re-ID, FishIoU yields HOTA $\lambda = \alpha^2 (n+\kappa) - n,$2 and IDF1 $\lambda = \alpha^2 (n+\kappa) - n,$3; with Re-ID, the same association family yields HOTA $\lambda = \alpha^2 (n+\kappa) - n,$4 and IDF1 $\lambda = \alpha^2 (n+\kappa) - n,$5. In motion-model comparisons, UKF outperforms KF, AKF, and STF when combined with FishIoU or HMIoU; the best reported setting is FishIoU plus Re-ID with UKF, giving HOTA $\lambda = \alpha^2 (n+\kappa) - n,$6 and IDF1 $\lambda = \alpha^2 (n+\kappa) - n,$7. Re-ID backbone analysis identifies SBS-S101 as the strongest option for identity preservation, while instance-batch normalization variants are “not consistently helpful.”

Performance varies sharply across sequences. PF-002 reaches HOTA $\lambda = \alpha^2 (n+\kappa) - n,$8 and IDF1 $\lambda = \alpha^2 (n+\kappa) - n,$9 with zero IDs, whereas SN-011 is explicitly identified as challenging, with HOTA $\alpha \in (10^{-3},1]$0 because of severe occlusions and erratic motion. The baseline also generalizes beyond underwater data: SU-T evaluated on MOT17 and MOT20 achieves HOTA $\alpha \in (10^{-3},1]$1 and $\alpha \in (10^{-3},1]$2, respectively.

The reported failure cases remain severe occlusions in dense schooling, low visibility and turbidity, strong lighting changes, and extreme non-linear maneuvers, which still cause identity switches and fragmentation. The paper lists several remedies: incorporating orientation or turn-rate into the state, using rotated boxes or instance masks with rotation- or shape-aware overlap, adding temporal appearance smoothing and occlusion-aware identity models, and pursuing domain-adaptive feature learning with underwater-specific augmentation and turbidity-invariant features.

A later paper on the Multi-Scaled Unscented Kalman Filter presents a concise technical synthesis specialized to a Scale-aware and Unscented Tracker, in which per-dimension scaling parameters $\alpha \in (10^{-3},1]$3 are assigned differently across state components while preserving the standard UKF moment-matching properties and order of accuracy. That work reports that MS-UKF retains zero-order mean, first-order mean, and second-order covariance matching, introduces negligible extra per-iteration cost relative to standard UKF, and improves TSTD by $\alpha \in (10^{-3},1]$4 in one nonlinear system and by $\alpha \in (10^{-3},1]$5 in another over the best standard UKF settings. This suggests a possible extension path for SU-T’s motion model—particularly for separating kinematic and scale-state spread control—rather than a component of the original MFT25 baseline (Levy et al., 6 Apr 2026).