FishIoU: Morphology-Aware Metric for Fish Tracking

• FishIoU is a specialized metric for underwater fish tracking that integrates a fish-specific central overlap, aspect-ratio and area consistency terms, and a scale-aware center-distance penalty.
• It addresses the shortcomings of standard IoU by emphasizing the head region and elongated body shapes of fish, thereby improving object association in non-linear motion contexts.
• When integrated into the SU-T tracker, FishIoU has shown measurable improvements in HOTA and IDF1 on the MFT25 dataset, demonstrating its practical impact on tracking accuracy.

Fish-Intersection-over-Union (FishIoU) is a spatial matching metric introduced in "When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking" for underwater multiple fish tracking. It augments standard Intersection-over-Union (IoU) with a fish-specific central-overlap term, aspect-ratio and area consistency terms, and a scale-aware center-distance penalty. In the same work, FishIoU is integrated into the Scale-aware and Unscented Tracker (SU-T) and evaluated on Multiple Fish Tracking Dataset 2025 (MFT25), where it is used to address elongated body shape, head-region salience, and non-linear motion patterns that are characteristic of fish in underwater scenes (Li et al., 8 Jul 2025).

1. Context within underwater multiple fish tracking

The underlying paper presents MFT25 as the first comprehensive dataset specifically designed for underwater multiple fish tracking, with 15 diverse video sequences, 408,578 meticulously annotated bounding boxes, and 48,066 frames. It also introduces SU-T, a specialized tracking framework featuring an Unscented Kalman Filter (UKF) optimized for non-linear fish swimming patterns and FishIoU matching for association (Li et al., 8 Jul 2025).

Within that framework, FishIoU is positioned as a response to a limitation of terrestrial MOT assumptions. Standard IoU treats all pixels in a box equally and assumes roughly compact, symmetric object shapes. The paper states that fish have elongated bodies and most of their distinguishing features, including eyes, mouth, and fins, concentrated near the head. It further notes that two long, thin, crossing fish can have high IoU yet actually be mismatched, and that drastic orientation changes between frames are not adequately captured by a simple overlap measure.

This establishes FishIoU as a morphology-aware association score rather than a generic box-overlap criterion. A plausible implication is that the metric is designed not merely to rank overlap quality, but to encode object-specific priors directly into the matching stage.

2. Formal mathematical definition

FishIoU is defined for two axis-aligned bounding boxes

$$B_1 = [x_1, y_1, x_2, y_2], \qquad B_2 = [x_1', y_1', x_2', y_2'].$$

Their widths, heights, centers, areas, and aspect ratios are

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$

$$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$

$$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$

The construction begins with the standard IoU:

$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$

A normalized center-distance penalty is then defined as

$$d_c = \frac{(c_{x_1}-c_{x_2})^2 + (c_{y_1}-c_{y_2})^2}{d_{\mathrm{diag}}^2},$$

where $d_{\mathrm{diag}}^2$ is the squared diagonal length of the minimal enclosing box of $B_1$ and $B_2$ (Li et al., 8 Jul 2025).

To encode fish morphology, the method defines an asymmetric central sub-box for each detection:

$$B_i^c = \bigl[ x_{1i} + \alpha w_i,\; y_{1i} + \beta h_i,\; x_{2i} - \gamma w_i,\; y_{2i} - \beta h_i \bigr], \quad i=1,2,$$

with empirically set constants

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$0

The corresponding central IoU is

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$1

Two additional consistency terms are included:

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$2

Small targets are down-weighted in the center-distance penalty through

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$3

The final metric is the weighted sum

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$4

with empirically chosen weights

$$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$5

3. Morphological assumptions and region parameterization

FishIoU is explicitly motivated by the claim that standard IoU is insufficient for fish because it assumes compact, symmetric object shapes, whereas fish are elongated and visually discriminative cues are concentrated near the head (Li et al., 8 Jul 2025).

The central design choice is the asymmetric central sub-box $$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$6. The insets are not symmetric along the horizontal extent: the left boundary is shifted by $$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$7 and the right boundary by $$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$8, with $$w_i = x_{2i} - x_{1i}, \qquad h_i = y_{2i} - y_{1i},$$9. The paper describes this as carving out a smaller "central" region biased toward the fish's head via the asymmetric inset parameters $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$0, $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$1, and $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$2. The associated cIoU therefore emphasizes overlap in a region intended to better reflect fish-specific morphology.

The paper also states that no rotated boxes or ellipses are used. Each fish is represented by a standard axis-aligned rectangle

$$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$3

The only additional shape information is the central sub-box

$$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$4

This is presented as sufficient to emphasize the morphological head region without incurring the complexity of full oriented bounding boxes or keypoint models. A common misconception is therefore that FishIoU depends on rotated-box geometry or articulated pose estimation; the paper states the opposite.

4. Computational procedure

The paper provides a step-by-step computation of FishIoU for two detections $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$5 and $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$6 (Li et al., 8 Jul 2025).

1. Compute area of intersection and union to obtain standard IoU.
2. Compute centers $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$7 and the diagonal penalty $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$8.
3. Define the central sub-boxes $$(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),$$9 via $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$0, $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$1, and $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$2.
4. Compute

$$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$3

1. Compute the aspect-ratio term $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$4.
2. Compute the area-ratio term $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$5.
3. Compute the scale-adjustment factor

$$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$6

1. Combine the terms with the specified weights:

$$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$7

The ordering of these steps makes clear that FishIoU preserves the conventional IoU computation as a base term and then introduces morphology- and scale-aware corrections. This suggests that the metric is intended as an extension of box overlap rather than a replacement of overlap geometry by an unrelated distance measure.

5. Use in SU-T association

In SU-T, FishIoU is used as the cost matrix in all three stages of the Hungarian assignment. After predicting each track's box $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$8 via UKF, the tracker extracts high-confidence detections $$a_i = w_i \cdot h_i, \qquad r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.$$9 and forms the cost matrix

$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$0

for $$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$1 (Li et al., 8 Jul 2025).

When a Re-ID module is enabled, the paper states that the cost matrix is optionally updated as

$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$2

The assignment is then solved as

$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$3

after which matched tracks are updated.

The same procedure is repeated in two further cascaded stages with lower-confidence detections and "last-chance" detections, always using FishIoU to compute the spatial-only cost. In this role, FishIoU functions as the primary geometric compatibility measure for track-detection association, while Re-ID, when enabled, contributes an additional embedding-based term.

6. Empirical behavior on MFT25

The paper compares Center distance, IoU, CIoU, DIoU, HMIoU, GIoU, and FishIoU in Table VI. The key reported rows are as follows (Li et al., 8 Jul 2025).

Method	HOTA $$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$4	IDF1 $$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$5
IoU	32.790	40.098
GIoU	32.885	39.957
HMIoU	32.258	38.421
FishIoU	33.351	41.717
FishIoU$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$6	33.581	43.268

Here, FishIoU$$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$7 denotes FishIoU + Re-ID. The paper states that replacing standard IoU with FishIoU yields an approximately $$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$8 point absolute gain in HOTA and an approximately $$\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.$$9 point gain in IDF1. It further reports that ablations in Tables VII–IX show that FishIoU consistently outperforms other IoU variants under all motion models, namely KF, AKF, STF, and UKF.

The best combination is reported as UKF + FishIoU + Re-ID, which achieves 34.1 HOTA and 44.6 IDF1 on the MFT25 test set. In the broader framing of the paper, these results are presented alongside the claim that the experiments reveal fundamental differences between fish tracking and terrestrial object tracking scenarios. A plausible implication is that morphology-aware association contributes materially to those differences, especially when combined with a motion model tailored to non-linear fish swimming patterns.