---
title: 'FishIoU: Morphology-Aware Metric for Fish Tracking'
url: https://www.emergentmind.com/topics/fish-intersection-over-union-fishiou
type: topic
---

# FishIoU: Morphology-Aware Metric for Fish Tracking

Fish-Intersection-over-Union (FishIoU) is a spatial matching metric introduced in "When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking" for underwater multiple fish tracking. It augments standard Intersection-over-Union (IoU) with a fish-specific central-overlap term, aspect-ratio and area consistency terms, and a scale-aware center-distance penalty. In the same work, FishIoU is integrated into the Scale-aware and Unscented Tracker (SU-T) and evaluated on Multiple Fish Tracking Dataset 2025 (MFT25), where it is used to address elongated body shape, head-region salience, and non-linear motion patterns that are characteristic of fish in underwater scenes [2507.06400].

## 1. Context within underwater multiple fish tracking

The underlying paper presents MFT25 as the first comprehensive dataset specifically designed for underwater multiple fish tracking, with 15 diverse video sequences, 408,578 meticulously annotated bounding boxes, and 48,066 frames. It also introduces SU-T, a specialized tracking framework featuring an Unscented Kalman Filter (UKF) optimized for non-linear fish swimming patterns and FishIoU matching for association [2507.06400].

Within that framework, FishIoU is positioned as a response to a limitation of terrestrial MOT assumptions. Standard IoU treats all pixels in a box equally and assumes roughly compact, symmetric object shapes. The paper states that fish have elongated bodies and most of their distinguishing features, including eyes, mouth, and fins, concentrated near the head. It further notes that two long, thin, crossing fish can have high IoU yet actually be mismatched, and that drastic orientation changes between frames are not adequately captured by a simple overlap measure.

This establishes FishIoU as a morphology-aware association score rather than a generic box-overlap criterion. A plausible implication is that the metric is designed not merely to rank overlap quality, but to encode object-specific priors directly into the matching stage.

## 2. Formal mathematical definition

FishIoU is defined for two axis-aligned bounding boxes
$$
B_1 = [x_1, y_1, x_2, y_2], \qquad
B_2 = [x_1', y_1', x_2', y_2'].
$$

Their widths, heights, centers, areas, and aspect ratios are
$$
w_i = x_{2i} - x_{1i}, \qquad
h_i = y_{2i} - y_{1i},
$$
$$
(c_{x_i}, c_{y_i}) = \left(\frac{x_{1i}+x_{2i}}{2}, \frac{y_{1i}+y_{2i}}{2}\right),
$$
$$
a_i = w_i \cdot h_i, \qquad
r_i = \frac{w_i}{h_i}, \qquad i \in \{1,2\}.
$$

The construction begins with the standard IoU:
$$
\mathrm{IoU} = \frac{\lvert B_1 \cap B_2 \rvert}{\lvert B_1 \cup B_2 \rvert}.
$$

A normalized center-distance penalty is then defined as
$$
d_c = \frac{(c_{x_1}-c_{x_2})^2 + (c_{y_1}-c_{y_2})^2}{d_{\mathrm{diag}}^2},
$$
where $d_{\mathrm{diag}}^2$ is the squared diagonal length of the minimal enclosing box of $B_1$ and $B_2$ [2507.06400].

To encode fish morphology, the method defines an asymmetric central sub-box for each detection:
$$
B_i^c =
\bigl[
x_{1i} + \alpha w_i,\;
y_{1i} + \beta h_i,\;
x_{2i} - \gamma w_i,\;
y_{2i} - \beta h_i
\bigr], \quad i=1,2,
$$
with empirically set constants
$$
\alpha = 0.15, \qquad \beta = 0.3, \qquad \gamma = 0.25.
$$

The corresponding central IoU is
$$
\mathrm{cIoU} =
\frac{\lvert B_1^c \cap B_2^c \rvert}{\lvert B_1^c \cup B_2^c \rvert}.
$$

Two additional consistency terms are included:
$$
\alpha_r = \frac{\min(r_1,r_2)}{\max(r_1,r_2)},
\qquad
\alpha_a = \frac{\min(a_1,a_2)}{\max(a_1,a_2)}.
$$

Small targets are down-weighted in the center-distance penalty through
$$
s = 1 - \exp\!\bigl(-\min(a_1,a_2)/1000\bigr).
$$

The final metric is the weighted sum
$$
\mathrm{FishIoU}
=
\omega_1\,\mathrm{IoU}
+\omega_2\,\mathrm{cIoU}
+\omega_3\,\alpha_r
+\omega_4\,\alpha_a
-\omega_5\,s\,d_c,
$$
with empirically chosen weights
$$
\omega_1=1.0,\quad
\omega_2=0.3,\quad
\omega_3=0.1,\quad
\omega_4=0.2,\quad
\omega_5=0.4.
$$

## 3. Morphological assumptions and region parameterization

FishIoU is explicitly motivated by the claim that standard IoU is insufficient for fish because it assumes compact, symmetric object shapes, whereas fish are elongated and visually discriminative cues are concentrated near the head [2507.06400].

The central design choice is the asymmetric central sub-box $B^c$. The insets are not symmetric along the horizontal extent: the left boundary is shifted by $\alpha w$ and the right boundary by $\gamma w$, with $\alpha \neq \gamma$. The paper describes this as carving out a smaller "central" region biased toward the fish's head via the asymmetric inset parameters $\alpha$, $\beta$, and $\gamma$. The associated cIoU therefore emphasizes overlap in a region intended to better reflect fish-specific morphology.

The paper also states that no rotated boxes or ellipses are used. Each fish is represented by a standard axis-aligned rectangle
$$
B = [x_1, y_1, x_2, y_2].
$$
The only additional shape information is the central sub-box
$$
B^c = [x_1+\alpha w,\; y_1+\beta h,\; x_2-\gamma w,\; y_2-\beta h].
$$

This is presented as sufficient to emphasize the morphological head region without incurring the complexity of full oriented bounding boxes or keypoint models. A common misconception is therefore that FishIoU depends on rotated-box geometry or articulated pose estimation; the paper states the opposite.

## 4. Computational procedure

The paper provides a step-by-step computation of FishIoU for two detections $B_1$ and $B_2$ [2507.06400].

1. Compute area of intersection and union to obtain standard IoU.  
2. Compute centers $(c_{x_i}, c_{y_i})$ and the diagonal penalty $d_c$.  
3. Define the central sub-boxes $B_i^c$ via $\alpha$, $\beta$, and $\gamma$.  
4. Compute
   $$
   \mathrm{cIoU} = \frac{|B_1^c \cap B_2^c|}{|B_1^c \cup B_2^c|}.
   $$
5. Compute the aspect-ratio term $\alpha_r$.  
6. Compute the area-ratio term $\alpha_a$.  
7. Compute the scale-adjustment factor
   $$
   s = 1-\exp\!\bigl(-\min(a_1,a_2)/1000\bigr).
   $$
8. Combine the terms with the specified weights:
   $$
   \mathrm{FishIoU}
   = \omega_1\,\mathrm{IoU}
   + \omega_2\,\mathrm{cIoU}
   + \omega_3\,\alpha_r
   + \omega_4\,\alpha_a
   - \omega_5\,s\,d_c.
   $$

The ordering of these steps makes clear that FishIoU preserves the conventional IoU computation as a base term and then introduces morphology- and scale-aware corrections. This suggests that the metric is intended as an extension of box overlap rather than a replacement of overlap geometry by an unrelated distance measure.

## 5. Use in SU-T association

In SU-T, FishIoU is used as the cost matrix in all three stages of the Hungarian assignment. After predicting each track's box $\hat B_j$ via UKF, the tracker extracts high-confidence detections $\mathcal D_{\rm high}$ and forms the cost matrix
$$
\mathbf C_{ij} = \mathrm{FishIoU}(d_i,\hat B_j)
$$
for $d_i \in \mathcal D_{\rm high}$ [2507.06400].

When a Re-ID module is enabled, the paper states that the cost matrix is optionally updated as
$$
\mathbf C \leftarrow \omega_1\,\mathbf C + \omega_2\,\mathrm{EmbeddingDistance}.
$$
The assignment is then solved as
$$
\mathcal M_1,\;\mathcal U_{\mathcal D},\;\mathcal U_{\mathcal T}
\gets
\mathrm{Hungarian}(-\mathbf C),
$$
after which matched tracks are updated.

The same procedure is repeated in two further cascaded stages with lower-confidence detections and "last-chance" detections, always using FishIoU to compute the spatial-only cost. In this role, FishIoU functions as the primary geometric compatibility measure for track-detection association, while Re-ID, when enabled, contributes an additional embedding-based term.

## 6. Empirical behavior on MFT25

The paper compares Center distance, IoU, CIoU, DIoU, HMIoU, GIoU, and FishIoU in Table VI. The key reported rows are as follows [2507.06400].

| Method | HOTA $\uparrow$ | IDF1 $\uparrow$ |
|---|---:|---:|
| IoU | 32.790 | 40.098 |
| GIoU | 32.885 | 39.957 |
| HMIoU | 32.258 | 38.421 |
| FishIoU | 33.351 | 41.717 |
| FishIoU$^\dagger$ | 33.581 | 43.268 |

Here, FishIoU$^\dagger$ denotes FishIoU + Re-ID. The paper states that replacing standard IoU with FishIoU yields an approximately $0.56$ point absolute gain in HOTA and an approximately $1.62$ point gain in IDF1. It further reports that ablations in Tables VII–IX show that FishIoU consistently outperforms other IoU variants under all motion models, namely KF, AKF, STF, and UKF.

The best combination is reported as UKF + FishIoU + Re-ID, which achieves 34.1 HOTA and 44.6 IDF1 on the MFT25 test set. In the broader framing of the paper, these results are presented alongside the claim that the experiments reveal fundamental differences between fish tracking and terrestrial object tracking scenarios. A plausible implication is that morphology-aware association contributes materially to those differences, especially when combined with a motion model tailored to non-linear fish swimming patterns.

Source: https://www.emergentmind.com/topics/fish-intersection-over-union-fishiou