Motion-Adaptive IoU (Mo-IoU) in Object Tracking
- Mo-IoU is a dynamic adaptation of standard IoU that adjusts spatial expansion and height alignment based on motion cues in multi-object tracking.
- It combines Expansion IoU (EIoU) and Height IoU (HIoU) to better handle occlusions, fast motion, and scale inconsistencies.
- Empirical results from frameworks like MeMoSORT and HIT show that Mo-IoU significantly improves tracking accuracy and reduces identity switches.
Searching arXiv for the cited Mo-IoU papers and related context. Motion-adaptive IoU (Mo-IoU) denotes an IoU-based association metric for multi-object tracking that modifies overlap computation in response to motion characteristics rather than treating all track–detection pairs with a rigid geometric criterion. In the MeMoSORT formulation, Mo-IoU is defined as the product of an Expansion IoU (EIoU) term and a Height IoU (HIoU) term, with both terms modulated by motion-derived parameters estimated from track history (Wang et al., 13 Aug 2025). In the HIT framework, the same label refers more broadly to IoU computed after motion- and scale-adaptive preprocessing, specifically through size-consistent expansion, camera-motion compensation, and motion bootstrapping within a hierarchical tracklet association procedure (Du et al., 2024). Across these formulations, the common objective is to improve association robustness under fast motion, camera motion, occlusion, and scale inconsistency while retaining the computational simplicity of IoU-based matching.
1. Concept and problem setting
Mo-IoU arises in the context of tracking-by-detection, where detections must be associated across frames or across short tracklets. Standard IoU between two axis-aligned bounding boxes and is
This measure is effective when localization is accurate and displacement between corresponding instances is modest. However, both MeMoSORT and HIT identify conditions in which rigid IoU becomes unreliable: large post-occlusion displacement, detector jitter, fast target motion, strong ego-motion, and the disproportionate IoU penalty incurred by small objects for identical pixel errors (Wang et al., 13 Aug 2025, Du et al., 2024).
In MeMoSORT, Mo-IoU is introduced for online, real-time multi-person tracking in human-dominant scenes, where complex motion and severe occlusions degrade both filtering and association (Wang et al., 13 Aug 2025). In HIT, Mo-IoU is embedded in a unified hierarchical tracking framework that links short and long temporal gaps using only IoU-based association, without heavy appearance models or learning-based association modules (Du et al., 2024).
A plausible implication is that “Mo-IoU” is best understood not as a single universally fixed formula, but as a family of motion-conditioned IoU constructions whose defining feature is adaptive overlap evaluation rather than rigid geometric intersection alone.
2. MeMoSORT formulation
In MeMoSORT, a detection at time is represented as
and each track’s Memory-assisted Kalman filter (MeKF) produces a prediction
The coordinates are box centers with width and height (Wang et al., 13 Aug 2025). MeMoSORT defines Mo-IoU as
$\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$
Here, 0 controls spatial expansion and 1 controls the emphasis on vertical-height similarity (Wang et al., 13 Aug 2025).
The EIoU component expands both boxes symmetrically before computing IoU:
2
with expanded boxes
3
4
This factor 5 recovers standard IoU when 6, introduces symmetric scale expansion around the box center, and increases overlap likelihood under large position errors typical of fast motion and camera motion (Wang et al., 13 Aug 2025).
The HIoU component measures vertical overlap as a one-dimensional IoU on the 7-axis and raises it to 8:
9
where 0 is the vertical intersection height between the two boxes. If the top and bottom extents of the prediction are
1
and those of the detection are
2
then
3
The base term is the one-dimensional IoU on the vertical axis, and the exponent modulates the impact of height similarity depending on temporal reliability (Wang et al., 13 Aug 2025).
MeMoSORT emphasizes that product fusion imposes a joint constraint: high Mo-IoU requires both sufficient spatial overlap, after adaptive expansion, and compatible vertical alignment and height. No extra weighting coefficients are added; adaptivity is carried entirely by 4 and 5 (Wang et al., 13 Aug 2025).
3. Motion adaptivity and parameterization
The defining feature of MeMoSORT’s Mo-IoU is the Motion-Adaptive Technique (MAT), which sets 6 and 7 from normalized motion speeds derived from track history:
8
The normalized speeds are
9
The dot denotes a temporal derivative estimated from track states by finite differences along the trajectory, and normalization by width and height yields scale-independent speeds (Wang et al., 13 Aug 2025).
The default hyperparameters used in MeMoSORT experiments are reported as 0, 1, 2, and 3, with dataset-dependent thresholds selected from percentiles of the training distribution of normalized speeds. For DanceTrack, 4 at the 50th percentile and 5 at the 70th percentile; for SportsMOT, 6 at the 50th percentile and 7 at the 70th percentile (Wang et al., 13 Aug 2025).
The paper interprets 8 as “spatial tolerance” and 9 as “height reliability.” Faster center motion leads to larger expansion to counter larger prediction–detection misalignments, while faster height change leads to a smaller exponent to temper height influence. Discrete levels are used to avoid heavy computation from arbitrary fractional exponents and to ensure real-time throughput (Wang et al., 13 Aug 2025).
This suggests that MeMoSORT’s adaptivity is explicitly local and trajectory-conditioned: it depends on recently observed motion magnitude rather than on a static dataset-wide rule or on covariance estimates from the motion filter.
4. Association pipeline and coupling with motion filtering
MeMoSORT’s association is inspired by Deep OC-SORT and proceeds in two confidence-stratified stages (Wang et al., 13 Aug 2025). The inputs are track predictions from MeKF, detections with confidence scores, and optional ReID cosine distances from an appearance encoder. In the high-score stage, detections with confidence at least 0 are selected, Mo-IoU is computed for each track–detection pair using MAT-derived 1 and 2, and Mo-IoU is combined with ReID to form the cost matrix for Hungarian matching, with the precise weighting following Deep OC-SORT defaults (Wang et al., 13 Aug 2025).
In the low-score stage, remaining detections with confidence in 3 are matched using standard IoU, without height emphasis or adaptive expansion. The stated reason is to reduce the chance of associating spurious low-confidence detections via aggressive expansions (Wang et al., 13 Aug 2025). After association, matched tracks are updated through MeKF using the associated detection, while unmatched tracks and detections are handled according to tracking-by-detection conventions such as new-track creation and grace-period retention (Wang et al., 13 Aug 2025).
Mo-IoU is tightly coupled to MeKF in two respects. First, it uses 4, the predicted state from MeKF’s State Prediction Gate, as one input to the IoU computations. Improved MeKF predictions therefore directly improve spatial matching. Second, MAT computes 5 and 6 from the trajectory history maintained by MeKF, so the motion filter supplies the cues that determine adaptation (Wang et al., 13 Aug 2025). The paper explicitly notes that Mo-IoU does not use MeKF uncertainty covariances; adaptivity is driven by observed filtered motion magnitudes.
A common misconception would be to treat Mo-IoU in MeMoSORT as merely an expanded IoU. That description is incomplete. The metric is not only expansion-based; it also introduces a multiplicative height-aware term and a motion-conditioned parameter schedule, both of which are central to its intended behavior under occlusion and height-stable interactions (Wang et al., 13 Aug 2025).
5. The HIT interpretation of Mo-IoU
In HIT, Mo-IoU has a broader meaning. Rather than a single closed-form product such as EIoU times HIoU, it denotes IoU computed after three consistency-oriented adaptations: consistent-IoU for small objects, consistent-camera for ego-motion, and consistent-motion for initializing motion cues in the first hierarchy (Du et al., 2024).
HIT’s tracker operates on tracklet intervals. A tracklet 7 is a temporally contiguous sequence of detections, and only tracklet pairs satisfying 8 are eligible for association at hierarchy 9, where 0 and the threshold set 1 increases with 2 (Du et al., 2024). At each hierarchy, candidate generation is followed by bidirectional motion prediction via a standard linear Kalman filter, computation of IoU-based similarity using predicted and true boxes after motion-adaptive mechanisms, gating with a unified IoU threshold 3, Hungarian matching with cost 4, and tracklet merging (Du et al., 2024).
The size-consistent component addresses the fact that a fixed localization error can yield substantially lower IoU for small objects than for large ones. If both widths are below the threshold 5, both boxes are rescaled by a shared factor
6
with 7, and IoU is computed on the expanded boxes (Du et al., 2024).
The camera-motion component estimates per-frame global translation from initial adjacent-frame matches in hierarchy 8. If the average IoU over those matches, denoted 9, is below 0, then sequence-level camera motion is considered significant. Per-frame offsets are then estimated as
1
and compensated boxes are defined as 2 for subsequent hierarchies (Du et al., 2024).
The consistent-motion component addresses the absence of motion history in the first hierarchy, where each tracklet has length one. After running adjacent-frame IoU association, instantaneous velocities are derived from matched pairs and used to initialize tracklet velocities for later Kalman prediction (Du et al., 2024).
HIT summarizes its effective Mo-IoU similarity as
3
with gating by 4 and cost 5 (Du et al., 2024). In this framework, Mo-IoU remains “IoU-only in spirit,” but the coordinate frame, scale, and motion initialization are adapted before overlap is computed.
6. Empirical behavior and reported impact
MeMoSORT reports ablation results on the DanceTrack validation set that isolate the effect of Mo-IoU. A baseline without MeKF, Mo-IoU, or ReID achieves HOTA 6 at 7 FPS; adding MeKF yields HOTA 8 at 9 FPS; adding Mo-IoU on top of MeKF yields HOTA 0 at 1 FPS; and adding ReID further reaches HOTA 2 at 3 FPS (Wang et al., 13 Aug 2025). The paper states that Mo-IoU alone on top of MeKF yields a gain of 4 HOTA, indicating a strong contribution to association robustness.
A second ablation in MeMoSORT compares IoU variants with MeKF fixed. Standard IoU yields HOTA 5, AssA 6, IDF1 7, MOTA 8, and DetA 9; EIoU reaches HOTA 0; HMIoU reaches HOTA 1; HA-EIoU reaches HOTA 2; and adaptive Mo-IoU reaches HOTA 3, AssA 4, IDF1 5, MOTA 6, and DetA 7 (Wang et al., 13 Aug 2025). The reported margins are 8 HOTA over rigid IoU and 9 over HA-EIoU, which the paper interprets as evidence that adaptivity to motion is crucial.
On full benchmarks, MeMoSORT reports on the DanceTrack test set HOTA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$0, AssA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$1, DetA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$2, and IDF1 $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$3, and on SportsMOT HOTA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$4, AssA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$5, and IDF1 $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$6 (Wang et al., 13 Aug 2025). The paper attributes the higher association metrics, particularly under occlusion and complex interactions, to the proposed association mechanism.
HIT likewise reports systematic gains from its motion-adaptive IoU mechanisms. On MOT17 validation, the baseline without CI, CC, or CM has HOTA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$7, MOTA $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$8, and IDF1 $\mathrm{Mo\mbox{-}IoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t,q_t) = \mathrm{EIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,p_t) \times \mathrm{HIoU}(\hat{\mathbf{b}}'_t,\widetilde{\mathbf{b}}_t,q_t).$9; adding CI yields HOTA 00 and IDF1 01; adding CC yields HOTA 02, MOTA 03, and IDF1 04; adding CM yields HOTA 05, MOTA 06, and IDF1 07; and combining all three yields HOTA 08, MOTA 09, and IDF1 10 (Du et al., 2024). HIT further reports higher association quality than offline ByteTrack on MOT17 test, with HOTA 11, AssA 12, IDF1 13, and IDSW 14, compared with ByteTrack’s AssA 15 and IDSW 16 (Du et al., 2024).
The following comparison summarizes the two principal Mo-IoU instantiations described in the cited literature.
| Framework | Core Mo-IoU mechanism | Reported emphasis |
|---|---|---|
| MeMoSORT (Wang et al., 13 Aug 2025) | Product of EIoU and HIoU with motion-derived 17 | Online, real-time multi-person tracking |
| HIT (Du et al., 2024) | IoU after size-consistent expansion, camera-motion compensation, and motion bootstrapping | Hierarchical tracklet association without appearance cues |
A plausible implication is that the performance gains stem less from abandoning IoU than from conditioning IoU on the failure modes most responsible for association error in a given tracker design.
7. Practical interpretation, limitations, and relation to neighboring IoU variants
Mo-IoU is designed to retain the favorable properties of IoU-based association—simplicity, interpretability, and low computational cost—while compensating for known deficiencies of rigid overlap criteria. MeMoSORT’s implementation details quantify the overhead: on DanceTrack validation, throughput drops from 18 FPS with MeKF alone to 19 FPS with MeKF plus Mo-IoU, with the added cost attributed primarily to box scaling and one-dimensional-IoU exponentiation; the method remains lighter than a full appearance pipeline (Wang et al., 13 Aug 2025). HIT similarly characterizes its design as appearance-free and training-free, with overall complexity 20 under interval-gated hierarchies, where cost-matrix construction is 21 and Hungarian assignment is 22 (Du et al., 2024).
The two frameworks also articulate boundary conditions. MeMoSORT notes that under extreme scale changes, height becomes unreliable and MAT lowers the influence of HIoU by selecting 23; in very dense crowds of similar heights, one should avoid overemphasizing HIoU; and with detector jitter or large camera motion, increasing expansion can compensate for spatial noise (Wang et al., 13 Aug 2025). HIT notes that CI can improve IDF1 while potentially increasing false positives, slightly hurting MOTA; that CC is particularly useful for UAV or other strong ego-motion scenarios; and that interval priors bridge long gaps more effectively than sliding windows (Du et al., 2024).
Mo-IoU also sits in a broader lineage of modified IoU criteria. MeMoSORT compares standard IoU, EIoU, HMIoU, HA-EIoU, and adaptive Mo-IoU, presenting Mo-IoU as the strongest among these tested variants when paired with MeKF (Wang et al., 13 Aug 2025). HIT notes that for MOT17 and DanceTrack persons, a height-modulated IoU from Hybrid-SORT is employed for association, although HIT remains IoU-only in spirit (Du et al., 2024). This indicates that height-aware overlap and scale-aware overlap are not unique to Mo-IoU, but MeMoSORT’s contribution lies in explicitly motion-conditioned parameter selection, while HIT’s contribution lies in integrating several adaptive IoU corrections into a unified hierarchical framework.
A frequent misconception is that IoU-based trackers are necessarily rigid and incapable of handling complex motion without appearance features. The cited work complicates that claim. HIT shows that IoU-only association, if equipped with interval priors and motion-adaptive consistency mechanisms, can remain competitive with appearance- and learning-based trackers on several benchmarks (Du et al., 2024). MeMoSORT shows that even in an online real-time tracker, substantial gains can be obtained by adapting overlap geometry to motion rather than relying solely on a conventional Kalman filter and rigid IoU (Wang et al., 13 Aug 2025).
In summary, Motion-adaptive IoU refers to a class of association strategies in which the geometric overlap test is conditioned on estimated motion, scale behavior, or camera dynamics. In MeMoSORT, this idea takes the form of a multiplicative metric combining adaptive expansion and height-aware alignment, parameterized by motion-derived discrete states. In HIT, it denotes IoU after consistency-driven motion and scale adaptation within a hierarchical interval-based tracker. Both formulations preserve IoU as the core similarity measure while altering the conditions under which overlap is evaluated, thereby improving association stability under the motion regimes that most often cause identity switches and target loss (Wang et al., 13 Aug 2025, Du et al., 2024).