AdvTraj: Adversarial ID-Transfer in MOT

• AdvTraj is an ID-transfer attack that manipulates tracking-by-detection MOT systems by transferring the attacker’s ID to a target without perturbing the detection stage.
• It leverages motion prediction vulnerabilities by optimizing physical trajectories using Kalman filtering and DIoU cost minimization in an online, real-world setting.
• Empirical results demonstrate high success rates across various trackers, highlighting a critical vulnerability in the association phase of MOT for autonomous and surveillance applications.

Searching arXiv for the exact AdvTraj paper and closely related trajectory-adversarial work for contextual support. AdvTraj is an adversarial method for tracking-by-detection multi-object tracking (MOT) in which an attacker uses adversarial trajectories to transfer its own tracker ID to a target object without attacking object detection. In the exact formulation introduced in “Physical ID-Transfer Attacks against Multi-Object Tracking via Adversarial Trajectory” (Wang et al., 1 Dec 2025), the attack is online and physical: the adversary perturbs only its own motion in the scene so as to exploit weaknesses in motion prediction and bipartite association, causing the tracker to preserve the attacker’s historical identity while reassigning it to the target. This distinguishes AdvTraj from adversarial methods that perturb pixels, detections, or detector outputs, and places it in a broader trajectory-adversarial literature that includes physically realistic trajectory attacks on downstream prediction systems such as AdvDO (Cao et al., 2022).

1. Definition and scope

AdvTraj is defined as an ID-transfer attack on tracking-by-detection MOT systems. The attacker does not seek merely to induce a generic ID switch; rather, the goal is to make the tracker reassign the attacker’s original ID to a target object while the attacker itself loses that ID (Wang et al., 1 Dec 2025). The paper formalizes this objective through the assignment function $f(O_i^t)$, where $O_a$ denotes the attacker and $O_b$ the target. Initially,

$$f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,$$

and after the attack time $\tau$,

$$f(O_a^t)\neq ID_a,\; f(O_b^t)=ID_a,\quad \forall t\ge \tau.$$

This notion of ID transfer is more specific than the broader notion of an ID switch. A generic switch can simply destroy or reinitialize identities. AdvTraj instead preserves the attacker’s historical identity as a live track and transfers it to the target, which is especially consequential for surveillance and autonomous-driving stacks that consume trajectory histories for downstream inference (Wang et al., 1 Dec 2025).

The method is explicitly framed as the first online and physical ID-manipulation attack against tracking-by-detection MOT. Its novelty lies in isolating the association stage as the attack surface. In this respect, AdvTraj differs sharply from prior MOT attacks that focused on detector manipulation, digital perturbation, adversarial patches, or offline video tampering. The same emphasis on trajectory realism appears in adversarial work on motion forecasting, where AdvDO studies physically plausible perturbations of observed trajectories rather than unconstrained feature-space attacks (Cao et al., 2022). A plausible implication is that AdvTraj and AdvDO expose a common systems vulnerability: trajectory-processing modules can be destabilized through feasible motion manipulation alone.

2. Threat model and attack objective

The victim setting is standard tracking-by-detection MOT. At time $t$, the detector outputs a detection state

$\mathbf{d}_i=(x_1,y_1,x_2,y_2)_i,$

and the tracker maintains a latent state

$\mathbf{x}=(u,v,s,r,\dot{u},\dot{v},\dot{s})^T,$

where $(u,v)$ is the bounding-box center, $O_a$0 is scale, $O_a$1 is aspect ratio, and $O_a$2 are first-order derivatives (Wang et al., 1 Dec 2025). Motion prediction is performed with a Kalman filter, with prior estimate

$O_a$3

The attacker is a physical object visible to the camera, such as a pedestrian or a vehicle, and controls only its own movement. The attacker is assumed to know the camera location and viewing angle and to have access to the target’s trajectory. The attacker and target are assumed to belong to the same class so that their detections are comparable in size. The attack therefore targets the association logic under physically bounded motion rather than the detector itself (Wang et al., 1 Dec 2025).

The paper distinguishes two attacker classes. $O_a$4 is an optimized adversary, such as an autonomous robot or vehicle, capable of precise online optimization and control. $O_a$5 is a heuristic adversary, such as a human walker or driver, who instead performs simple universal maneuvers. This distinction is central because the full AdvTraj optimization is designed for real-time receding execution, whereas the heuristic version is intended to be human-executable in ordinary scenarios (Wang et al., 1 Dec 2025).

The attack objective is expressed through sufficient conditions on association costs. Let $O_a$6 be the distance between predicted and detected boxes. The paper gives the following conditions for ID transfer: $O_a$7 The first inequality enforces that the swapped matching is cheaper than the correct one; the latter two exclude interference from other objects. Under the paper’s assumptions, the attack largely focuses on driving the system toward $O_a$8 (Wang et al., 1 Dec 2025).

3. Attack mechanism and optimization

The core idea is to manipulate motion prediction inertia. Tracking-by-detection pipelines commonly combine Kalman-filter motion prediction with IoU-style cost computation and Hungarian matching. AdvTraj exploits the fact that abruptly non-linear motion can create a mismatch between an attacker’s true position and the tracker’s predicted position, thereby making a swapped assignment globally cheaper than the correct one (Wang et al., 1 Dec 2025).

The physical optimization is posed over the attacker’s ground-plane position: $O_a$9 This objective minimizes the swapped assignment cost while enforcing a bounded physical displacement per step (Wang et al., 1 Dec 2025).

Because the attacker controls a 3D body but the tracker consumes 2D detections, the paper introduces a differentiable world-to-image projection. Let

$O_b$0

be the 3D bounding-box vertices and $O_b$1 the camera projection matrix. The projected 2D box is

$O_b$2

with

$O_b$3

The attacker’s control variable is a ground-plane displacement

$O_b$4

applied as $O_b$5 with constant height (Wang et al., 1 Dec 2025).

The differentiable adversarial objective is

$O_b$6

Here the first term makes the attacker’s current detection resemble the target’s current predicted state, and the second term makes the attacker’s next predicted track state resemble the target’s next predicted state (Wang et al., 1 Dec 2025). This two-term structure is essential: minimizing only the first term would not account for the global Hungarian assignment.

For optimization, the paper uses DIoU rather than raw IoU because IoU has zero gradient when boxes do not overlap: $O_b$7 The victim tracker remains unchanged; DIoU is used only as a differentiable surrogate during attack generation (Wang et al., 1 Dec 2025).

Algorithmically, AdvTraj operates online. At each frame it predicts attacker and target tracker priors, checks whether the swapped cost still exceeds the normal cost, and if so performs a small number of gradient steps on $O_b$8, clips $O_b$9 to the admissible range, and physically executes the resulting movement. The reported implementation uses Adam with learning rate $$f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,$$0, $$f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,$$1, and $$f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,$$2, corresponding to a walker speed limit of $$f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,$$3 (Wang et al., 1 Dec 2025).

4. Victim trackers and empirical results

AdvTraj is optimized in a white-box setting against SORT, then evaluated for transfer against ByteTrack, OC-SORT, Deep OC-SORT, BoT-SORT, and StrongSORT. These trackers share the design principles that make transfer plausible: Kalman-filter motion prediction, IoU-style association, and Hungarian matching, with some methods additionally using ReID features (Wang et al., 1 Dec 2025).

Tracker	Motion model	ReID
SORT	Std. Kalman Filter	N
ByteTrack	Std. Kalman Filter	N (default best model)
OC-SORT	Std. Kalman Filter	N
Deep OC-SORT	Std. Kalman Filter	Y
BoT-SORT	Std. Kalman Filter	Y
StrongSORT	NSA Kalman Filter	Y

In CARLA simulation, white-box attacks against SORT achieve 100% success rate in various scenarios, including pedestrian surveillance and autonomous-driving settings (Wang et al., 1 Dec 2025). Transfer is also strong. In pedestrian surveillance, the reported attack success rates are 89% for ByteTrack, 92% for OC-SORT, 93% for Deep OC-SORT, 74% for BoT-SORT, and 84% for StrongSORT. In AD-perpendicular settings, the corresponding rates are 45%, 69%, 74%, 78%, and 77%. In AD-parallel settings they are 66%, 69%, 80%, 79%, and 85% (Wang et al., 1 Dec 2025).

These results support the paper’s central claim that trajectories optimized against SORT transfer because many SOTA MOT algorithms share common association logic. The attack is therefore not merely exploiting an implementation bug in SORT; it is exploiting a structural weakness in the motion-association stage (Wang et al., 1 Dec 2025).

The paper also reports vehicle-surveillance results in the appendix. In CARLA, transfer success rates are 83% for SORT, 74% for ByteTrack, 80% for OC-SORT, 83% for Deep OC-SORT, 56% for BoT-SORT, and 40% for StrongSORT. A plausible implication is that the phenomenon is not limited to pedestrian tracking, although the evaluation remains within controlled scenarios (Wang et al., 1 Dec 2025).

5. Adversarial trajectory patterns and universal maneuvers

A distinctive contribution of the paper is its characterization of the optimized trajectories. Across randomized initial positions relative to a target moving at constant velocity, two recurring patterns appear. First, the attacker greedily closes distance to the target. Second, once near the target, the attacker moves in the same direction but with deliberately non-linear speed variation, especially abrupt acceleration or deceleration (Wang et al., 1 Dec 2025).

From these patterns, the authors derive two universal adversarial maneuvers that can be executed by human attackers.

The first is Go-and-Stop, used when the attacker starts behind the target. The attacker accelerates to catch up, aligns direction with the target, and then abruptly decelerates so that its actual detection lags behind while its Kalman-filter prediction overshoots toward the target. This creates favorable conditions for swapped assignment (Wang et al., 1 Dec 2025).

The second is Stop-and-Go, used when the attacker starts ahead of the target. The attacker approaches the target’s projected path, slows or waits, and then turns and accelerates in the target’s direction as the target passes. This produces the complementary prediction/detection mismatch needed for the transfer (Wang et al., 1 Dec 2025).

The paper validates these maneuvers in real-world surveillance and autonomous-driving scenarios. With human-executed heuristic attacks, the reported maximum success reaches 45%, substantially above baseline natural misassignment rates. For example, in pedestrian surveillance with similar appearance, the reported rates are 30% for SORT, 40% for ByteTrack, 27.5% for OC-SORT, 45% for Deep OC-SORT, 6.5% for BoT-SORT, and 0% for StrongSORT. In AD-parallel with similar appearance, the corresponding rates are 30%, 20%, 40%, 45%, 20%, and 15% (Wang et al., 1 Dec 2025).

These real-world results are notably lower than the optimized CARLA results, which the paper attributes to imperfect human timing, limited control precision, detector noise, and occlusion. Still, the fact that simple maneuvers can induce ID transfer or ID switch without any detector attack is itself the substantive empirical point (Wang et al., 1 Dec 2025).

6. Implications, limitations, and relation to adjacent trajectory-adversarial work

AdvTraj shows that even if object detection is accurate, MOT remains vulnerable through the association stage alone. This is operationally significant because trajectory histories are used downstream. The paper explicitly notes that wrong ID association can lead to wrong trajectory predictions, which in autonomous-driving stacks can propagate into unsafe planning (Wang et al., 1 Dec 2025). This connects AdvTraj to broader trajectory-adversarial work such as AdvDO, where physically plausible trajectory perturbations induce large forecasting degradation and unsafe downstream planning behavior (Cao et al., 2022).

Several limitations are also explicit. The full optimized attack assumes strong attacker knowledge and control: camera geometry, target trajectory, white-box access to the victim tracker, and the ability to execute precise online motion. The real-world efficacy gap between optimized and heuristic attackers reflects this. The formulation also assumes accurate detections, which isolates the association vulnerability but leaves open how detector failures interact with the attack in more cluttered environments (Wang et al., 1 Dec 2025).

The defense discussion is suggestive rather than algorithmically complete. The paper points to more expressive motion models such as Extended Kalman Filter and Particle Filter, more robust use of ReID, adaptive covariance handling, and stronger temporal consistency reasoning as plausible mitigations. StrongSORT, which uses an NSA Kalman Filter and Gaussian-smoothed interpolation during short occlusions, emerges as relatively more robust in heuristic real-world settings, though not immune in simulation transfer (Wang et al., 1 Dec 2025).

A common misconception is that adversarial vulnerability in MOT is mainly a detector problem. AdvTraj directly refutes that view: the attack succeeds without attacking object detection, by exploiting shared design principles in motion prediction and assignment. In that sense, AdvTraj is best understood not as an object-detection attack, nor as a generic trajectory-forecasting attack, but as a physical adversarial attack on identity maintenance in tracking-by-detection MOT that uses motion itself as the adversarial channel (Wang et al., 1 Dec 2025).