---
title: 'AdvTraj: Adversarial ID-Transfer in MOT'
url: https://www.emergentmind.com/topics/advtraj
type: topic
---

# AdvTraj: Adversarial ID-Transfer in MOT

Searching arXiv for the exact AdvTraj paper and closely related trajectory-adversarial work for contextual support.
AdvTraj is an adversarial method for **tracking-by-detection multi-object tracking (MOT)** in which an attacker uses **adversarial trajectories** to transfer its own tracker ID to a target object **without attacking object detection**. In the exact formulation introduced in “Physical ID-Transfer Attacks against Multi-Object Tracking via Adversarial Trajectory” [2512.01934], the attack is **online** and **physical**: the adversary perturbs only its own motion in the scene so as to exploit weaknesses in motion prediction and bipartite association, causing the tracker to preserve the attacker’s historical identity while reassigning it to the target. This distinguishes AdvTraj from adversarial methods that perturb pixels, detections, or detector outputs, and places it in a broader trajectory-adversarial literature that includes physically realistic trajectory attacks on downstream prediction systems such as AdvDO [2209.08744].

## 1. Definition and scope

AdvTraj is defined as an **ID-transfer attack** on tracking-by-detection MOT systems. The attacker does not seek merely to induce a generic ID switch; rather, the goal is to make the tracker reassign the attacker’s original ID to a target object while the attacker itself loses that ID [2512.01934]. The paper formalizes this objective through the assignment function \(f(O_i^t)\), where \(O_a\) denotes the attacker and \(O_b\) the target. Initially,
\[
f(O_a^t)=ID_a,\; f(O_b^t)=ID_b,\quad \forall t<\tau,
\]
and after the attack time \(\tau\),
\[
f(O_a^t)\neq ID_a,\; f(O_b^t)=ID_a,\quad \forall t\ge \tau.
\]

This notion of ID transfer is more specific than the broader notion of an ID switch. A generic switch can simply destroy or reinitialize identities. AdvTraj instead preserves the attacker’s historical identity as a live track and transfers it to the target, which is especially consequential for surveillance and autonomous-driving stacks that consume trajectory histories for downstream inference [2512.01934].

The method is explicitly framed as the **first online and physical ID-manipulation attack against tracking-by-detection MOT**. Its novelty lies in isolating the **association stage** as the attack surface. In this respect, AdvTraj differs sharply from prior MOT attacks that focused on detector manipulation, digital perturbation, adversarial patches, or offline video tampering. The same emphasis on trajectory realism appears in adversarial work on motion forecasting, where AdvDO studies physically plausible perturbations of observed trajectories rather than unconstrained feature-space attacks [2209.08744]. A plausible implication is that AdvTraj and AdvDO expose a common systems vulnerability: trajectory-processing modules can be destabilized through feasible motion manipulation alone.

## 2. Threat model and attack objective

The victim setting is standard tracking-by-detection MOT. At time \(t\), the detector outputs a detection state
\[
\mathbf{d}_i=(x_1,y_1,x_2,y_2)_i,
\]
and the tracker maintains a latent state
\[
\mathbf{x}=(u,v,s,r,\dot{u},\dot{v},\dot{s})^T,
\]
where \((u,v)\) is the bounding-box center, \(s\) is scale, \(r\) is aspect ratio, and \(\dot{u},\dot{v},\dot{s}\) are first-order derivatives [2512.01934]. Motion prediction is performed with a Kalman filter, with prior estimate
\[
\hat{\mathbf{x}}_{\text{prior}}^t=\mathbf{F}\hat{\mathbf{x}}_{\text{posterior}}^{t-1}.
\]

The attacker is a physical object visible to the camera, such as a pedestrian or a vehicle, and controls **only its own movement**. The attacker is assumed to know the camera location and viewing angle and to have access to the target’s trajectory. The attacker and target are assumed to belong to the same class so that their detections are comparable in size. The attack therefore targets the association logic under physically bounded motion rather than the detector itself [2512.01934].

The paper distinguishes two attacker classes. \((\mathtt{T1})\) is an optimized adversary, such as an autonomous robot or vehicle, capable of precise online optimization and control. \((\mathtt{T2})\) is a heuristic adversary, such as a human walker or driver, who instead performs simple universal maneuvers. This distinction is central because the full AdvTraj optimization is designed for real-time receding execution, whereas the heuristic version is intended to be human-executable in ordinary scenarios [2512.01934].

The attack objective is expressed through sufficient conditions on association costs. Let \(d(\cdot,\cdot)\) be the distance between predicted and detected boxes. The paper gives the following conditions for ID transfer:
\[
\begin{aligned}
&d(\mathbf{x}_a,\hat{\mathbf{x}}_{ID_b}) + d(\mathbf{x}_b,\hat{\mathbf{x}}_{ID_a}) < d(\mathbf{x}_a,\hat{\mathbf{x}}_{ID_a}) + d(\mathbf{x}_b,\hat{\mathbf{x}}_{ID_b}) \quad &&\mathtt{(C1)},\\
&d(\mathbf{x}_b,\hat{\mathbf{x}}_{ID_a}) < d(\mathbf{x}_i,\hat{\mathbf{x}}_{ID_a}), \forall i: O_i \in \mathcal O \setminus \{O_a,O_b\} \quad &&\mathtt{(C2)},\\
&d(\mathbf{x}_a,\hat{\mathbf{x}}_{ID_b}) < d(\mathbf{x}_i,\hat{\mathbf{x}}_{ID_b}), \forall i: O_i \in \mathcal O \setminus \{O_a,O_b\} \quad &&\mathtt{(C3)}.
\end{aligned}
\]
The first inequality enforces that the swapped matching is cheaper than the correct one; the latter two exclude interference from other objects. Under the paper’s assumptions, the attack largely focuses on driving the system toward \(\mathtt{C1}\) [2512.01934].

## 3. Attack mechanism and optimization

The core idea is to manipulate **motion prediction inertia**. Tracking-by-detection pipelines commonly combine Kalman-filter motion prediction with IoU-style cost computation and Hungarian matching. AdvTraj exploits the fact that abruptly non-linear motion can create a mismatch between an attacker’s true position and the tracker’s predicted position, thereby making a swapped assignment globally cheaper than the correct one [2512.01934].

The physical optimization is posed over the attacker’s ground-plane position:
\[
\begin{aligned}
&\arg\min_{(x^t_a,y^t_a)} d(\mathbf{x}_a, \hat{\mathbf{x}}_{ID_b}) + d(\mathbf{x}_b, \hat{\mathbf{x}}_{ID_a})\\
&\text{subject to } \|(x^t_a,y^t_a)-(x^{t-1}_a,y^{t-1}_a)\| \le \epsilon.
\end{aligned}
\]
This objective minimizes the swapped assignment cost while enforcing a bounded physical displacement per step [2512.01934].

Because the attacker controls a 3D body but the tracker consumes 2D detections, the paper introduces a differentiable world-to-image projection. Let
\[
\mathbf{S}=\{(x_i,y_i,z_i)^T \mid 1\le i\le 8\}
\]
be the 3D bounding-box vertices and \(\mathbf{P}\) the camera projection matrix. The projected 2D box is
\[
\mathbf{d}=\mathcal{P}\big(\mathbf{S}\big)
= \left( \min_{1\le i\le 8} \frac{x_i'}{z_i'}, \min_{1\le i\le 8} \frac{y_i'}{z_i'}, \max_{1\le i\le 8} \frac{x_i'}{z_i'}, \max_{1\le i\le 8} \frac{y_i'}{z_i'} \right),
\]
with
\[
\begin{pmatrix} x_1' & \dots & x_8'\\ y_1' & \dots & y_8'\\ z_1' & \dots & z_8' \end{pmatrix}
=
\mathbf{P}
\begin{pmatrix} \mathbf{S}\\ \mathbf{1} \end{pmatrix}.
\]
The attacker’s control variable is a ground-plane displacement
\[
\boldsymbol{\delta}=(\Delta x,\Delta y),
\]
applied as \(\mathbf{S}+\boldsymbol{\delta}\) with constant height [2512.01934].

The differentiable adversarial objective is
\[
\begin{aligned}
\mathcal{L}(\mathcal{P}(\mathbf{S}^{t-1}+\boldsymbol{\delta})) =
d(g(\mathbf{d}^t), \hat{\mathbf{x}}_b^t) + d(\hat{\mathbf{x}}_b^{t+1},\hat{\mathbf{x}}^{t+1})
\quad \text{subject to }\|\boldsymbol{\delta}\|\le \epsilon.
\end{aligned}
\]
Here the first term makes the attacker’s **current detection** resemble the target’s **current predicted state**, and the second term makes the attacker’s **next predicted track state** resemble the target’s **next predicted state** [2512.01934]. This two-term structure is essential: minimizing only the first term would not account for the global Hungarian assignment.

For optimization, the paper uses **DIoU** rather than raw IoU because IoU has zero gradient when boxes do not overlap:
\[
\mathcal{L}_{DIoU} = 1-\text{IoU} + \frac{\rho^2(\mathbf{b}_1,\mathbf{b}_2)}{c^2}.
\]
The victim tracker remains unchanged; DIoU is used only as a differentiable surrogate during attack generation [2512.01934].

Algorithmically, AdvTraj operates online. At each frame it predicts attacker and target tracker priors, checks whether the swapped cost still exceeds the normal cost, and if so performs a small number of gradient steps on \(\boldsymbol{\delta}\), clips \(\boldsymbol{\delta}\) to the admissible range, and physically executes the resulting movement. The reported implementation uses Adam with learning rate \(\alpha=0.1\), \(\mathtt{iter}=5\), and \(\epsilon=0.35\), corresponding to a walker speed limit of \(3.5\,\mathrm{m/s}\) [2512.01934].

## 4. Victim trackers and empirical results

AdvTraj is optimized in a white-box setting against **SORT**, then evaluated for transfer against **ByteTrack**, **OC-SORT**, **Deep OC-SORT**, **BoT-SORT**, and **StrongSORT**. These trackers share the design principles that make transfer plausible: Kalman-filter motion prediction, IoU-style association, and Hungarian matching, with some methods additionally using ReID features [2512.01934].

| Tracker | Motion model | ReID |
|---|---|---:|
| SORT | Std. Kalman Filter | N |
| ByteTrack | Std. Kalman Filter | N (default best model) |
| OC-SORT | Std. Kalman Filter | N |
| Deep OC-SORT | Std. Kalman Filter | Y |
| BoT-SORT | Std. Kalman Filter | Y |
| StrongSORT | NSA Kalman Filter | Y |

In CARLA simulation, white-box attacks against SORT achieve **100% success rate in various scenarios**, including pedestrian surveillance and autonomous-driving settings [2512.01934]. Transfer is also strong. In pedestrian surveillance, the reported attack success rates are **89%** for ByteTrack, **92%** for OC-SORT, **93%** for Deep OC-SORT, **74%** for BoT-SORT, and **84%** for StrongSORT. In AD-perpendicular settings, the corresponding rates are **45%**, **69%**, **74%**, **78%**, and **77%**. In AD-parallel settings they are **66%**, **69%**, **80%**, **79%**, and **85%** [2512.01934].

These results support the paper’s central claim that trajectories optimized against SORT transfer because many SOTA MOT algorithms share common association logic. The attack is therefore not merely exploiting an implementation bug in SORT; it is exploiting a structural weakness in the motion-association stage [2512.01934].

The paper also reports vehicle-surveillance results in the appendix. In CARLA, transfer success rates are **83%** for SORT, **74%** for ByteTrack, **80%** for OC-SORT, **83%** for Deep OC-SORT, **56%** for BoT-SORT, and **40%** for StrongSORT. A plausible implication is that the phenomenon is not limited to pedestrian tracking, although the evaluation remains within controlled scenarios [2512.01934].

## 5. Adversarial trajectory patterns and universal maneuvers

A distinctive contribution of the paper is its characterization of the optimized trajectories. Across randomized initial positions relative to a target moving at constant velocity, two recurring patterns appear. First, the attacker **greedily closes distance** to the target. Second, once near the target, the attacker moves in the **same direction** but with deliberately **non-linear speed variation**, especially abrupt acceleration or deceleration [2512.01934].

From these patterns, the authors derive two **universal adversarial maneuvers** that can be executed by human attackers.

The first is **Go-and-Stop**, used when the attacker starts behind the target. The attacker accelerates to catch up, aligns direction with the target, and then abruptly decelerates so that its actual detection lags behind while its Kalman-filter prediction overshoots toward the target. This creates favorable conditions for swapped assignment [2512.01934].

The second is **Stop-and-Go**, used when the attacker starts ahead of the target. The attacker approaches the target’s projected path, slows or waits, and then turns and accelerates in the target’s direction as the target passes. This produces the complementary prediction/detection mismatch needed for the transfer [2512.01934].

The paper validates these maneuvers in real-world surveillance and autonomous-driving scenarios. With human-executed heuristic attacks, the reported maximum success reaches **45%**, substantially above baseline natural misassignment rates. For example, in pedestrian surveillance with similar appearance, the reported rates are **30%** for SORT, **40%** for ByteTrack, **27.5%** for OC-SORT, **45%** for Deep OC-SORT, **6.5%** for BoT-SORT, and **0%** for StrongSORT. In AD-parallel with similar appearance, the corresponding rates are **30%**, **20%**, **40%**, **45%**, **20%**, and **15%** [2512.01934].

These real-world results are notably lower than the optimized CARLA results, which the paper attributes to imperfect human timing, limited control precision, detector noise, and occlusion. Still, the fact that simple maneuvers can induce ID transfer or ID switch without any detector attack is itself the substantive empirical point [2512.01934].

## 6. Implications, limitations, and relation to adjacent trajectory-adversarial work

AdvTraj shows that even if object detection is accurate, MOT remains vulnerable through the **association stage alone**. This is operationally significant because trajectory histories are used downstream. The paper explicitly notes that wrong ID association can lead to **wrong trajectory predictions**, which in autonomous-driving stacks can propagate into unsafe planning [2512.01934]. This connects AdvTraj to broader trajectory-adversarial work such as AdvDO, where physically plausible trajectory perturbations induce large forecasting degradation and unsafe downstream planning behavior [2209.08744].

Several limitations are also explicit. The full optimized attack assumes strong attacker knowledge and control: camera geometry, target trajectory, white-box access to the victim tracker, and the ability to execute precise online motion. The real-world efficacy gap between optimized and heuristic attackers reflects this. The formulation also assumes accurate detections, which isolates the association vulnerability but leaves open how detector failures interact with the attack in more cluttered environments [2512.01934].

The defense discussion is suggestive rather than algorithmically complete. The paper points to more expressive motion models such as **Extended Kalman Filter** and **Particle Filter**, more robust use of ReID, adaptive covariance handling, and stronger temporal consistency reasoning as plausible mitigations. StrongSORT, which uses an **NSA Kalman Filter** and Gaussian-smoothed interpolation during short occlusions, emerges as relatively more robust in heuristic real-world settings, though not immune in simulation transfer [2512.01934].

A common misconception is that adversarial vulnerability in MOT is mainly a detector problem. AdvTraj directly refutes that view: the attack succeeds **without attacking object detection**, by exploiting shared design principles in motion prediction and assignment. In that sense, AdvTraj is best understood not as an object-detection attack, nor as a generic trajectory-forecasting attack, but as a **physical adversarial attack on identity maintenance in tracking-by-detection MOT** that uses motion itself as the adversarial channel [2512.01934].

Source: https://www.emergentmind.com/topics/advtraj