Motion-Aware Transformer (MATR)

• Motion-Aware Transformer (MATR) is a design family that explicitly incorporates motion signals into Transformer architectures to improve temporal precision.
• It employs tailored modifications—such as LSTM-augmented self-attention, motion-aware positional encoding, and query updates—to better capture motion dynamics.
• Empirical results across tasks like action localization, multi-object tracking, occluded re-identification, and 4D reconstruction demonstrate significant performance gains.

Searching arXiv for the specified papers to ground the article in current literature. Searching arXiv for "(Yang et al., 2022) Motion-aware Transformer". “Motion-Aware Transformer” (MATR) denotes a class of Transformer-based architectures that incorporate explicit motion modeling into representation learning, attention, positional encoding, or query propagation. In the literature provided here, the name has been used for distinct systems addressing language-driven action localization, multi-object tracking, action detection, and occluded person re-identification, while closely related work on 4D reconstruction adopts motion-aware Transformer components without using the same acronym (Yang et al., 2022, Yang et al., 26 Sep 2025, Korban et al., 2024, Zhou et al., 2022, Fang et al., 5 Mar 2026). Across these formulations, the common design principle is that standard Transformer operations are supplemented with motion-specific inductive structure in order to improve temporal precision, boundary estimation, association robustness, spatiotemporal semantics, or part alignment.

1. Scope and usage of the term

In the sources considered here, MATR is not a single canonical architecture but a recurring designation for task-specific Transformer designs that make motion an explicit computational object. In language-driven action localization, the Motion-aware Transformer is the second stage of a two-stage pipeline and refines start and end boundaries within a shrunken temporal window by modeling fine-grained motion dynamics via an LSTM-augmented self-attention Transformer guided by motion queries (Yang et al., 2022). In multi-object tracking, MATR is a module inserted between encoder and decoder to predict object movement across frames and update track queries in advance, thereby reducing “query collisions” (Yang et al., 26 Sep 2025). In action detection, the acronym denotes a Semantic and Motion-Aware Spatiotemporal Transformer Network whose motion-aware network encodes the locations of action semantics in video frames utilizing a motion-aware 2D positional encoding algorithm (Korban et al., 2024). In occluded person re-identification, MATR refers to a transformer encoder-decoder architecture coupled with self-supervised keypoint detection, affine motion estimation, and motion-assisted part segmentation (Zhou et al., 2022).

A related but differently named system is MoRe, a “Motion-aware Feed-forward 4D Reconstruction Transformer,” which extends a static-scene reconstruction Transformer with attention-forcing, grouped causal attention, and a bundle-adjustment-like global refinement to recover dynamic 3D scenes from monocular videos (Fang et al., 5 Mar 2026). This suggests that “motion-aware Transformer” functions less as a single method name than as a design family spanning several computer vision subfields.

2. Core architectural motifs

A shared architectural pattern is the modification of standard Transformer components so that motion is represented explicitly rather than left to emerge implicitly from generic self-attention. The precise mechanism differs by task.

For language-driven action localization, motion enters through both query construction and self-attention. The sentence is embedded with GloVe and a standard Transformer, and motion query features are defined by retaining only those word embeddings relevant to motion or “others,” using POS-derived one-hot tags. Within the video branch, each Q/K/V linear projection is replaced by a multi-scale LSTM so that the self-attention head operates on motion-sensitive projected features rather than conventional linear projections (Yang et al., 2022).

For multi-object tracking, the central architectural intervention is the Motion-Aware Transformer module placed between the encoder and the deformable decoder. Track queries from the previous frame are first updated through self-attention and cross-attention with the current encoder memory, and the residual update is applied both to the query features and to the positional box embeddings. The updated track queries are then concatenated with fixed detection queries and passed into a standard multi-layer Deformable Transformer Decoder (Yang et al., 26 Sep 2025).

For action detection, motion awareness is built into the spatiotemporal backbone at several levels. RGB frames are processed with a DETR-style detector to identify persons and objects, optical flow is extracted via FastFlowNet, and a Motion-Memory Module produces 2D motion offsets and motion memory features. These are fused with spatial and motion embeddings through motion-aware positional encoding, multi-feature selective semantic attention, and sequence-based temporal attention (Korban et al., 2024).

For occluded person re-identification, a CNN plus Transformer encoder-decoder first produces globally aware features. A self-supervised keypoint-detection branch predicts unsupervised keypoint heatmaps and local Jacobians, and a part-segmentation branch uses transformer features together with motion cues derived from the keypoints to generate refined segmentation maps for body parts plus background (Zhou et al., 2022).

For 4D reconstruction, motion awareness is realized through attention-forcing during training, grouped causal attention for streaming inference, incremental KV-caching, and one global “bundle-adjustment-like” attention pass over cached keys and values to refine camera poses (Fang et al., 5 Mar 2026).

3. Mechanisms for representing motion

The formulations differ substantially in how motion is encoded, supervised, and coupled to attention.

In the action-localization MATR, motion is represented linguistically and visually. Motion queries are derived from sentence tokens with POS-derived one-hot vectors $p_i=[p_e,p_m,p_o]$, and the motion query features are

$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$

so that $F_q^m\in\mathbb R^{N\times d}$ collects only those word embeddings relevant to motion or “others.” On the visual side, the model replaces each self-attention projection with a multi-scale LSTM:

$$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$

with

$$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$

At scale $\ell$, the recurrent unit processes temporally strided subsequences such as $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$ in order to capture motion at temporal stride $\ell$ (Yang et al., 2022).

In the multi-object-tracking MATR, motion is represented as an explicit update of track-query features and bounding-box embeddings. If $Q_{\mathrm{trk}}^t$ are track embeddings and $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$ are box embeddings, the MAT module computes

$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$0

and regresses a box update $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$1 that is added to $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$2. The paper also writes the motion view as

$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$3

This construction treats motion as a supervised pre-alignment step before the decoder (Yang et al., 26 Sep 2025).

In the action-detection MATR, motion is represented by learned 2D offsets in positional encoding. If $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$4 are initial patch indices and $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$5 are horizontal and vertical motion offsets from the Motion-Memory module, then the motion-aware positional encoding is

$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$6

with an analogous definition for $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$7. Concatenating these gives $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$8, which is added to both spatial and motion features (Korban et al., 2024).

In the re-identification MATR, motion is represented by local affine maps induced by self-supervised keypoints. If $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$9 are two images of the same identity in different poses, the local warp near keypoint $F_q^m\in\mathbb R^{N\times d}$0 is approximated by

$F_q^m\in\mathbb R^{N\times d}$1

and the dense motion field for pixel $F_q^m\in\mathbb R^{N\times d}$2 in part region $F_q^m\in\mathbb R^{N\times d}$3 is

$F_q^m\in\mathbb R^{N\times d}$4

The target feature map is then warped as

$F_q^m\in\mathbb R^{N\times d}$5

and this motion-warped feature is fed back to the segmentation branch (Zhou et al., 2022).

In MoRe, motion is encoded through motion masks during training and through constrained temporal attention during inference. The attention-forcing strategy defines patch-level staticity scores

$F_q^m\in\mathbb R^{N\times d}$6

and penalizes camera-token attention on moving regions via

$F_q^m\in\mathbb R^{N\times d}$7

Streaming inference applies grouped causal attention:

$F_q^m\in\mathbb R^{N\times d}$8

followed by a global camera-query refinement over the full cache after all frames have been processed (Fang et al., 5 Mar 2026).

4. Attention design and temporal reasoning

The attention mechanisms used in these systems show several distinct strategies for incorporating motion priors into Transformer computation.

The action-localization MATR modifies the self-attention head itself by replacing the standard linear Q/K/V projections with multi-scale LSTMs, then performing

$F_q^m\in\mathbb R^{N\times d}$9

The resulting motion-aware visual features $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$0 are fused with motion queries by context-query attention,

$$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$1

where $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$2 is the action-relevant score from the first stage (Yang et al., 2022). This design localizes motion modeling at the point where temporal neighborhoods are projected into attention space.

The action-detection MATR uses a more heterogeneous attention stack. Its Multi-Feature Selective Semantic Attention defines four pairings—GG, MM, GM, and MG—between spatial and motion streams:

$$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$3

with analogous terms for $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$4 and $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$5. Queries are formed only from person embeddings, while keys and values come from all semantics. The model then applies a sequence-based temporal attention whose correlation matrix is designed to penalize differences between frame features with respect to the global covariance of the sequence, rather than relying only on similarity (Korban et al., 2024). A plausible implication is that this formulation aims to prioritize frames that are discriminative for action identity even when they are not maximally self-similar.

The multi-object-tracking MATR retains standard Transformer operations but changes when and where they are applied. Motion-aware self-attention and cross-attention are executed on track queries before those queries enter the shared detection-and-tracking decoder. The stated purpose is to reduce the gap between a stale query and its true object before Hungarian matching, thereby lowering query collisions (Yang et al., 26 Sep 2025).

MoRe adopts a different temporal attention regime oriented toward real-time reconstruction. Within each frame, image tokens attend bidirectionally for spatial consistency; across frames, each token at time $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$6 may only attend to keys and values from frames $$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$7, enforcing temporal causality. After all frames, camera queries receive one global attention pass over the entire cache to recover long-range consistency in camera estimation (Fang et al., 5 Mar 2026). This combination separates streaming efficiency from global pose refinement.

5. Optimization objectives and supervision

The various MATR formulations are also distinguished by their supervisory signals, which reflect the kind of motion information each task requires.

For language-driven action localization, boundary precision is optimized with a start-end classification loss and an auxiliary inner-frame prediction loss:

$$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$8

$$f_{\nu} = \mathrm{LSTM}^S_{\nu}(V^{in}),\quad \nu\in\{Q,K,V\},$$9

and the total loss is

$$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$0

with $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$1 and $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$2 (Yang et al., 2022).

For multi-object tracking, the overall objective is

$$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$3

where $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$4 is the original MOTR loss with 1-to-1 Hungarian matching, per-query classification cross-entropy, and per-matched-query L1 plus GIoU box regression, while

$$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$5

The weighting $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$6 is set to $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$7 (Yang et al., 26 Sep 2025).

For action detection, the training loss combines frame-level and sequence-level cross-entropy:

$$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$8

with $$\mathrm{LSTM}^S(V^{in}) = \bigl[L_1(V^{in});\,L_2(V^{in});\cdots;L_S(V^{in})\bigr].$$9 (Korban et al., 2024).

For occluded person re-identification, training is end-to-end with six losses: global and local ID losses, global and local triplet losses, an equivariance loss on keypoints, and a feature-consistency loss on warped features. The full objective is

$\ell$0

with $\ell$1, $\ell$2, and $\ell$3 (Zhou et al., 2022).

For MoRe, the supervision combines confidence-weighted regression for depth and point maps, motion mask classification, attention-forcing, and relative camera-pose loss. The depth and point-map term is

$\ell$4

the motion-mask term is binary cross-entropy, and the camera term averages geodesic rotation error and translation error over frame pairs (Fang et al., 5 Mar 2026).

6. Empirical behavior across tasks

The reported results indicate that motion-aware Transformer designs improve performance in several distinct regimes, but the gains are tied to different evaluation criteria and should not be conflated across tasks.

For language-driven action localization, the pure motion-aware module was compared on Charades-STA using R@1 at IoU $\ell$5. The Standard Transformer (“FC Trans”) obtained $\ell$6, Temporal-Conv + Trans obtained $\ell$7, the Multi-scale LSTM layer without self-attention obtained $\ell$8, and the Full LSTM-augmented Transformer obtained $\ell$9. The same source reports that TACoS improves to $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$0 at R@1 and IoU $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$1 (Yang et al., 2022).

For multi-object tracking, ablations on DanceTrack show a progression from the MOTR baseline at HOTA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$2, DetA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$3, AssA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$4, MOTA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$5, and IDF1 $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$6, to a strengthened baseline with DAB-style box propagation at HOTA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$7, and then to the MAT module with a single decoder layer at HOTA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$8, AssA $$\{\ldots,v_{i-2\ell}^{in},v_{i-\ell}^{in},v_i^{in},v_{i+\ell}^{in},v_{i+2\ell}^{in},\ldots\}$$9, and IDF1 $\ell$0. The reported state-of-the-art results include HOTA $\ell$1 on DanceTrack with Swin-Tiny, HOTA $\ell$2 on SportsMOT, and mTETA $\ell$3, mHOTA $\ell$4, mAssocA $\ell$5 on the BDD100k validation set (Yang et al., 26 Sep 2025).

For action detection, the Semantic and Motion-Aware Spatiotemporal Transformer Network, denoted “SMAST” in the paper, reports state-of-the-art performance on AVA v2.2, AVA v2.1, UCF101-24, and EPIC-Kitchens. The listed figures are $\ell$6 on AVA v2.2, $\ell$7 on AVA v2.1, $\ell$8 on UCF101-24, and on EPIC-Kitchens top-1 scores of Action $\ell$9, Verb $Q_{\mathrm{trk}}^t$0, and Noun $Q_{\mathrm{trk}}^t$1. Its ablations attribute gains to motion-aware positional encoding, multi-feature attention, selective attention, and sequence-based temporal attention (Korban et al., 2024).

For occluded person re-identification, MATR reports on Occluded-Duke $Q_{\mathrm{trk}}^t$2 Rank-1 and $Q_{\mathrm{trk}}^t$3 mAP, on Occluded-REID $Q_{\mathrm{trk}}^t$4 and $Q_{\mathrm{trk}}^t$5, on Partial-REID $Q_{\mathrm{trk}}^t$6 Rank-1, on Partial-iLIDS $Q_{\mathrm{trk}}^t$7, and on Market-1501 and DukeMTMC-reID mAP values of $Q_{\mathrm{trk}}^t$8 and $Q_{\mathrm{trk}}^t$9. An ablation on Occluded-Duke shows a Baseline ResNet50 at $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$0, then gains from segmentation plus self-supervised keypoints, then from a Transformer encoder, and finally the full encoder-decoder at $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$1 (Zhou et al., 2022).

For 4D reconstruction, MoRe reports camera pose estimation on Sintel, Bonn, TUM-dynamics, and ScanNet using ATE, RPE$$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$2, and RPE$$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$3. On Sintel, the streaming version reports ATE $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$4 m, RPE$$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$5 m, and RPE$$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$6. For video depth estimation on Sintel, the same source gives AbsRel $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$7 and $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$8. For inference speed on KITTI at $$PE_{\mathrm{trk}}^t\in\mathbb R^{N_{\mathrm{trk}}\times 4}$$9 on NVIDIA A800, the table lists VGGT at $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$00 FPS, CUT3R at $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$01 FPS, Stream3R$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$02 at $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$03 FPS, 3R$f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$04 window $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$05 at $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$06 FPS, and MoRe with grouped causal attention plus window $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$07 at $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$08 FPS. Ablations report that removing attention-forcing changes Sintel ATE from $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$09 to $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$10, removing BA-refinement changes it to $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$11, and removing grouped causal attention degrades Sintel depth AbsRel from $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$12 to $f_{q,i}^m = f_{q,i}\cdot (p^m_i + p^o_i),$13 (Fang et al., 5 Mar 2026).

7. Interpretation, recurring advantages, and limitations

Across the formulations surveyed here, the principal recurring claim is not that Transformers alone suffice for motion understanding, but that explicit motion structure improves performance when standard attention is otherwise poorly matched to the task. In action localization, the stated motivation is that standard linear projections in Transformer cannot capture local, successive frame-to-frame motion changes, so Q/K/V are replaced by multi-scale LSTMs (Yang et al., 2022). In multi-object tracking, the problem is that stale queries drift and collide under Hungarian matching, so motion prediction is used to pre-shift track queries before decoding (Yang et al., 26 Sep 2025). In action detection, the motion-aware mechanism is introduced because standard 1D patch positional encoding cannot adapt when an object or person moves from one patch to another (Korban et al., 2024). In 4D reconstruction, the motivation is that static backbones confuse moving objects with background when estimating camera pose, which leads to attention-forcing and grouped causal attention (Fang et al., 5 Mar 2026). In occluded person re-identification, motion cues are used to identify major human body components and refine part segmentation under occlusion (Zhou et al., 2022).

A common misconception would be to treat MATR as a single reusable architecture. The literature here indicates the opposite: the acronym is attached to substantially different mechanisms, including LSTM-augmented attention, motion-aware query updates, motion-aware positional encoding, affine keypoint warping, and training-time attention forcing. Another potential misconception is that “motion awareness” always implies optical flow. That is not the case in these sources: optical flow is explicit in the action-detection system, but motion is represented elsewhere through POS-filtered motion queries, track-box deltas, self-supervised affine fields, or binary motion masks.

The limitations are also task-specific. The occluded person re-identification paper notes that under extreme occlusions where few keypoints are visible, the affine approximation may degrade, that the current model processes still images in pairs, and that transformer plus motion-flow estimation is heavier than pure CNN pipelines (Zhou et al., 2022). For the other systems, the reported ablations imply that performance is sensitive to whether motion-aware components are retained: removing attention-forcing or BA-style refinement in MoRe degrades ATE, and replacing the full motion-aware module in action localization with simpler variants lowers R@1 (Fang et al., 5 Mar 2026, Yang et al., 2022).

Taken together, these works define Motion-Aware Transformer not as a single standardized model but as a family of Transformer adaptations in which motion is injected into the computational graph as a first-class signal. The consistent pattern is that explicit motion-aware structure is used to correct a specific failure mode of generic attention: imprecise temporal boundaries, stale track queries, motion-insensitive positional encodings, unstable part grouping under occlusion, or camera-pose corruption from dynamic content.