Motion Token Pruning (MTP)

• Motion Token Pruning (MTP) is a method that reduces computational burdens in video Transformers by retaining tokens with vital motion cues and structural information.
• It uses temporal variance, recency weighting, and diversity-aware greedy selection to prioritize dynamic features over static background redundancies.
• In formulations like EventPrune, MTP leverages event-camera data to provide continuous motion signals, achieving significant token reduction and faster inference.

Motion Token Pruning (MTP) is the principle of reducing the visual tokens in a video Transformer by prioritizing tokens that carry motion-relevant and geometry-preserving information, so that spatial reasoning over dynamic scenes remains accurate while the quadratic attention cost is cut (Ma et al., 19 May 2026). In the 2026 literature provided here, the term appears in two closely related but distinct senses. In ST-Prune, MTP denotes Motion-aware Temporal Pruning, a training-free module for multi-view, multi-frame vision-LLMs in autonomous driving that prunes tokens within each camera view by combining motion volatility, temporal recency, and diversity-aware greedy selection (Sha et al., 21 Apr 2026). In EventPrune, MTP denotes a broader pruning principle operationalized through Event Cascade Pruning (ECP), where synchronized event streams provide a continuous motion prior for temporal anchoring, token filtering, and attention calibration in first-person dynamic spatial reasoning (Ma et al., 19 May 2026). Across both formulations, the central premise is that temporally dynamic and structurally informative tokens should be preserved preferentially under strict token budgets.

1. Terminological scope and research context

Within the supplied literature, MTP is not a single algorithmic template but a family resemblance across training-free token selection mechanisms that explicitly privilege motion cues over static redundancy. ST-Prune introduces MTP as one of two complementary modules in a complete spatio-temporal pruning framework for autonomous driving; its companion module, Ring-view Spatial Pruning (RSP), removes cross-view redundancy that temporal pruning alone cannot suppress (Sha et al., 21 Apr 2026). EventPrune, by contrast, presents Event Cascade Pruning as, to its knowledge, the first training-free instantiation of MTP for first-person dynamic spatial reasoning, using event-camera measurements as a motion prior (Ma et al., 19 May 2026).

The two papers share several commitments. Both are training-free, plug-and-play, and designed to reduce the computational burden induced by dense visual tokens in Transformer-based systems (Sha et al., 21 Apr 2026, Ma et al., 19 May 2026). Both also position MTP against token pruning paradigms developed for static images or flat token sets, arguing that such approaches fail to preserve temporally ordered, motion-dependent cues. ST-Prune states that methods such as FAST-V, ToMe, DivPrune, VisPruner, and PACT operate on flat token sets or single images and, when applied naively per frame, fail to prioritize current-frame content and lose temporal diversity (Sha et al., 21 Apr 2026). EventPrune similarly argues that attention-only and discrete snapshot strategies miss continuous motion causality and are sensitive to blur and appearance changes (Ma et al., 19 May 2026).

A plausible implication is that “motion” in MTP is not limited to explicit kinematic estimation. In ST-Prune it is represented by temporal feature volatility and recency bias, whereas in EventPrune it is represented by event activity, event-aligned token saliency, and rank-fused attention correction.

2. Motion-aware Temporal Pruning in ST-Prune

In ST-Prune, MTP operates on multi-view, multi-frame visual tokens produced by a vision encoder plus projector. Let $V$ synchronized camera views be indexed by $v \in \{1,\ldots,V\}$, each providing $T$ consecutive frames, and let each frame yield $P$ patch tokens with embedding dimension $C$. The full visual token tensor is

$X \in \mathbb{R}^{V \times T \times P \times C}.$

Within a view $v$, the temporal tensor is $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$, with tokens $x_{v,t,p} \in \mathbb{R}^{C}$, where $t$ indexes time and $v \in \{1,\ldots,V\}$0 indexes patch (Sha et al., 21 Apr 2026).

A defining design choice is the per-view constraint: tokens are pruned independently within each view to preserve camera identity, and selection is applied per view on the flattened sequence of $v \in \{1,\ldots,V\}$1 tokens (Sha et al., 21 Apr 2026). For each view $v \in \{1,\ldots,V\}$2, the goal is to select a subset $v \in \{1,\ldots,V\}$3 of size $v \in \{1,\ldots,V\}$4, thereby reducing tokens from $v \in \{1,\ldots,V\}$5 to $v \in \{1,\ldots,V\}$6 prior to spatial pruning (Sha et al., 21 Apr 2026). Cross-view relations are deferred until RSP; MTP is therefore temporally local to each camera stream.

ST-Prune defines motion by a temporal variance proxy over a fixed patch location $v \in \{1,\ldots,V\}$7 in a view: $v \in \{1,\ldots,V\}$8 This quantity captures appearance change at a fixed image location and is described as favoring evolving regions, such as moving actors and transient events, over temporally static background (Sha et al., 21 Apr 2026).

Temporal recency is injected through an exponential weighting

$v \in \{1,\ldots,V\}$9

with default $T$0. When $T$1 is the current frame, this bias prioritizes current-frame content (Sha et al., 21 Apr 2026). Both volatility and recency are independently min–max normalized before combination: $T$2 The paper uses equal weighting between volatility and recency inside $T$3, while the trade-off to diversity is controlled externally by $T$4, with default $T$5 and a reported stable range $T$6 (Sha et al., 21 Apr 2026).

This formulation is notable for not requiring explicit ego-motion compensation. ST-Prune states that volatility reflects appearance changes due to both object motion and ego-motion, and that the method does not require optical-flow warping or stabilization (Sha et al., 21 Apr 2026). Robustness is instead attributed to recency prioritization, diversity-based downselection of redundant background tokens, and subsequent cross-view pruning by RSP.

3. Diversity-aware objective and greedy optimization

MTP in ST-Prune extends standard greedy max–min diversity selection by integrating temporal importance as a soft constraint (Sha et al., 21 Apr 2026). The seed token is chosen to be simultaneously informative in feature norm and temporally important: $T$7 Subsequent selection uses a penalized redundancy score

$T$8

and chooses

$T$9

The similarity function is cosine similarity: $P$0 Minimizing the maximum similarity discourages the retention of tokens highly correlated with already selected ones, especially static background regions that barely change over time (Sha et al., 21 Apr 2026).

The algorithm is greedy, training-free, and plug-and-play. Per view, it proceeds by flattening $P$1 tokens, computing motion volatility and recency, forming $P$2, selecting the seed, iterating the argmin rule for $P$3 steps, and returning the retained set $P$4 (Sha et al., 21 Apr 2026). The reported computational complexity per view is $P$5, with the dominant term arising from computing max similarity to the evolving set (Sha et al., 21 Apr 2026).

The paper’s position study reports that inserting MTP inside the LLM phase yields the best performance, although the method is model-agnostic and can also be applied between projector and LLM (Sha et al., 21 Apr 2026). It further recommends applying MTP first and then RSP, preserving per-view identities throughout.

4. Event-assisted MTP in Event Cascade Pruning

EventPrune generalizes MTP beyond pure RGB feature volatility by introducing a continuous event-guided motion prior derived from event cameras (Ma et al., 19 May 2026). Each event is represented as

$P$6

with pixel location $P$7, timestamp $P$8, and polarity $P$9, triggered when

$C$0

The event stream $C$1 provides microsecond-level motion signals sensitive to edges and temporal transitions (Ma et al., 19 May 2026).

Event Cascade Pruning comprises three stages. Event-Triggered Causal Sampling (ETCS) partitions the event stream into fixed windows $C$2, applies a spatiotemporal density filter $C$3, computes window activity

$C$4

and activity change

$C$5

Given a target number of windows $C$6, ETCS selects a union of high-$C$7 windows and top-ranked $C$8 windows, then refines for temporal coverage and maps the result to key RGB frames (Ma et al., 19 May 2026). This stage is described as causal, lightweight, and robust to event jitter.

Event-guided Motion Saliency Filtering (EMSF) then aligns an event window $C$9 to each keyframe $X \in \mathbb{R}^{V \times T \times P \times C}.$0 and aggregates token-level event counts: $X \in \mathbb{R}^{V \times T \times P \times C}.$1 where $X \in \mathbb{R}^{V \times T \times P \times C}.$2 is the image support of token $X \in \mathbb{R}^{V \times T \times P \times C}.$3, $X \in \mathbb{R}^{V \times T \times P \times C}.$4 is the density-filtered event stream, and $X \in \mathbb{R}^{V \times T \times P \times C}.$5 is per-keyframe min–max normalization (Ma et al., 19 May 2026). Under retention ratio $X \in \mathbb{R}^{V \times T \times P \times C}.$6, EMSF keeps $X \in \mathbb{R}^{V \times T \times P \times C}.$7 tokens per keyframe by maximizing total motion saliency: $X \in \mathbb{R}^{V \times T \times P \times C}.$8

Event-Attention Ranking Fusion (EARF) fuses event saliency with text-conditioned attention. For pruning layer $X \in \mathbb{R}^{V \times T \times P \times C}.$9 and keyframe $v$0, attention scores are read from head-averaged post-softmax attention of the previous layer: $v$1 To handle scale mismatch and long tails, both attention and motion are projected to normalized ranks: $v$2 Fusion uses a layer-wise weight $v$3: $v$4 Under layer retention ratio $v$5, the method keeps

$v$6

tokens by maximizing summed calibrated score (Ma et al., 19 May 2026).

EventPrune identifies a specific failure mode of attention-only pruning, the Peripheral Sink, in which border tokens receive disproportionate attention and structure-relevant regions are under-retained under high sparsity (Ma et al., 19 May 2026). In this formulation, MTP is therefore not only motion-sensitive but also geometry-preserving through event-calibrated attention ranks.

5. Computational properties and integration patterns

Across both papers, MTP is presented as a mechanism for reducing the quadratic cost of self-attention by lowering the active visual token count before or during Transformer computation. In EventPrune, pruning from $v$7 to $v$8 reduces attention from $v$9 to $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$0; with uniform retention ratio $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$1, this becomes $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$2 (Ma et al., 19 May 2026). In ST-Prune, the relevant per-view temporal selection complexity is $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$3 (Sha et al., 21 Apr 2026).

The integration patterns differ in accordance with task and modality. ST-Prune is model-agnostic and can be inserted either between projector and LLM or inside the LLM, with the paper’s position study favoring pruning inside the LLM because of richer semantics in LLM token representations (Sha et al., 21 Apr 2026). MTP is applied per view before explicit multi-view fusion, and its output feeds into RSP for ring-view spatial redundancy removal (Sha et al., 21 Apr 2026). EventPrune operates at score and mask level; events are never injected into hidden states, and no event encoder or parameter updates are needed (Ma et al., 19 May 2026). The retained token set is then used in standard Transformer attention and standard greedy decoding, with KV-cache maintained only for retained tokens (Ma et al., 19 May 2026).

The following table summarizes the two MTP instantiations documented in the supplied literature.

Work	MTP formulation	Primary redundancy addressed
ST-Prune (Sha et al., 21 Apr 2026)	Motion-aware Temporal Pruning using volatility, recency, and diversity-aware greedy selection	Temporal redundancy within each camera view
EventPrune (Ma et al., 19 May 2026)	Event Cascade Pruning using ETCS, EMSF, and EARF	Temporal anchoring, token-level motion inactivity, and attention bias under dynamic egocentric motion

A plausible implication is that the broader category “MTP” now spans both purely feature-statistical formulations and sensor-assisted formulations, provided that motion priors guide token retention under a training-free regime.

6. Empirical behavior, misconceptions, and limitations

ST-Prune validates its complete spatio-temporal pruning pipeline on four benchmarks spanning perception, prediction, and planning: DriveLM, LingoQA, NuInstruct, and OmniDrive (Sha et al., 21 Apr 2026). At 10% retention, corresponding to approximately 90% token reduction, the reported results are: DriveLM Average 57.87 versus 59.11 for the full model, with Language improving to 52.81 versus 51.33; LingoQA Lingo-Judge 65.20, outperforming all pruning baselines; NuInstruct Average 27.78 versus 28.35, with mAP 19.74 while other pruning baselines collapse near zero; and OmniDrive Average 64.46 versus 64.41, essentially matching or slightly surpassing full performance (Sha et al., 21 Apr 2026). On the single-modality temporal benchmark LingoQA, the paper reports that MTP drives the improvement, reaching 68.20 at 25% retention (Sha et al., 21 Apr 2026). It also reports that adding temporal scores alone yields a $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$4 average gain over vanilla diversity, and combining temporal with spatial scores yields $$X_{\text{tem}}(v) \in \mathbb{R}^{T \times P \times C}$$5, indicating complementarity between MTP and RSP (Sha et al., 21 Apr 2026).

EventPrune reports that at 20% visual-token retention, ECP achieves 37.62% Avg.Acc versus 36.31% for the full-token baseline, along with 1.89x inference speedup and 52% GFLOPs reduction to 141.9 GFLOPs (Ma et al., 19 May 2026). On ESR-Real, a synchronized RGB-event benchmark with 700+ QA pairs, it reports 54.58% Avg.Acc at 50% retention versus 51.90% for the full-token baseline, and 50.07% at 20% retention versus 46.40% for the best baseline DTD (Ma et al., 19 May 2026). Component ablations averaged across retention ratios report latency improving from 2.08 s to 1.58 s with ETCS and a full-cascade trade-off of 1.16 s and 37.85% (Ma et al., 19 May 2026).

Several misconceptions are directly addressed by the supplied papers. First, MTP is not restricted to learned pruning modules: both works emphasize training-free operation (Sha et al., 21 Apr 2026, Ma et al., 19 May 2026). Second, MTP is not synonymous with explicit motion compensation: ST-Prune explicitly does not require optical-flow warping or stabilization (Sha et al., 21 Apr 2026). Third, MTP is not merely per-frame token pruning under another name: both papers contrast their methods with static-snapshot or flat-token approaches that discard temporal ordering or continuous motion causality (Sha et al., 21 Apr 2026, Ma et al., 19 May 2026).

The limitations are also explicit. ST-Prune notes that extreme camera motion can globally inflate volatility because motion is measured through feature volatility without explicit ego-motion compensation; it suggests that incorporating optical flow or stabilization could further refine volatility attribution (Sha et al., 21 Apr 2026). It also notes that closed-loop VLA benchmarks remain to be explored, where finer temporal ordering and action-conditioned selection may benefit from adaptive windowing or trajectory-aware scoring (Sha et al., 21 Apr 2026). EventPrune assumes synchronized RGB-event inputs and is not designed as a pure RGB-only accelerator; it identifies future work in distilling event priors into RGB-only signals (Ma et al., 19 May 2026).

Taken together, the available literature frames Motion Token Pruning as a training-free strategy for dynamic visual token selection that preserves temporally actionable and geometrically meaningful information while suppressing redundant background and attention-inefficient tokens. In ST-Prune, this is realized through volatility, recency, and diversity-aware per-view greedy selection for autonomous driving (Sha et al., 21 Apr 2026). In EventPrune, it is realized through causal event-triggered sampling, token-aligned motion saliency, and rank-fused attention calibration for first-person dynamic spatial reasoning (Ma et al., 19 May 2026). This suggests that MTP is emerging less as a single algorithm than as a design principle for efficient video-language inference under motion-sensitive workloads.