ReconPruner: Plug-and-Play Visual Token Pruning

• ReconPruner is a reconstruction-based, foreground-aware token pruner that reduces computational overhead in Vision-Language-Action systems for autonomous driving.
• It uses MAE-style pixel reconstruction combined with an adversarial foreground-background strategy to selectively retain task-critical tokens.
• Designed as a plug-and-play module, it achieves state-of-the-art performance on nuScenes benchmarks with significant reductions in FLOPs and latency.

ReconPruner is a plug-and-play visual token pruner introduced within FastDriveVLA for efficient Vision-Language-Action (VLA) models in end-to-end autonomous driving. It addresses the high computational overhead induced by long visual token sequences by prioritizing foreground information through MAE-style pixel reconstruction, rather than relying on visual token similarity or visual-text attention. In the reported formulation, ReconPruner is trained with a novel adversarial foreground-background reconstruction strategy and then reused, without retraining, across different VLA models that share the same visual encoder; the accompanying FastDriveVLA system is evaluated on the nuScenes closed-loop planning benchmark and reported to achieve state-of-the-art results across different pruning ratios (Cao et al., 31 Jul 2025).

1. Problem setting and design rationale

ReconPruner is situated in the specific setting of autonomous driving VLA systems, where long visual token sequences increase FLOPs and latency. The motivating premise is that conventional visual token pruning methods developed for generic Vision-LLMs perform poorly in driving scenarios because they emphasize either token similarity or visual-text attention, whereas driving requires retention of task-critical foreground regions. The method is explicitly motivated by the assertion that human drivers concentrate on relevant foreground areas while driving, and that retaining visual tokens containing this foreground information is essential for effective decision-making (Cao et al., 31 Jul 2025).

This design rationale distinguishes ReconPruner from pruning schemes that are agnostic to scene semantics. In the FastDriveVLA formulation, pruning is not treated as a generic compression step but as a foreground-aware selection problem coupled to downstream action reasoning. A plausible implication is that token importance is defined less by redundancy reduction in the embedding space than by reconstructive sufficiency for physically meaningful scene content.

2. Architecture and token scoring mechanism

ReconPruner is described as lightweight. Its core components are a PrunerLayer, implemented as a single transformer decoder layer, and a Scorer, implemented as a single-layer MLP with shape $\mathbb{R}^{D \times 1}$, for a total of 0.07B parameters. It takes as input visual tokens $V \in \mathbb{R}^{N \times D}$ and a learnable query token $Q \in \mathbb{R}^{1 \times D}$, and it outputs saliency scores $S \in \mathbb{R}^{N \times 1}$ for the visual tokens (Cao et al., 31 Jul 2025).

Component	Role	Specification
PrunerLayer	Jointly processes query and visual tokens	Single transformer decoder layer
Scorer	Produces per-token saliency	Single-layer MLP, $\mathbb{R}^{D \times 1}$
Query token $Q$	Learnable relevance probe	$\mathbb{R}^{1 \times D}$
Visual tokens $V$	Tokenized visual input	$\mathbb{R}^{N \times D}$
Saliency scores $S$	Token selection signal	$V \in \mathbb{R}^{N \times D}$0

The scoring pipeline is given by

$V \in \mathbb{R}^{N \times D}$1

where $V \in \mathbb{R}^{N \times D}$2 denotes the Hadamard product. The query and each token are therefore fused before scoring, so that relevance is computed with respect to task-critical regions rather than token content in isolation. This coupling of token embeddings to a learnable query is central to the method’s foreground-selective behavior (Cao et al., 31 Jul 2025).

3. Reconstruction objective and adversarial foreground-background training

ReconPruner is trained through MAE-style pixel reconstruction. The training loop assigns a saliency score to every token, selects foreground tokens above a saliency threshold, and uses those tokens to reconstruct masked foreground regions at the pixel level. The supervision signal is based on the premise that high-quality reconstruction is only possible if the selected tokens genuinely encode salient foreground content, thereby incentivizing high scores on tokens needed for task-relevant reconstruction. The formulation is explicitly inspired by Masked AutoEncoder methods (He et al., 2021).

A central problem identified for this training regime is degeneracy: without additional constraints, the pruner could assign high saliency to all tokens and recover trivial perfect reconstruction. FastDriveVLA addresses this with an adversarial foreground-background reconstruction strategy. In addition to reconstructing the foreground from high-saliency tokens, the model reconstructs the background using only the low-saliency, pruned tokens. The binary mask is defined by $V \in \mathbb{R}^{N \times D}$3 if $V \in \mathbb{R}^{N \times D}$4, else $V \in \mathbb{R}^{N \times D}$5, and for backpropagation the Straight-Through Estimator is used:

$V \in \mathbb{R}^{N \times D}$6

Foreground and background token partitions are then formed as

$V \in \mathbb{R}^{N \times D}$7

and decoded by a reconstruction decoder $V \in \mathbb{R}^{N \times D}$8 into predicted foreground and background images (Cao et al., 31 Jul 2025).

The foreground and background reconstruction losses are weighted sums of MSE and SSIM, with the total objective

$V \in \mathbb{R}^{N \times D}$9

typically using $Q \in \mathbb{R}^{1 \times D}$0 and $Q \in \mathbb{R}^{1 \times D}$1. The reported effect of this dual objective is to prevent universal high scoring and to enforce a sharper separation between retained foreground information and discarded background content. In the method’s own terms, good pruning is judged not only by what is kept but also by how little foreground information remains reconstructible from the pruned tokens (Cao et al., 31 Jul 2025).

4. Training data, deployment workflow, and plug-and-play reuse

To train ReconPruner, FastDriveVLA introduces nuScenes-FG, a large-scale dataset consisting of 241K image-mask pairs with annotated foreground regions. This explicit supervision is an important part of the method’s design: unlike pruning approaches that infer saliency indirectly, ReconPruner is trained with ground-truth foreground supervision and is thereby oriented toward scene elements identified as critical to driving, including lanes, vehicles, and pedestrians (Cao et al., 31 Jul 2025).

Its deployment procedure is defined as plug-and-play. Once trained on a labeled dataset such as nuScenes-FG, ReconPruner can be inserted without re-training into any VLA model sharing the same vision encoder architecture, with Qwen2.5-VL given as an example. At inference time, for pruning ratio $Q \in \mathbb{R}^{1 \times D}$2, the method retains the top

$Q \in \mathbb{R}^{1 \times D}$3

tokens according to saliency,

$Q \in \mathbb{R}^{1 \times D}$4

and passes the retained visual tokens together with text tokens into the downstream policy:

$Q \in \mathbb{R}^{1 \times D}$5

The stated significance of this interface is modularity: token pruning is inserted before the downstream language/action module and does not require architectural modification or end-to-end retraining of the recipient VLA model (Cao et al., 31 Jul 2025).

5. Benchmark results and comparative performance

ReconPruner is evaluated through FastDriveVLA on the nuScenes closed-loop planning benchmark, using Impromptu-VLA as the tested VLA model. The reported metrics are trajectory error (L2, cm), collision rate (%), and intersection with road boundary (%). At 25%, 50%, and 75% pruning, FastDriveVLA with ReconPruner is reported to outperform all attention-based baselines, specifically FastV and SparseVLM, and all similarity-based baselines, specifically VisPruner and DivPrune, across L2, collision, and intersection metrics (Cao et al., 31 Jul 2025).

The reported efficiency gains are up to 7.5$Q \in \mathbb{R}^{1 \times D}$6 reduction in FLOPs and 3.7$Q \in \mathbb{R}^{1 \times D}$7 reduction in latency at standard pruning ratios, while maintaining high trajectory accuracy and safety. The exposition further notes that, in some cases, pruning with ReconPruner exceeds the performance of the unpruned baseline. The stated interpretation is that removing background or redundant tokens can benefit downstream planning by filtering out irrelevant visual context. This does not imply that pruning is uniformly accuracy-preserving in every regime; rather, it indicates that the method’s pruning criterion is aligned closely enough with driving-relevant semantics to make selective compression occasionally beneficial rather than merely tolerable (Cao et al., 31 Jul 2025).

6. Relation to reconstruction-based pruning in other domains

ReconPruner belongs to a broader family of reconstruction-based pruning methods, but its reconstruction target and pruning granularity are domain-specific. In LLM pruning, reconstruction commonly refers to matching dense-model predictions or activations on calibration data. "Rethinking Pruning LLMs: Benefits and Pitfalls of Reconstruction Error Minimization" shows that block-wise reconstruction, global propagation, and cross-block reconstruction can reduce cumulative reconstruction error by approximately 87–94% relative to standard layerwise schemes, while also showing that aggressive error minimization can overfit calibration data and worsen perplexity or downstream accuracy (Shin et al., 2024). In reasoning-model compression, "Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction" replaces prompt-only calibration with joint reconstruction over prompt and on-policy chain-of-thought activations, because reasoning is decode-dominated and standard pruning can induce longer, worse chains of thought (Lucas et al., 15 Sep 2025).

In computer vision and tracking, reconstruction-based pruning can operate at yet other levels. "Efficient Multi-Object Tracking on Edge Devices via Reconstruction-Based Channel Pruning" minimizes feature-map reconstruction error under channel removal, uses a greedy strategy inspired by ThiNet, and combines gated groups with DepGraph to manage architectural dependencies, achieving model-size reductions of up to 70% on edge-oriented MOT systems (Müller et al., 2024). "Neural Pruning for 3D Scene Reconstruction: Efficient NeRF Acceleration" instead studies neuron-level pruning in NeRF MLPs through uniform sampling, importance-based pruning, and coreset-based pruning, with coreset-driven pruning giving a 50% reduction in model size and a 35% speedup in training at moderate compression (Ding et al., 1 Apr 2025).

These comparisons clarify a frequent misconception: reconstruction-based pruning is not a single algorithmic template. ReconPruner reconstructs masked foreground and background pixels with annotated semantic supervision; LLM methods reconstruct dense-model activations or predictions; MOT channel pruning reconstructs feature maps; NeRF pruning reconstructs scene fidelity through compressed radiance-field MLPs. This suggests that the substantive novelty of ReconPruner lies less in the abstract use of reconstruction than in coupling reconstruction to foreground semantics specific to autonomous driving.

7. Scope, advantages, and interpretive boundaries

Within FastDriveVLA, ReconPruner is characterized by four recurring properties: foreground awareness, task relevance, discriminative pruning, and modular reuse. Foreground awareness comes from explicit supervision with nuScenes-FG; task relevance comes from MAE-style pixel-level reconstruction tied to driving-critical scene elements; discriminative pruning comes from the adversarial foreground-background reconstruction strategy; and modular reuse comes from the ability to transfer a trained pruner across VLA models with the same visual encoder, without retraining (Cao et al., 31 Jul 2025).

Its scope is nevertheless specific. ReconPruner is not presented as a universal compression mechanism for arbitrary multimodal models; the transfer condition is explicitly restricted to models sharing the same visual encoder architecture. Nor is it merely an attention approximation module, since its saliency scores are trained against reconstruction objectives rather than derived from text alignment or token similarity. A plausible implication is that its strongest applicability is in settings where foreground regions can be annotated or otherwise supervised and where downstream control depends on preserving a relatively small set of semantically dense visual tokens.

As used in FastDriveVLA, ReconPruner therefore occupies a distinct position in the pruning literature: it is a reconstruction-based, foreground-supervised, plug-and-play token selector for autonomous driving VLAs, designed to reduce visual-token cost while preserving—or in some cases improving—closed-loop planning performance under substantial pruning ratios (Cao et al., 31 Jul 2025).