LightVLA: Efficient Token Pruning for VLA Models

• The paper introduces a parameter-free differentiable pruning mechanism that reduces surplus visual tokens in VLA models while enhancing task performance.
• It employs a dynamic, task- and scene-conditioned strategy that retains core tokens (including the [CLS] token) without auxiliary losses or fixed pruning ratios.
• Empirical evaluations on LIBERO show a 59% reduction in FLOPs, 38% lower latency, and a 2.9-point increase in success rate compared to baseline methods.

LightVLA is a differentiable visual token pruning framework for vision-language-action (VLA) models that targets the dominant attention cost induced by large visual token sets in robotic policy inference. Its central design is to prune patch-level visual tokens adaptively, in a task- and scene-conditioned manner, before they enter the LLM backbone, while preserving the $[\mathrm{CLS}]$ token and the original position IDs of retained visual tokens. In the formulation reported by the authors, LightVLA is parameter-free in its main form, requires no auxiliary loss and no fixed pruning ratio hyperparameter, and is trained purely through the downstream VLA objective inherited from fine-tuning the base model (Jiang et al., 16 Sep 2025). The framework is presented as a response to the observation that, in many VLAs, visual tokens substantially outnumber language tokens, so attention over the visual sequence becomes a deployment bottleneck; the paper further argues that redundant visual tokens are not only computationally wasteful but can also distract the policy from task-relevant evidence (Jiang et al., 16 Sep 2025).

1. Definition and design objective

LightVLA is defined in the source paper as “a simple yet effective differentiable token pruning framework for vision-language-action (VLA) models” (Jiang et al., 16 Sep 2025). The intended problem setting is a standard VLA pipeline in which images are encoded into visual tokens, language instructions are encoded into language tokens, and the fused representation is used to predict robot actions. The framework operates on the premise that, because the visual sequence length $L_v$ is typically much larger than the language sequence length $L_l$, reducing $L_v$ is one of the most direct ways to reduce computational overhead in the decoder layers of the multimodal backbone (Jiang et al., 16 Sep 2025).

The paper’s conceptual claim is stronger than acceleration alone. It argues that efficiency and task performance are not inherently opposed in VLA acceleration, because many visual patches correspond to background, irrelevant objects, or duplicated information. Under this view, adaptive pruning is not merely a compression heuristic; it is a performance-driven mechanism by which the model learns which regions matter for control, and therefore which regions can be removed without harming—and in some reported settings while improving—task execution (Jiang et al., 16 Sep 2025).

Formally, the base VLA decomposition used in the paper is: $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$

$H = f_{\phi}(H_v, H_l), \qquad A = f_a(H),$

with the pruning module defined as

$H'_v = f_p(H_v) \subseteq H_v.$

Only patch-level visual tokens are pruned; the $[\mathrm{CLS}]$ token is always retained (Jiang et al., 16 Sep 2025).

2. Pruning mechanism and mathematical formulation

The main LightVLA mechanism is inserted after the vision encoder and projector and before the LLM backbone. Instead of using learnable compression queries, the framework generates parameter-free dynamic queries directly from the current visual and language token embeddings. The query-generation equation given in the paper is

$$Q = \text{softmax}\!\left(\frac{H_v H_l^T}{\sqrt{D}}\right) H_l,$$

where $Q \in \mathbb{R}^{L_v \times D}$ (Jiang et al., 16 Sep 2025).

These dynamic queries then score all visual tokens: $L_v$0 with $L_v$1. The ideal hard selection rule is written as

$L_v$2

Because there are $L_v$3 queries, pruning emerges through query collisions: if many queries select the same token, the unique retained set shrinks; if selections are diverse, more tokens survive. The resulting retention count is therefore adaptive rather than fixed by a hand-designed ratio (Jiang et al., 16 Sep 2025).

To make this differentiable, the paper replaces the non-differentiable $L_v$4 with a Gumbel-softmax-style straight-through construction: $L_v$5

$L_v$6

$L_v$7

$L_v$8

and the pruned token set is then

$L_v$9

Here $L_l$0 is sampling noise, and $L_l$1 denotes stop-gradient (Jiang et al., 16 Sep 2025).

A notable property of the training formulation is the absence of an explicit pruning loss. The paper states that LightVLA uses no auxiliary loss, no token supervision, and no fixed retention target. The pruning behavior is therefore learned indirectly through the original VLA fine-tuning objective: gradients from downstream action prediction determine which tokens are repeatedly selected and which are suppressed (Jiang et al., 16 Sep 2025). The paper also reports a modified noise schedule in which the upper bound $L_l$2 is decayed over training, with the stated motivation that large noise early encourages exploration of diverse pruning patterns and smaller noise later stabilizes token selection (Jiang et al., 16 Sep 2025).

3. Architecture, variants, and implementation

All main experiments in the LightVLA paper are built on OpenVLA-OFT. The reported base architecture consists of a two-branch vision encoder—DINOv2 and SigLIP—together with LLaMA-2-7B as the LLM backbone (Jiang et al., 16 Sep 2025). LightVLA interfaces with this model by pruning visual tokens between the vision encoder/projector and the LLM, thereby reducing the token sequence that all downstream decoder layers must process (Jiang et al., 16 Sep 2025).

The paper distinguishes two variants. The primary method, LightVLA, is parameter-free: it forms queries from current visual-language features and introduces no additional trainable parameters (Jiang et al., 16 Sep 2025). The secondary variant, LightVLA$L_l$3, uses learnable compression queries and therefore adds trainable parameters. Two LightVLA$L_l$4 formulations are described. At the vision encoder, the learnable-query score is

$L_l$5

where $L_l$6. At early LLM layers, the paper defines

$L_l$7

where $L_l$8 is learnable and $L_l$9 is a learnable trade-off scalar initialized to $L_v$0 (Jiang et al., 16 Sep 2025).

The implementation details reported for fine-tuning are specific. The experiments use 8 Nvidia H20 GPUs, start from an open-source OpenVLA-OFT checkpoint, apply LoRA rank 32 to the entire model—including the vision encoder, LLM backbone, and action head—and run for 40,000 steps. The learning rate is $L_v$1 initially and is decayed to $L_v$2 after 30,000 steps. Batch size is 8 per device, with global batch size 64 (Jiang et al., 16 Sep 2025). At inference time, no Gumbel noise is used; selection reverts to direct $L_v$3 (Jiang et al., 16 Sep 2025).

The paper emphasizes that LightVLA preserves the position IDs of retained visual tokens, because the LLM uses them to maintain spatial relationships among patches (Jiang et al., 16 Sep 2025). It also stresses compatibility with inference frameworks such as vLLM and SGLang, precisely because the main method does not rely on internal LLM attention maps (Jiang et al., 16 Sep 2025).

4. Empirical results on LIBERO

The evaluation is conducted on LIBERO, using the four standard suites—LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long—each with 500 expert demonstrations across 10 tasks and 50 trials per task, or 500 trials per suite (Jiang et al., 16 Sep 2025). The reported metric is success rate.

The central quantitative comparison is against OpenVLA-OFT. The paper reports that OpenVLA-OFT achieves an average success rate of 94.5, with 512 visual tokens, 8.8 TFLOPs, and 34 ms latency. Under the same benchmark, LightVLA achieves an average success rate of 97.4, with an average of 78 visual tokens, 3.6 TFLOPs, and 21 ms latency (Jiang et al., 16 Sep 2025).

Model	Efficiency	Avg SR
OpenVLA-OFT	512 tokens, 8.8 TFLOPs, 34 ms	94.5
LightVLA	78 tokens, 3.6 TFLOPs, 21 ms	97.4

The paper summarizes these changes as a 59.1% reduction in FLOPs, a 38.2% reduction in latency, and a 2.9-point improvement in task success rate (Jiang et al., 16 Sep 2025). It also states that, among the acceleration methods in its comparison table, LightVLA is the only one that improves performance while accelerating (Jiang et al., 16 Sep 2025).

Suite-level retention counts are also reported. Starting from 512 visual tokens in OpenVLA-OFT, LightVLA retains $L_v$4 tokens on Spatial, $L_v$5 on Object, $L_v$6 on Goal, and $L_v$7 on Long (Jiang et al., 16 Sep 2025). This corresponds to roughly 15% token retention on average. The paper interprets the variation across suites as evidence that the pruning ratio is adaptive rather than fixed (Jiang et al., 16 Sep 2025).

The score table in the paper gives the following suite-level results for LightVLA: Spatial 98.4, Object 98.4, Goal 98.2, and Long 94.6, compared with OpenVLA-OFT at 95.2, 94.2, 95.2, and 93.2, respectively (Jiang et al., 16 Sep 2025). It also reports that LightVLA outperforms all other listed baselines in average LIBERO score, including CogACT at 93.6, $L_v$8 at 94.2, SmolVLA at 88.8, and WorldVLA$L_v$9 at 81.8 (Jiang et al., 16 Sep 2025).

5. Ablations and interpretive findings

The ablation studies in the paper focus on the token-selection mechanism rather than on large-scale architectural changes. On the noise schedule, three settings are compared: full LightVLA, LightVLA without sampling noise, and LightVLA without the noise decay schedule. The reported averages are 97.4 with 78 tokens for full LightVLA, 97.0 with 72 tokens without noise, and 97.0 with 112 tokens without schedule (Jiang et al., 16 Sep 2025). The authors interpret the no-noise variant as pruning too aggressively and the constant-noise variant as failing to stabilize, leaving too many tokens (Jiang et al., 16 Sep 2025).

A second ablation tests whether the selected subset is close to optimal. Starting from LightVLA’s selected $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$0 tokens, the paper either adds $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$1 random tokens or removes 10% of the selected tokens. The resulting average success rates are 96.8 for $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$2 tokens and 96.6 for $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$3 tokens, compared with 97.4 for the original $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$4-token set (Jiang et al., 16 Sep 2025). The paper presents this as evidence that the retained set is not merely small, but specifically informative for control (Jiang et al., 16 Sep 2025).

The LightVLA$$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$5 comparison further clarifies the role of learned queries. OpenVLA-OFT scores 94.5. LightVLA$$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$6 at the vision encoder scores 96.2; at LLM layer #1, 97.0; at layer #2, 96.7; and at layer #3, 96.6. The parameter-free LightVLA still attains the best average result at 97.4 (Jiang et al., 16 Sep 2025). This is a notable outcome because it suggests that, within the reported setup, additional trainable query parameters are not necessary to obtain the strongest average score.

The qualitative token-visualization analysis is consistent with the paper’s performance-driven pruning thesis. In a LIBERO-Long task, “Put both moka pots on the stove,” retained tokens concentrate around the moka pots, the stove, and the robot arm, while most background patches are pruned (Jiang et al., 16 Sep 2025). The figure also shows that the number of retained tokens changes across frames depending on task phase. A plausible implication is that LightVLA behaves less like a fixed compressor and more like a task-conditioned perception bandwidth allocator, although the paper itself frames this observation descriptively rather than as a separate formal claim (Jiang et al., 16 Sep 2025).

6. Position within efficient VLA research

Within the broader efficient-VLA literature, LightVLA belongs to the class of token-level compression methods. Its most direct contrast in the provided corpus is with structural layer reduction and action-generation redesign. Shallow-$$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$7, for example, argues that, for flow-based VLAs, layer-level efficiency can yield stronger wall-clock benefits than token-level compression because transformer depth is sequential and action heads are repeatedly executed across denoising steps (Jeon et al., 28 Jan 2026). In that paper’s LIBERO comparison, “LightVLA” is listed as a token-compression baseline with average success 97, FLOPs 2.91 T, and CUDA time 22.0 ms, while a distilled $$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$8-L6 student is reported at average success 95, FLOPs 1.30 T, and CUDA time 11.3 ms (Jeon et al., 28 Jan 2026). This positions LightVLA as effective in success retention and token reduction, but not as the only path to low latency.

ReactVLA addresses a different efficiency bottleneck: iterative generative action sampling. It uses improved Mean Flow and Attention Residuals to reduce action-generation latency, reporting 18.3 ms inference latency and 88.0 average success on LIBERO for a 0.39B model (Guo et al., 12 Jun 2026). The relevant contrast is that LightVLA compresses perception tokens before the backbone, whereas ReactVLA redesigns the generative action mechanism itself (Guo et al., 12 Jun 2026). A plausible implication is that the two approaches are complementary rather than mutually exclusive, since they attack different cost centers.

LiteVLA-Edge occupies yet another part of the efficient-VLA landscape: on-device deployment through supervised fine-tuning, 4-bit GGUF quantization, llama.cpp runtime, and ROS 2 integration on Jetson Orin-class hardware, with a mean end-to-end latency of 150.5 ms and reasoning frequency of 6.64 Hz (Williams et al., 3 Mar 2026). Unlike LightVLA, which is evaluated on LIBERO simulation and focuses on token pruning, LiteVLA-Edge is a systems recipe for embedded execution (Williams et al., 3 Mar 2026). cVLA, by contrast, simplifies the action interface by predicting image-grounded trajectory waypoints rather than low-level controls, making training more efficient and more embodiment-agnostic in its stated scope (Argus et al., 2 Jul 2025).

These neighboring lines of work matter for interpreting the term “lightweight VLA.” In the recent literature represented here, the phrase refers to at least four distinct strategies: visual token pruning in LightVLA (Jiang et al., 16 Sep 2025), layer distillation in Shallow-$$H_v = f_v(X_I) \in \mathbb{R}^{L_v \times D}, \qquad H_l \in \mathbb{R}^{L_l \times D},$$9 (Jeon et al., 28 Jan 2026), low-step reactive generation in ReactVLA (Guo et al., 12 Jun 2026), and embedded quantized deployment in LiteVLA-Edge (Williams et al., 3 Mar 2026). LightVLA is therefore most precisely understood as the token-pruning branch of efficient VLA design, not as a generic label for all compact VLA systems.

7. Scope, limitations, and nomenclature

The LightVLA paper is explicit about several boundaries. Its experimental evidence is on LIBERO simulation rather than real-world hardware deployment (Jiang et al., 16 Sep 2025). The method is demonstrated mainly on OpenVLA-OFT, so cross-architecture generalization is suggested but not comprehensively established in the reported experiments (Jiang et al., 16 Sep 2025). It also depends on fine-tuning rather than serving as a purely training-free acceleration method (Jiang et al., 16 Sep 2025). For LightVLA$H = f_{\phi}(H_v, H_l), \qquad A = f_a(H),$0, some variants rely on decoder attention outputs and therefore lose compatibility with frameworks such as vLLM and SGLang (Jiang et al., 16 Sep 2025).

It is also important to distinguish LightVLA from similarly named systems. “LightVA” is a lightweight visual analytics framework based on LLM-agent task planning and execution, and the corresponding paper explicitly states that it does not introduce a system called “LightVLA” (Zhao et al., 2024). “LiteVLA-Edge” is a deployment-oriented on-device VLA pipeline for embedded robotics rather than the differentiable token pruning framework described here (Williams et al., 3 Mar 2026). The naming overlap is therefore potentially confusing, but the technical content is distinct.

The broader significance of LightVLA lies in its claim that, for robot action models, perceptual sparsification can be learned from the control objective itself. The paper’s title—“The Better You Learn, The Smarter You Prune”—encapsulates that position (Jiang et al., 16 Sep 2025). In its reported results, the framework reduces the average visual token count from 512 to 78, lowers FLOPs from 8.8 to 3.6, reduces latency from 34 ms to 21 ms, and improves average LIBERO success from 94.5 to 97.4 (Jiang et al., 16 Sep 2025). Within the efficient-VLA literature, that combination makes LightVLA a canonical example of task-conditioned, differentiable visual token pruning for robot control.