See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model

Abstract: Vision-Language-Action (VLA) models have shown remarkable promise in robotics manipulation, yet their high computational cost hinders real-time deployment. Existing token pruning methods suffer from a fundamental trade-off: aggressive compression using pruning inevitably discards critical geometric details like contact points, leading to severe performance degradation. This forces a compromise, limiting the achievable compression rate and thus the potential speedup. We argue that breaking this trade-off requires rethinking compression as a geometry-aware, continuous token resampling in the vision encoder. To this end, we propose the Differentiable Grid Sampler (GridS), a plug-and-play module that performs task-aware, continuous resampling of visual tokens in VLA. By adaptively predicting a minimal set of salient coordinates and extracting features via differentiable interpolation, GridS preserves essential spatial information while achieving drastic compression (with fewer than 10% original visual tokens). Experiments on both LIBERO benchmark and a real robotic platform demonstrate that validating the lowest feasible visual token count reported to date, GridS achieves a 76% reduction in FLOPs with no degradation in the success rate. The code is available at https://github.com/Fediory/Grid-Sampler.

• The paper introduces a novel differentiable grid sampler that adaptively selects salient visual tokens to preserve geometric fidelity in robotic tasks.
• It achieves a 76% reduction in FLOPs and retains fewer than 10% of the initial tokens without compromising manipulation success rates.
• This approach enables real-time deployment on commodity hardware and paves the way for future research in efficient multimodal representations.

Differentiable Grid Sample Pruning for Efficient and Generalizable Vision-Language-Action Models

Introduction

The paper "See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model" (2605.11817) addresses a critical challenge in the deployment of Vision-Language-Action (VLA) models for robotic manipulation: prohibitive computational cost during inference due to the high dimensionality of visual token streams. Standard vision transformers and their multimodal extensions for VLA encode high-resolution, dense features, which leads to heavy memory and compute requirements. While previous token pruning strategies reduce input dimensionality, they compromise geometric fidelity essential to robotic tasks, resulting in degraded manipulation performance. The authors propose a geometry-aware, differentiable resampling method—Differentiable Grid Sampler (GridS)—to break the trade-off between computational efficiency and task-critical detail retention.

Methodology

GridS is introduced as a plug-and-play vision encoder module that adaptively and continuously resamples the visual token grid. Instead of discrete hard-pruning or conventional token selection, GridS predicts a minimal, task-aware set of salient spatial coordinates for token extraction. Features at these coordinates are obtained through differentiable interpolation, ensuring geometric continuity and preserving crucial context, such as contact points and object boundaries.

The GridS pipeline consists of:

• Coordinate Prediction: A lightweight network infers the most informative sample locations on a 2D grid, conditioned on task, language, or intermediate visual features.
• Differentiable Resampling: Visual features corresponding to selected coordinates are interpolated from the dense feature map using bilinear or higher-order kernels, enabling gradients to flow through both the feature extractor and the coordinate predictor.
• Plug-and-Play Integration: GridS is designed for modular integration into state-of-the-art VLA frameworks, regardless of backbone architecture.

Compared to prior patch-sampling or hard token-pruning approaches [(Bai et al., 13 Oct 2025, Qu et al., 27 Jan 2025), 3a2ef31a], GridS redefines efficiency as a continuous spatial sampling problem, sidestepping the fundamentally lossy bottleneck of hard selection.

Experimental Results

Comprehensive empirical evaluation is conducted on both the LIBERO benchmark (Bai et al., 13 Oct 2025, Black et al., 2024) and real-robot platforms. Key numerical highlights:

• Compression Rate: GridS achieves retention of fewer than 10% of the original visual tokens without loss of downstream performance.
• FLOPs Reduction: The grid sample pruning results in a 76% reduction in floating-point operations during inference, representing the lowest visual token count reported to date for general-purpose VLA models.
• Task Success Rate: No statistically significant drop in manipulation success rate is observed when compared to unpruned or less aggressively pruned models, in contrast to previous token reduction methods that often incur notable accuracy loss at similar compression levels.

Ablations further demonstrate that task-aware, continuous sampling preserves geometric details—particularly contact points and fine spatial cues—better than any discrete token selection baseline.

Implications and Theoretical Considerations

The implications of GridS extend across practical and theoretical domains:

• Practical Deployment: By sustaining high task performance at drastically reduced computation, GridS facilitates real-time VLA model deployment on consumer-grade commodity hardware and edge devices. This advance aligns with 'Green AI' paradigms by substantively reducing energy consumption for robotic inference workloads.
• Generalization: GridS enhances the transferability of VLA models across novel tasks and environments by focusing on spatially and semantically salient features, rather than relying on input grid resolution or prior object-centric heuristics. This property is emphasized as critical for scaling to diverse robotic manipulation settings.
• Theory of Information Preservation: The proposed continuous tokenization framework reframes token compression as an optimization of spatial information preservation, potentially informing future studies on minimal sufficient representations within multimodal transformers.

Limitations and Future Directions

While GridS achieves unprecedented compression-accuracy trade-offs, several open research questions remain:

• Dynamic Task-Adaptive Sampling: Extending coordinate prediction modules to explicitly incorporate multi-step temporal history, language priors, and uncertainty modeling could further improve efficiency and robustness.
• End-to-End Learning Synergies: Integrating GridS with other token pruning (Pertsch et al., 16 Jan 2025), semantic sparsification [semanticvla2026], and dynamic attention mechanisms may yield further synergistic gains.
• Generalization across Backbone Architectures: While demonstrated on standard VLA encoders, systematic investigation is warranted for cross-modal transformers with intricate spatial-linguistic alignment, such as PaLM-E [driess2023palm] or X-VLA (Zheng et al., 11 Oct 2025).

Conclusion

The Differentiable Grid Sampler establishes a new paradigm for efficiency in VLA models by enabling geometry-aware, continuous spatial token pruning. It realizes unprecedented visual token compression—with over 76% FLOPs reduction—without degrading robotic manipulation success rates, validated across both synthetic benchmarks and real-world tasks. This advancement not only democratizes access to advanced VLA capabilities by making models deployable on modest hardware, but also suggests directions for rethinking spatial tokenization and information preservation in multimodal architectures. Future work should address dynamic adaptation, multi-modal coordination, and broader architecture generalization, toward fully scalable embodied intelligence.