GridS: Differentiable Grid Sampler

• GridS is a plug-and-play module that compresses visual tokens by learning salient geometric coordinates via differentiable interpolation.
• It reframes token compression as continuous, geometry-aware resampling, preserving essential details like object boundaries, contact points, and grasp locations.
• GridS achieves over 90% token reduction and a 76% FLOP decrease while maintaining manipulation success on both benchmark and real robotic platforms.

Differentiable Grid Sampler, abbreviated GridS, is a plug-and-play visual token compression module for vision-language-action (VLA) models introduced to reduce the computational cost of robotic manipulation without destroying the fine geometric information on which manipulation depends (Feng et al., 12 May 2026). In the reported formulation, GridS reframes compression from discrete token pruning to continuous, geometry-aware resampling inside the vision encoder: it predicts a minimal set of salient coordinates and extracts features at those coordinates via differentiable interpolation, with the stated goal of preserving object boundaries, contact points, grasp locations, and spatial relations while reducing the visual token count to fewer than 10% of the original tokens (Feng et al., 12 May 2026).

1. Definition and motivating problem

GridS was proposed against the background that existing VLA acceleration methods largely treat visual tokens as items to discard or keep, a design that can be acceptable for semantic tasks but is problematic for embodied manipulation (Feng et al., 12 May 2026). In robotic settings, the visual stream is not only a source of category-level evidence; it is also the medium through which the policy infers action-relevant geometry. The reported motivation is therefore that standard token pruning methods, while lowering compute, tend to destroy fine geometric information such as object boundaries, contact points, grasp locations, and spatial relations (Feng et al., 12 May 2026).

The central problem is described as a trade-off between aggressive token reduction and preservation of geometry. More pruning reduces FLOPs and speeds inference, but can degrade performance because spatially critical patches disappear; less pruning better preserves spatial structure, but remains too expensive for real-time deployment (Feng et al., 12 May 2026). GridS is introduced specifically to break the usual trade-off between aggressive token reduction and preserving geometry by replacing deletion with resampling (Feng et al., 12 May 2026).

A common misconception is to treat token compression in VLA as equivalent to generic semantic token selection. The reported argument is that this equivalence does not hold in contact-rich manipulation, because losing a small region can mean missing the gripper target, the object edge, or the placement zone (Feng et al., 12 May 2026). In that sense, GridS is positioned not merely as a speedup technique, but as a geometry-preserving compression mechanism.

2. Compression as continuous, geometry-aware resampling

The defining conceptual move in GridS is to recast compression as continuous, geometry-aware resampling rather than hard token pruning (Feng et al., 12 May 2026). Instead of selecting token indices and removing the rest, GridS learns a small set of salient coordinates in image space and samples features from those coordinates using differentiable interpolation (Feng et al., 12 May 2026). The resulting compressed representation remains tied to the original spatial layout.

This formulation addresses the failure mode of hard pruning directly. When a pruning method drops a token, the exact local spatial content represented by that token is lost. GridS instead predicts where geometry matters for the downstream task and resamples features at those locations, allowing compression while retaining a structured representation of important spatial regions (Feng et al., 12 May 2026). The method is therefore characterized in three ways in the source description: task-aware, because coordinates are learned for the downstream manipulation task; continuous, because coordinates are not restricted to discrete patch centers; and plug-and-play, because the module can be inserted into existing VLA pipelines (Feng et al., 12 May 2026).

The distinction between pruning and resampling is central. Hard pruning implements a brittle “drop low-score tokens” strategy, whereas GridS is designed to keep the vision encoder informative at very low token budgets by learning better token placement rather than merely keeping fewer tokens (Feng et al., 12 May 2026). This suggests that the method’s contribution lies as much in the representation change as in the reduction of token count.

3. Architecture and processing pipeline

The reported pipeline consists of five stages (Feng et al., 12 May 2026):

1. Input visual tokens from the VLA vision encoder.
2. A coordinate prediction module estimates a minimal set of salient spatial locations.
3. GridS forms a sampling grid from those coordinates.
4. Features are extracted via differentiable interpolation / grid sampling from the original visual feature map.
5. The resulting compressed visual tokens are passed to the downstream VLA policy.

A notable design choice is that GridS compresses within the vision encoder, not after the fact (Feng et al., 12 May 2026). Rather than pruning already-formed tokens independently of geometry, it transforms the feature map into a resampled representation. This internal placement matters because the module operates before spatial structure has been irreversibly discarded.

The coordinate predictor is described at a high level as a lightweight predictor that outputs coordinate parameters conditioned on the visual input and task context (Feng et al., 12 May 2026). The exact predictor architecture is not specified in the provided description, but the mechanism is stated as: take visual tokens or features, predict $K$ normalized 2D points, and use those points as the grid for resampling (Feng et al., 12 May 2026). Because the method is defined in terms of coordinates rather than discrete token indices, it is explicitly not limited to choosing among existing patch centers.

The source description notes a conceptual resemblance to spatial transformers or deformable attention, while emphasizing that GridS is specifically aimed at token reduction with geometry preservation (Feng et al., 12 May 2026). This should be understood as contextual comparison rather than an assertion of architectural identity.

4. Mathematical formulation and end-to-end optimization

The abstract explicitly states that GridS uses adaptive prediction of salient coordinates and differentiable interpolation (Feng et al., 12 May 2026). Although the full method equations are not given in the provided description, the standard formulation implied by “grid sampler” is presented as follows.

Let the visual feature map be $F \in \mathbb{R}^{H \times W \times C}$, and let the coordinate predictor output $K$ sampling points: $G = \{(x_k, y_k)\}_{k=1}^{K}.$

These coordinates define a sampling grid. The sampled feature at location $(x_k, y_k)$ is obtained by bilinear interpolation from the neighboring lattice points: $$\hat{F}(x_k,y_k) = \sum_{i,j} w_{ij}(x_k,y_k)\, F(i,j),$$ where $w_{ij}$ are interpolation weights determined by the relative distances from $(x_k,y_k)$ to the surrounding discrete positions (Feng et al., 12 May 2026).

In bilinear sampling, if $(x,y)$ lies between four pixels $(x_0,y_0), (x_1,y_0), (x_0,y_1), (x_1,y_1)$, then: $F \in \mathbb{R}^{H \times W \times C}$0 with

$F \in \mathbb{R}^{H \times W \times C}$1

The significance of this formulation is that the sampling operation is described as fully differentiable with respect to the coordinates, so the coordinate predictor can be trained end-to-end (Feng et al., 12 May 2026). Gradients flow from the VLA loss into the sampled features, through the interpolation weights, and back to the predicted coordinates (Feng et al., 12 May 2026). This is identified as the core advantage over hard pruning, whose token selection is discrete and usually non-differentiable or only weakly approximated.

Training is described as end-to-end with the downstream VLA objective (Feng et al., 12 May 2026). The provided description gives no indication that GridS requires a separate reconstruction loss; the stated emphasis is on being a plug-in module optimized for task success (Feng et al., 12 May 2026). A plausible implication is that the learned coordinates are supervised only indirectly, through their contribution to policy performance.

5. Distinction from prior token reduction strategies

The source description groups prior token reduction strategies under token pruning, sparsification, caching, or attention-based token reduction (Feng et al., 12 May 2026). These methods generally operate by ranking tokens by importance, dropping low-scoring tokens, merging tokens, or caching and reusing tokens. They are characterized as often semantic or attention-driven, but not explicitly geometry-preserving (Feng et al., 12 May 2026).

GridS differs along several stated dimensions (Feng et al., 12 May 2026):

• Continuous-space sampling: it samples in continuous space rather than selecting discrete tokens only.
• Preservation of spatial structure: it retains features at task-relevant coordinates.
• Learning where geometry matters: it does not rely only on global saliency scores.
• Differentiability: the sampling locations are optimized for the final manipulation objective.
• Extremely aggressive compression: it is designed to remain effective even at very low token budgets.

The resulting contrast is not simply methodological but representational. Prior pruning strategies remove information once a token is discarded; GridS attempts to preserve action-relevant geometry by relocating representational capacity to learned coordinates (Feng et al., 12 May 2026). This supports the paper’s framing of the method as a rethink of compression itself: not as pruning, but as resampling.

A related misconception is that aggressive compression necessarily implies severe manipulation failure. The reported argument is narrower: performance degradation is associated with compression mechanisms that lose geometry, not with low token count as such (Feng et al., 12 May 2026). GridS is presented as evidence that very low token budgets can remain viable when token placement is learned and geometry-aware.

6. Reported empirical results and practical significance

Experiments are reported on the LIBERO benchmark and on a real robotic platform (Feng et al., 12 May 2026). The headline claims are that GridS preserves essential spatial information while reducing the visual token count to fewer than 10% of the original tokens, achieves a 76% reduction in FLOPs, and does so with no degradation in the success rate (Feng et al., 12 May 2026). The paper further states that this validates the lowest feasible visual token count reported to date (Feng et al., 12 May 2026).

Setting	Reported outcome	Source
LIBERO benchmark	No degradation in success rate	(Feng et al., 12 May 2026)
Real robotic platform	No degradation in success rate	(Feng et al., 12 May 2026)
Visual token compression	Fewer than 10% of original visual tokens	(Feng et al., 12 May 2026)
Compute reduction	76% reduction in FLOPs	(Feng et al., 12 May 2026)

The description does not include full per-task success tables, but its stated interpretation is that GridS matches baseline success while running substantially faster on both simulated benchmark tasks and real-robot deployment (Feng et al., 12 May 2026). The reported “lowest feasible visual token count” claim is particularly notable because it suggests an empirical lower bound, in the authors’ evaluation, on how far visual perception can be compressed before manipulation ceases to remain reliable.

The practical significance is tied to real-time robotic control. The provided description states that sharply reducing FLOPs while keeping success unchanged can lower latency, reduce energy consumption, enable deployment on more modest hardware, and make advanced embodied policies more accessible outside large server-class setups (Feng et al., 12 May 2026). Because control loops often run under strict timing constraints, this suggests that the principal systems-level value of GridS is not only computational efficiency in isolation, but compatibility between VLA perception and deployment-time control requirements.

In summary, GridS is described as a differentiable, geometry-preserving visual resampling module for VLA systems whose core innovation is to replace brittle token dropping with task-aware coordinate prediction and differentiable grid sampling, allowing compression to under 10% of the original visual tokens while maintaining success rate in the reported evaluations (Feng et al., 12 May 2026).