SQAP-VLA: Efficient VLA Inference

• SQAP-VLA is a structured training-free vision-language-action acceleration framework that co-designs quantization and token pruning for efficient embodied intelligence.
• The approach integrates Hadamard-aided 4-bit quantization with a three-part token pruning strategy to maintain attention fidelity and reduce computational load.
• Empirical results show up to 1.93× speedup, a 4.5% improvement in average success rate, and over 46% reduction in GPU memory usage compared to FP16 baselines.

SQAP-VLA is a structured, training-free Vision-Language-Action (VLA) inference acceleration framework that jointly enables state-of-the-art quantization and token pruning for embodied intelligence models whose deployment is constrained by computational cost, memory footprint, and inference latency (Fang et al., 11 Sep 2025). It is defined by a co-design between a Hadamard-aided, channel-wise 4-bit quantization scheme and a three-part quantization-aware token pruning pipeline intended to preserve attention fidelity under aggressive low-bit inference. In the reported experiments on standard VLA models, the framework is presented as achieving a $\times 1.93$ speedup and up to a $4.5\%$ average success rate enhancement relative to the original model while remaining entirely post-training and training-free (Fang et al., 11 Sep 2025).

1. Problem setting and motivation

Vision-Language-Action models such as CogAct, “$\pi_0$,” and DiffusionPolicy combine a vision encoder, a LLM backbone, and often a diffusion-based action decoder (Fang et al., 11 Sep 2025). Within this architecture, the LLM backbone dominates both floating-point operations and intermediate activation memory, particularly during the prefill stage. On an edge GPU such as a 3090, a full-precision FP16 VLA model can occupy more than $14\,\mathrm{GB}$ of peak VRAM and incur high inference latency, which the source describes as too large for mobile or battery-powered robots (Fang et al., 11 Sep 2025).

The framework is motivated by a specific incompatibility between two common acceleration mechanisms. Quantization alone, exemplified by W4A4 with 4-bit weights and activations, can reduce model size by $4\times$ and accelerate matrix operations, but aggressive quantization distorts intermediate feature distributions, especially the attention scores used for pruning. Token pruning, by contrast, relies on attention magnitudes to rank and drop visual tokens in a ViT-style encoder. According to the reported observation, naively applying pruning after quantization causes the attention score maps to become “scattered” and “shifted,” which leads top-$k$ or threshold-based pruning to select the wrong tokens and degrade performance (Fang et al., 11 Sep 2025).

A common misconception addressed by the framework is that quantization and token pruning can simply be composed as independent post hoc modules. The reported formulation rejects that assumption and treats the two operations as mutually coupled. This suggests that, for VLA systems in particular, token saliency estimation is not robust to low-bit perturbations unless the quantizer and the pruning rule are designed together.

2. Co-designed quantization pipeline

SQAP-VLA resolves the incompatibility by co-designing quantization and pruning rather than applying them independently (Fang et al., 11 Sep 2025). The high-level pipeline begins from a pretrained VLA model checkpoint such as CogAct, enhances the quantizer for better attention fidelity, quantizes weights and activations to 4 bits, and then performs quantization-aware visual token pruning at each inference step.

The quantization stage modifies the Q/K projections in two ways. First, it applies a per-channel, described parenthetically as “tensor-wise,” min/max quantizer for activations in the query and key projections. Second, it inserts a Hadamard rotation $H$ before and its inverse after those projections:

$(W_q^T H^T)\cdot(HX)=W_q^T X.$

The stated purpose is to spread out activation energy and suppress outliers, thereby making the post-quantization activation distribution more uniform (Fang et al., 11 Sep 2025). After this modification, weights and activations are quantized to W4A4 using computed scales and zero points.

The associated quantization function is given as

$$s=\frac{x_{\max}-x_{\min}}{2^b-1}, \qquad z=\mathrm{round}\!\left(-\frac{x_{\min}}{s}\right),$$

followed by

$$Q(x)=\mathrm{clamp}\!\left(\mathrm{round}\!\left(\frac{x}{s}+z\right),\,0,\,2^b-1\right), \qquad \hat{x}=s\cdot(Q(x)-z),$$

with $4.5\%$0 for W4A4 (Fang et al., 11 Sep 2025). The clamp operation bounds the integer representation to $4.5\%$1.

The Hadamard component is formalized by letting $4.5\%$2 be an $4.5\%$3 Hadamard matrix satisfying $4.5\%$4, and replacing

$4.5\%$5

with

$4.5\%$6

At full precision, the source states that $4.5\%$7, while quantization of $4.5\%$8 yields a more uniform activation distribution (Fang et al., 11 Sep 2025). The intended effect is not merely compression but preservation of attention-score semantics under low precision, which is critical because those scores are subsequently used by the pruning mechanism.

3. Quantization-aware token pruning

The pruning stage operates at each inference step and is built from three criteria that are explicitly designed to be robust to residual quantization noise (Fang et al., 11 Sep 2025). The framework defines a final kept-token set

$4.5\%$9

where each component serves a distinct role in preserving task-relevant information under aggressive token reduction.

The first criterion is Quantization-Insensitive Preservation. An attention vector $\pi_0$0 is computed from the task query token to the $\pi_0$1 visual tokens, and the top-$\pi_0$2 indices are selected:

$\pi_0$3

The source states that these top-$\pi_0$4 entries remain very stable under quantization noise (Fang et al., 11 Sep 2025). The associated attention-score vector after quantization is

$\pi_0$5

The second criterion is Robot-Aware Protection. Using known world coordinates $\pi_0$6 of the robot end-effector, the framework projects the point into image coordinates through

$\pi_0$7

then maps the image position to the patch-token grid $\pi_0$8 and defines an $\pi_0$9 ring of radius $14\,\mathrm{GB}$0 around the central token:

$14\,\mathrm{GB}$1

This imposes a hard retention rule for tokens associated with the robot region (Fang et al., 11 Sep 2025).

The third criterion is Spatially-Aware Sampling. From the remaining set

$14\,\mathrm{GB}$2

Farthest Point Sampling is applied to select $14\,\mathrm{GB}$3 additional tokens:

$14\,\mathrm{GB}$4

The description specifies that FPS starts from a seed point and repeatedly picks the point farthest in Euclidean distance from the currently selected set until $14\,\mathrm{GB}$5 points have been chosen (Fang et al., 11 Sep 2025).

Taken together, the three-way strategy is described as ensuring that task-critical tokens from top-$14\,\mathrm{GB}$6 attention, robot regions from the ring constraint, and a spatially diverse background from FPS all survive pruning (Fang et al., 11 Sep 2025). A plausible implication is that the design separates semantic saliency, embodiment-specific safety, and scene coverage into distinct retention mechanisms rather than forcing a single saliency score to encode all three.

4. Algorithmic structure and computational properties

The high-level algorithm begins with QuantizerSetup(), in which each Q/K weight and activation receives a Hadamard rotation, per-channel min/max statistics, and stored scale and zero-point parameters. QuantizeModel() then replaces weights by their dequantized low-bit form and applies the same logic to activations at runtime. InferenceWithPruning(input image) computes the full patch-token set $14\,\mathrm{GB}$7, forms the attention vector to the special “task” query token, constructs $14\,\mathrm{GB}$8, computes the image projection of the robot pose to obtain the ring set, uses FPS to fill the remaining token budget, and prunes all tokens in $14\,\mathrm{GB}$9 before the forward pass (Fang et al., 11 Sep 2025).

The hyperparameters reported for this procedure are a bit-width of $4\times$0 for both weights and activations, a pruning ratio $4\times$1 corresponding to keeping $4\times$2 of tokens, top-$4\times$3, a ring radius $4\times$4 of approximately 1–2 tokens, and an FPS budget $4\times$5 equal to the target keep count minus $4\times$6 (Fang et al., 11 Sep 2025).

A notable property of the method is the absence of any explicit joint-loss objective or regularizer. The framework is described as entirely post-training and training-free (Fang et al., 11 Sep 2025). This distinguishes it from compression procedures that require fine-tuning to recover accuracy after quantization or pruning.

The theoretical analysis included in the source assigns specific roles to each component. The rotation plus channel-wise quantizer is said to preserve the ordering of the very top attention entries even after quantization, the robot-aware ring supplies a hard constraint that critical visuomotor tokens cannot be pruned, and FPS provides a provable 2-approximation to the maximum min-distance coverage problem in low dimensions, ensuring spatial diversity (Fang et al., 11 Sep 2025). The complexity per inference step is reported as

$4\times$7

which is described as comparable to standard pruning alone, while quantization adds only $4\times$8 overhead per linear layer (Fang et al., 11 Sep 2025).

5. Empirical performance

The empirical evaluation reports results in two scenarios, “Visual Matching” and “Variant Aggregation,” comparing the FP16 CogAct baseline, an FP16 EfficientVLA configuration, and SQAP-VLA in W4A4 form (Fang et al., 11 Sep 2025). The principal metrics are task-wise success rates, average success rate, speed-up, and BOPs.

Scenario	Method	Avg SR / Speed-up / BOPs
Visual Matching	CogAct (baseline), FP16	74.8 / 1.0× / 100%
Visual Matching	EfficientVLA, FP16	76.4 / 1.59× / 45.1%
Visual Matching	SQAP-VLA, W4A4	79.3 / 1.93× / 26.3%
Variant Aggregation	CogAct (baseline), FP16	61.3 / 1.0× / 100%
Variant Aggregation	EfficientVLA, FP16	63.2 / 1.57× / 45.1%
Variant Aggregation	SQAP-VLA, W4A4	64.4 / 1.93× / 26.3%

In the Visual Matching scenario, SQAP-VLA records task success rates of 94.7 for PickCoke, 85.5 for MoveNear, 72.2 for O/C Drawer, and 64.8 for PlaceApple, yielding an average success rate of 79.3 with a 1.93× speed-up and 26.3% BOPs (Fang et al., 11 Sep 2025). In the Variant Aggregation scenario, it reports 92.8 for PickCoke, 80.6 for MoveNear, 27.2 for O/C Drawer, and 57.0 for PlaceApple, with an average success rate of 64.4 at the same 1.93× speed-up and 26.3% BOPs (Fang et al., 11 Sep 2025).

The source further attributes the end-to-end system speedup of 1.93× over FP16 CogAct to a 2.56× LLM acceleration composed of 2.09× from W4A4 quantization and 1.21× from token pruning (Fang et al., 11 Sep 2025). Peak GPU memory is reported to decrease from 14.3 GB for the FP16 baseline to 7.6 GB for SQAP-VLA, described as a drop greater than 46%, with effectively more than 73% of original BOPs pruned (Fang et al., 11 Sep 2025).

These results are significant because the framework does not merely maintain performance while compressing the model; in the reported settings it exceeds the original FP16 baseline in average success rate. This suggests that the co-designed pruning policy may act as an effective structural regularizer at inference time, although that interpretation is not itself stated as a formal causal claim in the source.

6. Ablations, limitations, and interpretation

The ablation study reports that a pruning ratio sweep identifies $4\times$9, corresponding to 60% token retention, as the setting with the best average success rate (Fang et al., 11 Sep 2025). Component ablations show a progression: W4A4 alone hurts success rate by approximately 3%; adding quant-insensitive top-$k$0 preserves some success rate but remains below FP16; adding robot-aware protection recovers most of the loss; and adding spatially-aware FPS yields 79.3% average success rate, which is reported as $k$1 above baseline (Fang et al., 11 Sep 2025). This sequence makes the role of each module explicit and supports the claim that the gain arises from their combination rather than from quantization or pruning in isolation.

The limitations are also stated directly. The pruning rate is fixed across all layers, while dynamic or per-layer $k$2 is left as future work. The Hadamard transform must be inserted in all Q/K layers, which introduces additional engineering complexity for custom kernels. Although the framework is training-free, it does not exploit any fine-tuning of scale factors or token masks (Fang et al., 11 Sep 2025).

Within the broader VLA compression landscape, SQAP-VLA is characterized as the first structured, training-free VLA inference acceleration framework that simultaneously enables state-of-the-art quantization and token pruning (Fang et al., 11 Sep 2025). Existing approaches are described as performing quantization or token pruning in an ad hoc manner and failing to enable both for holistic efficiency improvement because of their incompatibility. The central technical claim is therefore not merely that both mechanisms can be combined, but that they can be combined only if the quantizer is redesigned for attention fidelity and the pruner is redesigned for quantization robustness.

A reasonable synthesis of the method is that it treats the attention map as the shared interface between low-bit arithmetic and token selection. The quantization stage acts to preserve that interface, and the pruning stage acts to avoid over-reliance on any single noisy estimate by adding robot-aware protection and spatially-aware sampling. On that reading, SQAP-VLA is best understood as a co-designed inference-time systems framework for embodied multimodal transformers rather than as a standalone quantizer or a standalone token-pruning heuristic (Fang et al., 11 Sep 2025).