DA-PTQ: Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models

Abstract: Vision-Language-Action models (VLAs) have demonstrated strong potential for embodied AI, yet their deployment on resource-limited robots remains challenging due to high memory and computational demands. While Post-Training Quantization (PTQ) provides an efficient solution, directly applying PTQ to VLAs often results in severe performance degradation during sequential control. We identify temporal error accumulation as a key factor, where quantization perturbations at the vision-language-to-action interface are progressively amplified, leading to kinematic drift in executed trajectories. To address this issue, we propose Drift-Aware Post-Training Quantization (DA-PTQ), which formulates quantization as a drift-aware optimization problem over sequential decision processes. DA-PTQ consists of two components: (1) Cross-Space Representation Compensation, which mitigates structured distortions between multimodal representations and action space to improve action consistency, and (2) Motion-Driven Mixed-Precision Allocation, which assigns bit-widths by minimizing trajectory-level motion errors. Extensive experiments show that DA-PTQ significantly reduces kinematic drift and achieves comparable performance to full-precision models under low-bit settings, enabling practical deployment of VLAs on resource-limited robotic platforms.

• The paper presents a drift-aware post-training quantization framework that analytically compensates multimodal distortions to mitigate trajectory drift in vision-language-action models.
• It employs a three-stage calibration pipeline using cross-space representation compensation and drift-aware mixed-precision allocation, achieving up to 42.5% memory reduction and 54.8% speedup.
• Empirical results show that DA-PTQ preserves kinematic fidelity across diverse robotic platforms while generalizing effectively to cross-embodiment shifts.

Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models

Introduction

Vision-Language-Action (VLA) models are central to scalable embodied AI, integrating multimodal perception and policy architectures for robotic manipulation based on visual and linguistic instructions. As such models scale in both backbone and action generation capacity, their deployment on resource-constrained platforms is hindered by substantial memory and inference cost. While Post-Training Quantization (PTQ) is established in reducing computational burdens in language and vision transformers, direct application to VLAs leads to degraded and unstable sequential control—manifested as compounding temporal error and kinematic drift. "DA-PTQ: Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models" (2604.11572) introduces a paradigm shift, reframing VLA quantization as a drift-aware sequential optimization rather than a static reconstruction constraint. This essay provides an expert analysis of DA-PTQ, its components, empirical contributions, and implications for efficient embodied AI.

Challenges in Quantizing Vision-Language-Action Models

VLAs implement policies $\Pi_\theta$ mapping multimodal observations to continuous control sequences, often via diffusion-based action decoders. In contrast to unimodal generation, quantization errors at the vision-language-to-action interface ($\mathbf{z}_t$) propagate and amplify both over iterative denoising (intra-step) and closed-loop control (inter-step), contravening assumptions of locality and independence underlying standard PTQ. Jacobian-based kinematic analyses reveal that quantization-induced errors—especially in highly coupled serial chains—induce geometric drift that is further magnified across time-steps, ultimately manifesting as trajectory deviation and task failure (Figure 1).

Figure 1: Conventional PTQ introduces distortion at the conditioning interface, driving drift in sequential execution; DA-PTQ stabilizes this by explicitly compensating cross-space representation and allocating precision based on motion-driven criteria.

Prior VLA quantization approaches (e.g., QuantVLA, QVLA, SQAP-VLA) have attempted to stabilize the conditioning interface or channel bit allocation but rely on static sensitivity heuristics, lacking explicit modeling of error propagation and interface misalignment. Consequently, aggressive quantization yields substantial degradation in dynamic tasks, especially under domain and embodiment shift.

DA-PTQ: Framework and Technical Contributions

DA-PTQ is formulated as a training-free, three-stage post-training calibration pipeline that (i) analytically compensates multimodal representation distortions, and (ii) allocates mixed-precision weights in action generation modules based on trajectory-level, drift-aware sensitivity.

Figure 2: DA-PTQ framework overview, jointly aligning cross-space representations and allocating precision using Jacobian-induced drift sensitivity.

1. Cross-Space Representation Compensation (CSRC)

CSRC rectifies distributional shifts between quantized and full-precision multimodal activations at the interface layer via analytic affine and low-rank transformations. Unlike prior methods based solely on diagonal/statistical matching, DA-PTQ jointly optimizes per-channel scaling and dense covariance alignment by solving an SVD-regularized matrix regression. These parameters are folded directly into quantized weights, introducing no runtime overhead yet providing fine-grained correction of both diagonal and structured off-diagonal distortions.

2. Drift-Aware Mixed-Precision Allocation (DA-MPA)

Error propagation is modeled via structural Jacobian analysis of a virtual serial chain corresponding to the robot's kinematics—a design-agnostic and differentiable mechanism. The objective function $\mathcal{L}_{\text{drift}}$ penalizes action-space deviations re-weighted by dimension-wise drift sensitivity, which is analytically estimated through Jacobian pseudo-inverse projection. Layers within the action decoder receive bit allocations (e.g., BF16 for top $k\%$ most drift-sensitive, 4-bit otherwise) proportional to their accumulated effect on trajectory-level error, drastically outperforming uniform or static channel-based heuristics.

3. Calibration Pipeline and Zero-Overhead Integration

The procedure is strictly post-training: calibration statistics—means, covariances, sensitivity scores—are estimated using a modest dataset of 512 trajectories. No task-specific fine-tuning is required, and both compensation and allocation steps are fused into the static model graph.

Empirical Results

Efficiency-Fidelity Trade-offs

DA-PTQ consistently matches or surpasses baselines across both efficiency and control fidelity, attaining a 42.5% memory reduction and a 54.8% inference speedup with only marginal loss in average success compared to full-precision models in SimplerEnv (see Figure 3). Notably, aggressive quantization using prior art (QuantVLA) suffers pronounced degradation, particularly in tasks demanding precision over long horizons.

Figure 3: Pareto frontier of success rate versus efficiency for ablation variants on SimplerEnv; DA-PTQ yields substantial gains without severe trade-off in task success.

Task Performance and Breakdown

On the WidowX embodiment—across diverse manipulation tasks—DA-PTQ mitigates the drop in success rate relative to full-precision CogACT, outperforming both VLA-Cache (inference caching, no quantization) and QuantVLA. On high-sensitivity tasks such as "Put Spoon on Towel", DA-PTQ nearly closes the gap to FP, evidencing effective preservation of kinematically sensitive control (Figure 4, Figure 5).

Figure 4: WidowX Sequential Execution — smooth, oscillation-free control signals confirm cross-space compensation efficacy.

Figure 5: WidowX—Eggplant in Basket; DA-PTQ maintains precise, drift-free trajectory over extended horizon.

Cross-Embodiment Generalization

A key claim is the ability of DA-PTQ to generalize calibration parameters trained on one embodiment (WidowX/BridgeData V2) to another (Google Robot), where kinematic properties, workspace, and sensory statistics shift nontrivially. DA-PTQ retains high success rates in both Visual Matching and Variant Aggregation regimes, outperforming prior quantization methods which destabilize under such cross-domain shifts (Figure 6).

Figure 6: Google Robot generalization—DA-PTQ preserves coherent action sequencing despite differing embodiment and workspace.

Ablation and Qualitative Analysis

Ablation experiments decompose DA-PTQ into CSRC and DA-MPA components, affirming that each independently yields non-trivial improvements, yet the integrated pipeline is strictly superior. CSRC alone stabilizes the conditioning signal but is inadequate in the presence of severe drift, while DA-MPA alone limits error amplification but suffers interface bias. Only their composition achieves optimal Pareto efficiency with robust trajectory fidelity.

Qualitative inspection of multi-DoF action curves further corroborates that DA-PTQ eliminates quantization-induced noise and oscillations, with continuous control traces remaining smooth and physically plausible even in long-horizon tasks (Figures 4–6).

Implications and Future Directions

DA-PTQ establishes that model quantization in sequential embodied control must address both representation-level and trajectory-level error simultaneously. By integrating cross-space compensation and kinematic-aware allocation, DA-PTQ provides a template for post-training optimization in sequential and feedback-sensitive architectures beyond robotics—e.g., neuro-symbolic agents, decision-transformers, and process control with hybrid action spaces. Its zero-overhead inference and modest calibration cost make it tractable for real-world deployment.

Limitations include reliance on simplified Jacobian-based error propagation, which may underrepresent error geometry in contact-rich or highly nonlinear domains. Fixed calibration datasets also limit domain adaptation; integrating distributionally robust or task-conditioned calibration is a key direction. Combining DA-PTQ with further pruning (e.g., token or parameter) and exploring synergy with prompt adaptation could yield even greater gains in real-world embodied systems.

Conclusion

DA-PTQ reframes post-training quantization for VLAs as a drift-aware sequential optimization, offering cross-modality representation alignment and analytically driven precision allocation without additional runtime cost. Empirical results demonstrate robust efficiency, stable long-horizon manipulation, and generalization across embodiments. The framework lays a foundation for efficient embodied AI under stringent resource constraints and invites future research into adaptive, theoretically grounded quantization and compression for sequential multimodal control (2604.11572).