VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation

Abstract: Multi-task robot manipulation policies are challenging to learn from demonstration because traditionally a single network must select among qualitatively different action modes from a multimodal demonstration distribution, conditioned on language and visual context. A wrong mode selection means executing the wrong task or an action infeasible in the scene. Tokenizing continuous actions into a learned discrete codebook separates these modes at the representation level, offering structural advantages for multi-task learning. We propose VQActFlow, a multi-task manipulation policy that tokenizes action chunks and generates code sequences via Variational Flow Matching. VQActFlow maintains an explicit preference over action modes throughout generation. Inference-time guidance acts on this preference to steer mode commitment. We instantiate this with classifier-free guidance over language conditioning, which steers the policy toward the instructed action mode, and a learned codebook critic that supplies a complementary feasibility signal. We evaluate VQActFlow on three platforms: the LIBERO simulation benchmarks, a Unitree G1 humanoid performing whole-body pick-and-place, and an ALOHA-style bimanual platform performing contact-rich tasks. Across these benchmarks, VQActFlow outperforms both continuous and discrete baselines.

• The paper introduces a discrete action mode interface via VQ-VAE tokenization and variational flow matching to improve task-specific steering.
• It integrates classifier-free guidance and a contrastive codebook critic to refine action selection, achieving significant performance gains on LIBERO benchmarks.
• The framework outperforms continuous and previous VQ-based policies on both simulated and real hardware, offering enhanced task disambiguation and scene consistency.

VQActFlow: Explicit Action Mode Steering with Vector-Quantized Flow Matching for Multi-Task Robot Manipulation

Problem Definition and Motivation

Multi-task robot manipulation from demonstration is constrained by modalities where disparate task instructions yield qualitatively different action sequences under similar visual observations. Conventional end-to-end policies—whether trained in continuous or discrete latent spaces—struggle to select the correct mode for ambiguous scenarios, often executing feasible yet incorrect actions or failing to align with task instructions. Existing diffusion and flow-matching approaches generate actions in latent spaces that lack explicit mode representations. Recent vector-quantized policies such as "Discrete Policy" [12] employ VQ-VAE tokenization but revert to continuous sampling, recovering discrete indices only post-hoc and losing explicit mode commitment during generation.

Model Architecture and Methodology

VQActFlow introduces a robust categorical action mode interface by tokenizing action chunks via a VQ-VAE encoder, mapping continuous actions to a compressed sequence of discrete codebook indices. The frozen codebook then provides structured supervision for the subsequent policy.

A Variational Flow Matching (VFM) policy transports Gaussian noise toward embeddings of the VQ-VAE codebook, maintaining categorical distributions over code indices at every generation step. Training utilizes cross-entropy loss over codebook indices, ensuring the policy discriminates among competing action modes position-wise. Inference integrates the predicted velocity via posterior-weighted averaging of codebook embeddings, quantizing terminal states to produce discrete action token outputs.

The backbone is a Diffusion Transformer (DiT) with cross-attention to visual (ResNet-18) and language (CLIP-B/32) context, combined with adaptive normalization for time embeddings.

Guidance Mechanisms: Classifier-Free Guidance and Codebook Critic

Inference-time steering exploits the explicit categorical interface through two modalities:

• Classifier-Free Guidance (CFG): Linear extrapolation of conditional and unconditional model outputs biases the categorical distribution toward task-consistent modes, effectively sharpening task identity at each integration step.
• Contrastive Codebook Critic: An independently trained transformer evaluates the feasibility of candidate code sequences against negative samples constructed via temporal shuffling, random replacement, and mismatched context. The critic steers the policy via feasibility gradients, amplifying scene consistency and temporal coherence absent in CFG.

Both guidance mechanisms operate on the categorical mode distribution, enabling synergistic yet non-redundant performance improvements.

Experimental Evaluation

VQActFlow is evaluated on LIBERO benchmarks (LIBERO-Goal, LIBERO-90) and two real-world hardware platforms: Unitree G1 humanoid and ALOHA-style bimanual arms.

• CFG on LIBERO-Goal: VQActFlow achieves 81.0% success rate at CFG weight $w=4$, outperforming Discrete Policy (61.5% at $w=6$). Performance is consistent across tasks, with substantial gains in scenarios with unguided headroom and minimal regression in near-ceiling tasks.
• Multi-task Scaling on LIBERO-90: The policy with combined CFG ($w=2$) and critic guidance ($\lambda=1.0$) attains 80.5% success, surpassing all continuous and VQ-based baselines. Critic guidance independently contributes +2.0% improvement; combined mechanisms provide non-redundant gains.
• Humanoid Whole-Body Manipulation: CFG elevates success from 23.8% to 57.5%, primarily by correcting wrong-task failures. Execution quality (missed grasps) remains unchanged, reflecting the sharp task identity encoded by the guidance interface.
• Bimanual Manipulation: CFG boosts VQActFlow to 73.8% (compared to Discrete Policy at 61.3%), and critic guidance further improves outcomes, especially for tasks requiring execution precision. The combined configuration achieves a 77.5% success rate.

The quantitative results demonstrate that explicit categorical mode representation enables more effective inference-time guidance, yielding higher success rates and consistent performance across heterogeneous task distributions.

Technical Implications

The explicit action mode interface resolves the ambiguity inherent in multimodal demonstration distributions, allowing real-time steering toward task-consistent and feasible actions at deployment. The categorical cross-entropy supervision ensures sharp mode discrimination; continuous embedding regression fails to provide comparable structure. The codebook critic's feasibility signal introduces a versatile mechanism for scene grounding and temporal coherence, adaptable to new constraints without retraining the policy.

The categorical structure facilitates integration of language-conditioned guidance (CFG), suggesting future VLA pipelines could leverage VFM as an action head with natural token alignment.

Practical and Theoretical Prospects

Practically, VQActFlow's modular guidance interface supports seamless deployment on contact-rich hardware platforms, overcoming limitations of continuous policies which exhibit instability and poor trajectory smoothness.

Theoretically, the information bottleneck imposed by VQ-VAE limits representational capacity; enhanced tokenization schemes may mitigate this while retaining discrete mode structure. The synergistic combination of CFG and feasibility critics provides a template for latent policy steering applicable to lifelong learning, knowledge transfer, and generalist robot policies.

Future research may explore more expressive codebooks, hierarchical tokenization, and tighter integration with vision-language-action models. Further, the categorical interface lends itself to scaling via large demonstration datasets and reinforcement from feedback or reward-guidance modules.

Conclusion

VQActFlow establishes that maintaining explicit preference over discrete action modes throughout generation delivers a superior interface for inference-time guidance in multi-task robot manipulation. Empirically, it consistently outperforms both continuous and VQ-based baselines across simulated and real hardware platforms. The categorical structure enables sharper task disambiguation and scene-consistent execution via classifier-free guidance and a contrastively-trained feasibility critic. The results indicate strong potential for discrete mode-based policies as foundational components in scalable multimodal robot architectures (2606.21600).