Unified Action Tokenization

• Unified action tokenization is a framework that transforms continuous, high-dimensional action sequences into discrete, semantically rich tokens for scalable embodied AI.
• It employs various methods—such as DCT, VQ, and B-spline encoding—to balance compression and decodability, achieving precise reconstruction and efficient policy execution.
• The approach enables the fusion of vision, language, and action signals within transformer architectures, boosting performance in complex, real-world robotic tasks.

Unified action tokenization refers to a set of representational schemes and algorithms that translate high-dimensional, temporally continuous action sequences into discrete token streams compatible with modern sequence models—most prominently, transformer-based architectures used for vision-language-action (VLA) reasoning and control. The unification aspect denotes frameworks that compress, structure, and semantically harmonize action representations to support generalization, expressivity, and interoperability across hierarchical reasoning, policy learning, and cross-modal integration. Unified action tokenization is central in scaling embodied AI from simple, end-to-end control to rich, semantically grounded decision-making, as it directly addresses the trade-off between precision in action execution and the global contextual reasoning needed for open-world robotic behavior (Liu et al., 30 Dec 2025, Pertsch et al., 16 Jan 2025, Zhou et al., 6 Jun 2025, Liu et al., 4 Dec 2025, Liu et al., 4 Feb 2026, Wang et al., 2024, Zhong et al., 2 Jul 2025).

1. Conceptual Foundations and Motivations

The challenge in embodied agent modeling lies in bridging continuous control signals (e.g., robot joint velocities, human body movements) with discrete, token-oriented architectures needed for language-based reasoning and generative planning. Early VLA approaches treated action outputs either as raw continuous vectors or naively discretized signals, failing to capture either semantic context or maintain reconstructive fidelity for high-frequency control (Zhong et al., 2 Jul 2025). Unified action tokenization aims to:

• Compress long, high-dimensional action streams to short, informative token sequences, maximizing compression without sacrificing precise decodability.
• Support autoregressive, bidirectional, or hybrid sequence modeling regimes, which underlie both instruction-conditioned reasoning and fine-grained policy execution.
• Provide fully decodable tokenization, avoiding issues of token sequence invalidity leading to execution errors.
• Align action tokens with the causal, left-to-right token emission order of autoregressive transformer models.
• Enable modular fusion of reasoning (e.g., chain-of-thought, visual-language cues) and control in a single token-centric agent framework (Liu et al., 30 Dec 2025, Liu et al., 4 Feb 2026, Wang et al., 2024).

A survey of action tokenization in VLA (Zhong et al., 2 Jul 2025) identifies linguistic, programmatic, spatial, trajectory, latent, and raw action tokens, arguing for a hierarchical or unified scheme to control trade-offs among expressiveness, interpretability, and end-to-end learnability.

2. Key Algorithms and Tokenization Techniques

Unified tokenization encompasses both analytical/parametric and fully learned schemes, often combining quantization, latent compression, and explicit causal ordering. Table 1 summarizes major tokenization approaches as reported in canonical works.

Method	Principle	Decodability	Ordering/Anytime	Causal Alignment
FAST(+/er)	DCT + BPE (or VQ)	Partial/Total	No/Yes (BAR in FASTer)	Partial/Yes
FACT	VQ + Flow-matching	Total	No	Yes
BEAST	B-spline encoding	Total	Yes (parallel)	Yes
OAT	Register, FSQ, causal mask	Total	Yes (prefix)	Yes
OmniJARVIS	FSQ latent tokens	Total	No	Yes (contextual)

2.1 Frequency-Space Action Sequence Tokenization (FAST/FAST+)

FAST maps continuous actions to tokens by:

• Quantile normalization of action dimensions.
• Discrete Cosine Transform (DCT) per dimension.
• Quantization and optional BPE to merge runs of zeros and common structures, yielding sequences of ≈20–60 tokens per second of control.
• Fully invertible for well-formed output, but variable-length and BPE partial tokenization can introduce invalid sequences under AR models.

FAST+ is a universal tokenizer trained on 1M cross-embodiment robot trajectories, supporting plug-and-play deployment (Pertsch et al., 16 Jan 2025).

2.2 Flow-matching Action Tokenizer (FACT)

FACT integrates a VQ encoder with a flow-matching decoder:

• Encodes $H \times S$ action trajectories via a VQ-encoder, compressing temporally ($L < H$) and across channels ($D < S$).
• Quantization is performed via a sign-based binarizer to form codes $c \in \{-1,+1\}^{L \times D}$.
• Decoding uses a rectified flow objective, training to reconstruct trajectories by predicting velocity fields along stochastic interpolants.
• FACT achieves sub-millimeter reconstruction error ($\text{MSE}\sim 10^{-3}$ for code length $N=20$), an order-of-magnitude improvement over FAST+ (Liu et al., 30 Dec 2025).

2.3 BEAST: B-Spline Encoded Action Sequence Tokenizer

BEAST projects actions onto B-spline bases:

• Approximates action chunks as degree-$P$ B-splines with $N$ control points per dimension.
• Control points are directly quantized (e.g., 8 bits per value) or encoded via a VAE for continuous latent tokens.
• Uniform token length ($D \times N$), supporting parallel decoding and guaranteeing $C^{P-1}$ smoothness at chunk boundaries.
• Enables fast inference (e.g., 617 Hz for BEAST-F) and trajectory smoothness unattainable by chunked binning (Zhou et al., 6 Jun 2025).

2.4 Ordered Action Tokenization (OAT)

OAT introduces left-to-right-ordered token spaces via:

• Transformer-with-registers encoders, with $H_l$ learnable register tokens causally masked.
• Per-coordinate scalar quantization (FSQ), packing each $d$-vector into a token in a finite vocabulary.
• Nested-dropout training, forcing the decoder to reconstruct from incomplete prefixes, imposing causally aligned information packing.
• Total decodability and prefix-based “anytime” decoding, offering a test-time trade-off between computation and action fidelity.
• Outperforms alternative schemes (e.g., FAST, binning) both in average task success and flexibility (e.g., monotonic improvement with prefix length) (Liu et al., 4 Feb 2026).

2.5 Unified Token Representations (UTR) in RL

UTR collapses return, state, and action modalities into a single token at each timestep:

• Significantly reduces sequence length and quadratic attention complexity in offline RL architectures.
• Provably tightens generalization bounds (via Rademacher complexity) and improves compute efficiency across transformer and CNN backbones without altering parameter count.
• Facilitates scaling of sequential decision models in resource-constrained scenarios (Tian et al., 24 Oct 2025).

3. Architectural Integration and Unified Agent Design

Unified action tokenization enables seamless fusion of vision, language, and action signals in large-scale autoregressive or prefix LM backbones:

• In GenieReasoner, image patch, instruction, and FACT action code streams are projected into the same embedding space, allowing a Transformer to process chain-of-thought (reasoning) and low-level execution jointly, aligning gradients from both sources (Liu et al., 30 Dec 2025).
• FASTerVQ’s 2D “patchified” action images and block-wise autoregressive decoding (BAR) afford highly compressed yet precise action representation. Block-wise AR decoding matches the RVQ code hierarchy, enhancing both training and inference speed (Liu et al., 4 Dec 2025).
• BEAST discrete tokens can be natively injected into the token space of large pretrained VLMs such as Florence-2, supporting generic encoder-decoder and parallelized robotic control (Zhou et al., 6 Jun 2025).
• OmniJARVIS extends the vocabulary of LLM-based Multimodal LLMs (MLMs) to behavior tokens, enabling a unified prefix LM objective over instructions, memories, thoughts, behavior tokens, and observations, with a learned IL policy decoding the behavioral tokens to executable agent actions (Wang et al., 2024).

These architectures support end-to-end training (e.g., cross-entropy on token prediction, policy loss on decoded actions), efficient inference, and transfer across embodiments, tasks, and data sources.

4. Empirical Benchmarks and Characteristic Results

Substantial empirical evidence demonstrates unified action tokenization’s advantages in compression, task success, reconstruction error, and cross-task/embodiment generalization:

• FACT attains an order of magnitude lower MSE (~0.002 vs. ~0.02 for FAST+) at the same code length, with real-robot grasp success rates of 0.73 (vs. 0.10 for $\pi_0$-FAST), and ERIQ accuracy gains of 24 points (action understanding from 65.50% to 96.67%) (Liu et al., 30 Dec 2025).
• FAST+ matches diffusion-based policies in zero-shot performance while reducing training cost by 5x; enables autoregressive VLAs to handle high-frequency, dexterous manipulation (Pertsch et al., 16 Jan 2025).
• BEAST achieves 0.0004±0.0005 MSE on 1D cubic spline fits, parallel inference at 617 Hz, and SOTA multi-task simulation scores (e.g., 86.4% LIBERO-Long split) (Zhou et al., 6 Jun 2025).
• OAT provides monotonic task success improvements with increasing prefix length and supports efficient “anytime” inference, yielding 56.3% LIBERO mean success at 8 tokens versus 23.0% for FAST at ~50 tokens (Liu et al., 4 Feb 2026).
• FASTer surpasses prior methods with a 97.9% Libero success (vs. 96.8% $\pi_{0.5}$), and only a 29% OOD drop (vs. FAST/Fast+ at 35–40%) (Liu et al., 4 Dec 2025).
• UTR achieves 67–92% normalized D4RL success rates with up to 75% reduction in FLOPs and sequence length, maintaining or improving SOTA performance (Tian et al., 24 Oct 2025).
• In OmniJARVIS, unified tokenization delivered a 16% absolute improvement in programmatic Minecraft tasks versus best LLM+controller (DEPS), reaching near-optimal skill-level rewards with minimal variance (Wang et al., 2024).

5. Practical Guidelines and Open Problems

Best practices for deploying unified action tokenization include:

• Balancing compression (tokens per chunk) against capacity, generally tuning the codebook size and token dimension for empirical reconstruction error and AR modelability (Liu et al., 4 Dec 2025, Liu et al., 4 Feb 2026).
• Employing total, prefix-decodable tokenization for safety-critical or latency-sensitive scenarios (OAT, BEAST), ensuring that all predicted sequences map to valid actions (Liu et al., 4 Feb 2026, Zhou et al., 6 Jun 2025).
• Using block-wise AR or parallel decoding to accelerate policy rollout (FASTer, BEAST-F) (Liu et al., 4 Dec 2025, Zhou et al., 6 Jun 2025).
• Modular vocabulary augmentation supports plug-in integration of action, vision, and language tokens into foundation models, enabling joint cross-modal learning (Wang et al., 2024, Zhou et al., 6 Jun 2025).
• Training with chain-of-thought and “reasoning” tokens for enhanced robustness in complex, multi-modal settings (Zhong et al., 2 Jul 2025, Liu et al., 30 Dec 2025).

Outstanding research directions include learning dynamic, data-adaptive tokenization horizons, codebook entropy regularization to prevent collapse, integrating diffusion or flow-based decoders over discrete tokens, and expanding to multi-modal (audio, tactile) or multi-agent action streams (Liu et al., 4 Dec 2025, Zhong et al., 2 Jul 2025, Liu et al., 4 Feb 2026).

6. Theoretical and Taxonomic Perspectives

Unified action tokenization can be formalized as a composition of encoding $$\mathcal{E}: \mathbb{R}^{H \times S} \rightarrow \mathcal{V}^N$$ and decoding $$\mathcal{D}: \mathcal{V}^N \rightarrow \mathbb{R}^{H \times S}$$ functions, where $\mathcal{V}$ is the token vocabulary, with the following properties:

• High compression ratio: $C = (H_a \cdot D_a)/H_l \gg 1$.
• Total decodability: $\forall T \in \mathcal{V}^{H_l},\, \mathcal{D}(T)$ is a valid action chunk.
• Causal structure: token sequence aligns with AR transformer order, enabling efficient prefix sampling and inference (Liu et al., 4 Feb 2026, Zhou et al., 6 Jun 2025).
• Alignment with cognitive and physical agent architectures, supporting hierarchical planner–controller decoupling, explicit reasoning, and robust control (Zhong et al., 2 Jul 2025).

The token taxonomy in VLA models (Zhong et al., 2 Jul 2025) establishes the representational space in which unified action tokenization operates, recommending hybrid schemes where different token types are layered or fused according to task and model demands.

7. Impact, Limitations, and Future Research

Unified action tokenization constitutes an enabling technology for scalable embodied intelligence, directly supporting generalization across embodiments, compositional reasoning, and high-frequency control with tractable sequence models. Remaining challenges include large-scale codebook learning, adaptive compression, safety and interpretability, and efficient integration into increasingly multimodal, open-world agent architectures (Liu et al., 30 Dec 2025, Liu et al., 4 Dec 2025, Zhou et al., 6 Jun 2025, Zhong et al., 2 Jul 2025).

Major directions involve on-policy RL fine-tuning of tokenized autoregressive policies, extensible fusion with audio/tactile modalities, agent memory augmentation, and development of total, non-autoregressive decoding regimes (e.g., diffusion, flow) over learned discrete action spaces. Hierarchical, unified action tokenization represents a synthesizing framework, connecting high-level semantic reasoning with precision control at a computationally and statistically efficient token level.