Mixture of Horizons (MoH) in Robotic Action

• Mixture of Horizons (MoH) is a strategy that injects multi-scale temporal prediction into full-attention modules, balancing fine-grained control with long-term planning.
• It partitions action chunks into overlapping horizons that are processed in parallel via a shared transformer, achieving efficiency with minimal computational overhead.
• MoH employs cross-horizon consensus for adaptive inference, dynamically selecting action prefixes to boost throughput and success rates in robotic tasks.

A Mixture of Horizons (MoH) is a plug-and-play strategy for injecting multi-scale temporal prediction into full-attention neural action modules, designed to address the inherent trade-off in vision-language-action (VLA) models between fine-grained temporal precision (short horizons) and global foresight (long horizons). Developed in the context of robotic manipulation, MoH dynamically partitions action chunks into overlapping horizons, processes these in parallel with a shared transformer, and fuses their predictions via a lightweight linear gate. This enables simultaneous exploitation of both short-term accuracy and long-term planning within a single architecture, with minimal added computational or memory overhead. MoH further supports dynamic inference through cross-horizon consensus, which adapts the executed action prefix based on inter-horizon agreement, thereby increasing throughput and robustness in both simulated and real-world robotic tasks (Jing et al., 24 Nov 2025).

1. Motivation: The Horizon Trade-Off in Action Chunking

In VLA models for robotic manipulation, action sequences are divided into fixed-length “chunks” for tractable prediction and execution. The horizon—the chunk length chosen for training and inference—affects model performance critically. Longer horizons offer better global foresight but often reduce fine-grained accuracy, as errors compound further into the future and local corrections become more difficult. Conversely, short action horizons facilitate precise local control, but limit the model’s ability to coordinate extended behaviors, resulting in poor performance on long-term tasks. Empirical results demonstrate that selecting any fixed horizon represents a trade-off that is suboptimal for complex mixed-horizon tasks (Jing et al., 24 Nov 2025).

2. Core Method: Multi-Horizon Chunking and Parallel Transformer Design

Let the model at timestep $t$ predict an action chunk $$A_t = (a_{t,1}, ..., a_{t,H}) \in \mathbb{R}^{H \times d_a}$$, where $d_a$ is the action dimension and $H$ is the maximum chunk length. MoH introduces a set of “candidate horizons” $\mathcal{H} = \{ h_1, h_2, ..., h_N \}$ such that $0 < h_1 < ... < h_N = H$. For each horizon $h \in \mathcal{H}$, the ground-truth chunk is truncated: $$A_t^{(h)} = (a_{t,1}, ..., a_{t,h}) \in \mathbb{R}^{h \times d_a}$$. Each chunk $A_t^{(h)}$ is zero-padded to length $H$ and masked so that positions $k > h$ are ignored in self-attention. This enables all horizon-augmented input variants to be processed in parallel by a shared action transformer head of depth $L$, leveraging tensorized computation and weight sharing for efficiency.

The VLM backbone encodes multi-modal context—including multi-view images $V_t$, past history $h_{<t}$, language $T$, and proprioceptive input $s_t$—once per step. The representation then feeds into the parallel transformer, with time and positional embeddings optionally applied (flow-matching: time embedding $\tau$; one-step regression: learnable query token). For each horizon and step, the transformer outputs $Z_t^{(h)} \in \mathbb{R}^{h \times d}$, masked appropriately.

3. Horizon Gating, Prediction, and Fusion

Above the shared transformer, a “gate head”—a single linear layer—computes unnormalized log-weights $g_{t,k,h}$ for each time step $k$ and horizon $h$. Softmax normalization is performed over the set $\mathcal{H} : h \geq k$, ensuring that gating is only performed among active horizons for each index:

$$\alpha_{t,k,h} = \frac{\exp(g_{t,k,h})}{\sum_{h' \in \mathcal{H}: k \leq h'} \exp(g_{t,k,h'})}$$

Horizon-specific action heads convert $Z_t^{(h)}$ into predictions $$\hat{A}_t^{(h)} = (\hat{a}_{t,1}^{(h)}, ..., \hat{a}_{t,h}^{(h)})$$. The fused prediction at step $k$ is:

$$\hat{a}_{t,k} = \sum_{h \in \mathcal{H}: k \leq h} \alpha_{t,k,h} \cdot \hat{a}_{t,k}^{(h)}$$

This fusion explicitly allows the model to arbitrate between predictions at varying temporal scales, increasing both accuracy and task generalization.

4. Adaptive Inference via Cross-Horizon Consensus

MoH supports a dynamic inference mode that adaptively selects how much of the predicted action chunk to execute, based on inter-horizon agreement. For each step $k$, the model computes the $\ell_1$ disagreement between the fused action and each horizon’s prediction, weighted by their gating value:

• $H_k = \{ h \in \mathcal{H} : k \leq h \}$
• $$\bar{d}_k = \sum_{h \in H_k} \alpha_{t,k,h} \|\hat{a}_{t,k} - \hat{a}_{t,k}^{(h)}\|_1$$

A threshold $$\mathrm{thresh} = (\text{mean}~\bar{d}_1... \bar{d}_n) \times r$$ is defined, with $n$ a minimum prefix length and $r>1$ a margin. The executed prefix $K_{exec}$ is the longest $K$ such that for all $k \leq K$, $\bar{d}_k \leq \mathrm{thresh}$ and $|H_k| \geq m$. This scheme yields increased throughput (up to $2.5 \times$ longer execution intervals with fewer replans) without sacrificing success rates (Jing et al., 24 Nov 2025).

Dynamic Inference Pseudocode

For k = 1…H: H_k ← {h∈𝓗 : k≤h} \bar d_k ← ∑_{h∈H_k} α_{k,h} ⋅ ‖\hat a_{k} − \hat a_{k}^{(h)}‖₁ thresh ← Mean(\bar d_1… \bar d_n) ⋅ r K_exec ← n For k = n+1…H: if |H_k|<m or \bar d_k>thresh: break else K_exec ← k Return actions {\hat a_1,…,\hat a_{K_exec}}

5. Training Objective and Computational Overhead

MoH is designed for efficiency. The architecture adds only $2 \times N$ parameters for the gating head (with $N = |\mathcal{H}|$). The VLM prefix is computed once per step, while the full-attention transformer head (approximately 300M parameters) benefits from parallelism. GPU overhead is under 5% for training; inference is virtually latency-free because all horizons are masked and batched together.

The composite training objective is:

$$L = L_{\text{mix}} + \lambda_{\text{ind}} L_{\text{ind}} + \lambda_{\text{bal}} L_{\text{bal}}$$

where

• $L_{\text{mix}}$: prediction loss on fused actions ($\hat{a}$), using either flow-matching or regression/classification,
• $L_{\text{ind}} = \sum_{h \in \mathcal{H}} L^{(h)}$: sum of per-horizon losses,
• $L_{\text{bal}}$: a coefficient-of-variation (CV) penalty promoting usage of all horizons, calculated interval-wise over the action chunk.

In experiments, $\lambda_{ind}=1$, $\lambda_{bal}=10^{-3}$. Ablation confirms that the CV penalty prevents gating collapse to the longest horizon, maximizing the benefits of multi-horizon fusion.

6. Empirical Evaluation and Impact

MoH was evaluated on the LIBERO benchmark—comprising the Spatial, Object, Goal, and Long suites—as well as the RoboTwin2.0 bimanual manipulation benchmark and three real-robot tasks. Performance metrics demonstrate consistent and significant improvements in action chunking policies. For example, on LIBERO, MoH with flow policy $\pi_{0.5}$ attains a new state of the art (99% average success rate) after 30,000 iterations. Table 1 summarizes main results:

Method	Spatial	Object	Goal	Long	Avg
$\pi_{\text{reg}}$	97.8	98.2	94.6	90.2	95.2
$\pi_{\text{reg}}$ + MoH	99.0	98.8	96.4	91.4	96.4
$\pi_0$	97.4	98.2	95.4	84.2	93.8
$\pi_0$ + MoH	97.6	98.8	96.4	87.4	95.1
$\pi_{0.5}$	98.8	99.0	97.6	95.4	97.7
$\pi_{0.5}$ + MoH	98.8	100.0	98.8	98.4	99.0

Ablation studies show diminishing gains beyond a moderate candidate set (e.g., $\{3,6,...,30\}$) and confirm that the gating-balance regularizer is critical for preventing collapse to the longest horizon. On RoboTwin2.0, $\pi_0$+MoH improves average success rates across both easy and hard bimanual tasks; on real-robot experiments, MoH consistently increases success rates over non-MoH baselines.

7. Broader Significance and Outlook

MoH resolves the widely observed trade-off in action horizon selection for sequential prediction in robotics and more broadly wherever multi-scale temporal modeling is critical. Its core plug-and-play nature, negligible training and inference overhead, and empirical robustness across simulated and real tasks support its deployment as a generic upgrade for full-attention action modules. A plausible implication is that similar multi-horizon strategies could benefit other domains—such as planning, policy learning, or even natural language generation—where fixed temporal scales limit performance (Jing et al., 24 Nov 2025).