---
title: 'Future Chunk Prediction: Definition and Examples'
url: https://www.emergentmind.com/topics/future-chunk-prediction-fcp
type: topic
---

# Future Chunk Prediction: Definition and Examples

Future Chunk Prediction (FCP) is a family of methods that predicts, represents, supervises, or adaptively executes temporally extended future chunks rather than restricting computation to a single next frame, token, action, or latent state. The term is used in several related but nonidentical senses: chunk-level residual pose forecasting, dynamic multi-token speech synthesis, multi-horizon video prediction, future latent-state prediction for model-based control, and adaptive selection of chunk commitment boundaries. Across these settings, FCP addresses the tension between temporal coherence, long-horizon prediction, inference cost, and feedback frequency.

## 1. Terminology and conceptual scope

FCP does not denote a single canonical objective or architecture. Its common structural element is a temporal abstraction in which a model predicts or reasons about a contiguous future interval. Depending on the domain, the predicted object may be:

- a sequence of absolute poses generated through residual changes between consecutive chunks;
- a variable-length block of speech tokens;
- several future video chunks at distinct temporal offsets;
- a future latent state at an action-chunk boundary;
- an action chunk together with a predicted future representation;
- or an execution boundary specifying how much of a predicted chunk should be trusted before replanning.

The term is therefore broader than next-token prediction, but narrower than arbitrary long-horizon prediction. In explicit FCP, the future target is usually temporally structured and aligned with a source context or action sequence. In adjacent approaches, the future object may instead be an execution duration or future context required by a downstream policy.

The acronym FCP is ambiguous in the literature. In “Meta Theorem for Hardness on FCP-Problem,” FCP means **Fewest Clues Problem**, not Future Chunk Prediction; that work defines an FCP variant of an NP problem as the problem of specifying at most \(k\) certificate entries so that exactly one solution remains compatible with the clues, and studies \(\Sigma_2^p\)-completeness under structured parsimonious reductions [2504.11859]. In the machine-learning literature considered here, FCP refers to Future Chunk Prediction.

A useful distinction is between **prediction horizon**, **chunk length**, and **execution horizon**. The prediction horizon is the temporal extent represented by a model output. The chunk length is the number of frames, tokens, actions, or latent states in that output. The execution horizon is the portion of a predicted chunk that is committed before new observations are obtained. These quantities may coincide, but methods such as Dynamic Execution Horizon Prediction explicitly separate them: a pretrained policy predicts an \(H\)-action chunk, while a learned branch selects an execution horizon \(h\in\{1,\ldots,H\}\), executes the first \(h\) actions, discards the remainder, and replans [2606.11408].

## 2. Chunk-level prediction and residual temporal structure

In human-motion forecasting, ResChunk formulates prediction as forecasting a future sequence of poses from an observed sequence. With \(J\) joints, \(D\) values per joint, and \(K=JD\), the observed sequence is \(x=\{X_1,\ldots,X_T\}\), while the target is \(y=\{X_{T+1},\ldots,X_{T+p}\}\). In the reported experiments, angle-axis coordinates give \(D=3\), with 50 observed frames and 50 future frames at 50 Hz.

The model divides the future into six temporal chunks. If \(ch_0\) is the final observed chunk, the first future chunk is generated as

$$
\widehat{ch}_1=ch_0+end_1,
$$

and subsequent chunks are generated recursively:

$$
\widehat{ch}_i=\widehat{ch}_{i-1}+end_i,\qquad i=2,\ldots,6.
$$

Equivalently,

$$
\widehat{ch}_i=ch_0+\sum_{r=1}^{i}end_r.
$$

The final outputs are absolute poses, but the model predicts learned residual pose changes between consecutive chunks. Thus, \(end_i\) approximates \(ch_i-ch_{i-1}\), rather than representing a single global residual from the entire input to the entire output. The chunk recursion enforces temporal continuity at boundaries and is intended to mitigate long-horizon over-smoothing and mode collapse in graph-based motion prediction. The architecture predicts the residual tensors in one feed-forward computation; “autoregressive” describes the residual dependency structure rather than a frame-by-frame recurrent loop [2308.16801].

ResChunk combines this temporal organization with graph processing at two body scales. The fine-scale path operates on the original joint representation, while a second path learns sequence-dependent body components. For each pair of joints \(i,j\), latent edge variables \(z_{ij}\in[0,1]\) are inferred by message passing over a fully connected graph without self-loops. A Concrete/Gumbel-softmax relaxation with temperature \(\tau=0.5\) makes categorical edge sampling differentiable. Hierarchical agglomerative clustering is applied to the resulting pairwise correlation matrix, and the pose of each learned group is computed by averaging the poses of its members. The grouping can vary across motion sequences and can associate distant joints when they participate similarly in an action.

The training objective combines fine- and coarse-scale prediction or reconstruction terms with a KL regularization term for the latent graph. The experiments on CMU Mocap and Human3.6M report MPJPE at multiple horizons. On Human3.6M, average MPJPE is \(9.15\), \(21.40\), \(45.69\), and \(56.41\) mm at \(80\), \(160\), \(320\), and \(400\) ms, respectively; at \(1000\) ms, the average is \(81.95\) mm. Ablations show degradation when the coarse scale is removed, learned groups are replaced by predefined groups, or chunk residuals are replaced by one global residual. Four chunks generally improve over one, whereas seven can improve some short-term results while harming long-term forecasting.

## 3. Dynamic future chunks in speech and video generation

In autoregressive speech synthesis, conventional next-token prediction requires one sequential decision per acoustic token. DCAR changes the transition granularity by predicting a variable-length block

$$
(s_t,s_{t+1},\ldots,s_{t+n_t-1}),
$$

where the lightweight policy selects \(n_t\) from a permitted set such as \(\{2,3\}\) or \(\{2,3,4\}\). The selected chunk is committed, appended to the prefix, and used to produce the next policy state. Tokens within a chunk are predicted internally in parallel, but chunks remain sequential and conditioned on the accepted prefix.

Training proceeds in two stages. First, multiple prediction heads are trained from the same prefix with a multi-token objective:

$$
\mathcal L_{\mathrm{CAR}}
=
-\frac{1}{T}
\sum_{t=0}^{T}
\sum_{i=0}^{N'}
\gamma^i
\log p_\theta(s_{t+i}\mid s_{<t},x),
$$

where \(N'=\min\{N,T-t\}\). The reported setting \(\gamma=1\) gives equal weight to the future heads. Second, a dynamic policy is trained on-policy using a clipped GRPO/PPO-style objective, group-relative advantages, KL regularization, and generated-speech WER as the principal reward. The “chase-then-exceed” strategy combines schedules from a profiled good range with schedules sampled from the learned policy, while an inner-chunk mask applies policy gradients only when a new chunk size is selected.

With HuBERT 50-Hz tokens, FAR obtains WER \(9.99\) and RTF \(0.320\), while DCAR with range \([2,3,4]\) obtains WER \(2.77\), RTF \(0.128\), and reported speedup \(2.61\times\). The stated \(72.27\%\) improvement is a relative WER reduction from \(9.99\%\) to \(2.77\%\), not a percentage-point increase in human intelligibility. With \(\mathcal S^3\)-Tokenizer tokens, the reported WER decreases from \(2.31\) for FAR to \(1.84\) for DCAR with range \([2,3]\), a relative reduction of approximately \(20.34\%\). The method preserves approximately the same SECS and UTMOS in the main HuBERT evaluation. Its policy overhead means that dynamic scheduling is not unconditionally faster than fixed chunking; the benefit is adaptive speed–quality control [2506.22023].

Next Forcing applies a related idea to causal video-world modeling. The standard model denoises the current \(M\)-frame chunk, while auxiliary Multi-Chunk Prediction modules predict the next\(^1\), next\(^2\), and next\(^3\) chunks. The future targets are independently noised, and each MCP module predicts the flow-matching velocity of its corresponding target. The modules form a causal chain across prediction depths: main-model features feed next\(^1\), whose internal representation feeds next\(^2\), which feeds next\(^3\). Features from main-model layers \(4,12,20,30\) are fused, and the default MCP loss weights are \(0.5\), \(0.2\), and \(0.1\). The main and MCP timestep-shift parameters are \(s_{\mathrm{main}}=5\) and \(s_{\mathrm{MCP}}=10\).

Next Forcing is hybrid during training: future targets are supervised simultaneously, but the auxiliary prediction chain is causal across depth. At inference, MCP modules can be discarded for zero-overhead deployment, or the next\(^1\) module can be retained to predict the next video chunk in parallel with the current one. The latter mode reports approximately \(2\times\) inference acceleration. On RoboTwin, Next Forcing obtains \(94.1\%\) Clean and \(93.5\%\) Random success, compared with \(92.9\%\) and \(91.5\%\) for LingBot-VA. At 50 fps and 5k training steps, it obtains \(70.2\%\) Clean and \(61.6\%\) Random success, compared with \(45.5\%\) and \(31.9\%\) for the baseline [2606.11187].

Prospective Forcing uses parallel offset-specific heads in a rolling-window autoregressive video generator. With window size \(W=5\), the newest hidden representation predicts future chunks at offsets \(1,\ldots,K\); the main configuration uses \(K=2\). Training aligns the predicted future features and velocities with detached backbone outputs when the corresponding chunks later enter the rolling window. At inference, learned candidate embeddings generate multiple next-chunk candidates, and a verifier either accepts one candidate or rejects all and falls back to ordinary rolling generation. On 30-second videos, standard inference uses 46 sampling and 46 cache-update passes, while verified inference uses 37 sampling and 46 cache-update passes, producing reported speed \(1.20\times\). The verifier accepts \(32.1\%\) of proposals. The method reports VBench Total \(81.40\), VBench Quality \(82.71\), and VBench-Long Quality \(83.71\) [2610.14654].

“Looking Ahead” defines FCP as an auxiliary training objective for autoregressive video diffusion that predicts a temporally separated target chunk \(\mathbf{x}_{k+\Delta}\) from a source representation \(\mathbf{h}_k\), while withholding the intervening chunks \(\mathbf{x}_{k+1:k+\Delta-1}\). Offset-specific auxiliary DiT branches operate in parallel and receive intermediate generator features through AdaLN. The total objective combines ordinary teacher-forced flow matching with clean- and noisy-source FCP losses. The auxiliary branches are discarded at inference, so FCP changes the learned representation rather than the autoregressive inference procedure [2610.29989].

On Physics-IQ Verified with Wan2.2-TI2V-5B, teacher forcing obtains I2V IQ \(30.4\) and V2V IQ \(40.6\), while FCP with five offsets obtains \(33.8\) and \(44.4\). Ablations favor parallel over chained prediction, combined clean and noisy source supervision, and source-chunk-only conditioning. Adding explicit text conditioning to the auxiliary predictor reduces IQ from \(30.9\) to \(29.8\), consistent with the objective of preventing prompt information from directly revealing future events. The reported training-only FCP leaves inference unchanged and requires no external pretrained visual encoder.

## 4. Future latent states and action chunks in control

FCP in control systems frequently predicts future latent states conditioned on action chunks. FlexiWorld samples goal spans of \(35\), \(55\), or \(75\) primitive actions and partitions each span into variable-length chunks with individual lengths in \(\{1,\ldots,10\}\). Each chunk \(A_i\) is paired with its start and end observations, defining a transition

$$
z_i \xrightarrow[A_i]{} \hat z_{i+1}.
$$

The action encoder is causal and maps arbitrary-length primitive action sequences to a fixed-dimensional representation augmented with a learned chunk-length embedding. The predictor operates at chunk boundaries rather than primitive-action resolution. The actor is autoregressive within each chunk, and Student Forcing replaces expert within-chunk prefixes with greedily generated prefixes with probability \(p_{\mathrm{SF}}=0.5\).

FlexiWorld combines local and final-goal latent intent, mixed-span supervision, variable-length chunks, and ARCEM residual search. ARCEM perturbs actor-generated primitive actions with residuals, feeds each perturbed action back into subsequent within-chunk action generation, and uses the world model to evaluate predicted boundary latents. With 128 candidates per iteration, three CEM iterations, 16 elites, and residual temperature \(T=0.2\), the method obtains mean success \(89.29\%\) across PushT, Cube, Reacher, and TwoRoom, compared with \(83.98\%\) for the strongest listed baseline. Changing the planning chunk length from \(k=5\) to \(k=10\) produces approximately \(1.3\times\) faster ARCEM planning with comparable mean success [2609.35138].

Chunk-wise Model-based Policy Optimization (CMPO) uses a fixed-length action chunk and a direct chunk world model:

$$
a_{t:t+H-1}=\pi_\theta(z_t),
$$

$$
z_{t+1:t+H}=f_\phi(z_t,a_{t:t+H-1}).
$$

The world model and actor jointly predict a hypothetical future in one forward computation. A value expansion is differentiated through the direct chunk mapping rather than through recurrent one-step model applications. At execution, only the first action is executed, after which a new chunk is generated from the next observation. Thus, CMPO uses FCP for training-time value expansion but retains closed-loop environment control.

The method is built on TD-MPC2 and evaluated on 19 HumanoidBench tasks. On an NVIDIA RTX 4090, reported inference time is \(1.4\) ms for CMPO versus \(40.8\) ms for TD-MPC2 on Walk, and \(1.3\) ms versus \(39.5\) ms on Run. Training time at 200k steps is approximately half that of TD-MPC2 in the listed settings. Performance declines at \(H=5\), demonstrating that direct chunk prediction mitigates recurrent compounding error without making long-horizon prediction intrinsically easy. The “without collaborative chunk modeling” variant, which uses a chunk policy with a single-step dynamics model, performs substantially worse, indicating that chunking the actor alone is insufficient [2610.14564].

JEPA Policy predicts an expert action chunk and a single future observation representation from the same demonstration segment:

$$
(o_t,\mathbf a_{t:t+H},o_{t+k}).
$$

The future target is a globally pooled image embedding of one future observation, with default \(k=4\), rather than a future action chunk, observation sequence, or multi-step rollout. Action and future-representation tokens interact through a shared Transformer, and the future loss shapes the action-generating representation through shared parameters. The main nine-task mean success is \(83.0\%\), compared with \(77.4\%\) for action-only MIP. The method adds \(0.29\) ms to MIP’s \(12.9\) ms latency, whereas the evaluated 100-step Diffusion Policy requires \(439.5\) ms [2609.09630].

FutureRTC predicts execution-time context for asynchronous VLA control. If a new action chunk is generated \(d\) control steps before it begins execution, the method predicts the future proprioceptive state and visual latent at the execution boundary. The State Correction Module rolls the state forward and predicts a residual correction, while the Observation Prediction Module transports and synthesizes visual features using committed robot motion as an explicit prior. A policy-consistency loss aligns actions produced from predicted context with actions produced from true execution-time context.

FutureRTC therefore predicts

$$
(\hat z_{t+K},\hat s_{t+K}),
$$

rather than directly predicting the next action chunk. The frozen VLA then generates the chunk from that predicted context and executes it from its first action. In LIBERO evaluations with \(\pi_{0.5}\), FutureRTC reports success rates of \(94.2\%\), \(92.4\%\), \(91.0\%\), and \(88.5\%\) at delays \(d=5,10,15,20\), respectively. The SmolVLA-450M results are \(75.8\%\), \(73.3\%\), \(71.6\%\), and \(69.4\%\). The visual-latent ablation shows that state correction alone is insufficient at large delays, while adding predicted visual features produces the dominant improvement [2607.24008].

## 5. Adaptive execution horizons and consequence-aware commitment

Several methods treat the execution boundary itself as a future-prediction problem. DEHP attaches a lightweight categorical horizon branch to a frozen chunk policy. Given the current state and an \(H\)-action chunk, the branch predicts \(h\in\{1,\ldots,H\}\). The first \(h\) actions are executed open-loop and the remaining actions are discarded. Training uses online PPO in a semi-Markov formulation, with variable-duration temporal-difference targets and step-discounted generalized advantage estimation. The method improves the trade-off between efficiency during free-space motion and reactivity during grasping, alignment, contact, and insertion. On multi-stage peg insertion, DEHP obtains \(93.17\%\) success at zero action noise versus \(71.50\%\) for fixed-horizon behavior cloning, and \(61.43\%\) versus \(37.73\%\) at action noise \(0.15\) [2606.11408].

CA\(^3\)C extends adaptive horizon selection by basing commitment on agreement among imagined consequences rather than similarity among actions. A frozen policy samples \(K=3\) candidate action chunks under the same context. A frozen action-conditioned world model generates candidate future latent sequences using shared sampling noise. Candidate disagreement is measured by

$$
D_k=\frac{1}{Kd}\sum_{i=1}^K
\left\|z_{t+k}^{(i)}-\bar z_{t+k}\right\|_2^2,
$$

and normalized by predicted motion from the current latent:

$$
U_k=\frac{D_k}{M_k+\varepsilon}.
$$

A width-three temporal average smooths the disagreement curve. The execution horizon is selected by Bayesian Information Criterion comparison between a single linear trend and a hinge model with a nonnegative post-change slope. If the hinge model is supported, the controller commits to the estimated change point; otherwise it commits to the longest valid prefix.

CA\(^3\)C also selects the candidate closest to consensus over the chosen prefix. The reported settings use \(h_{\min}=4\), \(h_{\max}=15\), ten world-model diffusion steps, and realized horizons approximately in \(\{4,\ldots,10\}\). On LIBERO-Plus with \(\pi_{0.5}\), average success increases from \(88.1\%\) for the fixed baseline to \(93.5\%\) for the complete method. Adaptive horizon selection alone produces \(91.3\%\), whereas consensus selection alone produces \(86.3\%\), indicating that adaptive execution-horizon estimation is the principal mechanism. On real-robot Block Separation and Parcel Sorting, average success increases from \(67.0\%\) to \(72.0\%\) [2610.07949].

ChunkFlow addresses a different boundary problem: inconsistency between overlapping predictions from successive action chunks. A length-\(L\) chunk has overlap \(O\) and stride \(S=L-O\). The method distinguishes frozen, editable seam, and future zones. In the overlap, it applies deterministic linear blending between the previous chunk’s tail and the current chunk’s prediction. Training adds boundary, total-variation, and second-order continuity losses to behavior cloning, while history corruption and scheduled sampling expose the policy to executed-history errors. An optional AWAC stage uses post-blending actions for reward evaluation while preserving structural regularizers on raw predictions.

Under local Lipschitz and input-drift assumptions, the expected pre-blending seam discrepancy scales as \(\mathcal O((L-O)^2)\). On CALVIN ABC–D, the reported default configuration has \(B_{\mathrm{jump}}=0.209\) and \(\mathrm{HF}_{\mathrm{ratio}}=0.431\); on LIBERO-Long it obtains \(93.4\%\) success and \(4.43\) ms amortized action inference latency. The method’s central claim is not that blending alone solves FCP, but that chunk execution, overlap structure, history semantics, and continuity losses should be trained as a coherent contract [2607.12992].

FutureRTC, ChunkFlow, DEHP, and CA\(^3\)C address different components of the chunk boundary problem:

| Method | Primary predicted or controlled object | Base policy modification |
|---|---|---|
| FutureRTC | Future execution-time visual latent and state | Frozen VLA with plug-in adapter |
| ChunkFlow | Seam-consistent overlapping action predictions | Training and deterministic execution framework |
| DEHP | Execution horizon \(h\) for a generated chunk | Frozen chunk policy; RL-trained length head |
| CA\(^3\)C | Consensus-supported prefix and replanning boundary | Frozen policy and world model |

These methods show that future prediction and future commitment are separable. A policy may predict a plausible long-horizon chunk while still needing to replan before executing its entire content. Conversely, an adaptive boundary controller may improve feedback cadence without predicting the content of the next chunk in advance.

## 6. Objectives, evaluation, and unresolved issues

FCP objectives differ according to the predicted domain. Residual chunk forecasting supervises target-to-target temporal differences; speech FCP uses multi-token cross-entropy and on-policy WER rewards; video FCP uses future flow-matching losses; latent world models use boundary-state prediction losses; CMPO uses differentiable value expansion; and adaptive commitment methods use policy or change-point objectives.

A recurring design choice is whether future predictions are independent, chained, or recursively rolled out. Parallel offset-specific predictors avoid allowing a farther-horizon head to consume an intermediate prediction that may leak withheld information. Chained predictors can impose structured temporal dependencies, as in Next Forcing, but may expose later predictions to intermediate representations. Recursive world-model rollouts preserve temporal feedback but introduce compounding model error and BPTT difficulties. Direct chunk models avoid recurrent invocation but must learn a more difficult long-horizon mapping.

Another issue is whether future prediction should remain active at inference. Training-only objectives, such as “Looking Ahead,” can improve representation quality while leaving inference unchanged. Prospective Forcing and Next Forcing retain auxiliary predictions for speculative acceleration. CMPO uses future prediction primarily during training-time optimization. FutureRTC uses future context prediction online to compensate for asynchronous execution. These choices produce different computational trade-offs: auxiliary modules can be discarded, retained for parallel prediction, or integrated into the control loop.

Evaluation also varies substantially. Motion forecasting uses MPJPE at multiple time horizons. Speech synthesis reports WER, UTMOS, SECS, RTF, and speedup. Video generation uses physical-plausibility, VBench, FVD, abnormal-ratio, and drift metrics. Robot control uses task success, execution steps, execution time, velocity and acceleration profiles, and robustness to delay or action noise. The literature therefore does not support a single universal FCP metric.

Several limitations recur across domains:

- **Longer horizons are not automatically better**: distant targets are less predictable, and excessive temporal recursion or auxiliary supervision can reduce quality.
- **Prediction error is not equivalent to control risk**: latent error, action error, and consequence disagreement measure different properties.
- **Direct multi-step prediction trades recurrent compounding for approximation difficulty**: CMPO’s degradation at \(H=5\) illustrates this trade-off.
- **Fixed chunk size is often inadequate**: the appropriate horizon depends on task phase, temporal resolution, scene dynamics, and execution latency.
- **World-model and representation dependence remains substantial**: false agreement, false disagreement, latent collapse, and representation insensitivity can mislead adaptive policies.
- **Boundary consistency requires execution-aware training**: inference-time blending alone can lower visible discontinuities without correcting the underlying predictions or histories.
- **Multimodality is unresolved**: deterministic regression or diagonal Gaussian action models may average incompatible futures, while candidate sampling and residual search provide only partial remedies.
- **Physical plausibility is not fully captured by visual similarity**: future-chunk supervision is motivated precisely by cases where local appearance continuity permits incorrect dynamics.
- **Calibration is incomplete**: BIC-selected change points and future-prediction errors are model-selection or diagnostic signals, not generally calibrated probabilities of safety or success.
- **Training cost can increase**: auxiliary heads, future targets, world-model rollouts, and online reinforcement learning add memory, computation, or environment-interaction requirements.

Taken together, the research defines FCP as a design space rather than a single algorithm. Its central principle is to make a model’s internal state, output structure, or execution policy sensitive to temporally extended consequences. This may be achieved by predicting residual changes between future chunks, supervising multiple future offsets, forecasting future latent states conditioned on action chunks, generating variable-length token blocks, predicting future execution contexts, or selecting commitment boundaries from imagined consequence agreement. The common objective is to reduce the limitations of purely local next-step prediction while preserving sufficient causal feedback for stable, efficient, and physically coherent long-horizon behavior.

Source: https://www.emergentmind.com/topics/future-chunk-prediction-fcp