Temporal Next-Scale Prediction (TENS)

• TENS is a framework that distributes predictions over multiple temporal scales, using distinct discount factors or chunked sequences to capture varying future horizons.
• It leverages shared representations and causal chaining across scales, enabling models to refine coarse predictions into precise details.
• TENS has been applied in reinforcement learning, video generation, and motion modeling, demonstrating improved inference efficiency and predictive accuracy.

Searching arXiv for the cited TENS-related papers and neighboring work. Temporal Next-Scale Prediction (TENS) denotes a family of predictive formulations in which a model learns structured forecasts across multiple temporal horizons or temporal resolutions rather than only a single immediate next step. In its earliest reinforcement-learning formulation, the idea appears as multi-timescale nexting: each measurable signal is treated as a pseudo-reward, and a bank of value functions predicts exponentially discounted future signals at distinct short horizons (Modayil et al., 2011). In later generative formulations, TENS is used more explicitly to describe multi-scale temporal objectives such as predicting next$^1$, next$^2$, and next$^3$ video chunks with a causal chain (Xu et al., 9 Jun 2026), or generating sequences coarse-to-fine across temporal scales in motion and occupancy models (Zheng et al., 4 Apr 2026, Jin et al., 4 Sep 2025). Across these usages, the unifying principle is that anticipatory structure is distributed over a set of “next-scales,” with coupling across horizons or resolutions used to improve temporal coherence, controllability, or sample efficiency.

1. Origins in multi-timescale nexting

The foundational formulation of TENS is implicit in multi-timescale nexting on a reinforcement-learning robot (Modayil et al., 2011). “Nexting” denotes the continual prediction of what will happen next in an immediate, local, and personal sense. In that setting, each measurable signal $x_t^i$ is treated as a pseudo-reward $r_t^i$, and the learner estimates a separate value function for each signal and each timescale. TENS, in this sense, is realized by a bank of temporal-difference predictors that share a common feature representation while differing in target signals and discount factors.

For each signal, the prediction target is the exponentially discounted return

$$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$

with value function

$$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$

Multiple temporal scales are obtained by assigning distinct $\gamma$ values per prediction. If data are sampled every $\Delta t$ seconds, the desired continuous-time horizon $\tau$ is approximated by

$^2$0

and the effective discrete-time horizon by

$^2$1

In the Critterbot experiments, $^2$2, and four timescales were used: $^2$3, corresponding approximately to horizons of $^2$4 s, $^2$5 s, $^2$6 s, and $^2$7 s (Modayil et al., 2011). The reported system learned 2160 predictions online, updated at better than $^2$8 Hz, with most dramatic gains within approximately $^2$9 minutes. Learned predictions for an ambient light sensor at $^3$0 s and $^3$1 s rose and fell in anticipation of saturation near a lamp, matching empirical returns and the offline-optimal least-squares solution closely. This establishes the canonical interpretation of TENS as short-horizon anticipatory prediction over a spectrum of timescales.

A broader implication suggested by later work is that multi-timescale nexting supplies a general template: select temporal scales, define a prediction target at each scale, and couple the resulting predictors through a shared representation rather than through a single monolithic output.

2. Formalizations of temporal next-scale prediction

Across the cited literature, TENS takes two mathematically distinct but conceptually related forms.

The first is the discounted-return formulation of multi-timescale nexting, where different $^3$2 values encode different effective horizons (Modayil et al., 2011). This formulation preserves a conventional value-function semantics: each predictor estimates an expectation of future signal accumulation over a chosen timescale.

The second is an explicit multi-depth or multi-resolution prediction objective, in which the model predicts several future chunks, frames, or scales jointly. In Next Forcing, TENS is defined as jointly predicting

$^3$3

with causal supervision at each depth and causal chaining across depths (Xu et al., 9 Jun 2026). In VideoAR, the factorization is over both time and intra-frame scale: $^3$4 so temporal next-scale prediction couples next-frame prediction with next-scale prediction inside each frame (Ji et al., 9 Jan 2026). In MoScale, the factorization becomes

$^3$5

where each scale spans the complete temporal horizon at a different resolution (Zheng et al., 4 Apr 2026). In ScaleMoGen, the same pattern appears as

$^3$6

with optional bitwise decomposition inside each scale (Hwang et al., 12 May 2026).

A distinct continuous-time generalization appears in a scale-invariant future representation (Tiganj et al., 2018). There, future outcomes are represented over a logarithmically compressed timeline, and multiplicative dilation in real time becomes an additive shift in log-time: $^3$7 On a discrete logarithmic grid, moving to the next larger temporal scale is therefore an index shift. This is not a chunked or coarse-to-fine generative model, but it instantiates TENS in the precise sense that “next-scale” prediction corresponds to moving along a structured temporal axis rather than rolling out one step at a time.

These formulations differ in target semantics—discounted returns, future chunks, residual token maps, occupancy scales, or log-time future timelines—but they share the same organizing principle: prediction is distributed over a set of explicitly parameterized temporal scales, and adjacent scales are allowed to inform one another.

3. Learning mechanisms and architectural patterns

In the reinforcement-learning setting, TENS is implemented with standard TD$^3$8 and linear function approximation (Modayil et al., 2011). With shared sparse tile-coded features $^3$9, each predictor uses

$x_t^i$0

$x_t^i$1

$x_t^i$2

and

$x_t^i$3

The reported implementation used $x_t^i$4, $x_t^i$5, and exactly $x_t^i$6 active binary features per time step. Practical efficiency came from sparse dot products, step-size scaling by the number of active features, multithreading, and per-$x_t^i$7 trace sharing (Modayil et al., 2011).

Later TENS systems replace value-function banks with hierarchical generative architectures. Next Forcing augments a flow-matching autoregressive video transformer with $x_t^i$8 lightweight MCP modules for depths $x_t^i$9, each receiving noisy target tokens for a future chunk and the previous depth’s representation (Xu et al., 9 Jun 2026). Hidden states from layers $r_t^i$0 are fused by a two-layer MLP into $r_t^i$1, and the per-depth fused input is

$r_t^i$2

This yields explicit causal chaining across temporal depths, with gradients from the multi-chunk losses flowing back into multiple backbone layers.

VideoAR uses a 3D multi-scale tokenizer and a causal Transformer backbone with block-wise causal masking to combine next-frame prediction and intra-frame next-scale prediction (Ji et al., 9 Jan 2026). ScaleMoGen and MoScale transpose the same principle to motion: both generate coarse global structure first, then finer temporal detail, using hierarchical tokenization and shared or scale-conditioned autoregressive predictors (Hwang et al., 12 May 2026, Zheng et al., 4 Apr 2026). OccTENS applies the pattern to occupancy world modeling by splitting the problem into temporal scene-by-scene prediction and spatial scale-by-scale generation within each frame, implemented by a TensFormer with scale-wise temporal causal attention and frame-wise spatial attention (Jin et al., 4 Sep 2025).

A recurrent architectural pattern is therefore visible across domains. Coarser scales encode global structure, finer scales encode residual refinement, and cross-scale coupling is deliberately asymmetric: later predictions depend on earlier scales, but not conversely. This suggests that TENS is best understood as a causal ordering principle as much as a loss design.

4. Representative realizations across domains

The following examples illustrate how the same temporal next-scale idea has been instantiated in substantially different tasks.

Paper	Prediction unit	Temporal next-scale structure
"Multi-timescale Nexting in a Reinforcement Learning Robot" (Modayil et al., 2011)	Sensor and derived signals	Parallel value functions at $r_t^i$3
"Next Forcing: Causal World Modeling with Multi-Chunk Prediction" (Xu et al., 9 Jun 2026)	Video chunks	next$r_t^i$4, next$r_t^i$5, next$r_t^i$6 with causal chain
"VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction" (Ji et al., 9 Jan 2026)	Frame-scale residual maps	Frame-by-frame plus coarse-to-fine intra-frame generation
"ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation" (Hwang et al., 12 May 2026)	Skeletal-temporal token maps	Coarse-to-fine scales preserving skeletal hierarchy
"Next-Scale Autoregressive Models for Text-to-Motion Generation" (Zheng et al., 4 Apr 2026)	Motion token groups at temporal scales	Full-horizon coarse-to-fine temporal resolutions
"OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction" (Jin et al., 4 Sep 2025)	Multi-scale occupancy tokens	Scene-by-scene temporal prediction and scale-by-scale spatial generation

In robotics, the emphasis was on real-time anticipatory knowledge. The Critterbot system learned 2160 predictions over $r_t^i$7–$r_t^i$8 s horizons, updated online at better than $r_t^i$9 Hz, with measured compute time of approximately $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$0 ms per $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$1 ms cycle and memory usage of approximately $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$2 MB (Modayil et al., 2011). In forecasting, the same broad multi-time-scale intuition appears in wind power prediction, where a TFT-based framework targets ultra-short-term, short-term, and medium-term horizons and reports nMAE reductions of $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$3 and $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$4 for 24-hour and 48-hour forecasting relative to the second-best model (Jiang et al., 2023). That paper does not introduce explicit hierarchical couplings across horizons; the data explicitly state that cross-scale information transfer occurs implicitly via decomposition and shared temporal processing. A plausible implication is that not all multi-horizon forecasting should be classified as TENS in the strong causal-coupling sense used by later generative papers.

In generative world models, the emphasis shifts to dense temporal supervision and efficient inference. Next Forcing reports a $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$5 relative improvement over LingBot-VA at $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$6k training steps on RoboTwin at $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$7 fps, $$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$8 faster convergence, and near-$$G_t^i \equiv \sum_{k=0}^{\infty} (\gamma^i)^k \, r_{t+1+k}^i,$$9 inference throughput when the depth-1 head is retained at inference (Xu et al., 9 Jun 2026). OccTENS reports mIoU $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$0 and IoU $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$1 with occupancy input on nuScenes val, and measured latency $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$2 s for the 6-scale model, compared with $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$3 s for OccWorld and approximately $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$4 s for OccSora (Jin et al., 4 Sep 2025). In text-to-motion, ScaleMoGen reports FID $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$5 on HumanML3D and CLIP Score $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$6 on SnapMoGen, while MoScale reports strong alignment metrics on HumanML3D and KIT-ML with a coarse-to-fine temporal hierarchy (Hwang et al., 12 May 2026, Zheng et al., 4 Apr 2026).

Taken together, these results show that TENS is not confined to one modality. It has been used for sensorimotor prediction, video generation, motion generation, occupancy world modeling, and multi-horizon forecasting, but the exact interpretation of “scale” varies from discount-controlled horizon to chunk depth, residual resolution, or hierarchical state granularity.

5. Computational properties, advantages, and trade-offs

One recurring motivation for TENS is efficiency relative to flat sequential prediction. In multi-timescale nexting, sparse shared features made thousands of online predictors practical: with $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$7 active features and per-$$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$8 eligibility sharing, complexity depended on $$v^i_\gamma(s_t) = \mathbb{E}\!\left[\sum_{k=0}^{\infty} \gamma^k \, r_{t+1+k}^i \;\middle|\; s_t\right].$$9 for updates plus $\gamma$0 for trace maintenance, rather than dense $\gamma$1 processing (Modayil et al., 2011). In scale-invariant future estimation, the entire future timeline is constructed in one parallel operation over a logarithmic temporal grid, avoiding forward simulation whose cost grows linearly with horizon (Tiganj et al., 2018).

In chunked and scale-wise generative models, the claimed advantage is not merely lower wall-clock cost but improved supervision geometry. Next Forcing argues that next-chunk-only training is myopic; its MCP losses inject dense multi-scale temporal supervision into several backbone layers through fused intermediate features (Xu et al., 9 Jun 2026). VideoAR argues that raster next-token autoregression over flattened tokens is poorly aligned with spatio-temporal structure, whereas next-scale generation reduces sequence length and better captures spatial correlations (Ji et al., 9 Jan 2026). MoScale makes the same argument for motion: local continuity makes next-token prediction easy, but it does not force commitment to global semantics such as counts, ordering of actions, and global trajectories (Zheng et al., 4 Apr 2026).

These advantages come with clear trade-offs. In discounted-return TENS, higher $\gamma$2 extends horizon but increases variance and can slow learning; lower $\gamma$3 learns faster but misses longer dependencies (Modayil et al., 2011). In coarse-to-fine generators, mistakes at coarse scales can propagate to fine scales. Several papers therefore introduce explicit correction mechanisms. MoScale perturbs coarser tokens during training through cross-scale hierarchical refinement and uses in-scale temporal refinement for selective bidirectional re-prediction (Zheng et al., 4 Apr 2026). VideoAR uses Cross-Frame Error Correction and Random Frame Mask to mitigate exposure bias and over-reliance on distant frames (Ji et al., 9 Jan 2026). Next Forcing keeps depth-2 and depth-3 predictions for training but uses only depth-1 at inference to avoid drift (Xu et al., 9 Jun 2026).

A common misconception is that TENS is simply multi-horizon prediction. The cited works suggest a narrower technical reading. TENS generally involves explicit organization of prediction across adjacent temporal scales, together with some mechanism by which one scale conditions, constrains, or refines another. Mere production of several horizons from a shared backbone, without such cross-scale structure, is closer to generic multi-horizon forecasting than to the stronger TENS formulations.

6. Generalizations, limitations, and research directions

The literature identifies several extensions that broaden TENS beyond its initial forms. In reinforcement learning, state-dependent discounting replaces constant $\gamma$4 with variable $\gamma$5, allowing event-conditioned horizons such as predicting power expenditure until a light sensor saturates or until a stochastic termination (Modayil et al., 2011). Off-policy prediction can be handled with gradient TD methods such as GTD2 and TDC, while non-linear representations such as kernels, random Fourier features, and neural networks may reduce residual error when linear tile coding underfits (Modayil et al., 2011). Continuous-time formulations based on Laplace transforms and the Post approximation offer a scale-invariant future timeline retaining “what will happen when,” rather than only a scalar discounted value (Tiganj et al., 2018).

In generative modeling, later work pushes TENS toward stronger causal world modeling and controllability. OccTENS integrates ego-motion as a $\gamma$6-th scale token and treats occupancy and motion in one autoregressive sequence, enabling controllable future occupancy generation aligned with planned pose tokens (Jin et al., 4 Sep 2025). MSTP with IG-MC extends the next-scale concept along both temporal and state axes, using synchronized multi-scale visual previews and a multi-agent hierarchy for general and surgical scenes (Zeng et al., 22 Sep 2025). This suggests that the “scale” in TENS need not be purely temporal resolution; it can also denote hierarchical state abstraction, provided that cross-scale consistency is maintained.

The limitations are similarly recurrent. Performance is bounded by representational capacity in linear nexting, by hierarchy design in motion tokenization, and by memory or compute overhead in multi-head or multi-agent systems [(Modayil et al., 2011); (Hwang et al., 12 May 2026); (Zeng et al., 22 Sep 2025)]. Very long horizons remain difficult. Next Forcing notes that depth-2 and depth-3 are excellent for training but risky for inference because far-future chunk prediction can accumulate drift (Xu et al., 9 Jun 2026). OccTENS notes that increasing the number of scales improves fidelity but raises latency, and its motion tokenizer omits the $\gamma$7-axis, which is reasonable for on-road driving but limits generalization (Jin et al., 4 Sep 2025). MoScale reports that excessive corruption or too many refinement iterations can degrade stability (Zheng et al., 4 Apr 2026).

The available evidence therefore supports a precise characterization of TENS. It is neither a single algorithm nor a synonym for forecasting. It is a design paradigm in which temporal prediction is decomposed into an ordered family of next-scales—discount horizons, chunk depths, coarse-to-fine resolutions, or logarithmic future lags—and these scales are tied together by shared representations, causal coupling, or refinement operators. The historical trajectory from multi-timescale nexting to modern generative world models suggests that TENS has evolved from a bank of short-horizon predictors into a broader framework for organizing temporal structure in learning systems [(Modayil et al., 2011); (Xu et al., 9 Jun 2026)].