---
title: 'ALTTS: Dual-Path AR/CR Forecasting'
url: https://www.emergentmind.com/topics/altts
type: topic
---

# ALTTS: Dual-Path AR/CR Forecasting

ALTTS, introduced in “AltTS: A Dual-Path Framework with Alternating Optimization for Multivariate Time Series Forecasting” [2602.11533], is a multivariate long-term time series forecasting framework built on the claim that two qualitatively different mechanisms drive forecasting performance: stable within-series autoregressive (AR) dynamics and intermittent cross-relation (CR) interactions. The method argues that fitting a single model to capture both effects creates an optimization conflict, because the high-variance updates needed for cross-dimension modeling can corrupt the gradients that support autoregression. ALTTS therefore separates AR and CR both architecturally and during training: the AR path is instantiated with a linear predictor, the CR path uses a Transformer with Cross-Relation Self-Attention (CRSA), and the two branches are coordinated by alternating optimization [2602.11533].

## 1. Conceptual basis and forecasting decomposition

ALTTS is formulated for multivariate forecasting with a look-back window of length \(L\) over \(D\) variables and a prediction horizon \(H\). The input at time \(t\) is
\[
X_t = [X_t^{(1)}, \dots, X_t^{(D)}] \in \mathbb{R}^{D \times L},
\]
with
\[
X_t^{(i)} = [x^{(i)}_{t-L+1}, \dots, x^{(i)}_t],
\]
and the target is
\[
Y_{t+1} = [Y_{t+1}^{(1)}, \dots, Y_{t+1}^{(D)}] \in \mathbb{R}^{D \times H},
\]
where
\[
Y_{t+1}^{(i)} = [x^{(i)}_{t+1}, \dots, x^{(i)}_{t+H}].
\]
The paper writes the forecasting operator as a matrix of projection functions \(F=(f_{ij})_{D\times D}\), where each \(f_{ij}: \mathbb{R}^L \mapsto \mathbb{R}^H\) models how the history of variable \(j\) contributes to the future of variable \(i\). The induced operator is
\[
F * X_t := \left(\sum_{j=1}^D f_{1j}(X_j), \dots, \sum_{j=1}^D f_{Dj}(X_j)\right) \in \mathbb{R}^{D\times H},
\]
and the forecasting equation is
\[
F * X_t + \varepsilon_{t+1} = Y_{t+1}.
\]
From this perspective, ALTTS decomposes the transition operator into diagonal self-effects and off-diagonal cross-effects:
\[
F = F_{\text{AR}} + F_{\text{CR}},
\]
with
\[
F_{\text{AR}} = \mathrm{diag}(f_{11}, \dots, f_{DD}), \qquad F_{\text{CR}} = F - F_{\text{AR}}.
\]
This decomposition is the method’s central premise. The paper argues that AR dynamics are usually stable, smooth, and persistent, whereas CR signals are more irregular, often weaker, more transient, and more susceptible to spurious correlations, especially at longer horizons [2602.11533].

## 2. Gradient entanglement and the optimization conflict

ALTTS does not motivate its dual-path structure solely by inductive bias; it also gives a gradient-based argument. If projection-specific residuals were observable, then each block \(\hat f_{ij}\) with parameters \(\theta_{ij}\) could be optimized using the “ideal” gradient
\[
\nabla_{\theta_{ij}} \mathcal{L}^* = -J_{ij}^\top r_{ij},
\]
where
\[
\mathcal{L}^* := \frac{1}{2}\sum_{i,j=1}^D \|r_{ij}\|_2^2,
\]
\(J_{ij}\) is the Jacobian of \(\hat f_{ij}\), and \(r_{ij} := f_{ij}(X_j)-\hat f_{ij}(X_j)\). In practice, however, only the aggregate residual for variable \(i\) is available:
\[
r_i := Y_i - \hat Y_i = \sum_{j=1}^D r_{ij},
\]
so the practical gradient becomes
\[
\nabla_{\theta_{ij}} \mathcal{L} = -J_{ij}^\top r_i,
\]
with
\[
\mathcal{L} := \frac{1}{2}\sum_{i=1}^D \|r_i\|_2^2.
\]
For the AR block, this means the update for \(\theta_{ii}\) is contaminated by the off-diagonal CR residuals. Writing
\[
r_{-ii} := \sum_{j\neq i} r_{ij},
\]
the AR gradient inherits a cross-block noise term that generally does not vanish in the channel-dependent setting. The paper further decomposes the covariance of the mixed AR gradient as
\[
\mathrm{Cov}(J_{ii}^\top r_i) = \Sigma_{ii} + \Sigma_{-ii} + 2\,\mathrm{Cov}(J_{ii}^\top r_{ii}, J_{ii}^\top r_{-ii}),
\]
and argues that the CR component contributes a higher-variance perturbation. Empirically, under joint training the CR branch exhibits much larger gradient variance than the AR branch on five of seven datasets, while alternating optimization stabilizes both branches [2602.11533].

## 3. Dual-path architecture

The implemented ALTTS architecture reflects the operator decomposition directly. Its final prediction is the additive combination
\[
\hat Y = \hat Y^{\text{AR}} + \hat Y^{\text{CR}}.
\]

The AR path is deliberately simple. After Reversible Instance Normalization (RevIN), each variable is forecast independently by a linear predictor:
\[
\hat Y_i^{\text{AR}} = \hat f_{ii}(X_i).
\]
The paper identifies this branch with RLinear. Parameters are not shared across variables, so the AR path is channel-independent and purely diagonal.

The CR path uses an inverted Transformer encoder in the style of iTransformer, but applied over variable tokens rather than time tokens. The normalized multivariate input is embedded as
\[
Z_0 = \mathrm{Embedding}(X_t), \qquad Z_0 \in \mathbb{R}^{D\times L_0}.
\]
Each token represents one variable’s temporal history. Cross-variable modeling is then performed by Cross-Relation Self-Attention:
\[
\mathrm{CRSA}(Z_0)=\mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_h}} + M\right)V,
\]
where
\[
Q,K,V \in \mathbb{R}^{D\times d_h},
\]
and
\[
M = \mathrm{diag}(-\infty,\dots,-\infty).
\]
The diagonal mask forces self-attention logits to \(-\infty\), so each variable is prohibited from attending to itself. This is the architectural mechanism by which the CR path is prevented from duplicating the AR path. After attention, the encoder follows a standard residual-Transformer pattern:
\[
Z_1 = \mathrm{LayerNorm}(Z_0 + \mathrm{MHSA}(Z_0)),
\]
\[
Z_2 = \mathrm{LayerNorm}(Z_1 + \mathrm{MLP}(Z_1)).
\]
A channel-independent linear head maps encoder outputs to \(\hat Y_i^{\text{CR}}\), and the final prediction is obtained by denormalized summation of the AR and CR outputs. The implementation uses parameter-free RevIN, so the two branches share no trainable parameters [2602.11533].

## 4. Alternating optimization

The defining methodological contribution of ALTTS is its training procedure. Rather than jointly optimizing all parameters, the paper alternates between AR and CR subproblems. The total objective is written as
\[
\mathcal{L}(\theta_{\text{AR}}, \theta_{\text{CR}}; B) =
\ell(\theta_{\text{AR}}, \theta_{\text{CR}}; B)
+ R_{\text{AR}}(\theta_{\text{AR}})
+ R_{\text{CR}}(\theta_{\text{CR}}),
\]
where \(B\) is a random minibatch, \(\ell\) is the data-fitting term, and \(R_{\text{AR}},R_{\text{CR}}\) are branch-specific regularizers. At iteration \(n\), ALTTS solves
\[
\theta_{\text{AR}}^{(n+1)} \gets \arg\min_{\theta_{\text{AR}}}
\left\| Y_{t+1} - (F_{\text{AR}} + F_{\text{CR}}^{(n)}) * X_t \right\|_2^2
+ R_{\text{AR}}(\theta_{\text{AR}}),
\]
then
\[
\theta_{\text{CR}}^{(n+1)} \gets \arg\min_{\theta_{\text{CR}}}
\left\| Y_{t+1} - (F_{\text{AR}}^{(n+1)} + F_{\text{CR}}) * X_t \right\|_2^2
+ R_{\text{CR}}(\theta_{\text{CR}}).
\]

The covariance argument for alternating optimization is explicit. Under joint training,
\[
\mathrm{Cov}(\nabla_{\theta_{ii}}\mathcal{L}) =
E_{\theta_{\text{CR}}}\!\left[ \mathrm{Cov}(\nabla_{\theta_{ii}}\mathcal{L}\mid \theta_{\text{CR}}) \right]
+ \mathrm{Cov}_{\theta_{\text{CR}}}\!\left[ E(\nabla_{\theta_{ii}}\mathcal{L}\mid \theta_{\text{CR}}) \right].
\]
Under alternating training, since \(\theta_{\text{CR}}\) is fixed while updating AR,
\[
\mathrm{Cov}_{\text{alt}}(\nabla_{\theta_{ii}}\mathcal{L}) =
E_{\theta_{\text{CR}}}\!\left[ \mathrm{Cov}(\nabla_{\theta_{ii}}\mathcal{L}\mid \theta_{\text{CR}}) \right],
\]
hence
\[
\mathrm{Cov}_{\text{alt}}(\nabla_{\theta_{ii}}\mathcal{L})
\preceq
\mathrm{Cov}(\nabla_{\theta_{ii}}\mathcal{L}).
\]
This is the paper’s formal statement that alternating optimization removes the extra covariance term induced by the changing CR block [2602.11533].

In implementation, ALTTS uses two independent AMSGrad optimizers. The AR path is always updated before the CR path, with a practical schedule of **10 AR updates and 2 CR updates per mini-batch**. The datasets are Weather, Traffic, Electricity, ETTh1, ETTh2, ETTm1, and ETTm2; the input length is \(L=512\); the prediction horizons are \(H\in\{96,192,336,720\}\); ETT datasets use a \(6{:}2{:}2\) train/validation/test split, and the others use \(7{:}1{:}2\) [2602.11533].

## 5. Empirical results and ablations

Across 7 datasets, 4 horizons, and 2 metrics, the paper reports 56 evaluation slots. ALTTS achieves **49 top-two results**, compared with **25** for OLinear and **19** for PatchTST. The strongest improvements appear on the ETT family and Weather, especially at long horizons. On Weather, ALTTS reports \(0.144 / 0.195\) at horizon 96 and \(0.310 / 0.329\) at horizon 720. On ETTh1, it reports \(0.360 / 0.391\) at horizon 96 and \(0.435 / 0.453\) at horizon 720. On ETTm2, it is best across all horizons on MSE, including \(0.160 / 0.249\) at horizon 96 and \(0.357 / 0.382\) at horizon 720 [2602.11533].

The most decisive ablation concerns alternating optimization. The paper compares ALTTS with AO and without AO. Representative differences are large: on ETTh1 at horizon 720, ALTTS with AO reports \(0.435/0.453\), versus \(0.510/0.506\) without AO; on ETTh2 at horizon 720, \(0.375/0.419\) versus \(0.440/0.463\); on ETTm2 at horizon 720, \(0.357/0.382\) versus \(0.381/0.401\). The paper notes a few narrower exceptions, such as Electricity at horizon 720, where simpler AR-dominant methods remain very competitive. This yields a consistent pattern: ALTTS is strongest where both persistent autoregression and useful cross-variable interactions coexist, while its relative advantage narrows in strongly AR-dominated regimes [2602.11533].

The paper also uses RLinear as an AR-only reference and iTransformer as a CR-only reference. This comparison supports the dual-path thesis: RLinear performs well on autoregression-dominated settings such as Electricity, iTransformer helps where cross-variable coupling matters, and ALTTS is designed to combine both without allowing the CR component to destabilize the AR component. The gradient-variance analysis reinforces this interpretation by showing that under joint training the CR branch is the noisier block, whereas alternating updates materially stabilize the optimization trajectory [2602.11533].

## 6. Scope, limitations, and acronymic ambiguity

ALTTS is specific to multivariate time series forecasting. The paper does **not** provide explicit big-\(O\) time or memory complexity formulas, and it omits several implementation details such as learning rate, batch size, number of Transformer layers, hidden dimension \(L_0\), number of heads, and dropout. Its gains are also not universal: Electricity and parts of Traffic remain settings where simpler linear or Transformer baselines can match or slightly exceed it. These omissions do not alter the central claim, but they delimit the method’s current level of specification [2602.11533].

The term “ALTTS” is also easy to confuse with unrelated acronyms. In adjacent literature, **ATLAS** denotes “Agentic Test-time Learning-to-Allocate Scaling” [2606.01667], **LATTS** denotes “Locally Adaptive Test-Time Scaling” [2509.20368], and **aLTT** denotes “Adaptive Learn-then-Test” [2409.15844]. These are distinct methods in distinct domains. In the forecasting literature, the exact referent is the dual-path AR/CR framework of AltTS [2602.11533].

In that sense, ALTTS is best understood not as a generic acronym family but as a specific forecasting architecture whose main contribution is optimization-theoretic: it argues that multivariate forecasting quality depends not only on representing AR and CR structure, but on preventing the higher-variance CR block from corrupting the lower-variance AR block during training. Its broader implication is that, in long-horizon multivariate forecasting, carefully designed optimization strategies may be as consequential as architectural expressivity itself [2602.11533].

Source: https://www.emergentmind.com/topics/altts