Autoregressive Unrolling

• Autoregressive unrolling is a technique that reformulates latent-state recursions into explicit input–output filters, enabling clear, finite or infinite representations of dynamic systems.
• It encompasses classical linear filters via Kalman unrolling, operator-theoretic z-transform methods, and modern neural forecasting regimes that emphasize error propagation and stability.
• The approach is applied in diverse settings—from time-series prediction and iterative optimization to ADMM-based denoising—highlighting practical trade-offs in stability, computational depth, and gradient propagation.

Searching arXiv for relevant papers on autoregressive unrolling and adjacent formulations. Autoregressive unrolling denotes the explicit expansion, iteration, or repeated application of an autoregressive update rule across time or algorithmic depth, so that the evolution of a predictor or solver is represented as a sequence of state-dependent computations rather than as an implicit fixed-point or latent-state mechanism. In the literature, the term appears in several technically distinct but structurally related settings: unrolling the Kalman filter of a linear dynamical system into an autoregressive input–output filter; training forecasting models by rolling them forward on their own predictions across multiple horizon blocks; interpreting iterative optimization algorithms as recurrent updates and truncating or nesting those updates; and analyzing bilateral autoregressive laws of motion through forward, backward, and outward temporal decompositions (Lee et al., 2019, Li et al., 2 Feb 2026, Mehmood et al., 23 Feb 2026, Beare et al., 2024).

1. Classical linear formulation

In its classical control-theoretic form, autoregressive unrolling arises when the optimal predictor of a linear dynamical system with hidden state is rewritten entirely in terms of past inputs and outputs. For the linear Gaussian state-space model

$$\begin{aligned} h(t) &= A h(t-1) + B x(t-1) + \xi(t), \ y(t) &= C h(t) + \eta(t), \end{aligned}$$

the steady-state Kalman filter is the optimal linear one-step predictor. That predictor can be expressed as an infinite autoregressive filter

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$

with coefficients

$$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$

Here unrolling means replacing latent-state recursion by a representation over the observed trajectory, so that the predictor becomes an autoregressive filter driven jointly by past controls and past outputs (Lee et al., 2019).

The same paper formulates the learning problem directly at the filter level: $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$ with $\eta(t)\sim \mathcal{N}(0,\sigma^2)$ i.i.d. and finite-support or truncated filters $g^*, h^*$. One then estimates finite-length filters $g,h\in\mathbb{R}^r$ for one-step-ahead prediction, bypassing identification of the underlying state-space matrices. This is an improper learning strategy in the sense that the learned object is an input–output operator rather than a recovered latent-state model. A plausible implication is that autoregressive unrolling can be viewed not only as an analysis device, but also as a target representation for learning when state identification is unnecessary or undesirable.

A second, broader linear perspective is provided by the complete solution theory for the autoregressive law of motion

$x_t = \phi x_{t-1} + \varepsilon_t$

in a finite-dimensional complex vector space. There the set of all solutions is decomposed by spectral projections associated with eigenvalues inside, outside, and on the unit circle. Every solution is the sum of three temporally directed components: one flowing forward from the arbitrarily distant past, one backward from the arbitrarily distant future, and one outward from time zero (Beare et al., 2024). This bilateral formulation generalizes the standard causal forward recursion and makes explicit that “unrolling” an autoregressive dynamic need not be purely forward in time.

2. Operator-theoretic representation

A central representation in the linear filter setting is obtained by passing to $z$-transforms. If

$$G^*(z)=\sum_{k\ge0} g^*(k) z^{-k}, \qquad H^*(z)=\sum_{k\ge0} h^*(k) z^{-k},$$

then the autoregressive model can be written as

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$0

hence

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$1

Defining the unrolled filter

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$2

one obtains

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$3

or in the time domain

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$4

In this representation, the feedback recursion has been expanded into a convolution operator on the entire past trajectory, directly analogous to unrolling a recurrent network into a feedforward computation across time (Lee et al., 2019).

The same paper uses $$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$5 and $$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$6 norms to distinguish worst-case and average-case behavior. For a filter $$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$7 with transfer function $$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$8,

$$\hat y(t+1) = \sum_{k=0}^{\infty} g^*(k)\, x(t-k) + \sum_{k=0}^{\infty} h^*(k)\, y(t-k),$$9

Under the stability assumption $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$0, the paper derives a bound on prediction error over a rollout horizon $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$1 in which the signal-dependent term is governed by an $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$2-type quantity and the noise term by an $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$3-type quantity. This makes explicit that, once an autoregressive system is unrolled, stability and robustness are determined by operator norms of the induced trajectory map rather than by coefficient-wise estimation error alone (Lee et al., 2019).

A related but more general operator picture appears in the fixed-point analysis of unrolling. If $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$4 defines a solution mapping $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$5, then unrolling $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$6 iterations of

$$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$7

and differentiating through them yields a Jacobian recursion

$$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$8

where $$g^*(k) = C_{KF} A_{KF}^k B_{KF,x}, \qquad h^*(k) = C_{KF} A_{KF}^k B_{KF,y}.$$9 and $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$0. The true Jacobian is

$y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$1

This fixed-point formulation is not itself an autoregressive model of observed time series; however, it places iterative unrolling and recurrent differentiation in the same formal class of repeatedly applied state updates (Mehmood et al., 23 Feb 2026).

3. Training-time rollout and temporal causality

In contemporary time-series forecasting, autoregressive unrolling often refers to repeated application of a short-horizon predictor to its own outputs. A recent formulation considers a model

$y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$2

with input length $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$3, native prediction horizon $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$4, and optional overlap $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$5. The first block is

$y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$6

and subsequent blocks are generated recursively: $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$7 The model is therefore trained and evaluated in the same prediction-fed regime in which it is later deployed (Li et al., 2 Feb 2026).

The distinctive feature of that work is not merely autoregressive rollout, but a horizon-structured objective. For the $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$8-th block of length $y(t+1) = g^* * x(t) + h^* * y(t) + \eta(t+1),$9,

$\eta(t)\sim \mathcal{N}(0,\sigma^2)$0

The paper begins from the constrained formulation

$\eta(t)\sim \mathcal{N}(0,\sigma^2)$1

then relaxes the monotonicity condition through an RL-style reward

$\eta(t)\sim \mathcal{N}(0,\sigma^2)$2

with discounted loss

$\eta(t)\sim \mathcal{N}(0,\sigma^2)$3

The explicit intention is to enforce temporal causality in the sense that autoregressive prediction errors should not decrease with horizon. When later-block errors are smaller than earlier-block errors, the corresponding gradient is down-weighted by a factor $\eta(t)\sim \mathcal{N}(0,\sigma^2)$4, whereas monotone growth retains the full gradient magnitude (Li et al., 2 Feb 2026).

This treatment differs from teacher forcing and one-shot direct forecasting. In direct multi-horizon prediction, a model maps a past window to the entire future window in one pass, and the loss imposes no causal relation between early and late prediction errors. In teacher-forced sequence-to-sequence training, the decoder sees ground-truth previous future steps during training but predicted ones at inference, generating exposure bias. By contrast, the unrolled regime above trains directly on predicted contexts. This suggests a broader definition of autoregressive unrolling as an optimization regime in which the unrolled trajectory, rather than only the next-step predictor, is the training object.

4. Algorithmic realizations and variants

Autoregressive unrolling appears in neural forecasting architectures both as an explicit linear component and as recursive state evolution. The “Autoregressive Convolutional Recurrent Neural Network” combines a multi-scale causal CNN feature extractor, three GRU encoders, and a linear autoregressive shortcut. Given an input window $\eta(t)\sim \mathcal{N}(0,\sigma^2)$5, the model constructs downsampled streams $\eta(t)\sim \mathcal{N}(0,\sigma^2)$6 and $\eta(t)\sim \mathcal{N}(0,\sigma^2)$7, applies two layers of causal convolutions with ReLU to each scale, encodes them by GRUs, concatenates the final hidden states, and maps them to a direct multi-step forecast $\eta(t)\sim \mathcal{N}(0,\sigma^2)$8. In parallel, for each variable $\eta(t)\sim \mathcal{N}(0,\sigma^2)$9, it computes a linear prediction

$g^*, h^*$0

implemented effectively with only the last five time steps, so that

$g^*, h^*$1

The final prediction is

$g^*, h^*$2

Although the model is trained for direct multi-step prediction rather than recursive free-running rollout, its GRU state dynamics and explicit lag-based head make it naturally interpretable as defining an autoregressive mapping that can itself be unrolled on a sliding window (Maggiolo et al., 2019).

A different use of unrolling appears in graph signal restoration. There, ADMM iterations for graph denoising are treated as layers: $g^*, h^*$3 followed by proximal and dual updates. In the unrolled GraphDAU architecture, the layer-specific scalars $g^*, h^*$4, $g^*, h^*$5, and $g^*, h^*$6 are learnable, while graph operators remain fixed. NestDAU further places this ADMM-based denoiser inside an outer PnP-ADMM loop, creating a nested unrolling in which an outer iterative solver contains an inner unrolled iterative module (Nagahama et al., 2021). The paper explicitly describes these constructions as iterative or autoregressive updates in the sense that each layer computes the next state from the current one. Although this is not an autoregressive time-series model, it is a structurally relevant expansion of the term “autoregressive unrolling” to optimization-derived recurrent maps.

A third variant arises in learning variational models by unrolling gradient descent on a quadratic denoising objective. With step size $g^*, h^*$7,

$g^*, h^*$8

and initialization $g^*, h^*$9, the $g,h\in\mathbb{R}^r$0-step unrolled estimator is

$g,h\in\mathbb{R}^r$1

The analysis characterizes the estimator class reachable by finite-depth unrolling and shows that the step size matters substantially, while the number of unrolled iterations plays a minor role once the step size is learned (Brauer et al., 2022). A plausible implication is that, in many unrolled autoregressive or iterative systems, per-step scaling parameters may determine practical behavior more strongly than nominal depth.

5. Stability, robustness, and the curse of unrolling

A recurring issue in autoregressive unrolling is error propagation. In the linear filter setting, the prediction error for candidate filters $g,h\in\mathbb{R}^r$2 relative to the true conditional mean is

$g,h\in\mathbb{R}^r$3

Under the unrolled representation and the assumption $g,h\in\mathbb{R}^r$4, robust performance over a horizon $g,h\in\mathbb{R}^r$5 can be controlled by an $g,h\in\mathbb{R}^r$6 quantity involving the mismatch in frequency response and by an $g,h\in\mathbb{R}^r$7 quantity governing the noise term. The same paper argues that ordinary least squares only yields Frobenius-type control and can therefore be suboptimal by a $g,h\in\mathbb{R}^r$8 factor when translated into worst-case operator norms (Lee et al., 2019). In this precise sense, the central difficulty of autoregressive unrolling is not merely local next-step fitting but control of the operator induced by repeated rollout.

The same principle appears in modern forecasting objectives through the insistence that long-horizon errors should increase with horizon. The RL-style loss described above treats later blocks that appear “too good” relative to earlier ones as suspicious and dampens their gradient contribution. The empirical claim reported there is that, under the new loss, MSE grows roughly linearly with the number of AR steps $g,h\in\mathbb{R}^r$9 across models and datasets, rather than exploding erratically (Li et al., 2 Feb 2026). This suggests a practical notion of stable unrolling in which error accumulation is predictable rather than absent.

A distinct but closely related issue is the “curse of unrolling” in derivative computation. For fixed-point iterations $x_t = \phi x_{t-1} + \varepsilon_t$0, the derivative iterates converge asymptotically to the implicit Jacobian, but the non-asymptotic bound

$x_t = \phi x_{t-1} + \varepsilon_t$1

contains a growing-then-decaying term $x_t = \phi x_{t-1} + \varepsilon_t$2. The paper identifies this term as the source of transient overshoot in Jacobian error: more unrolling can initially worsen gradients before eventual convergence (Mehmood et al., 23 Feb 2026). The effect is absent in a fully linear setting with constant $x_t = \phi x_{t-1} + \varepsilon_t$3 and $x_t = \phi x_{t-1} + \varepsilon_t$4, where Jacobian errors decay monotonically. This creates an important distinction between stable forward rollout and stable gradient propagation: contractive forward dynamics do not by themselves preclude transient derivative deterioration.

The same work shows that truncating early derivative iterations mitigates the curse: $x_t = \phi x_{t-1} + \varepsilon_t$5 so the harmful term gains an extra factor $x_t = \phi x_{t-1} + \varepsilon_t$6. In reverse mode this reduces memory from $x_t = \phi x_{t-1} + \varepsilon_t$7 to $x_t = \phi x_{t-1} + \varepsilon_t$8. In autoregressive language, truncated backpropagation is therefore not just a computational convenience; it can be a principled response to transient instability in unrolled gradients (Mehmood et al., 23 Feb 2026).

6. Regime switching, bilateral time, and scope of the concept

Autoregressive unrolling is not restricted to a single stationary forward recursion. In the AR-AsLG-HMM, the hidden state process $x_t = \phi x_{t-1} + \varepsilon_t$9 is first-order Markov, but the emission law in each state $z$0 and variable $z$1 is

$z$2

with

$z$3

The state-specific autoregressive order $z$4 is chosen by structural EM under a BIC-type penalty, and the forward–backward and Viterbi recursions are modified so that the emission likelihood at time $z$5 depends on the lagged observations $z$6 (Puerto-Santana et al., 2020). Here the unrolling is probabilistic and regime-dependent: at each time step the active autoregressive structure changes with the latent state.

An even broader scope is provided by the bilateral solution theory for $z$7. There, forward unrolling is valid on stable eigenspaces, backward unrolling on unstable eigenspaces, and unit-circle modes require outward cumulation from time zero (Beare et al., 2024). This resolves a common misconception that autoregressive unrolling is inherently causal and one-sided. In the general finite-dimensional setting, purely forward-time representation of all solutions is impossible unless there are no unstable or unit-circle modes. Backward and outward components are mathematically intrinsic whenever the spectrum demands them.

Taken together, these formulations indicate that “autoregressive unrolling” is best understood as a family of constructions rather than a single method. In control and signal processing, it often means converting latent recursion into a filter on observed histories. In forecasting, it denotes iterated prediction-fed rollout and loss design across the resulting blocks. In algorithmic learning, it denotes finite-depth differentiation through recurrent optimization maps, possibly nested or truncated. In spectral autoregressive theory, it includes forward, backward, and outward temporal resolutions of the same law of motion. This suggests that the unifying feature is not the specific architecture but the replacement of an implicit dynamic law by an explicit finite or infinite composition over time or iteration index.

7. Limitations, trade-offs, and open directions

Across these literatures, strong assumptions remain central. The robust filter-learning analysis assumes linear time-invariant dynamics, Gaussian noise, stability in $z$8, designed sinusoidal inputs, multiple rollouts, and primarily a single-input single-output presentation (Lee et al., 2019). The time-series forecasting method with monotonic error constraints is empirical in its characterization of near-linear error growth and does not provide a rigorous propagation bound for ultra-long horizons (Li et al., 2 Feb 2026). The curse-of-unrolling analysis is carried out under contraction and smoothness assumptions for fixed-point maps, so its direct transfer to arbitrary noncontractive sequence models requires caution (Mehmood et al., 23 Feb 2026).

A second set of limitations concerns what is actually learned. In variational-model learning by unrolling, finite-depth unrolling can outperform exact bilevel optimization in the quadratic toy model, but the result depends sensitively on the step size and on the spectral properties induced by parity of the depth $z$9 (Brauer et al., 2022). This does not imply that more unrolling is universally useless; rather, it shows that depth, parameter sharing, and update scaling interact in structured and sometimes counterintuitive ways.

A third limitation is definitional breadth. Some works use “autoregressive” in the classical lag-on-observation sense; others extend the term to any iterative map whose next state depends on previous iterates, including ADMM and PnP-ADMM layers (Nagahama et al., 2021). This broader usage is structurally natural but not universal. A plausible editorial distinction is between trajectory autoregression—explicit dependence on past predicted or observed sequence values—and iterative autoregression—state recursion across algorithmic depth. The two are closely related in unrolled computation graphs but not identical.

Open directions are visible in the source material itself. The robust filter-learning work leaves open whether comparable guarantees can be obtained from a single rollout rather than multiple independent ones (Lee et al., 2019). The forecasting work leaves future theoretical analysis of AR error propagation and broader exploration of ultra-long horizons (Li et al., 2 Feb 2026). The curse-of-unrolling work points to warm-starting as an implicit form of truncation, suggesting further study of stateful training and equilibrium initialization in recurrent and autoregressive systems (Mehmood et al., 23 Feb 2026). The bilateral spectral theory suggests that stable, unstable, and unit-root components may merit separate treatment in practical sequence models whenever long-horizon rollout is required (Beare et al., 2024).

Autoregressive unrolling therefore occupies a technically important intersection of system identification, forecasting, optimization, and dynamical-systems theory. Its central questions are how a recursive law is expanded into an explicit trajectory operator, how approximation and learning error propagate through that operator, which norms or objectives govern robust long-horizon behavior, and how much of the unrolled computation should actually be differentiated or learned.