Autoregressive Buffering in Sequential Models

• Autoregressive buffering is a mechanism that couples explicit, memory-retaining buffers with autoregressive processes for efficient sequential modeling in areas like queueing theory and neural transformers.
• Computational strategies, including transformer buffer mechanisms and recurrent buffering units, reduce complexity by enabling efficient one-pass inference and dynamic state updates.
• Empirical studies show that autoregressive buffering enhances simulation reliability and model stability in applications ranging from financial time series to network simulation.

Autoregressive buffering refers to a broad spectrum of mechanisms in which buffered (memory-retaining) state is dynamically coupled with an autoregressive process for the purpose of efficient sequential modeling, queueing theory, regime-switching time series, queue simulation, and high-throughput joint inference. These models are distinguished from classical autoregressive formulations by introducing explicit buffer systems—states or data structures that mediate the influence of past observations, input signals, or contextual information on the evolution of the current and future outputs, frequently under causality or reflection constraints. The applications and formal models are diverse, spanning neural transformers with causal buffer layers, queuing systems with AR-driven net inputs, time series with sticky regime buffers, recurrent neural buffering units for dynamical system emulation, and reflected AR(1) processes with general dependencies.

1. Mathematical Formulations of Autoregressive Buffering Systems

The core mathematical archetype is an evolution equation—deterministic or stochastic—where the next state depends on both past autoregressive states and the buffer, possibly with nontrivial dependence structure, and subject to boundary or causality constraints.

Transformer Buffer Mechanism

In transformer-based models for joint probabilistic inference, a causal autoregressive buffer is introduced to separate immutable context encoding from mutable target memory. Let the context set be $\mathcal{C} = \{(x_n, y_n)\}_{n=1}^N$, with a cached encoder $r_{\mathcal{C}}(\mathcal{C})$, and buffer states $\mathcal{B}_{1:k} = \{(x_j, y_j)\}_{j=1}^k$ encoded as $\mathbf{b}_{1:k}$. The autoregressive factorization,

$$p(y_{1:K} \mid x_{1:K}; \mathcal{C}) = \prod_{k=1}^K p(y_k \mid x_k ; [r_{\mathcal{C}}(\mathcal{C}), \mathbf{b}_{1:k-1}])$$

with buffer update,

$$\mathbf{b}_k = r_{\mathcal{B}}((x_k,y_k), [r_{\mathcal{C}}(\mathcal{C}), \mathbf{b}_{1:k-1}]), \quad \mathbf{b}_{1:0} = \emptyset$$

bridges set- and sequence-based prediction (Hassan et al., 10 Oct 2025).

AR-Driven Buffering in Queueing and Networks

Consider the buffer update in queuing theory: $Q_n = \max\{0, Q_{n-1} + Z_{n,1}\}$ where $Z_{n,1}$ is a net-input increment governed by an AR($p$) model on $Y_n = Z_{n,1} - \mu$: $Y_n = \sum_{i=1}^p \varphi_i Y_{n-i} + \epsilon_n$ and $\epsilon_n$ i.i.d. noise (Fendick, 2011). In network simulation, the Recurrent Buffering Unit (RBU) evolves as: $$b_t = z_t \odot b_{t-1} + (1-z_t) \odot \tilde{b}_t$$ where $z_t$ is an update gate and $\tilde{b}_t$ a candidate state; the output is made autoregressive via feedback of $\hat{d}_{t-1}$ into the input (Anshumaan et al., 2022).

Buffered Regime Switching and Double AR

The BDAR($p$) model formalizes regime "stickiness" via a buffer zone $[r_L, r_U]$ on the threshold variable. If $y_{t-d}$ is in this zone, the regime indicator $R_t$ persists; otherwise, it switches:

$$R_t = \begin{cases} 1, & y_{t-d} \leq r_L \ 0, & y_{t-d} > r_U \ R_{t-1}, & r_L < y_{t-d} \leq r_U \end{cases}$$

and $y_t = \mu_{R_t, t} + \sigma_{R_t, t} \varepsilon_t$ with $\varepsilon_t \sim N(0,1)$ (Liu, 2018).

2. Computational Strategies and Algorithms

Autoregressive buffering often targets reduction of computational complexity and memory overhead, and enables efficient batch inference or exact simulation.

Efficient Autoregressive Sampling and One-Pass Joint Log-Likelihood

Using cached key/values for the context, and a dynamically growing buffer block for targets, the transformer-based causal buffer approach performs joint inference via a single forward pass with a causal mask. For a context of size $N$ and $K$ targets, per-layer cost transitions from naive $\mathcal{O}(K(N+K)^2)$ to buffered $\mathcal{O}(N^2+NK+K^2)$ (Hassan et al., 10 Oct 2025).

Algorithmic Skeletons:

• Sampling with Buffer: For $k=1,\dots,K$, sample $y_k$ conditioned on $$[\mathrm{KV}_{\mathcal{C}}, \mathrm{KV}_{\mathcal{B}_{1:k-1}}]$$, update the buffer.
• One-Pass Log-Likelihood: Evaluate all $K$ targets with proper causal masking in a single forward, summing log probabilities.

Grey-Box Simulation with Autoregressive Buffering

The RBU integrates queuing-inspired semantics in a gated RNN, predicting buffer occupancy using past delay feedback. It optimizes through regularized MSE and supports interpretability by aligning $b_t$ with learned queue length (Anshumaan et al., 2022).

AR Queue Simulation and Distributional Minimization

Queue simulation with AR-driven increments and censored demand employs alternating latent variable sampling (E-step) and least-squares refit (M-step) in the “Distribution-Minimization" algorithm, exploiting buffer state and observed queue lengths (Fendick, 2011).

3. Regimes of Stability, Ergodicity, and Stationarity

Autoregressive buffering models have specific structural constraints to ensure stable long-run behavior.

• Causal Transformer Buffer: Context-cached conditioning is fixed, and causal masking in the buffer admits efficient, stable joint inference (Hassan et al., 10 Oct 2025).
• BDAR(p): Geometric ergodicity holds if model parameters satisfy norm contraction involving both AR coefficients and conditional variance loadings:

$$\sum_{j=1}^p \sup_{k=1,2} |\phi_{kj}|^r + \sum_{j=1}^p \sup_{k=1,2} \alpha_{k j}^{r/2} E|\varepsilon_t|^r < 1$$

for $r \in (0,1]$ (Liu, 2018).

• Reflected AR(1) Queues: Under $E[\ln|\alpha_n|]<0$ and $E[\ln^+|D_n|]<\infty$, a unique stationary regime obtains for $X_n = \max\{0, \alpha_n X_{n-1} + D_n\}$ (Dimitriou et al., 2023).

4. Generalizations and Extensions

Autoregressive buffering methodology encompasses a range of models differing in memory mechanism, feedback, and regime switching.

• Threshold and Proportional Dependence: Reflected AR models allow input dependence on past service times, arrival times, or buffer occupancy, using functional and iterative representations of stationary transforms (Dimitriou et al., 2023).
• Graduated Censoring: Queue boundaries can be softened by a graduation parameter $\omega$ in the update law; simulation and parameter estimation generalize accordingly (Fendick, 2011).
• Multidimensional Buffers: Priority-retrial queues and multidimensional buffer processes yield boundary value problems reducible to mixed-AR functional equations (Dimitriou et al., 2023).

5. Empirical and Comparative Results

The practical impact of autoregressive buffering is evaluated via synthetic experiments, real-world data, and comparative modeling.

• Transformer Buffering: On large-context tasks, the buffered method achieves up to $20\times$ speedup over fully autoregressive transformer neural processes with negligible predictive loss (log-likelihoods matching within statistical error). For synthetic GPs and EEG interpolation (with $M=16$), log-likelihoods are: | Method | GP | Sawtooth | EEG Int. | |-------------------|--------|----------|----------| | TNP-D AR | 2.57 | 1.05 | 0.51 | | TNP-A | 2.24 | 0.98 | 0.58 | | TNP w/ buffer | 2.51 | 1.00 | 0.52 | (SEM in parentheses) (Hassan et al., 10 Oct 2025).
• Recurrent Buffering Unit: RBU reduces per-packet delay MSE by 30–50% over LSTM and 15–25% over Transformer on challenging network simulation benchmarks; generalizes to new protocols and delivers sub-millisecond per-packet simulation (Anshumaan et al., 2022).
• BDAR Simulations and Finance: Simulation confirms estimator consistency and sharpness; empirical use on Hang Seng Index data uncovers strong regime asymmetry and volatility leverage effects, with BIC selection accurately identifying model order (Liu, 2018).
• Queueing Case Studies: Application to U.S. nonfarm jobs identifies autoregressive demand for unfilled positions and captures macroeconomic stress via estimated unsatisfied demand (Fendick, 2011).

6. Conceptual Scope, Limitations, and Applications

Autoregressive buffering is a unifying abstraction in sequential, regime-switching, and queueing models, stimulating progress in probabilistic deep learning, network systems, and econometric regime modeling. Its strength is in trading off full autoregressive expressivity with efficiency, stability, and interpretability. Main limitations include modeling choices for buffer granularity, the necessity to tune memory or boundary parameters, and, in some queue models, Gaussian increment assumptions. Potential applications extend to inventory management, network path emulation, tabular probabilistic modeling, financial time series, call-center staffing, and limit-order book simulation.

The literature consolidates autoregressive buffering as a principle for scalable and interpretable structure in modern statistical and machine learning models (Hassan et al., 10 Oct 2025, Anshumaan et al., 2022, Liu, 2018, Fendick, 2011, Dimitriou et al., 2023).