---
title: 'ARMA: Theory, Estimation, and Applications'
url: https://www.emergentmind.com/topics/arma
type: topic
---

# ARMA: Theory, Estimation, and Applications

Searching arXiv for recent and foundational ARMA papers relevant to a comprehensive synthesis.
ARMA, or autoregressive moving average, denotes a class of linear stochastic models in which a process is represented through an autoregressive polynomial and a moving-average polynomial, equivalently through a rational transfer function. In the standard stationary univariate setting, an ARMA\((p,q)\) model is written as
\[
X_t - \sum_{i=1}^{p} \phi_i X_{t-i} = \varepsilon_t + \sum_{j=1}^{q} \theta_j \varepsilon_{t-j},
\]
or, with the backshift operator \(B\), as \(\Phi(B)X_t=\Theta(B)\varepsilon_t\). Across the literature, the same rational-filter viewpoint extends beyond scalar time series to two-dimensional random fields, graph filters, quaternion-valued signals, and non-Gaussian or stochastic relaxations used for missing data, heavy tails, and biomedical imaging [2408.10610].

## 1. Classical formulation and spectral viewpoint

In its classical form, ARMA combines an autoregressive part, governed by coefficients \(\phi_1,\dots,\phi_p\), with a moving-average part, governed by \(\theta_1,\dots,\theta_q\). Using
\[
\Phi(B)=1-\phi_1B-\cdots-\phi_pB^p,\qquad
\Theta(B)=1+\theta_1B+\cdots+\theta_qB^q,
\]
the model is \(\Phi(B)X_t=\Theta(B)\varepsilon_t\), where \(\varepsilon_t\) is white noise with \(\mathbb{E}[\varepsilon_t]=0\) and variance \(\sigma^2\). Stationarity requires the zeros of \(\Phi(z)\) to lie outside the closed unit disc, and invertibility requires the zeros of \(\Theta(z)\) to lie outside the unit disc [2408.10610].

This representation is equivalent to viewing ARMA as a linear time-invariant filter with rational transfer function
\[
\Psi(B)=\frac{\Theta(B)}{\Phi(B)}.
\]
In the moving-average representation \(X_t=\sum_{k=0}^\infty c_k\varepsilon_{t-k}\), the generating function \(C(z)=\sum_{k=0}^\infty c_k z^k\) determines the second-order structure, and the spectral density is
\[
f(\lambda)=\sigma^2\left|C(e^{i\lambda})\right|^2.
\]
For ARMA processes, \(C(z)\) is rational; this is the core link between ARMA modeling and rational approximation on the unit circle [2408.10610].

From the forecasting perspective, the one-step-ahead predictor in the classical Gaussian formulation is
\[
\hat{Y}_t=\zeta+\sum_{j=1}^{q}\beta_j E_{t-j}+\sum_{i=1}^{p}\alpha_iY_{t-i},
\]
with one-step predictive variance \(\gamma\) when \(\beta_0=1\). This standard formulation underlies exact likelihood methods, state-space implementations, and later graphical-model and generalized extensions [1207.4162].

## 2. Likelihood, estimation, and inference

Classical ARMA estimation is typically performed by maximum likelihood, nonlinear optimization, or innovations/Kalman-filter methods. In widely used software, the exact Gaussian likelihood is commonly evaluated through a state-space representation and Kalman filtering, often initialized by conditional sum-of-squares (CSS). This workflow is effective but not benign: likelihood surfaces can be strongly multimodal, and single-start optimization can converge to suboptimal local maxima [2310.01198].

A systematic study of these optimization failures shows that standard single-start likelihood maximization improved in at least \(20.8\%\) of simulated cases when replaced by a root-based random-initialization algorithm, and in \(55.1\%\) of cases for \(p=q=3\) with \(n=50\). The same study reports that \(45.6\%\) of AIC tables built from single-start fits exhibit nested-model inconsistencies, and that profile confidence intervals provide superior confidence intervals to those based on the Fisher information matrix [2310.01198]. This suggests that practical ARMA inference depends as much on optimization strategy and interval construction as on the likelihood itself.

A separate computational line replaces repeated exact ARMA likelihood evaluations by a likelihood-based AR approximation. The proposed approximation requires \(O(n)\) work only once to precompute a small Champernowne matrix, after which repeated likelihood evaluations cost \(O(1)\) in the series length for fixed approximation order. In most cases, the resulting estimates are identical to or very close to the exact maximum likelihood estimate, making high-level implementations in environments such as Mathematica, Matlab, and R practical for long series [1611.00965].

## 3. Stochastic, graphical, and non-Gaussian generalizations

Expressing ARMA as a directed graphical model exposes a structural obstacle: in the classical model, \(Y_t\) is deterministic given its parents, and this degeneracy makes EM effectively unusable. A stochastic relaxation replaces the deterministic observation equation by
\[
Y_t \mid E_{t-q:t}, Y_{t-p:t-1} \sim \mathcal{N}(\mu_t,\sigma^2),
\]
with small user-fixed \(\sigma^2\), yielding the \(\sigma\)-ARMA and \(\sigma\)-ARMA* models. This modification restores EM, supports missing-data inference, and accommodates cross predictors for multiple time series and nontemporal covariates [1207.4162].

Within this framework, EM alternates between Gaussian inference over latent innovations and regression-style M-step updates for \((\zeta,\phi,\beta,\eta)\) and \(\gamma\). The graphical-model formulation also gives closed-form one-step predictive distributions of the form
\[
Y_{T+1}\mid \text{data}\sim\mathcal{N}(\mu^*,\sigma^{*2}),
\]
with \(\sigma^{*2}=\sigma^2+\beta^T\Sigma\beta+\gamma\). On complete economic time series, \(\sigma\)-ARMA and \(\sigma\)-ARMA* outperform classical ARMA in sequential predictive score; on US-Econo, for example, the average sequential predictive scores reported are \(-4.928\) for ARMA, \(-4.599\) for smoothed ARMA, \(-4.520\) for \(\sigma\)-ARMA, and \(-4.533\) for \(\sigma\)-ARMA* [1207.4162].

When Gaussian innovations are inadequate, ARMA has also been extended to symmetric \(\alpha\)-stable noise and to ARMA–GARCH with stable innovations. In that setting, autocovariances are undefined for \(\alpha<2\), so estimation is built on normalized autocovariation,
\[
NCV(X_t,X_{t-k})=\frac{\mathbb{E}\left(X_t\,\mathrm{sign}(X_{t-k})\right)}{\mathbb{E}|X_{t-k}|},
\]
a modified Yule–Walker step, and LAD regression in a modified Hannan–Rissanen procedure. For stable GARCH components, the paper proposes a modified empirical characteristic function method. The reported simulations show that LAD and the modified Hannan–Rissanen method substantially outperform least squares under heavy tails, and the methodology is applied to IBM log returns via an MA(1)–S\(\alpha\)S–GARCH(1,1) specification [1911.09985].

## 4. Multidimensional and spatial ARMA

For images and spatial lattices, ARMA generalizes to two-dimensional random fields. In the breast-imaging formulation, an image is modeled as a field \(\{x[n,m]\}\) on \(\mathbb{Z}^2\) with a total order
\[
(i,j)\le (s,t)\Longleftrightarrow i\le s \text{ and } j\le t,
\]
and a 2D ARMA\((p_1,p_2,q_1,q_2)\) model
\[
x[n,m]+\sum a_{ij}x[n-i,m-j]
=
w[n,m]+\sum b_{ij}w[n-i,m-j].
\]
The model is estimated by a two-stage Yule–Walker least-squares procedure: first, a finite AR approximation to the inverse filter is estimated by 2D Yule–Walker; second, the resulting innovation field is inserted into a least-squares system for joint AR and MA estimation [0906.3722].

In that application, local ARMA parameters are used as texture features on \(16\times 16\) windows of \(256\times256\) ultrasound images, and k-means with \(k=3\) segments healthy tissue, benign tumor, and cancerous tumor. The reported overall detection/classification accuracy is \(82\%\), with the method motivated by the claim that breast images can be accurately modeled by two-dimensional ARMA random fields and that the estimated coefficients provide a compact parametric descriptor of local tissue texture [0906.3722].

A different spatial generalization is the 2-D Rayleigh ARMA model for SAR amplitudes, where the observation at \((n,m)\) is conditionally Rayleigh with mean \(\mu[n,m]\), and a link-transformed mean \(g(\mu[n,m])\) follows a unilateral 2D ARMA recursion. The model is estimated by conditional maximum likelihood, with score recursion, Fisher information, and Wald tests derived explicitly. On CARABAS II forest data, the reported anomaly detector based on 2-D RARMA(1,1) detects \(24\) of \(25\) military vehicles with \(5\) false alarms, whereas a Gaussian 2-D ARMA(1,1) detects \(16\) vehicles with \(2\) false alarms; for image modeling on the same data, the reported MSE and MAPE are \(0.0562\) and \(0.4277\) for RARMA versus \(0.1241\) and \(0.7499\) for Gaussian ARMA [2208.03615].

## 5. Graph, spectral, and neural ARMA filters

On graphs, ARMA becomes a rational filter in the eigenvalues of a graph operator. One formulation defines a first-order graph ARMA recursion
\[
y_{t+1}=\psi My_t+\varphi x,
\]
where \(M\) is a shifted Laplacian, producing the graph-frequency response
\[
g(\mu)=\frac{r}{\mu-p}.
\]
Parallel and periodic ARMA\((K)\) constructions then realize higher-order rational graph filters. A central feature of this line of work is graph-independent design: coefficients are chosen for a spectral interval rather than for a particular graph, and the resulting filters are robust to changes in the signal and/or graph [1508.05808].

A centralized design perspective writes the graph filter as
\[
{\bf G}=\left(\sum_{p=0}^P a_p{\bf S}^p\right)^{-1}\left(\sum_{q=0}^Q b_q{\bf S}^q\right),
\]
with frequency response
\[
H(\lambda_n)=\frac{\sum_{q=0}^Q b_q\lambda_n^q}{\sum_{p=0}^P a_p\lambda_n^p}.
\]
Because the true least-squares design problem is nonlinear, the literature develops Prony-inspired modified-error methods and iterative true-error minimization. On both synthetic and real graph data, the reported results show that ARMA graph filters outperform FIR graph filters in approximation accuracy and are suitable for interpolation, compression, and prediction [1711.09086].

In graph neural networks, ARMA filters appear as recursive convolutional layers. A Graph Convolutional Skip layer updates
\[
\bar{\mathbf{X}}^{(t+1)}=\sigma\!\left(\tilde{\mathbf{L}}\bar{\mathbf{X}}^{(t)}\mathbf{W}+\mathbf{X}\mathbf{V}\right),
\]
and several such stacks approximate an ARMA\(_K\) spectral response. The reported experiments show improvements over polynomial graph filters on semi-supervised node classification, graph signal classification, graph classification, and graph regression, while preserving locality, trainability, and transferability across graphs [1901.01343]. The same rational-filter idea has also been transplanted to dense prediction in Euclidean CNNs: ARMA layers replace ordinary convolutions by combining a moving-average convolution with output-to-output autoregressive couplings, yielding an adjustable receptive field governed by learnable AR coefficients and a stable re-parameterization for training [2002.11609]. More recently, ARMA graph convolutions have been used as the encoder in a contrastive and modularity-regularized biomedical node-classification framework, ARMA-C3, for ADNI, NIFD, BreastMNIST, PneumoniaMNIST, and liver ultrasound data [2605.25657].

## 6. Online, distributed, and other operational settings

ARMA structure also governs distributed detection in temporally correlated environments. In the distributed detection setup where the signal is the impulse response of one ARMA filter and the noise is the output of another ARMA filter driven by white Gaussian noise, the paper extends the running consensus detector by whitening each local stream through fixed-order ARMA filters. The asymptotic theory yields two regimes: either local error probabilities decay exponentially fast to zero, or they converge to a strictly positive error floor. Necessary and sufficient conditions are given in terms of the poles and zeros of the composite ARMA models, and the threshold level influences the asymptotics in the positive-floor regime [2304.07225].

Another extension replaces real or complex coefficients by quaternion-valued coefficients. The quaternion ARMA\((p,q)\) model
\[
x_t=\sum_{i=1}^{p}\alpha_i x_{t-i}+\sum_{i=1}^{q}\beta_i\epsilon_{t-i}+\epsilon_t
\]
is learned online by reducing the hidden-noise problem to a full-information quaternion AR\((p+m)\) approximation. The resulting qARMA-QOGD and qARMA-QONS algorithms use quaternion GHR calculus and achieve logarithmic regret bounds; the reported synthetic experiments show that both approaches converge toward the optimal MSE, with the Newton-style method typically converging faster [1904.11830].

A common ambiguity is purely terminological: a separate system named “Arma” is a Byzantine Fault Tolerant consensus system that separates dissemination and validation of client transactions from consensus and orders only metadata of batches of transactions. Despite the name, that usage is unrelated to autoregressive moving-average modeling [2405.16575].

Source: https://www.emergentmind.com/topics/arma