---
title: Distributed ARX Estimation Techniques
url: https://www.emergentmind.com/topics/distributed-arx-estimation
type: topic
---

# Distributed ARX Estimation Techniques

Distributed ARX estimation addresses the collaborative identification of both model order and parameters for autoregressive systems with exogenous inputs (ARX) in multi-agent sensor networks. This problem is fundamental in scenarios where networked agents must learn the dynamics of an unknown stochastic system using only local and neighboring information, and where the model complexity (orders) is also unknown. Modern distributed ARX estimation schemes integrate local statistical model selection, recursive least squares (RLS), information diffusion, and cooperative excitation concepts to achieve strong convergence guarantees under minimal stochastic assumptions, without requiring global data centralization or independent input processes [2110.09826], [1301.5047].

## 1. ARX Model Structures and Distributed Observation Setting

In the prototypical distributed ARX context, each of $n$ sensors (agents) observes, at discrete time $t$,
$$
y_{t+1,i} = \sum_{j=1}^{p_0} b_j\,y_{t+1-j,i} + \sum_{l=1}^{q_0} c_l\,u_{t+1-l,i} + w_{t+1,i},
$$
where $p_0$ and $q_0$ are the (unknown) orders of the autoregressive and exogenous-input components, $b_j$ and $c_l$ are the unknown system parameters, and $w_{t+1,i}$ is zero-mean observation noise. The problem is to jointly estimate both $(p_0,q_0)$ and $\theta(p_0,q_0) = [b_1,\ldots,b_{p_0},c_1,\ldots,c_{q_0}]^T$ in a distributed fashion, leveraging the inter-agent communication graph for cooperation.

Model compactness for arbitrary candidate orders $(p, q)$ is achieved by defining the regression vector
$$
\phi_{t,i}(p,q) = [y_{t,i},\ldots,y_{t+1-p,i}, u_{t,i},\ldots,u_{t+1-q,i}]^T,
$$
such that
$$
y_{t+1,i} = \theta(p,q)^T \phi_{t,i}(p,q) + w_{t+1,i}.
$$
Each agent maintains local estimates for candidate orders and parameters, and exchanges information with its neighborhood $N_i$ as specified by the network topology [2110.09826].

## 2. Local Information Criteria for Distributed Order Selection

Selection of the correct ARX order pair $(p_0,q_0)$ is realized via a distributed Local Information Criterion (LIC) framework. At each time $t$, agent $i$ computes, for each candidate $(p,q)$,
$$
L_{t,i}(p,q) = \sigma_{t,i}\bigl(p,q,\theta_{t,i}(p,q)\bigr) + (p+q) a_t,
$$
with
$$
\sigma_{t,i}(p,q,\beta) = \sum_{j \in N_i} a_{ij} \left[ \sigma_{t-1,j}(p,q,\beta) + \left(y_{t,j} - \beta^T \phi_{t-1,j}(p,q)\right)^2 \right], \quad \sigma_{0,i}(\cdot) = 0,
$$
where $(a_{ij})$ are neighbor weights and $a_t$ is a non-decreasing penalty sequence, typically $a_t \sim \log t$. The first term accumulates squared prediction errors (locally and from neighbors), while the penalty controls model complexity, suppressing overfitting as $t$ increases. The current model order estimate at node $i$ is
$$
(p_{t,i},q_{t,i}) = \underset{0\leq p \leq p^*, 0\leq q \leq q^*}{\operatorname{argmin}} L_{t,i}(p,q),
$$
where $(p^*, q^*)$ are known upper bounds, or—if unknown—are replaced by an expanding search set [2110.09826].

## 3. Distributed Recursive Least Squares and Information Diffusion

Given an order selection $(p, q)$, each agent implements a consensus-type distributed Recursive Least Squares (RLS) algorithm. The "adaptation" step at sensor $i$ reads:
\[
\begin{aligned}
\bar\theta_{t+1,i} &= \theta_{t,i} + d_{t,i} P_{t,i} \phi_{t,i} \left[ y_{t+1,i} - \phi_{t,i}^T \theta_{t,i} \right], \\
\bar P_{t+1,i} &= P_{t,i} - d_{t,i} P_{t,i} \phi_{t,i} \phi_{t,i}^T P_{t,i}, \\
d_{t,i} &= \left[1 + \phi_{t,i}^T P_{t,i} \phi_{t,i}\right]^{-1},
\end{aligned}
\]
followed by a "diffusion" (consensus) step:
\[
\begin{aligned}
P_{t+1,i}^{-1} &= \sum_{j\in N_i} a_{ij} \bar P_{t+1,j}^{-1}, \\
\theta_{t+1,i} &= P_{t+1,i} \left( \sum_{j\in N_i} a_{ij} \bar P_{t+1,j}^{-1} \bar\theta_{t+1,j} \right).
\end{aligned}
\]
Alternatively, gradient-form stochastic approximation updates or consensus$+$innovations laws can be used, as in the general distributed exponential family estimation framework [1301.5047]. Here, the update at agent $n$ is:
\[
\mathbf{x}_n(t+1) = \mathbf{x}_n(t) - \beta_t \!\! \sum_{l\in\Omega_n(t)}\!\! [\mathbf{x}_n(t) - \mathbf{x}_l(t)] + \alpha_t K_n(t) \nabla_\theta \log p_n(y_n(t)|\mathbf{x}_n(t)),
\]
with innovation stepsize $\alpha_t$, consensus stepsize $\beta_t$, and adaptive gain $K_n(t)$.

## 4. Cooperative Excitation and Global Identifiability

The cooperative excitation condition is devised to ensure identifiability of system orders and parameters even under regressors that are correlated and/or nonstationary, i.e., weakening classical persistent excitation. Formally, there exists a scalar sequence $a_t \to \infty$ such that, for the maximally over-parameterized settings $(p^*,q_0)$ and $(p_0,q^*)$,
\[
\begin{aligned}
& \frac{\log r_t(p^*,q^*)}{a_t} \to 0, \\
& a_t\, \lambda_{\min}^{p,q}(t) \to \infty,
\end{aligned}
\]
for all $(p, q)\in \{(p^*, q_0), (p_0, q^*)\}$ almost surely, with $r_t(p,q)$ and $\lambda_{\min}^{p,q}(t)$ defined as
\[
r_t(p,q) = \lambda_{\max}\big[ P_0^{-1}(p,q) \big ] + \sum_{i=1}^n \sum_{k=0}^{t-1} \| \phi_{k,i}(p,q) \|^2,
\]
\[
\lambda_{\min}^{p,q}(t) = \lambda_{\min} \left( \sum_{j=1}^n P_{0,j}^{-1}(p,q) + \sum_{j=1}^n \sum_{k=0}^{t-D_G-1} \phi_{k,j}(p,q) \phi_{k,j}(p,q)^T \right).
\]
Collective network excitation, even in the presence of individually weak sensors, guarantees the statistical growth of the covariance matrices in all directions, ensuring convergence of both order and parameter estimates [2110.09826].

## 5. Statistical Guarantees and Convergence Theory

Under the martingale difference noise model and graph connectivity, the following consistency results are established:
- **Order Consistency:** $(p_{t,i},q_{t,i}) \rightarrow (p_0,q_0)$ almost surely for all $i$ (Theorem 3.1).
- **Parameter Consistency:** $\theta_{t,i}(p_{t,i},q_{t,i}) \rightarrow \theta(p_0,q_0)$ almost surely for all $i$ (Theorem 3.2).

Proof strategies combine martingale convergence arguments, stochastic Lyapunov techniques for RLS-type updates, and careful analysis of the local information criteria under correct and incorrect model orders. The double-array martingale limit theorem is crucial for establishing convergence when the model order itself is time-varying [2110.09826].

For fixed-order estimation, the consensus$+$innovations estimator achieves the asymptotic efficiency (inverse centralized Fisher information) under global observability and mean connectivity of the network. The estimate at each node attains
\[
\sqrt{t}\left( \hat\theta(t) - \theta^* \right) \to_d \mathcal{N}(0, I_c^{-1}),
\]
with centralized Fisher information $I_c = \frac{1}{\sigma^2} \sum_{i=1}^N \mathbb{E}\big[ \phi_i(t)\phi_i(t)^T \big]$ [1301.5047].

## 6. Order and Parameter Estimation Without Upper Bounds

When prior upper bounds $(p^*,q^*)$ are unavailable, the order search space is incrementally enlarged, e.g., to $\{0, \ldots, \lfloor \log t \rfloor\}$. A nested minimization is applied:
1. For $s = 0, \ldots, \lfloor \log t \rfloor$, run the diffusion-RLS at order $(s,s)$, compute
\[
\bar L_{t,i}(s,s) = \sigma_{t,i}(s,s,\theta_{t,i}(s,s)) + (2s)\bar a_t.
\]
2. Select $\hat m_{t,i} = \operatorname{argmin}_{0 \leq s \leq \lfloor \log t \rfloor} \bar L_{t,i}(s,s)$, then minimize over $p,q \leq \hat m_{t,i}$.
3. Rerun RLS at the chosen order $(\hat p_{t,i}, \hat q_{t,i})$.

A modified cooperative excitation condition and double-array martingale arguments yield that $\hat m_{t,i} \to m_0 = \max(p_0, q_0)$ and ultimately $(\hat p_{t,i}, \hat q_{t,i}) \to (p_0, q_0)$ almost surely [2110.09826].

## 7. Applications, Practical Considerations, and Extensions

Distributed ARX estimation is robust to stochastic feedback and correlated input scenarios, as it does not require independence or stationarity of the regression process. The cooperative excitation framework enables the network to succeed even when individual nodes fail to satisfy classical persistent excitation, highlighting the advantage of sensor cooperation.

Potential extensions of the distributed ARX estimation paradigm include:
- Distributed ARMAX (inclusion of moving-average terms),
- Time-varying parameter ARX models,
- Nonlinear or kernelized ARX estimators (adapting the LIC penalty and local recurrence structures accordingly).

For the distributed consensus$+$innovations method, parameter stepsizes $\alpha_t=1/(t+1)$ and consensus weights $\beta_t = b/(t+1)^{\tau_2}$ with $0 < \tau_2 < 1/2$ and $\beta_t/\alpha_t\to\infty$ are recommended for achieving optimal rates and covariance properties. Adaptive gain tuning via Fisher information consensus is practical when sensor models are heterogeneous [1301.5047].

---

**Summary Table of Key Elements in Distributed ARX Estimation**

| Component                 | Key Equation/Concept                               | Reference         |
|---------------------------|----------------------------------------------------|-------------------|
| ARX Model (node $i$)      | $y_{t+1,i} = \theta^T \phi_{t,i} + w_{t+1,i}$     | [2110.09826]      |
| Local Information Criterion| $L_{t,i}(p,q) = \sigma_{t,i} + (p+q)a_t$          | [2110.09826]      |
| Distributed RLS Update    | Adaptation + Diffusion (consensus)                 | [2110.09826]      |
| Consensus$+$Innovations   | $\mathbf{x}_n(t+1) = ...$                          | [1301.5047]       |
| Cooperative Excitation    | $a_t \lambda_{\min}^{p,q}(t) \to \infty$           | [2110.09826]      |
| Statistical Guarantees    | Strong consistency, efficiency                     | [2110.09826], [1301.5047] |

Distributed ARX estimation presents a unified framework for decentralized system identification in networked environments with unknown dynamics and is substantiated by rigorous convergence analysis, with broad applicability to adaptive signal processing, control, and sensor networks.

Source: https://www.emergentmind.com/topics/distributed-arx-estimation