Residual Variance Matching RLS Filter

Updated 12 December 2025

The RVM-RLS filter integrates Residual Variance Matching into the RLS framework, dynamically tuning the forgetting factor to align residual variance with assumed noise levels.
It employs online variance estimation and stochastic gradient descent to adapt to non-stationary, nonlinear system dynamics in real time.
Empirical studies, such as in UAV terrain following, show significant reductions in mean squared error and improved stability under impulsive noise.

A Residual Variance Matching Recursive Least Squares (RVM-RLS) filter is an adaptive estimation algorithm that integrates the Residual Variance Matching Estimation (@@@@2@@@@) criterion into the classical Recursive Least Squares (RLS) paradigm. Its central innovation is online adjustment of the RLS forgetting factor such that the estimated residual variance of the filter matches the assumed measurement noise variance. This forms the basis for robust, self-tuning operation of RLS-type filters in nonlinear or time-varying systems—particularly in real-time applications, such as UAV terrain following with fluctuating or unknown noise properties (Wu et al., 5 Dec 2025).

1. Foundational Principles of Residual Variance Matching

Residual Variance Matching (RVM), as encapsulated by RVME, seeks to synchronize the second moment of filter residuals with the measurement noise variance. This is implemented by constructing a cost function that penalizes deviations between the recursively estimated variance of the RLS residuals and a nominal or estimated noise variance $\sigma^2$ :

$J(\lambda_i) = c \, (\hat{\mathrm{Var}}[r_{i+1}] - \sigma^2)^2,$

where $r_{i+1}$ is the prediction residual at time $i+1$ , $\hat{\mathrm{Var}}[r_{i+1}]$ the exponentially smoothed variance estimate, $\lambda_i$ the forgetting factor, and $c$ a positive constant controlling adaptation rate. Minimization of $J(\lambda_i)$ with respect to $\lambda_i$ ensures that residuals remain commensurate with the process noise, enabling the filter to maintain a dynamic trade-off between agility (low $\lambda$ ) and stability (high $\lambda$ ) (Wu et al., 5 Dec 2025).

2. RVM-RLS Algorithmic Structure

The RVM-RLS filter augments standard RLS with a dedicated inner loop for variance matching and outer loop for parameter tracking, summarized as follows:

Prediction: Compute the one-step-ahead prediction using the current parameter estimate $\hat{\theta}_i$ and the regressor vector $\Phi_{i+1}$ :

$\hat{y}_{i+1} = \Phi_{i+1}^T \hat{\theta}_i$

Residual Calculation: $r_{i+1} = y_{i+1} - \hat{y}_{i+1}$ .
Online Variance Update: Apply exponential smoothing:

$\hat{\sigma}_{i+1}^2 = \lambda_i \hat{\sigma}_i^2 + (1-\lambda_i) r_{i+1}^2$

Outlier Rejection: If $|r_{i+1}| > 3\sqrt{\hat{\sigma}_i^2}$ , set $r_{i+1} := 0$ .
Forgetting Factor Update: Perform stochastic gradient descent on $J$ :

$\lambda_{i+1} = \lambda_i - 2 c \eta (\hat{\sigma}_{i+1}^2 - \sigma^2)(\hat{\sigma}_i^2 - r_{i+1}^2)$

with projection $\lambda_{i+1} \in [\lambda_{\min}, \lambda_{\max}]$ .

RLS Gain and Update:
- Compute gain $K_{i+1}$ .
- Update parameters: $\hat{\theta}_{i+1} = \hat{\theta}_i + K_{i+1} r_{i+1}$ .
- Update covariance: $P_{i+1} = (1/\lambda_{i+1}) [P_i - K_{i+1} \Phi_{i+1}^T P_i]$ .

These steps repeat at each time step to perpetually adapt both model parameters and the filter's time-constant via $\lambda$ (Wu et al., 5 Dec 2025).

3. Theoretical Properties and Connections

Variance Matching Principle: RVM's core principle mirrors the minimum mean-squared error (MMSE) orthogonality condition for linear estimators under Gaussian noise, where the filter residual covariance should asymptotically equal the noise covariance matrix ( $E[rr^T] = R = \sigma^2$ ). The online enforcement of this condition using sample moments and a tunable learning rate $\eta$ facilitates MMSE-like behavior even in time-varying, nonlinear, or weak-prior regimes.
Convergence Properties: Under bounded input, persistent excitation, and sufficiently small adaptation parameters, $\lambda_i$ converges toward a value that enforces $\hat{\mathrm{Var}}[r] = \sigma^2$ (Wu et al., 5 Dec 2025).
Robustness: Outlier rejection and rapid adaptation of $\lambda$ enable the RVM-RLS filter to suppress transients due to disturbances or abrupt noise changes, dynamically modulating memory depth.

4. Implementation and Practical Usage

The incremental cost of the RVME step is $O(1)$ per iteration, negligible compared to the $O(m^2)$ per-sample cost of standard RLS with $m$ parameters. Empirical runtimes in UAV terrain-following (Monte Carlo, $m=4$ ) are approximately $0.024$ ms for RVM-RLS, remaining well within real-time constraints. Empirical performance in scenarios with 10% impulsive outliers and $\sigma^2 = 0.09$ demonstrates significant gains:

MSE reduction $\approx 88\%$ versus standard RLS under outlier conditions.
Post-filter variance ratio reduction from $1.467$ (RLS) to $0.173$ (RVM-RLS).
Maximum error reduction (from $1.154$ m to $0.524$ m) (Wu et al., 5 Dec 2025).

The following table highlights central procedural elements:

Step	Description	Formula/Operation
Prediction	One-step-ahead output	$\hat{y}_{i+1} = \Phi_{i+1}^T \hat{\theta}_i$
Residual Variance Update	Exp. smoothing of squared residuals	$\hat{\sigma}_{i+1}^2 = \lambda_i\hat{\sigma}_i^2 + (1-\lambda_i) r_{i+1}^2$
Forgetting Factor Update	Gradient descent on residual variance error	$\lambda_{i+1} = \lambda_i - 2c\eta(\hat{\sigma}_{i+1}^2 - \sigma^2)(\hat{\sigma}_i^2 - r_{i+1}^2)$

5. Relationship to Residual Variance Matching in Other Domains

The RVME paradigm has independent roots and equivalent forms in nonparametric regression, random forest variance estimation, and compressed sensing:

In random forests, RVME is implemented by matching the variance of out-of-bag residuals to estimate noise variance; under $L_2$ -consistent forests, this yields a consistent estimator (Ramosaj et al., 2018).
Asymptotic residual matching (ARM) applies analogous principles to $\ell_1$ -norm minimization in compressed sensing, equating empirical and predicted residuals to estimate unknown noise variance (Hayakawa, 2020).
In nonparametric regression, variance can be estimated optimally by regressing squared differences of paired observations on their covariate separations; the intercept yields an asymptotically efficient estimate of the noise variance (Tong et al., 2013).

A plausible implication is that the variance-matching motif is both theoretically justified and practically advantageous across diverse estimation settings where the true noise model is only partially known or rapidly changing.

6. Assumptions, Regularity, and Limitations

Key assumptions include:

Additive Gaussian measurement noise with known (or nominal) variance.
Local linearizability or explicit linear-in-parameter model form (with nonlinear systems handled via local approximation).
Weak prior, i.i.d disturbance structure, and persistent excitation needed for stability and convergence.

While RVME attenuates the deleterious impact of mis-specified or nonstationary noise, mismatches between true and nominal $\sigma^2$ can bias adaptation. Use of outlier rejection (3 $\sigma$ -rule) introduces some bias in $\hat{\sigma}$ for heavy-tailed noise, which may reduce estimator efficiency but enhances stability (Wu et al., 5 Dec 2025).

7. Applications and Broader Impact

The RVM-RLS filter is particularly suited to real-time, safety-critical adaptive filtering applications where environmental noise properties are not constant or known a priori. In UAV terrain following, it demonstrably increases waypoint estimation accuracy and robustness compared to static RLS. More generally, the residual variance matching framework unifies the design of adaptive filters, smoothing procedures, and variance estimators under a common objective: empirical alignment of residual statistics with theoretically justified reference values, thereby ensuring model adaptivity and estimator consistency across a broad array of regression and signal recovery problems.

References: