Optimal Mean-Square Filtering

• Optimal mean-square filtering is a method for estimating unknown process values by minimizing the mean-square error through spectral projection.
• It leverages explicit spectral characterization and Hilbert space projections to derive optimal filters for both stationary and nonstationary signals.
• The approach underpins applications in signal recovery, forecasting, and quantum measurement while ensuring robust performance against spectral uncertainties.

Optimal mean-square filtering refers to the problem of estimating unknown values or functionals of a random process with the objective of minimizing the mean square error (MSE) of the estimate. This class of problems is foundational in signal processing, control, time series analysis, and quantum measurement, and encompasses a range of classical and modern methods including Wiener/Kolmogorov filtering, Kalman filtering, robust/minimax frameworks, and generalizations to nonstationary, non-Gaussian, and quantum regimes. The methods and results characterize the structure of the optimal filter (or estimator), error performance, and the effect of spectral characterization and uncertainty.

1. Fundamental Principles of Optimal Mean-Square Filtering

At its core, optimal mean-square filtering is the construction of a linear or nonlinear estimator for a target random variable (or functional) in a stochastic process, such that the MSE between the estimate and target is minimized. The archetypal example is the estimation of $A\xi = \int_0^\infty a(t)\xi(-t)\,dt$ or its discrete variant, given noisy observations $y(t) = \xi(t) + \eta(t)$, where $\xi$ and $\eta$ are stochastic processes and $a(t)$ is a known function.

The classical solution under spectral certainty (i.e., when spectral densities $f(\lambda)$ and $g(\lambda)$ of $\xi$ and $\eta$ are known) is given by projection in a Hilbert space of square-integrable random variables. The optimal estimate is the orthogonal projection onto the closed subspace generated by the observations. This reduces to analytically derived formulas in the frequency (spectral) domain, where the so-called spectral characteristic $h(\lambda)$ of the estimator appears explicitly in terms of $f(\lambda)$, $g(\lambda)$, and $A(e^{i\lambda}) = \int a(t) e^{-i\lambda t} dt$ (Luz et al., 23 Jun 2024, Luz et al., 15 Oct 2025).

2. Spectral Characterization and Explicit Filter Structure

The spectral methods offer explicit representations for the filter and its error. For processes with stationary increments, or more general nonstationary/cyclostationary structures, the analysis often employs transformations (e.g., to increments or vectorized stationary sequences) to enable spectral representation.

The key spectral characteristic of the optimal (mean-square) estimate for functionals of stationary processes is

$$h(\lambda) = A(e^{i\lambda}) \frac{f(\lambda)}{f(\lambda) + g(\lambda)} - \frac{C(e^{i\lambda})}{f(\lambda) + g(\lambda)},$$

where $C(e^{i\lambda})$ is determined via a Fourier series with unknown coefficients that enforce causality or “past-data” dependence; these are found by solving a finite or infinite system of linear equations generated from the orthogonality (projection) conditions. For processes with stationary $n$th increments, similar structures hold with additional polynomial or increment-related multipliers in the frequency domain (Luz et al., 15 Oct 2025).

When the total spectral density allows canonical factorization, the involved operators in the Hilbert space take a particularly tractable form, and inversion for the coefficients becomes explicit via dual sequences (Luz et al., 23 Jun 2024).

3. Mean-Square Error Evaluation and Performance Guarantees

The mean-square error (MSE) of the optimal filtering process is computed as an inner product in an appropriate function space, combining operators built from the spectral densities and the structure of the functional being estimated. In general,

$$\Delta(f, g; \widehat{A\xi}) = \langle R a, P^{-1} R a \rangle + \langle Q a, a \rangle,$$

where $R$, $P$, and $Q$ are operators with matrix elements defined by the Fourier coefficients of functions of $f(\lambda)$ and $g(\lambda)$ (typically $[1/(f+g)]$, $[f/(f+g)]$, and $[fg/(f+g)]$) (Luz et al., 23 Jun 2024, Luz et al., 15 Oct 2025). This MSE quantifies the accuracy improvement due to filtering, and these analytic forms are central both for theoretical understanding and practical computation.

4. Robust Filtering: Minimax and Least-Favorable Spectra

Real-world filtering often encounters uncertainty in the spectral densities. The minimax-robust approach addresses this by considering a class of admissible spectral densities (e.g., bounded integrals or pointwise bounds) and constructing the estimator/filter that minimizes the worst-case MSE over this class.

Formally, given a class $\mathcal{D} = D_f \times D_g$,

• The least favorable spectral densities $(f^0, g^0)$ solve

$$\Delta(f^0, g^0) = \max_{(f, g) \in \mathcal{D}} \Delta(h(f,g); f,g),$$

i.e., among all candidate spectra, they maximize the error of their own optimal filters.

• The minimax-robust spectral characteristic $h^0(\lambda) = h(f^0, g^0)(\lambda)$ gives the estimator with the best possible worst-case performance within $\mathcal{D}$.

The least favorable densities are found by solving constrained optimization problems, often via the method of Lagrange multipliers. The pointwise forms of the solutions, usually as equalities between norms of certain frequency-domain corrections and scaled spectral densities, are explicit and derived from necessary saddle-point conditions (Luz et al., 23 Jun 2024, Luz et al., 15 Oct 2025). This methodology provides a rigorous means to hedge filter design against spectral uncertainties.

5. Extensions: General Increment Structures, Canonical Factorization, and Observational Limitations

The theoretical framework extends to processes whose increments are stationary of arbitrary order (including fractional/integrated/cyclostationary and multi-periodic models), vector-valued multidimensional sequences, or periodically correlated increments (Luz et al., 2 Feb 2024, Luz et al., 2023). By suitable transformation (e.g., via differencing operators or block vectorization), these processes are reduced to formats amenable to spectral projection methods.

In cases of incomplete or missing observations, or where only partial information is available, the optimal estimator structure is modified via additional correction terms in the spectral characteristic determined again by orthogonality and linear algebraic systems reflecting observation patterns (Moklyachuk et al., 2016, Masyutka et al., 2018).

When the spectral densities admit canonical factorization, the theoretical and computational framework simplifies significantly, allowing for explicit formulae for filter operators, causality enforcement, and mean-square error calculations (Luz et al., 23 Jun 2024, Luz et al., 2023).

6. Applications, Implications, and Scope

Optimal mean-square filtering as described enables a spectrum of applications:

• Signal recovery and denoising in stochastic and deterministic systems (classical time series, communication signals, quantum phase estimation).
• Forecasting in economics and climatology where long-memory, seasonal, or cyclostationary effects are present.
• Robust control and system identification where model or observation uncertainty is significant.
• Advanced quantum measurement and adaptive estimation where mean-square minimization is vital for precision (Roy et al., 2012).

The minimax-robust methods provide guarantees that filtering performance degrades gracefully even in unknown or adversarial scenarios. Canonical factorization and operator methods facilitate efficient implementations, especially for real-time and high-resolution problems.

7. Summary Table: Core Elements of Optimal Mean-Square Filtering

Aspect	Deterministic/Specified Spectra	Spectral Uncertainty/Minimax Approach
Spectral characteristic $h$	Function of $f(\lambda),\, g(\lambda)$ per (2)	Based on least favorable $(f^0, g^0)$
Error computation $\Delta$	Explicit formulas in Fourier/operator domains	Maximized over admissible density set
Solution method	Hilbert space projection, operator equations	Constrained optimization, Lagrange multipliers
Extensions	Arbitrary increment/statistical structures	Same, with robustification

The full development provides not only concrete formulation and solution procedures for the optimal filter and its performance but generalizes classical results to a range of process structures and offers a mathematically rigorous approach to robustness in the presence of uncertainty (Luz et al., 23 Jun 2024, Luz et al., 15 Oct 2025, Luz et al., 2023, Luz et al., 2 Feb 2024, Masyutka et al., 2018, Moklyachuk et al., 2016).