Fixed & Structured Covariance Overview

• Fixed and Structured Covariance (FS) is a set of modeling techniques that impose algebraic structures like Toeplitz, banded, or sparse forms on covariance matrices to simplify estimation and inversion.
• FS methods accelerate computations by leveraging techniques such as FFT-based linear solves and structured matrix approximations in applications like fast detector learning.
• The framework enables robust covariance estimation and consistent low-dimensional modeling through convex optimization and support-preserving strategies that encode prior structural knowledge.

Searching arXiv for the cited papers and topic scope. arXiv search: "Fixed and Structured Covariance" Fixed and Structured Covariance (FS) denotes a class of covariance modeling and estimation strategies in which covariance, scatter, or shape matrices are not treated as arbitrary positive semidefinite objects. Instead, either a single covariance is fixed and reused across tasks, or the admissible matrices are restricted to a fixed structural family such as Toeplitz, banded, sparse, low-rank-plus-diagonal, block-diagonal, circulant, or low-dimensional affine models. Across object detection, robust covariance estimation, high-dimensional regularization, spatial statistics, operator estimation, and geometric learning, the common purpose is to reduce statistical and computational complexity by encoding prior algebraic structure and then exploiting that structure in estimation, inversion, optimization, or inference [(Valmadre et al., 2014); (1311.0594); (Said et al., 2016)].

1. Meanings and scope of FS

The literature uses “fixed” and “structured” in several closely related senses. In some settings, “fixed” means that a covariance matrix or its negative-class statistics are estimated once and then reused for many downstream models. In others, “fixed” means that the structure class is known a priori, while the numerical parameters remain unknown. “Structured” may refer to Toeplitz, banded, sparse, low-rank-plus-diagonal, linear parameterization, block-diagonal, block-Toeplitz, or low-dimensional affine-subspace constraints.

Context	Fixed aspect	Structured aspect
Fast detector learning	One global negative mean/covariance reused	Block two-level Toeplitz or circulant
Robust elliptical scatter estimation	Structure class known a priori	Toeplitz, banded, low-rank-plus-diagonal, linear models
High-dimensional PD correction	Support of first-stage estimator preserved	Linear shrinkage with unchanged off-diagonal support
Joint multi-group estimation	Shared family across groups	Low-dimensional affine subspace
Geometric statistics	Structured manifold fixed in advance	Complex, Toeplitz, block-Toeplitz spaces

This breadth is explicit in the cited work. Structure is encoded as a convex subset $\mathcal{S}\subset\mathcal{P}(p)$ in convex covariance matching, as a fixed support pattern in FSPD, as a shared affine subspace in joint covariance estimation, and as a structured Riemannian symmetric space in geometric models of covariance matrices [(1311.0594); (Choi et al., 2016); (Soloveychik et al., 2015); (Said et al., 2016)].

2. Fixed global covariance and fast structured linear solves

A canonical FS construction appears in fast sliding-window detection. For linear detectors

$f(x)=w^T x + c,$

LDA gives

$$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$

The FS step is to estimate the negative mean $\mu_{\text{neg}}$ and a single global negative covariance $S$ once from a large negative image pool, then reuse them for every detector. Under a stationary negative distribution,

$S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$

so $S$ is block two-level Toeplitz in space, with full cross-channel block structure (Valmadre et al., 2014).

This Toeplitz structure makes matrix–vector products equivalent to sums of channel-wise $2$-D cross-correlations, which can be computed by FFTs. The exact Toeplitz system can then be solved by CG or PCG, with $O(k^2 d\log d)$ matrix–vector multiplies and $O(d)$ memory aside from FFT buffers. The same paper further replaces the Toeplitz covariance by the nearest block two-level circulant approximation in Frobenius norm, which diagonalizes in the $f(x)=w^T x + c,$0-D Fourier domain and reduces the solve to $f(x)=w^T x + c,$1 independent $f(x)=w^T x + c,$2 linear systems. It reports that Toeplitz LDA essentially matches the precision–recall of HNM-based SVMs, while circulant LDA remains close to HNM and reduces detector learning from tens of seconds to fractions of a second, without storing explicit $f(x)=w^T x + c,$3 factors (Valmadre et al., 2014).

The same algebra establishes a direct connection to correlation filters. In that setting, covariance induced by circular shifts is circulant, and the corresponding normal equations in the Fourier domain have the same per-frequency $f(x)=w^T x + c,$4 block form. This identifies FS detector learning as a setting in which a globally estimated stationary covariance yields correlation-filter-style training without revisiting the negative set.

3. Convex and robust estimation under fixed structural classes

In robust covariance estimation for elliptical and generalized elliptical models, the target is the shape or dispersion matrix, defined up to positive scale: $f(x)=w^T x + c,$5 The structural prior is encoded by a convex subset $f(x)=w^T x + c,$6, often Toeplitz, banded, low-rank-plus-diagonal, or a linear nonnegative combination of known rank-one atoms. Scale is fixed by constraints such as $f(x)=w^T x + c,$7 [(1311.0594); (Soloveychik et al., 2014)].

The starting point is the elliptical moment identity

$f(x)=w^T x + c,$8

or its complex analog. Enforcing the empirical version inside $f(x)=w^T x + c,$9 yields a nonconvex GMM objective because of $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$0 inside the denominator. COCA replaces the nonlinear equalities

$$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$1

by inequalities

$$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$2

which are equivalent, by Schur complement, to the LMIs

$$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$3

The resulting estimator is an SDP over $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$4 and $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$5 [(1311.0594); (Soloveychik et al., 2014)].

Two theoretical facts are central. In the unstructured case, the relaxation is tight and COCA coincides with Tyler’s estimator up to scale. In the structured case, the relaxation is asymptotically tight or consistent when the true shape matrix lies in the structural set. The framework explicitly accommodates Toeplitz, banded, low-rank-plus-diagonal, known sparsity patterns, block-diagonal structure, and SPICE-type linear parameterizations. In synthetic compound Gaussian experiments with Toeplitz and banded structure, COCA attains lower MSE than sample covariance, Tyler’s estimator, and projection-based alternatives [(1311.0594); (Soloveychik et al., 2014)].

A recurring implication is that FS structure is not merely a post hoc projection. In COCA, the moment identity and the structural prior are imposed in one convex program, so the estimator is not simply “Tyler plus projection,” but the structured covariance that best matches the elliptical fixed-point relation.

4. Support-preserving and shared-structure FS models

In high-dimensional covariance estimation, a common first-stage estimator $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$6 may already encode the desired structure—sparsity, banding, or block pattern—but may fail to be positive definite. FSPD corrects this by the linear shrinkage family

$$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$7

Because the off-diagonal entries become $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$8, the support is preserved for any $$w = S^{-1}(\mu_{\text{pos}}-\mu_{\text{neg}}), \qquad S w = b,\quad b:=\mu_{\text{pos}}-\mu_{\text{neg}}.$$9. With smallest eigenvalue $\mu_{\text{neg}}$0, target margin $\mu_{\text{neg}}$1, and $\mu_{\text{neg}}$2,

$\mu_{\text{neg}}$3

For suitable $\mu_{\text{neg}}$4, this raises the minimum eigenvalue to $\mu_{\text{neg}}$5, achieves the minimal spectral-norm perturbation $\mu_{\text{neg}}$6, and preserves the first-stage convergence rate up to constant factors in spectral and Frobenius norms. The same deterministic argument applies to precision matrices (Choi et al., 2016).

A different FS generalization appears when several covariance matrices share a common but unknown linear family. If $\mu_{\text{neg}}$7 are vectorized covariances for $\mu_{\text{neg}}$8 groups and

$\mu_{\text{neg}}$9

has rank $S$0, then the covariances lie in an $S$1-dimensional affine subspace of $S$2. The proposed estimator computes sample covariances, centers them, applies SVD in covariance space, and reconstructs a rank-$S$3 approximation by TSVD. The CRB scales with $S$4, rather than $S$5, and the TSVD upper bound matches that scaling up to condition-number and eigenvalue factors. This suggests an FS regime in which the structural family is fixed across groups but learned from heterogeneous data rather than specified a priori (Soloveychik et al., 2015).

These two directions show that FS can mean either preserving a discovered structure exactly, as in FSPD, or estimating a common low-dimensional structural family jointly, as in mutual linear structure models.

5. Geometric, asymptotic, and inferential formulations

A geometric FS viewpoint treats structured covariance spaces as metric spaces or manifolds in their own right. For Toeplitz covariance estimation, one may project an empirical covariance $S$6 onto the structured set $S$7 using the Bures/Hellinger metric, which coincides with the $S$8-Wasserstein distance between the associated zero-mean Gaussian laws. The resulting projection problem can be written as the SDP

$S$9

and linear prior constraints such as Toeplitz, banded, or state-covariance equations enter as linear equalities on $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$0 (Ning et al., 2011).

A broader geometric construction defines Gaussian distributions directly on structured covariance manifolds. Spaces of complex, Toeplitz, and block-Toeplitz covariance matrices are treated as Riemannian symmetric spaces with metrics induced by the Hessian of $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$1. On such a space $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$2, the Gaussian density is

$S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$3

The mean parameter $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$4 is exactly the Riemannian barycentre of the distribution, the MLE of $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$5 is the sample barycentre, and mixture models lead to EM algorithms whose M-step requires weighted barycentres and a scalar scale update. Because all computations occur inside the structured manifold, the estimated means remain complex, Toeplitz, or block-Toeplitz by construction (Said et al., 2016).

On the inferential side, asymptotic theory for structured covariance estimators can be expressed as projection of an unstructured radial-type limit onto the linear covariance model. For a linear structure $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$6, the limiting variance of $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$7 takes the form of a projected radial covariance, and the corresponding variance-component estimator $S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$8 inherits a matched projected covariance. For homogeneous order-zero maps such as shape

$S_{(u,v,p),(i,j,q)} = g_{pq}[i-u,j-v],$9

or normalized variance components, the $S$0-term drops out and efficiency comparisons reduce to a single scalar $S$1. The same paper derives influence functions and gross-error sensitivity indices for these structured functionals (Lopuhaä, 2024).

A complementary robust regression framework considers balanced linear models

$S$2

where $S$3 is any identifiable structured covariance family. S-estimators jointly minimize a robust scale of Mahalanobis residuals over $S$4, and the paper establishes existence, consistency, asymptotic normality, breakdown-point, and influence-function results under mild assumptions that go beyond elliptically contoured densities. Linear mixed models, multiple regression with correlated errors, multivariate regression, and multivariate location–scatter models are all included as special cases (Lopuhaä et al., 2022).

6. Contemporary extensions across spatial, operator, and sensing settings

Recent work extends FS ideas to infinite-dimensional and nonstationary settings. For Gaussian processes on $S$5, covariance operators with banded off-diagonal decay or $S$6-sparse kernels admit minimax-optimal operator-norm rates governed by dimension-free quantities such as the effective dimension

$S$7

the decay profile $S$8, and sparsity indices $S$9. Tapering attains the optimal banded rate, and thresholding attains the optimal $2$0-sparse rate (Al-Ghattas et al., 2024). In nonstationary spatial modeling, a different FS mechanism uses fixed, observable covariates to drive variance, scale, anisotropy, and smoothness in a generalized nonstationary Matérn covariance. There the covariance is fully determined by covariates and a finite parameter vector, rather than by latent deformation fields (Blasi et al., 2024).

A related post hoc construction appears in the Spatial Adapter. With a frozen first-stage predictor, residual fields are modeled as

$2$1

and the induced residual covariance is

$2$2

Smoothness, sparsity, and orthogonality constrain the learned basis $2$3, while $2$4 and $2$5 are obtained in closed form once $2$6 is fixed. The result is a fixed low-rank-plus-noise residual covariance that supports kriging-style prediction and plug-in uncertainty quantification (Wang et al., 12 May 2026).

In sensing and communications, FS structure is often combined with aggressive measurement constraints. The survey on covariance estimation under multiple structures and coarse quantization emphasizes Toeplitz, sparse, low-rank, and combined Toeplitz-plus-sparsity or Toeplitz-plus-low-rank models, together with one-bit and dithered one-bit estimators based on arcsine-law identities and masked operator-norm bounds (Maly et al., 2021). In radar, geometric projection of the sample covariance onto

$2$7

yields distribution-free shrinkage-type estimators under any unitary invariant norm, with the eigenvalue update reduced to a one-dimensional convex problem and almost closed-form solutions for Frobenius and spectral norms (Aubry et al., 2017). For noise-type radars, the covariance is fixed to a $2$8 parametric family $2$9; minimizing Frobenius distance and maximum likelihood produce the same closed-form estimators for $O(k^2 d\log d)$0, together with exact and approximate PDFs and ROC formulas for detection (Luong et al., 2022).

Taken together, these developments show that FS is not tied to a single estimator or a single algebraic constraint. It is a general strategy for imposing reusable covariance statistics or preassigned covariance geometry so that estimation remains statistically stable, computationally tractable, and structurally faithful to the underlying application domain.