Min-Max Grassmannian Optimization for Online Subspace Tracking

Abstract: This paper discusses robustness guarantees for online tracking of time-varying subspaces from noisy data. Building on recent work in optimization over a Grassmannian manifold, we introduce a new approach for robust subspace tracking by modeling data uncertainty in a Grassmannian ball. The robust subspace tracking problem is cast into a min-max optimization framework, for which we derive a closed-form solution for the worst-case subspace, enabling a geometric robustness adjustment that is both analytically tractable and computationally efficient, unlike iterative convex relaxations. The resulting algorithm, GeRoST (Geometrically Robust Subspace Tracking), is validated on two case studies: tracking a linear time-varying system and online foreground-background separation in video.

• The paper introduces GeRoST, a robust online subspace tracking algorithm that leverages min-max optimization on the Grassmannian to handle noise and structured outliers.
• It utilizes a closed-form solution for worst-case subspace estimation and efficient Riemannian gradient updates, guaranteeing exponential convergence under mild drift and noise conditions.
• Experiments in system identification and video separation validate GeRoST's superior tracking accuracy and robustness compared to existing methods like GREAT and GRASTA.

Min-Max Grassmannian Optimization for Online Subspace Tracking

Introduction and Problem Formulation

The paper introduces GeRoST, a robust algorithm for online subspace tracking leveraging min-max optimization on the Grassmannian. The motivation is to address robustness in the online estimation of a time-varying low-dimensional subspace from high-dimensional streaming data contaminated with both stochastic noise and structured outliers. Unlike prior approaches focused on statistical or ADMM-based convex relaxations, the method models data uncertainty directly as a Grassmannian ball, recasting subspace tracking as a min-max optimization problem over this manifold.

Given an incoming data matrix $W_t$ (with a sliding window of $T$ columns), the nominal data subspace $\widehat{\mathcal{W}}_t$ is computed via a top-$d$ SVD. To address adversarial corruption in the window, the true subspace $\mathcal{U}_t$ is assumed to lie within radius $\rho_t$ (in the chordal metric), yielding the minimax problem:

$$\min_{\mathcal{Y} \in \mathrm{Gr}(k, n)} \max_{\mathcal{W} \in \mathcal{B}_{\rho_t}(\widehat{\mathcal{W}}_t)} f(\mathcal{Y},\mathcal{W}),$$

where $f$ quantifies misalignment via projection error. The inner maximization quantifies worst-case geometric deviation given bounded perturbations to the nominal window subspace.

Theoretical Contributions

A closed-form solution to the inner maximization is established, generalizing recent results in static robust least-squares to the online, sliding-window setting. For any fixed estimate $\mathcal{Y}$, the adversarial subspace achieving the worst-case error within the uncertainty ball is:

$$\mathcal{W}_t^*(\mathcal{Y}) = \mathcal{V}_d(B_t(\mathcal{Y},\lambda^*)),$$

where $T$0, and $T$1 enforces the boundary constraint $T$2. This step requires only a bisection search over a spectral parameter and a low-rank eigendecomposition, yielding computational complexity matching GREAT while eliminating the inner convex optimization loop required in GRASTA.

A Riemannian gradient formula is derived for efficient manifold optimization on $T$3 with Lipschitz guarantees. The essay establishes tracking error bounds and exponential convergence under mild drift, noise, and over-parameterization conditions, exhibiting explicit dependence on the key parameters: subspace drift $T$4, noise-to-signal ratio $T$5, the design uncertainty $T$6, and the gap $T$7 between the modeling and true subspace rank.

Robustness is theoretically certified: if $T$8 exceeds a computable lower bound determined by $T$9 and $\widehat{\mathcal{W}}_t$0, then all true subspaces are feasible and the tracking error bound is preserved.

Algorithmic Realization

The final GeRoST algorithm consists of a sliding window update, SVD for nominal subspace extraction, and $\widehat{\mathcal{W}}_t$1 Riemannian gradient steps employing the closed-form worst-case subspace. The minimax update is realized as a single eigendecomposition with analytical sensitivity guarantees, and gradient descent is performed on the Grassmannian using the Manopt toolbox.

The adaptivity of GeRoST is predicated on online, data-driven selection of $\widehat{\mathcal{W}}_t$2 based on the current noise-to-signal estimate, with a mechanism to cap this value in the presence of gross outliers, ensuring the tracker resists corruption while avoiding excessive conservatism.

Experimental Validation

Online System Identification under Sensor Faults

The primary numerical experiment is subspace identification for a time-varying mass-spring-damper system. The metric is the relative prediction error of the estimated system models before and after a severe sensor fault (Gaussian spike $\widehat{\mathcal{W}}_t$3 the nominal noise). As depicted in Figure 1, GeRoST matches GREAT in the nominal regime but offers dramatically improved post-fault performance, with lower mean and variance in predictive error across 50 trajectories.

Figure 1: Relative prediction error of system models estimated by GeRoST and GREAT. The bold lines represent the average over 50 trajectories.

Robust Tracking for Foreground-Background Video Separation

A more challenging scenario is foreground-background separation in video, wherein sparse moving occlusions enter a dynamic, low-rank background. The primary metric is the error in tracking the evolving background subspace, with GeRoST compared to GREAT and GRASTA.

Tracking error remains lower for GeRoST throughout foreground occlusion, as shown in Figure 2. This behavioral improvement is quantified by ROC/AUC analysis (Figure 3), where GeRoST (AUC = 0.989) outperforms GREAT (AUC = 0.932) and remains competitive with GRASTA (AUC = 0.995), despite the latter's additional computational overhead and lack of geometric robustness guarantees.

Figure 2: Tracking error of the estimated background subspace.

Figure 3: ROC curve for foreground-background separation. The curve plots the True Positive Rate (correctly identified foreground pixels) against the False Positive Rate (background pixels incorrectly identified as foreground) across a range of detection thresholds.

An analysis of the adaptive uncertainty radius $\widehat{\mathcal{W}}_t$4 and the associated Lagrangian $\widehat{\mathcal{W}}_t$5 (Figure 4) shows that bounding $\widehat{\mathcal{W}}_t$6 in the presence of structured outliers (here, dynamic foreground objects) is essential to prevent the data window from absorbing the foreground into the background subspace, a limitation of both unconstrained and lightly regularized algorithms.

Figure 4: Evolution of $\widehat{\mathcal{W}}_t$7 and $\widehat{\mathcal{W}}_t$8 over time. Capping the instantaneous noise-to-signal ratio limits the ball radius $\widehat{\mathcal{W}}_t$9 during occlusion $d$0, required to separate the sparse foreground from the background subspace.

A qualitative snapshot at frame 250 (Figure 5) demonstrates that GeRoST resists intensity absorption, preserving background structure without introducing the severe ghosting present in GREAT or the catastrophic model corruption exhibited by GRASTA.

Figure 5: Qualitative comparison of foreground-background separation at frame 250. While the causal nature of online tracking introduces ghosting artifacts in all methods, GeRoST resists severely absorbing the moving outlier into the background subspace, unlike GRASTA and GREAT.

Implications and Future Directions

The analytical formulation of the minimax update on the Grassmannian offers a unification and significant practical improvement over iterative robust PCA, recursive compressive sensing, and purely geometric trackers without robustness guarantees. The operational principle—geometric data uncertainty—yields interpretable parameters and reliable, data-driven tuning strategies.

In practical terms, GeRoST is highly relevant in video surveillance (e.g., separating dynamic events from structured backgrounds), real-time fault isolation in control systems, and online unsupervised representation learning in dynamic environments where outliers and model misspecification are prevalent.

On the theoretical side, this framework opens avenues for nonconvex-nonconcave min-max optimization on manifolds with explicit worst-case certificates, and adaptive regularization directly informed by system residuals.

Future developments could include extensions to nonlinear manifolds, applications to higher-order tensor subspace tracking, and integration with Bayesian or distributionally robust learning to further model nonstationary or non-Gaussian perturbations in real-world streaming applications.

Conclusion

This work delivers a principled, computationally efficient algorithm for online subspace tracking with explicit geometric robustness, validated by strong experimental results in system identification and video processing. GeRoST advances the state-of-the-art by providing not only empirical gains but also theoretical error bounds and convergence guarantees, with practical implications for robust, adaptive signal processing in high-dimensional, contaminated streaming data contexts.