Optimized Projection Functions (OPF)

• OPF techniques are variational methods that optimize projection maps to preserve specific response or structural features in high-dimensional spaces.
• They are applied in data visualization, electronic structure, stochastic filtering, and model reduction, outperforming classical methods.
• OPF implementations leverage constrained optimization on Stiefel or Grassmann manifolds with gradient-based methods and statistical null-model assessments.

Optimized Projection Functions (OPF) constitute a class of variational techniques for constructing projections or embedding maps tailored to preserve specific target characteristics of high-dimensional objects or systems. OPF methodology has emerged in diverse domains, including high-dimensional data visualization, electronic structure theory (automated Wannier function construction), filtering of stochastic differential equations, and nonlinear model reduction of dynamical systems. By optimizing projection functions to maximize the fidelity of chosen response or structural features—rather than generic geometric or energetic criteria—OPF methods offer automated, statistically sound, and scalable approaches that can outperform classical alternatives in structure discovery, filter accuracy, and model reduction.

1. Core Principles of OPF Methodology

OPF encompasses any approach that casts projection construction as an explicit optimization problem over a class of projection functions (often linear or semi-unitary) and their parameters, with objective functions directly tied to a user-specified response, statistical or physical property, or approximation error. The typical OPF framework involves:

• A high-dimensional source space, such as data points, quantum states, probability densities, or state trajectories.
• Target properties or responses, such as regression functions, spread measures, conditional densities, or dynamical outputs.
• A parametrized map (projection operator) P—often required to satisfy constraints like orthonormality, semi-unitarity, or membership in a Grassmann manifold.
• Objective function J(P, θ) which quantifies loss with respect to the target property, possibly including penalties or constraints.
• Joint or alternating optimization, frequently via gradient-based algorithms, with hard or soft enforcement of geometric or structural constraints.

This general principle supports application-specific variants, differing in technical details of the projection space, the target functional, and optimization procedure.

2. OPF in Data Visualization: Function Preserving Projections

A concrete instantiation of OPF in data science is the Function Preserving Projection (FPP) technique for interpretable exploration of high-dimensional data (Liu et al., 2019). Given $N$ samples $\{x_i\} \subset \mathbb{R}^D$ and user responses $\{f_i\}$, FPP seeks a 2D projection $y_i = P^\top x_i$ (with $P \in \mathbb{R}^{D\times 2}$, $P^\top P = I_2$) and a regression or classification model $g:\mathbb{R}^2\to \mathbb{R}^L$ to minimize

$$\min_{P,\theta} \frac{1}{N}\sum_{i=1}^N \mathcal{S}\Big(f_i, g(P^\top x_i; \theta)\Big), \quad \text{s.t. } P^\top P = I_2$$

with $\mathcal{S}$ a suitable loss (e.g., squared error for regression, cross-entropy for classification). FPP captures nonlinear patterns in the response via $g$, enabling the 2D embedding to expose highly nonlinear or curved manifolds, which classical linear methods (PCA, PLS) cannot.

Notable elements of the approach include:

• Joint gradient-based optimization over $P$ and $g$-parameters $\theta$, with periodic orthonormalization of $P$ via SVD projection.
• Use of mini-batch stochastic updates for scalability to millions of data points.
• Extension to multiple responses by averaging losses and, if needed, assigning per-response weights.
• Overfitting control through permutation-based generation of null distributions and empirical $p$-value assessment of discovered structures.

Empirical demonstrations show FPP's effectiveness on synthetic data, large physical simulation outputs, classification tasks (e.g., MNIST, RNA-seq), and visualization of neural network feature spaces, with interpretable axes and provable statistical significance (Liu et al., 2019).

3. OPF in Electronic Structure: Automated Wannier Function Construction

In electronic structure calculations, OPF methods have been adopted for automating the construction of maximally localized Wannier functions (MLWFs), which are essential for analyzing electronic bonding, polarization, and for accurate interpolation (Mustafa et al., 2015, Tillack et al., 5 Feb 2025).

The OPF methodology replaces the traditional Marzari–Vanderbilt gauge optimization over $N_k$ Bloch $k$-points by introducing a single semi-unitary matrix $X$ (or $W$) mapping a redundant set of $M$ trial orbitals to $J$ target bands. The optimization problem seeks to minimize the Wannier spread functional

$\Omega[X]=\Omega\big(\{U^k_{AX}(X)\}\big)$

where, at each $k$-point, the closest $J\times J$ unitary $U^k_{AX}$ to the projected overlap $A^k X$ is computed via SVD.

Advances in OPF for Wannier functions include:

• Derivation of the exact gradient of $\Omega$ w.r.t $X$ using SVD differentiation, removing earlier approximations and eliminating empirical parameters (Tillack et al., 5 Feb 2025).
• Self-projection extension, in which partially localized WFs are recycled as additional trial orbitals, significantly tightening the spread towards the optimal Marzari–Vanderbilt minimum.
• Efficient, robust convergence achieved by moving on the Stiefel manifold with unit-length retraction, and demonstrated on both isolated and entangled bands.
• Reduction of manual guesswork: No initial projections are needed per band, only sufficiently spanning trial orbitals.

Numerical studies confirm that this approach yields MLWFs with spread differing by less than $2\%$ from the global minimum, further reduced below $1\%$ with simple extension strategies. For heavily entangled bands, OPF provides a near-optimal starting point, accelerating subsequent gauge refinement (Mustafa et al., 2015, Tillack et al., 5 Feb 2025).

4. OPF in Nonlinear Filtering and Stochastic Systems

In the context of nonlinear filtering for stochastic differential equations, OPF denotes Optimal Projection Filters derived via optimal projections of the filtering SPDE onto finite-dimensional manifolds (e.g., families of densities parameterized by moments) (Brigo, 2022).

There exist distinct projection strategies:

• Itô-vector projection: Minimizes the leading order (O($\Delta t$)) mean-square error between the true and projected processes, by projecting both drift and diffusion onto the manifold's tangent space.
• Itô-jet projection: Kills the O($\Delta t$) term and minimizes the O($\Delta t^2$) coefficient in the expansion, achieving truly second-order strong mean-square optimality.
• Stratonovich projection (classical approach): Projects vector fields in Stratonovich form; shown to minimize a symmetrized error and is suboptimal compared to Itô-based projections.

The resulting finite-dimensional SDE for parameters $\theta$ has drift and diffusion explicitly constructed to best approximate the original infinite-dimensional filtering evolution within the chosen metric (either $L^2$ or Hellinger). Applications demonstrate that OPF-based filters systematically outperform classical counterparts (including extended Kalman and assumed-density filters) for nonlinear, non-Gaussian settings in both short- and moderate-time regimes, achieving superior mean-square performance (Brigo, 2022).

5. OPF in Model Reduction of Nonlinear Dynamical Systems

OPF strategies have been applied to reduced-order modeling (ROM) of nonlinear dynamical systems far from equilibrium (Otto et al., 2021). The approach casts the ROM subspace selection as an optimization over pairs of $r$-dimensional subspaces $(V,W)$ (the trial and test spaces in Petrov–Galerkin projection). The objective

$$J(V, W) = \frac{1}{L} \sum_{i=0}^{L-1} \|y(t_i) - \hat y(t_i; V, W)\|^2 + \gamma \rho(V, W)$$

is minimized over the product manifold $\mathrm{Gr}(n, r)\times \mathrm{Gr}(n, r)$ with Riemannian conjugate gradient, where $\rho$ penalizes approach to ill-posed projections. This guarantees:

• Trajectory-level fidelity: Matching empirical outputs over ensembles of observed trajectories.
• Fully nonlinear and non-equilibrium applicability: Not restricted to systems linearizable around equilibria.
• Provably convergent optimization: Under mild conditions, gradient-based searches converge to critical points.
• Superior accuracy and stability: Demonstrated on axisymmetric jet flow (dimension $n\sim 10^5$), strongly reducing mean-square errors and maintaining stability across regimes where classical POD and balanced truncation algorithms fail (Otto et al., 2021).

6. Implementation and Scalability Aspects

Across domains, OPF methods share robust engineering principles:

• Optimization over (constrained) projection spaces: Stiefel or Grassmann manifolds, semi-unitary or orthonormal matrices.
• Gradient-based algorithms: Using analytic gradients, automatic differentiation, or adjoint-state methods, followed by projection or retraction steps.
• Statistical or physical null-model assessment: For overfitting control and statistical significance, permutation tests or null distributions are invoked (notably in FPP for data structure discovery).
• Modular regressor/function approximators: Neural networks, polynomial functions, or analytic density families are routinely embedded in the loss function for maximum flexibility.
• Scalability to high dimensionality and sample count: Techniques such as mini-batch stochastic updates, streaming data access, and SVD-based orthogonalization are universally applied.

Typical computational complexity and memory footprints are kept proportional to the ambient and projected space dimensions, and iterative schemes converge in practical time for millions of samples or tens of thousands of system states.

7. Impact, Advantages, and Limitations

OPF approaches yield several domain-independent advantages:

• Automation and objectivity: They systematically remove human-in-the-loop parameter guessing, e.g., for initial Wannier projections or dimension reduction axes.
• High statistical or physical fidelity: Direct optimization with respect to target properties; null-model checks reduce spurious discoveries.
• Scalable to modern data and physical system sizes: Suitable for datasets, quantum calculations, or flows with millions of degrees of freedom.
• Improved interpretability and feature discovery: Especially in settings where classical methods (e.g., PCA, linear projections) are blind to nonlinear or functionally significant structures.

Principal limitations arise from the requirement for sufficiently expressive projection spaces (e.g., trial orbitals in Wannier construction) and the reliance on efficient gradient or adjoint computation. In cases of extreme feature entanglement or near-singular projection spaces, a final refinement (e.g., classical gauge minimization) may be needed. Nonetheless, OPF methods represent a paradigm shift toward function- and property-aware projection construction in modern scientific and engineering workflows (Mustafa et al., 2015, Liu et al., 2019, Otto et al., 2021, Brigo, 2022, Tillack et al., 5 Feb 2025).