Data-Driven Finite-Dimensional Predictors

• Data-driven finite-dimensional linear predictors are techniques that use state, input, and output trajectories to create reduced linear models from complex, often nonlinear systems.
• They employ methods like Koopman operator lifting, EDMD, ARX paradigms, and structured realization to optimize model accuracy and ensure robust prediction across varied dynamics.
• These predictors facilitate efficient linear control implementations, enabling strategies such as MPC and LQR to achieve reliable performance even in high-dimensional or nonlinear settings.

Data-driven construction of finite-dimensional linear predictors refers to a suite of methodologies that leverage observed data—typically in the form of state, input, and output trajectories of a dynamical or stochastic system—to synthesize linear models that optimally predict system evolution or outputs. These predictors are "finite-dimensional" in that they act on reduced (finite-dimensional) representations of potentially complex or nonlinear systems, enabling deployment of linear estimation and control tools even in highly nonlinear or infinite-dimensional contexts. The landscape includes operator-theoretic approaches such as lifting via Koopman operator theory, set membership and ARX paradigms, structured realization via interpolation of transfer function data, and low-rank regularized regression for systems with combinatorial interaction effects.

1. Koopman Operator and Lifted Linear Predictors

Data-driven lifted linear predictors, prominently based on Koopman operator theory, address nonlinear controlled discrete-time systems of the form $x_{k+1} = f(x_k, u_k)$. The key concept is to construct a finite set of lifting functions or "observables" $\psi_1,\ldots,\psi_N$ mapping the state space $\mathbb R^n$ into a higher-dimensional space $\mathbb R^N$. Defining the lifted mapping $\psi(x) = [\psi_1(x), \ldots, \psi_N(x)]^\top$, one seeks a linear predictor

$$z_{k+1} = A z_k + B u_k, \qquad \hat y_k = C z_k, \qquad z_0 = \psi(x_0),$$

where $(A, B, C)$ are identified via least squares in the lifted space. The extended dynamic mode decomposition (EDMD) algorithm, or variants thereof, are used to find $A, B$ to minimize

$\|Y_\text{lift} - A X_\text{lift} - B U\|_F^2,$

with $X_\text{lift}$ and $Y_\text{lift}$ the collections of lifted current and next states, and $U$ the input matrix. Output mapping $C$ is solved similarly, often trivially if the identity coordinates are included in $\psi$ (Korda et al., 2016, Do et al., 2024).

Finite-dimensional approximation leverages the linearity of the (infinite-dimensional) Koopman operator, whose action on observables translates, after lifting, to a high-dimensional linear system that approximates the nonlinear flow. The accuracy over finite prediction horizons improves with richness of the lifting dictionary and abundance/diversity of training data, with cross-validation over held-out validation sets used to tune the dictionary (N), regularization, and other hyperparameters (Do et al., 2024).

2. Construction of Predictors: Algorithms and Methodologies

EDMD-Based Lifting and Regression

The canonical workflow involves:

1. Dictionary Selection: Choosing lifting functions tailored to the nonlinearities of interest—polynomial, trigonometric, or radial basis functions, as well as problem-informed custom features.
2. Data Collection: Recording tuples $(x_i,u_i, x_i^+)$, where $x_i^+ = f(x_i, u_i)$, over the relevant regions of state-input space.
3. Lifted Regression: Forming $X_\text{lift}$ and $Y_\text{lift}$, then solving for $(A,B)$ (and optionally $C$) via least squares or regularized regression.
4. Validation: Evaluating prediction errors with physically meaningful metrics, notably projected state error $\varepsilon_\text{proj}$ and multi-step prediction error $\varepsilon_\text{pred}$, which are more indicative for control purposes than lifted-state error.
5. Iteration: Updating dictionary size, regularization, or data coverage as dictated by validation metrics (Do et al., 2024).

Set Membership and Multi-Step ARX Predictors

An alternative paradigm is to fit independent $p$-step-ahead ARX predictors using set membership identification. Here, each predictor is characterized by a feasible parameter set (FPS) defined by polyhedral constraints induced by measurement noise bounds and observed regressor/output pairs. Online or offline (for reduced computation) algorithms intersect these parameter sets for prediction, yielding a filtered output and minimal worst-case error bound (Lauricella et al., 2020).

Structured Realization via Interpolatory Model Reduction

For LTI and structured systems (including delay or second-order), data-driven structured realization leverages measured transfer function data at interpolation points. Generalized Loewner matrix pencils or Hermite-augmented systems enforce the desired dynamical structure, solving for the state-space realization through direct or least-squares methods. This framework accommodates prescribed structure via basis functions $h_k(s)$ in the transfer function and can exactly match interpolation data (Schulze et al., 2016).

Low-Rank Latent Variable Linear Predictors

In settings dominated by high-dimensional feature interactions, as in large-scale regression or factorization machine contexts, the LIT-LVM methodology introduces structured regularization for the interaction coefficient matrix $\Theta$ by parameterizing it as $UU^\top$ (latent variable model with $r \ll d$), reducing the number of free parameters from $O(d^2)$ to $O(rd)$. Optimization proceeds via block coordinate or proximal gradient descent, jointly minimizing prediction error and low-rank/sparsity penalties on $w$ and $U$ (Nemati et al., 18 Jun 2025).

3. Practical Implementation and Computational Aspects

Key considerations in implementation of data-driven finite-dimensional linear predictors include:

• Numerical Linear Algebra: Solving least squares via QR or SVD is recommended for stability (especially as $N$ increases). For massive $N$ or sample size $K$, kernel-based EDMD or randomized sketching circumvents explicit lifting (Korda et al., 2016, Do et al., 2024).
• Regularization: Tikhonov (ridge) or sparsity-inducing penalties address overfitting and ill-conditioning inherent in high-dimensional regression.
• Sample Complexity: The effectiveness of the lifted predictor depends crucially on the ratio $K \gg N$ and the diversity (persistence of excitation) of sampling trajectories, particularly in control-relevant regions (Korda et al., 2016).
• Scaling and Truncation: In functional linear processes, the dimension $d$ of the retained basis determines the bias-variance tradeoff, with convergence rates optimized by balancing truncation and estimation error (Klepsch et al., 2016).
• Offline/Online Tradeoffs: Methods such as global set membership filtering shift complexity offline by precomputing conservative bounds at the expense of minimally looser error bounds, whereas local approaches yield tighter bounds through online optimization (Lauricella et al., 2020).
• Interpretability: For LIT-LVM and similar factors models, the learned low-dimensional embeddings provide interpretable structure in interaction effects (Nemati et al., 18 Jun 2025).

4. Applications in Control and Estimation

A principal motivation for constructing such predictors is their direct utility in synthesis of linear controllers—most notably Model Predictive Control (MPC) and Linear Quadratic Regulation (LQR):

• Koopman-MPC: Substituting the true nonlinear dynamics in MPC optimization with the finite-dimensional linear predictor yields a convex quadratic program whose computational burden scales with the input and prediction horizon, not the lift dimension. Linear and even nonlinear state/input constraints can be embedded via augmentation of the lifting dictionary (Korda et al., 2016).
• Stochastic MPC and Chance Constraints: Identified multi-step predictors with quantified parameter uncertainty can be incorporated into robust or stochastic MPC formulations, using ellipsoidal or tail bounds for constraint tightening (Balim et al., 2024).
• Real-Time Feasibility and Closed-Loop Guarantees: Across a range of low- and high-dimensional systems (including high-dimensional PDEs), lifted linear predictors have delivered order-of-magnitude reductions in prediction error and control computation time compared with local linearization or Carleman approximations (Korda et al., 2016, Do et al., 2024).
• Functional Time Series: The finite-dimensional innovations approach enables prediction of infinite-dimensional processes (e.g., curves or spatial-temporal fields) with explicit rates and error guarantees (Klepsch et al., 2016).

5. Empirical Results, Theoretical Guarantees, and Performance

Empirical benchmarks consistently demonstrate that data-driven finite-dimensional predictors, especially those exploiting operator-theoretic lifting or structured low-rank regularization, surpass conventional predictors (local linearization, Carleman) both in long-horizon accuracy and control performance.

• Koopman vs. Local/Carleman Linearization: On canonical benchmarks (Van der Pol oscillator, bilinear DC motor, KdV PDE), the lifted "Koopman-MPC" approach reduced long-range RMSE by orders of magnitude and enabled feasible, reliable MPC tracking under constraints where other linearizations failed (Korda et al., 2016).
• LIT-LVM in High-Dimensional Regression: LIT-LVM maintained stable, low prediction error as the interaction-to-sample ratio $d^2/n$ increased, where unstructured elastic net devolved into severe overfitting (Nemati et al., 18 Jun 2025).
• Set Membership Multistep Predictors: The set-membership ARX filtering approach provided the tightest certified worst-case prediction bounds among all possible estimators of the form proposed, outperforming classical Kalman filter-based schemes, especially under bounded measurement noise (Lauricella et al., 2020).
• Convergence and Generalization Guarantees: Statistical theory attests to consistency and generalization rates, e.g., EDMD predictors on Koopman-invariant subspaces approach vanishing error as the number of samples and dictionary size increase, subject to ergodicity and identifiability conditions (Korda et al., 2016, Do et al., 2024).

6. Model Selection, Hyperparameters, and Best Practices

Effective model construction depends on disciplined choices of hyperparameters and evaluation strategies:

• Dictionary Selection: Tailoring the set of lifting functions to the known structure of the dynamics or the control objective often outperforms generic large bases. Heuristic rules suggest starting with the identity and key nonlinear terms, expanding until validation errors plateau (Do et al., 2024).
• Hyperparameter Tuning: Grid or random search over lifting dimension $N$, regularization strengths, or latent dimension $r$ (in LIT-LVM) is standard, guided by validation on physically meaningful prediction errors rather than lifted-state loss (Nemati et al., 18 Jun 2025, Do et al., 2024).
• Outlier and Noise Handling: Robust regression or outlier discarding before pseudoinversion improves conditioning, while normalization of observables prior to regression is essential for numerically stable fitting (Do et al., 2024).
• Interpretability and Diagnostics: Inspection of identified matrices (eigenvalues, block structure) can reveal unphysical couplings or instability, motivating model refinement prior to downstream usage (Do et al., 2024).

7. Representative Algorithms and Model Classes

Approach	Model Class / Structure	Sample/Parameter Complexity
Koopman-EDMD	Nonlinear → lifted linear (MPC/LQR)	O(N × K), N = lift dim, K = samples
Set Membership ARX	LTI/ARX with bounded noise	O(P̄ × N), P̄ = # predictors, N = samples
Structured Realization	LTI/delays/second-order	O(n^2 K), n = realization order, K = basis size
LIT-LVM	High-dim. regression w/ interactions	O(n d^2 + n r d), d = features, r = latent dim
Functional Innovations	Hilbert-space time series	O(n^3) (eigendecomp), O(d^3 m) (recursion)

Optimally, the method is chosen to match system characteristics, data regime, and target application.

---

The data-driven construction of finite-dimensional linear predictors has matured into a technically rigorous, computationally practical discipline, bridging nonlinear system identification, operator-theoretic analysis, robust control, and high-dimensional statistics. It enables principled reduction of complex dynamical phenomena into tractable linear surrogates, fostering advances in predictive control, estimation, and interpretable machine learning over wide-ranging domains (Korda et al., 2016, Do et al., 2024, Nemati et al., 18 Jun 2025, Lauricella et al., 2020, Schulze et al., 2016, Klepsch et al., 2016).