Linear Representation Identification Techniques

• Linear representation identification is a framework that extracts interpretable linear models or approximations from complex system input–output data using state-space and transfer function formulations.
• It employs diverse techniques including subspace methods, optimization with regularizers (e.g., ℓ1 and group-Lasso), and kernel-based approaches to ensure numerical robustness and physical consistency.
• The methodology provides practical guidelines for model selection, stability constraints, and finite-sample error guarantees, enhancing applications in control, system analysis, and signal processing.

Linear representation identification refers to the class of theoretical frameworks, computational methodologies, and algorithmic procedures for identifying a model that admits a linear structure, or an interpretable linear approximation, from system input–output data. This concept occupies a critical position in system identification, model reduction, and learning theory, as many physical, engineered, or inferential systems are represented either exactly or approximately in linear form—even when originating from nonlinear or high-dimensional settings. The term encompasses techniques for both direct estimation of linear models (in time or frequency domain), local or global extraction of linear approximations, and principled approaches for model selection, structure discrimination, and the imposition of regularization or sparsity constraints.

1. Model Classes, Parametrization, and Problem Definitions

The central structure in linear representation identification is the discrete- or continuous-time linear state-space model: $$x_{k+1} = A x_k + B u_k, \qquad y_k = C x_k + D u_k$$ where $x_k \in \mathbb{R}^{n_x}$, $u_k \in \mathbb{R}^{n_u}$, $y_k \in \mathbb{R}^{n_y}$, and $\theta \in \mathbb{R}^{n_\theta}$ is formed by stacking the vectorizations of $A, B, C, D$, and possibly the initial state $x_0$ (Bemporad, 2024). In continuous-time, transfer functions with additive modal decompositions

$G^*(p) = \sum_{i=1}^K \frac{B_i^*(p)}{A_i^*(p)}$

allow identification of interpretable, physically consistent models (González et al., 2024).

Parametric and nonparametric frameworks exist, ranging from impulse response (Markov parameter) estimation to realization-theoretic approaches using Hankel matrices, as well as direct identification of transfer functions or frequency response functions (FRFs) (Bensoussan et al., 2020).

Crucially, linear representation identification also encompasses identification within broader classes (e.g., block-oriented systems, nonlinear systems, LTV/LPV structures) by searching for a best linear approximation (BLA) or a local linearization (Schoukens et al., 2018, Xin et al., 8 May 2025, Xin et al., 2023).

2. Algorithms and Regularization Schemes

Subspace and Realization-Based Methods

Subspace identification procedures (e.g., N4SID, Ho–Kalman algorithms) rely on estimates of block Hankel matrices constructed from Markov parameters, with model order selection performed via singular value decomposition (SVD). These algorithms deliver numerically robust minimal realizations given persistently exciting data and sufficient data length (Bensoussan et al., 2020).

Optimization-Based and Regularized Formulations

Convex and nonconvex optimization play central roles. Classical experiments minimize a simulation or prediction error loss

$L(\theta) = \frac{1}{N}\|Y - \hat{Y}(\theta)\|_F^2$

subject to possible regularizers:

• $\ell_1$-norm: $R_1(\theta) = \lambda_1 \|\theta\|_1$
• group-Lasso: $$R_2(\theta) = \lambda_2 \sum_{g=1}^G \|\theta_g\|_2$$ which impose sparsity or encourage selection of meaningful groups of parameters (Bemporad, 2024).

Bound constraints for model stability ($\rho(A) < 1$), nonnegativity, or parameter bounding ($\ell \leq \theta \leq u$) are enforced directly via methods such as L-BFGS-B with automatic differentiation, enabling robust handling of non-smooth regularization terms and efficient parameter splitting (e.g., $\theta = y - z$, $y,z \geq 0$) (Bemporad, 2024).

Kernel-based and Robust Statistical Approaches

Kernel-based GP regression, using structured (e.g., stable spline) priors for the impulse response, can be robustified by heavy-tailed noise models (e.g., Laplace, Student-$t$) using EM algorithms for joint MAP estimation of the kernel and noise hyperparameters, effectively addressing the detrimental effect of outliers (Bottegal et al., 2014).

Nonconvex and Latent-Variable Methods

Nonconvex reformulations—including Burer–Monteiro factorizations for Hankel nuclear-norm minimization or direct "atomic norm" parameterization of eigenvalues and system modes—permit computationally efficient, statistically optimal identification, with global optimality guaranteed via dual (polar) certificates (Tadipatri et al., 26 Apr 2025).

3. Model Complexity, Structure Discrimination, and Regularization

Group-Regularization for Complexity Selection

Group-Lasso penalties enable automatic model order determination and mode/input selection by driving whole blocks of parameters to zero. State-group and input-group structures are induced to promote sparsity across states or channels, delivering both order reduction and optimal channel identification (Bemporad, 2024).

Structure Discrimination in Block-Oriented and Nonlinear Systems

When identifying block-oriented (e.g., Hammerstein, Wiener–Hammerstein, LFR) or LPV models, local linear approximations ("$\varepsilon$-linearizations") around multiple setpoints—and tracking of pole/zero loci—enable discrimination between architectures. Movements of poles/zeros with setpoint variations provide necessary (not sufficient) constraints for ruling out incompatible structures prior to full nonlinear estimation (Schoukens et al., 2018).

In LPV embedding, factorization of static nonlinearities in NLFR architectures generates affine LPV state-space representations with parametrically generated scheduling variables, enabling application of LPV subspace or instrumental-variable algorithms (Schoukens et al., 2018, Cox et al., 2020, Schoukens et al., 2018).

4. Finite-Sample Guarantees, Local Linearization, and Identifiability

Linearization of Nonlinear Systems

When the system is nonlinear, linear representation identification focuses on local estimation via Taylor expansion: $$x_{k+1} = f(x_k, u_k) \approx A(x_k-x^*) + B(u_k-u^*) + \mathcal{O}(\|x_k-x^*, u_k-u^*\|_1^2)$$ Multiple short, strategically designed experiments, with randomized coordinate excitations, guarantee persistent excitation while bounding the Taylor remainder. Regularized least-squares estimators deliver explicit non-asymptotic error bounds that separate bias (nonlinearity) from variance (noise), with sample complexity scaling optimally in state, input, and data dimensions (Xin et al., 8 May 2025, Xin et al., 2023).

A summary error bound expresses the tradeoff: $$\|\hat{\Theta} - \Theta\| \leq \sqrt{\text{nonlinearity error}} + \sqrt{\text{noise error}} + \sqrt{\text{regularization error}}$$ where nonlinearity error is $\mathcal{O}(q)$ in the excitation radius, and noise error scales as $\mathcal{O}(1/\sqrt{N})$.

Compressive and Blind Identification

In operator-theoretic settings, compressive identification enables blind recovery of continuous linear operators from minimal data, achieving stable identifiability of delay–Doppler kernels of area $D \leq \frac 12$ even without prior knowledge of the support region, unifying classical system identification and modern sparse sampling theory (Heckel et al., 2011).

5. Extensions: Nonlinear, Time-Varying, Latent, and Approximately Linearizable Systems

Best Linear Approximation and Nonlinear Distortion Quantification

In mildly nonlinear systems, the BLA minimizes the mean-square output error with respect to the stochastic input and output, with spectrum-based methods (random-phase multisines, "detection lines") quantifying both coherent and stochastic nonlinear distortion. Nonparametric estimation yields accurate uncertainty measures, warning against typical variance underestimation when nonlinearity is present (Schoukens et al., 2018).

Latent Representations and Self-Supervised Approaches

Self-supervised contrastive learning can recover a linear representation of latent dynamics from nonlinear, high-dimensional observations via contrastive InfoNCE losses. The optimal encoder and linear predictor jointly identify the latent space and dynamics matrix up to an unknown affine (orthonormal) transformation, generalizing identification to settings such as switching and locally linear systems (Laiz et al., 2024).

Time-Varying and Information-State Representations

For linear time-varying (LTV) systems, an "information-state" approach exploits ARMA regression of outputs on past inputs and outputs, enabling direct realization of a time-varying state-space model without explicit state estimation provided uniform observability is met. This yields a lossless route to LTV control via dynamic programming (Mohamed et al., 2022).

Approximately Linearizable Nonlinear Models

Identification procedures can also enforce structure (e.g., output feedback linearizability) on neural network–parametrized nonlinear models, yielding systems that—after transformation—take an affine-linear form with bounded nonlinear remainder. This permits direct application of robust LTI control synthesis with theoretical performance guarantees (Thieffry et al., 2024).

6. Benchmarks, Performance, and Practical Guidelines

Linear representation identification frameworks have been validated on a range of synthetic, industrial, and experimental benchmarks:

• L-BFGS-B with $\ell_1$ and group-Lasso yielded $R^2 \sim 94\%$ (train) and $92\%$ (test) in the classical cascade-tanks system, outperforming standard subspace methods, which can fail or suffer numerical instability (Bemporad, 2024).
• Additive continuous-time models showed lower mean-squared error and greater interpretability than unfactored transfer function approaches in mechanical/structural applications (González et al., 2024).
• LPV embeddings achieve performance within $<0.3\%$ RMSE of fully nonlinear block-oriented models in typical circuit/robotics benchmarks and rival the best robust LPV controllers, while fully removing scheduling-variable noise bias (Cox et al., 2020, Schoukens et al., 2018).

Optimal parameter selection (e.g., model order, group structure, regularization weights) is guided by inspecting singular value spectra, cross-validation, or input design using random-phase excitations. Model selection must account for the potential underestimation of uncertainty bounds in the presence of nonlinear distortion, and practitioners are advised always to evaluate the amplitude of nonlinear artifacts relative to noise and linear signal levels (Schoukens et al., 2018).

---

In summary, linear representation identification encompasses broad, mathematically rigorous methodologies for efficiently extracting linear structures and linear approximations from diverse data sources, unifying control, signal processing, and machine learning perspectives. Modern advances in regularization, robust statistics, nonconvex optimization, and representation learning have further enhanced both interpretability and statistical efficiency, supporting applications from robust controller synthesis to experimental system analysis in regimes where linear or quasi-linear structure prevails.