Nonlinear Embedding Converter (NEC)

• NEC is a framework that transforms complex, high-dimensional nonlinear systems into reduced, interpretable forms using rigorous mathematical constructions.
• It employs techniques like LPV conversion, manifold embedding, and neural compression to preserve system dynamics, local uniqueness, and global injectivity.
• The methodology provides scalable solutions for model reduction, control synthesis, and efficient data compression across various engineering and data science applications.

The Nonlinear Embedding Converter (NEC) denotes a class of methodologies leveraging nonlinear mapping and embedding principles to transform, compress, or parameterize high-dimensional and nonlinear structures. NEC arises in various mathematical and engineering contexts, including model reduction for nonlinear dynamical systems, global structure-preserving embeddings for complex data manifolds, and information-theoretic neural compression for efficient machine learning pipelines. This umbrella term encompasses a variety of rigorous techniques for constructing low-dimensional, interpretable, and computationally tractable representations, frequently underpinned by precise mathematical guarantees.

1. Foundational Principles and Notions

NEC methodologies target the nonlinear embedding of objects or systems such that essential structural or dynamical information is preserved under the transformation. The principal settings include:

• Nonlinear dynamical system embedding: Any smooth nonlinear state-space system of the form

$\dot{x} = f(x, u), \quad y = h(x, u)$

can be globally embedded into a linear parameter-varying (LPV) form via exact nonlinear-to-affine factorization relying on the Second Fundamental Theorem of Calculus. The embedding introduces a measurable scheduling signal $\rho(t)$ so that

$$\dot{x} = A(\rho)x + B(\rho)u, \quad y = C(\rho)x + D(\rho)u$$

with $A(\cdot), B(\cdot), C(\cdot), D(\cdot)$ explicit functions of $\rho$ (Olucha et al., 18 Feb 2025).

• Manifold and subspace embedding: Embedding a high-dimensional manifold $M\subset\mathbb{R}^n$ into a lower-dimensional space (possibly via selection of intrinsic coordinates) such that local (immersion) or global (embedding) properties are preserved. This can include extension of the Discrete Empirical Interpolation Method (DEIM) to nonlinear analogues, thereby ensuring that the reduced representation distinguishes all dynamical states (Otto et al., 2019).
• Compression of nonlinear or high-dimensional data embeddings: Transforming neural feature representations, particularly for multi-task settings, into compact forms compatible with downstream tasks while retaining core functional utility. Here, end-to-end neural compression is tailored to the structure of downstream utility rather than pointwise data fidelity (Gomes et al., 26 Mar 2024).

2. NEC for Nonlinear-to-LPV System Conversion

The NEC strategy for nonlinear-to-LPV embedding employs exact factorization of nonlinear mappings via integral transforms. For a smooth system $f(x, u)$, the NEC constructs matrices $A(x, u)$ and $B(x, u)$ as

$$A(x, u) = \int_0^1 \frac{\partial f}{\partial x}(\lambda x, \lambda u) \, d\lambda, \quad B(x, u) = \int_0^1 \frac{\partial f}{\partial u}(\lambda x, \lambda u) \, d\lambda.$$

Expressing $f(x, u)$ in the exact affine form

$f(x, u) = A(x, u)x + B(x, u)u,$

where each entry is explicitly computable via numerical or symbolic quadrature, enables the construction of a globally valid LPV embedding. The set of scheduling variables $p = \eta(x, u)$ consists of the nonlinear scalar functions required to express $A$ and $B$ as functions of $p$, effectively capturing all system nonlinearities.

This approach requires no Taylor approximations, local linearizations, or combinatorial expansions, and yields LPV systems that are behaviourally equivalent to the original nonlinear system across the entire operating domain. Implementations such as LPVcore provide automated symbolic or numeric computation of the required integrals and extraction of scheduling coordinates, interfacing seamlessly with controller synthesis and simulation environments (Olucha et al., 18 Feb 2025).

3. Nonlinear Empirical Embedding for Manifold and Model Reduction

In manifold learning and model reduction, NEC denotes an algorithmic approach to embedding nonlinear manifolds in lower-dimensional ambient or measurement spaces while preserving (i) local uniqueness (immersion) and (ii) global injectivity (embedding):

• Simultaneously Pivoted QR (SimPQR): The immersion step selects a common set of coordinates distinguishing all local tangent spaces across multiple patches covering the manifold. This is accomplished by a block-row QR decomposition with shared pivoting, ensuring each local tangent space is fully represented (Otto et al., 2019).
• Branch-resolving SimPQR (for global embedding): After immersion, additional coordinates are identified to separate "fiber" branches, ensuring the measurement map is globally injective. This guarantees the selected coordinate set provides a true embedding of the underlying manifold.

The algorithm iteratively maximizes a criterion balancing the number of patches covered and robustness to noise, controlled by a user-set parameter $\gamma$. The final coordinate set is interpretable, often corresponding to physically meaningful sensor locations (e.g., regions of high shear in cylinder wake flows or pulse centers in PDE solutions).

The typical NEC pipeline for nonlinear manifold embedding is as follows:

1. Estimate tangent spaces at multiple representative points.
2. Perform SimPQR to find immersion coordinates.
3. For each immersion, find branch-difference directions and perform a second SimPQR to guarantee global embedding.
4. The union of coordinate sets forms the NEC embedding.

This framework delivers low-cardinality, theory-backed variable selection for interpretable monitoring, control, or simulation on nonlinear manifolds (Otto et al., 2019).

4. Randomized Nonlinear Embedding for Norm Preservation

The problem of embedding nonlinear transformations of linear subspaces into lower dimensions with quantifiable distortion properties is central in modern signal processing and data science.

• Additive-error embeddings: For entrywise nonlinear functions $f$ with bounded curvature and linear tails, there exist random linear maps $\Pi$ of size $$m = O\left(\frac{k \log(n/\epsilon)}{\epsilon^2}\right)$$ preserving $\ell_2$ norms with additive error for all $y = f(x), x \in Z$ (a $k$-dimensional subspace) (Gajjar et al., 2020).
• Relative-error embeddings: Under further constraints (local linearity near zero), pure $(1\pm\epsilon)$-relative embeddings are achievable. The results generalize the Johnson-Lindenstrauss lemma for linear subspaces to broad classes of nonlinear activations.

These embedding results underpin sample-optimal compressed sensing and learning algorithms relying on generative models or neural activations, and they allow for principled random sketching in nonlinear regimes. In compressed sensing via generative priors, NEC-type sketches enable linear sample complexity in $k$ (intrinsic dimension) without explicit bounds on network Lipschitz constants (Gajjar et al., 2020).

5. Neural Embedding Compression for Multi-Task Learning

In the context of large-scale machine learning and Earth Observation (EO) tasks, Neural Embedding Compression (NEC) provides an end-to-end, rate-distortion-optimized framework for efficient data sharing and storage:

• Architecture: NEC integrates a pre-trained masked autoencoder (typically ViT-B based) with a learned compressor and a lightweight decompressor. Training involves only a small fraction ($\sim$10%) of original parameters and runs for $<1.25\%$ of pretraining duration (Gomes et al., 26 Mar 2024).
• Objective: For each data sample $x$, the loss is

$$\mathcal{L}(x; \theta, \phi, \psi) = \lambda\,D(x, \hat{x}) + R(\tilde y; \phi)$$

where $D$ is a masked-reconstruction MSE and $R$ is an entropy-coded estimate of compression rate. Quantization during inference is realized by rounding, with uniform noise added during training for differentiability.

• Empirical performance: On EO scene classification and segmentation tasks, NEC achieves 75%–99.7% compression relative to raw data with $<5\%$ drop in accuracy. Unlike ad hoc quantization, NEC jointly optimizes the embedding space and supports extreme compressions (e.g., 0.47 KB/embedding at 94% accuracy) (Gomes et al., 26 Mar 2024).
• Utility and generality: The single rate-distortion weight $\lambda$ presents a tunable compression–utility frontier. The pipeline is agnostic to specific backbones and readily adapts to domains with multi-task requirements and tight bandwidth/storage constraints (e.g., medical imaging, video analytics).

6. Nonlinear Embedding Constructions in Physical and Mathematical Models

NEC provides frameworks for unifying objects of differing dimensionalities through continuous nonlinear embeddings:

• Boxcar and $\kappa$-deformed functions: Arbitrary vectors and matrices are embedded using the \emph{boxcar} function $\mathcal{B}(x, y)$ and its $\kappa$-deformed counterpart $\mathcal{B}_\kappa(x, y)$, which smoothly interpolates between discrete (Kronecker-delta-like) and continuous representations. Two principal embedding modes (Mode I and Mode II) enable smooth morphing between vectors, matrices, scalars, and higher tensors as $\kappa$ varies (García-Morales, 2017).
• Applications: These embedding strategies are employed for modeling warped compactifications in supergravity, interpolating between cellular automata and coupled map lattices, and deriving nonlinear diffusion equations as continuous limits of embedded discrete systems. The parameter $\kappa$ serves as a morphogenetic scale, allowing continuous deformation between objects or dynamical rules of disparate structure.
• Key properties: NEC embeddings via these mechanisms exhibit smoothness in $\kappa$, algebraic consistency, and controllable invertibility, with endpoints $\kappa \to 0, \infty$ acting as fixed points corresponding to physical limits (full vs. collapsed dimensionality) (García-Morales, 2017).

7. Interpretability, Algorithmic Structure, and Applications

A defining feature of NEC approaches—both in physical model embedding and machine learning—is interpretability anchored in explicit variable selection or scheduling. In manifold reduction, NEC yields sensor or coordinate sets with direct physical meaning (i.e., spatial locations, parameterizations). In nonlinear-LPV conversion, the extracted scheduling map $\eta(x, u)$ isolates the essential nonlinearities governing system behaviour.

Typical computational steps in algorithmic NEC pipelines include:

• Extraction of ranked bases or feature-localizations (e.g., tangent space estimation, Jacobian calculation).
• Multi-stage QR or global sketching for physically or statistically optimal coordinate selection.
• Closed-form or numerically tractable integration or projection steps, depending on analytical tractability.

The NEC paradigm is broadly applicable for model reduction, real-time simulation, controller synthesis, compressed sensing, information compression, and theoretical analysis of complex systems, providing rigorous mathematically guaranteed frameworks for reduction and morphing in nonlinear and high-dimensional regimes (Gomes et al., 26 Mar 2024, Olucha et al., 18 Feb 2025, Otto et al., 2019, Gajjar et al., 2020, García-Morales, 2017).