Nonlinear Compressive Sensing

• Nonlinear generalizations of compressive sensing are extensions that apply to nonlinear measurement models and structured sparsity, enabling robust signal recovery in complex settings.
• They leverage techniques like lifting, convex relaxations, and iterative linearization to handle challenges posed by nonlinearity and discontinuity in measurement processes.
• Theoretical advances extend classical RIP to uniform recovery guarantees and identifiability criteria, supporting applications across imaging, system identification, and deep generative models.

Nonlinear generalizations of compressive sensing (CS) encompass the extension of foundational CS principles from strictly linear measurement models to broader classes of nonlinear, structured, or implicitly nonlinear observation processes. Key research in this area includes model-based CS for structured sparsity, nonlinear recovery methods for general and quasi-linear operators, polynomial and rational system identification, recovery from nonlinear and quantized observations, and principled algorithmic and theoretical treatments for nonlinear inverse problems. The resulting framework allows for robust signal recovery in cases where the measurement process or the signal model exhibits nonlinearity, discontinuity, or complex structural dependencies, as codified across multiple lines of recent arXiv literature.

1. Extensions Beyond Linear Measurement Models

Classical CS focuses on recovery of sparse signals from linear measurements using random projections and sparse regularization, with provable guarantees based on the Restricted Isometry Property (RIP). Nonlinear generalizations address both nonlinearities in the measurement process—as in quadratic or arbitrary analytic mappings—and richer signal models incorporating structured sparsity, such as wavelet trees or block sparsity.

Several foundational directions include:

• Nonlinear Measurement Operators: Models of the form $y = f(Ax)$, $y = A\,f(x)$, or $y = \mathcal{F}(x)$ where $f$ is a possibly unknown or discontinuous nonlinearity, and $A$ is a sensing matrix or linear operator (Yi et al., 2015, Blumensath, 2012, Plan et al., 2015, Chen et al., 2023).
• Quasi-Linear and Composite Mappings: Operators of the form $A(x) = F(x)x$, with $F(x)$ Lipschitz, or $y = A(x) + e$. These include "locally linear" models encountered in phase retrieval and asteroseismology (Ehler et al., 2013).
• Polynomial and Rational Dynamics: Representation and recovery of unknown nonlinear systems by expanding vector fields or observation processes in polynomial or rational bases, leading to sparse recovery over extremely large dictionaries (Wang et al., 2011, Pan et al., 2012).
• Nonlinear Generative Models: Signal priors are imposed via deep generative networks, leading to signal recovery over the image of a nonlinear Lipschitz mapping under general nonlinear observations, including 1-bit and quantized models (Chen et al., 2023, Genzel et al., 2020).

2. Algorithmic Methodologies in Nonlinear Compresssive Sensing

Algorithmic approaches must address both the loss of linearity and the presence of structured or complex signal models. Several canonical methodologies have been developed:

• Lifting Techniques and Convex Relaxations: Quadratic and higher-order measurement models are lifted into higher-dimensional spaces where the constraints become linear in outer product or tensorized variables. Rank-1 constraints are relaxed to semidefinite constraints with added trace penalties and $\ell_1$ regularization for sparsity (Quadratic Basis Pursuit, Nonlinear Basis Pursuit) (Ohlsson et al., 2013, Ohlsson et al., 2013).
• Generalized Lasso/Least-Squares Programs: Recovery under nonlinear measurements, including single-index models and discontinuous $f$, can be robustly performed via generalized Lasso, treating nonlinear observations as noisy linear estimates with functional scaling. This allows for efficient computation and nearly minimax error bounds under generic structured priors, e.g., sparsity or generative models (Plan et al., 2015, Genzel et al., 2020, Chen et al., 2023).
• Iterative Linearization and Hard Thresholding: For general nonlinear observation models, iterative algorithms use local affine approximations (via Jacobians) and projected updates onto combinatorial constraint sets (e.g., unions of subspaces, block sparsity) (Blumensath, 2012).
• Exact Gradient Probabilistic Reformulations: In scenarios such as best subset selection or sparse network recovery, introducing a probabilistic reparameterization enables closed-form computation of expected losses, facilitating exact gradients and highly efficient gradient-based optimization, largely sidestepping issues of combinatorial explosions (Barth et al., 18 Sep 2025).
• Adaptive and Online Algorithms: Stochastic or online approaches such as RZA-NLMF and adaptive sparse sensing leverage high-order error statistics and reweighted zero-attraction to drive robust recovery in nonstationary, nonlinear, or strongly noise-afflicted scenarios (Gui et al., 2014).

3. Theoretical Recovery Guarantees, RIP Extensions, and Identifiability

The theory supporting nonlinear CS encompasses generalizations of RIP, non-uniform/uniform recovery guarantees, and deep connections to geometric and probabilistic complexity measures.

• Restricted Amplification and Isometry Properties: Classic RIP is extended to capture model-based and nonlinear settings. The Restricted Amplification Property (RAmP) weakens the constraint, allowing controlled amplification in the norm of structured residual subspaces, notably for structured compressible signals (0808.3572). Quasi-linear RIPs guarantee near-isometric behavior of nonlinear/composite measurement operators on sparse or structurally constrained sets (Ehler et al., 2013).
• Uniform Recovery Frameworks: For a wide class of nonlinear (even discontinuous) measurements, uniform signal recovery is guaranteed for all elements in a structured set (e.g., a generative model's range) from a single realization of the random measurement ensemble, provided sufficient sample complexity as set by localized Gaussian mean width or metric entropy (Chen et al., 2023, Genzel et al., 2020).
• Identifiability in Nonlinear Networks: For nonlinear neural networks, parameter recovery is possible (in the infinite data limit) only up to inherent symmetries (permutations, sign flips). Normal-form algorithms can select canonical representatives, but persistent non-identifiability is empirically observed as a "rebound" effect in the parameters, indicating a decoupling between low test error and parameter convergence (Barth et al., 18 Sep 2025).
• Geometric and Empirical Process Analysis: Modern analyses use empirical process theory, local Gaussian mean width, and incremental conditions to quantify recovery error as a function of model complexity and the observation nonlinearity (Genzel et al., 2020).

Setting	Key Guarantee/Property	Sample Complexity / Error Bound
Model-based CS	RAmP + structured RIP	$M = O(K)$ or $O(JK)$ (block), error $\to$ best model approx.
Quadratic/NLBP	Convex relaxation of lifting	Exact recovery if (ε, k)-RIP holds; mutual coherence bounds
Nonlinear Lasso	Tangent cone/intrinsic dim.	$$\|\hat{x}-\mu x\|_2 \lesssim \frac{\sqrt{d(K)}\,\sigma+\eta}{\sqrt{m}}$$
Uniform (GCS)	Metric entropy/mean width	$m \geq \tilde{O}(k/\epsilon^2)$ for all $x^* \in G(B_2^k(r))$

4. Representative Applications and Practical Impact

Nonlinear compressive sensing frameworks find application across structured signals, physics-inspired inverse problems, dynamic systems, and modern machine learning architectures:

• Structured Natural Signals: Model-based CS and block/tree-sparse recovery yield substantial gains in signal/image reconstruction, sharply reducing measurement requirements for structured natural images or sensor-network data (0808.3572).
• Nonlinear Dynamical Systems Identification: Sparse recovery from nonlinear system identification (e.g., biochemical reaction networks, chaotic systems) enables learning full polynomial/rational ODE models, circumventing the need for manual model selection, and directly inferring dynamic equations from time series (Wang et al., 2011, Pan et al., 2012).
• Generative Model Priors: Compressive recovery under deep generative priors, e.g., variational autoencoders or GANs, supports high-dimensional image or signal recovery under severe quantization, 1-bit measurements, or other nonlinear sensing scenarios—with exact uniform error bounds (Chen et al., 2023).
• Quantized/Simulated Sensing: Application in 1-bit, multi-bit, or modulo measurement systems, with robust recovery and phase transitions matching or improving upon classical CS theory with linear measurements (Plan et al., 2015, Genzel et al., 2020).
• High-Dimensional Inference and Learning: Probabilistic reformulations unlock scalable, efficient ℓ₀ regularization for neural network pruning, sparse coding, and dictionary learning, even in the presence of severe nonlinearity or under physical constraints (Barth et al., 18 Sep 2025, Rencker et al., 2018).

5. Numerical Experiments and Empirical Findings

Validation across the literature employs extensive simulation and real-data benchmarks:

• Model-based CS yields RMSE near the best model-approximation with significantly fewer measurements compared to standard basis pursuit (e.g., near-perfect recovery for $M \approx 3K$ in tree-based models) (0808.3572).
• Quadratic and NLBP methods achieve exact signal recovery where traditional methods fail, especially under polynomial/high-order measurement models; NLBP demonstrates 100% success in numerically challenging instances (Ohlsson et al., 2013, Ohlsson et al., 2013).
• Generative uniform CS frameworks corroborate theoretically predicted error rates on datasets such as MNIST and CelebA—reconstructing images with $m\ll n$ even under discontinuous or random quantization (Chen et al., 2023).
• Adaptive sparse sensing via RZA-NLMF consistently outperforms classical BPDN and OMP, achieving MSE better than the CRLB for NSS, especially in low SNR and highly sparse regimes (Gui et al., 2014).
• In teacher-student experiments for nonlinear networks, the empirical "rebound" shows that parameter convergence can reverse despite continual decrease in test loss—highlighting a separation of functional and parametric recovery in nonlinear regimes (Barth et al., 18 Sep 2025).

6. Open Problems and Future Directions

The frontier of nonlinear generalizations in compressive sensing is shaped by several outstanding challenges and avenues for research:

• Sharp Characterization of Recovery Conditions: Unifying the various RIP-like, metric entropy, and empirical process theoretic conditions across nonlinear, generative, and high-dimensional models remains an active area.
• Algorithmic Scalability and Non-convexity: While exact gradient probabilistic and lifting-based methods provide tractable relaxations, scaling to extremely large models—especially deep architectures—will require further advances in optimization and representation.
• Robustness to Model Mismatch and Uncertainty: Approaches handling mixed operators, adversarial noise, or unknown nonlinearity (e.g., the generalized least squares/Lasso) are critical for practical deployments in systems with measurement uncertainty (Herman et al., 2010, Plan et al., 2015).
• Identifiability and Interpretability: Nonlinear CS demonstrates fundamental differences in parameter recovery versus function approximation, motivating research into unique identifiability, canonical normal forms, and the design of statistically efficient learning algorithms for nonlinear inverse problems (Barth et al., 18 Sep 2025).
• Integration with Learning and Data-driven Sensing: Incorporating model-based recovery and structured nonlinear constraints into learning pipelines—both for enhanced performance in resource-limited sensing and for interpretability in scientific discovery—represents a promising direction.

Nonlinear generalizations thus expand the scope, complexity, and practical reach of compressive sensing, unifying diverse models under principled algorithmic and theoretical frameworks while revealing new phenomena and research challenges.