Degree-2 Nonlinear Approximation

• Degree-2 nonlinear approximation techniques are defined as methods that explicitly incorporate quadratic nonlinearities to enhance convergence and representational power over linear methods.
• They employ diverse frameworks such as compositional dictionaries, NNLS-based sum-of-atoms, implicit quadratic formulations, splines, and quadratic polyflow to tackle complex function approximation challenges.
• These methods demonstrate improved convergence rates and computational accuracy, making them valuable for dynamic system modeling, signal processing, and other advanced applications.

A degree-2 nonlinear approximation technique refers to methods in which the approximation of a function, signal, or dynamic system utilizes interactions or compositions involving second-degree (quadratic) nonlinearities. This concept traverses several subfields—including function approximation, neural networks, rational and quadratic representations, spline approximation, and the model reduction of nonlinear dynamical systems. A variety of rigorously analyzed frameworks realize degree-2 nonlinear approximations, yielding convergence, expressivity, or computational benefits over classical linear or degree-1 (affine) approaches.

1. Foundational Definitions and Models

Degree-2 nonlinear approximation involves constructing approximants in which quadratic nonlinear structure is explicitly modeled. Several disparate representations exist:

• Compositional (Layered) Approximants: The dictionary $\mathcal{D}_2$ comprises functions expressed as $T(x) = T^{(2)}(T^{(1)}(x))$, where each $T^{(i)}$ is typically a one-hidden-layer neural network or basis function. Linear combinations of $N$ such compositional atoms are employed to approximate a target $f$ in a chosen norm, minimizing the error $\varepsilon_{2, f}(N)$ (Shen et al., 2019).
• Parameteric Nonlinear Terms: Approximants $F(x) = \sum_{j=1}^{N} a_j \phi(x;\theta_j)$ use basis functions $\phi$ that are linear in $a_j$ but nonlinear (often rational or exponential) in $\theta_j$ (Vabishchevich, 2023).
• Implicit Quadratic Manifold Approximations: An algebraic variety $a(x) f^2(x) - b(x) f(x) - c(x) = 0$ is constructed, with $a$, $b$, and $c$ chosen as smooth functions (typically polynomials). The root of this quadratic in $f$ defines the approximation; a sign or index function $\zeta(x)$ selects the relevant local branch (He et al., 6 Dec 2025).
• Quadratic Spline Approximation: Functions on $\mathbb{R}^2$ are approximated by globally continuous, piecewise polynomial splines of degree 2 (i.e., $k=3$), supported on nonnested rings (Lind et al., 2015).
• Quadratic Polyflow for Nonlinear ODEs: Dynamical systems $\dot{x} = f(x)$ are approximated by a $2n$-dimensional linear system constructed to match the first two Lie derivatives of $f(x)$ along the flow (Jungers et al., 2019).

These methods all embed a form of quadratic nonlinearity, either in the structure of the basis, the combinatorics of composition, or the underlying algebraic variety.

2. Structured Frameworks for Degree-2 Nonlinear Approximation

2.1. Compositional (Depth-2) Dictionaries

The degree-2 compositional dictionary $\mathcal{D}_2$ is central to approximation theory inspired by neural networks. Each atom $T \in \mathcal{D}_2$ is constructed as $T(x) = T^{(2)}(T^{(1)}(x))$ where each $T^{(i)}$ is typically realized as a one-hidden-layer ReLU network (or more generally, a continuous piecewise-linear function). The best $N$-term approximation error for a target $f$ is defined as:

$$\varepsilon_{2,f}(N) = \min_{\substack{a_n \in \mathbb{R}, T_n \in \mathcal{D}_2}}\left\| f - \sum_{n=1}^N a_n T_n \right\|.$$

This compositional construction enables the "squaring" of the effective piecewise linear resolution, enhancing the rate of convergence of the best $N$-term nonlinear approximation (Shen et al., 2019).

2.2. Two-Parameter Nonlinear Approximants via NNLS

Another widely-studied setting is the sum-of-atoms model:

$$F(x) = \sum_{j=1}^N a_j \phi(x; \theta_j), \quad a_j \ge 0, \, \theta_j \in [c,d].$$

Here, $\phi(x; \theta)$ is a nonlinear kernel (e.g., $\frac{1}{1+\theta x}$, $e^{-\theta x}$), and parameters $\theta_j$ are chosen adaptively. The weights $a_j$ are optimized via non-negative least squares (NNLS), while $\theta_j$ are sought, for example, using grid search and refinement strategies (Vabishchevich, 2023).

2.3. Quadratic Formula-based Implicit Representation

A recently introduced framework constructs an implicit quadratic relationship:

$a(x) f^2(x) - b(x) f(x) - c(x) = 0,$

with $a(x)$, $b(x)$, and $c(x)$ chosen (frequently via least squares) to minimize the approximation residual. The solution $f(x)$ is then reconstructed via the quadratic formula, with an index function $\zeta(x)$ selecting the relevant root at each $x$ (He et al., 6 Dec 2025).

2.4. Spline Approximation of Degree 2

The space $S(n,3)$ of piecewise quadratic ($k=3$) splines, defined on nonnested rings, yields a nonlinear degree-2 spline approximation paradigm for functions in $L^p(\Omega)$, combining local polynomial fits subject to global continuity and smoothness constraints (Lind et al., 2015).

2.5. Quadratic Polyflow Reduction for ODEs

Quadratic polyflow approximations for $\dot{x} = f(x)$ involve augmenting the state to $z = [x, f(x)]$ and seeking a linear ODE $\dot{z} = A z$ whose trajectories reproduce the first and second Lie derivatives of $f$. The matrix $A$ is selected to match these derivatives over a region of interest in state space (Jungers et al., 2019).

3. Convergence Rates and Expressivity

Degree-2 nonlinear approximation techniques consistently demonstrate provable or numerically validated improvements in convergence rates or representational power relative to degree-1 methods.

• Compositional improvement theorem (Shen et al., 2019): If the one-layer (degree-1) best $N$-term rate is $\varepsilon_{1,f}(N) = \mathcal{O}(N^{-\eta})$, the two-layer compositional analog satisfies $\varepsilon_{2,f}(N) = \mathcal{O}(N^{-2\eta})$. For $f\in \mathrm{Lip}(\alpha)$ on $[0,1]^d$, this yields $$\varepsilon_{2, f}(N) = \mathcal{O}\left(N^{-2\alpha/d}\right)$$.
• Quadratic formula-based error bounds (He et al., 6 Dec 2025): The degree-2 (quadratic) implicit representation achieves global exponential convergence for functions exhibiting a single jump or discontinuity, surpassing both degree-0 (polynomial) and degree-1 (rational) approaches, where best $N$-term errors for the latter are algebraic or root-exponential at best.
• Nonlinear spline approximation: The best error in $L^p$-norm for $n$ rings in $S(n,3)$ decays as $\mathcal{O}(n^{-s/2})$ for $f$ in the Besov space $B_{\tau}^{s,3}$, where $s \in (0,2)$ (Lind et al., 2015).
• NNLS-based sum-of-atoms (Vabishchevich, 2023): For analytic targets, the residual error typically decays exponentially with $N$; practical applications (such as rational approximation of fractional powers or exponential kernel sums) exhibit rapid convergence.
• Polyflow quadratic ODE approximation: Locally matches the second-order Taylor expansion ($O(t^3)$ error for $t$ in the Taylor radius) while introducing global exponential stability absent in standard Taylor truncations (Jungers et al., 2019).

4. Algorithmic and Practical Considerations

Degree-2 nonlinear approximation techniques typically involve greater computational complexity or storage than their degree-1 counterparts, balanced by superior accuracy per degree of freedom.

• Compositional Nets: Layer-2 networks of width $O(N)$ require $O(N^2)$ parameters. This increases the cost per function evaluation but is manageable for $N$ up to a few thousand, particularly under parallelization (Shen et al., 2019).
• NNLS Sum-of-Atoms: The dominant cost is solving repeated NNLS subproblems over large design matrices ($n$ sample points by $l$ candidate parameters). Grid refinement improves accuracy at the expense of additional computation (Vabishchevich, 2023).
• Quadratic Formula-based Least Squares: Basis selection may employ greedy or rank-revealing QR strategies; numerical stability requires attention to normalization (e.g., ensuring $a(x)$ does not approach zero) and the economic encoding or denoising of the index function $\zeta(x)$ (He et al., 6 Dec 2025).
• Spline Construction: The nonlinear selection of optimal rings and polynomial fits is combinatorially challenging, typically requiring greedy search or adaptive partitioning; global continuity constraints must be enforced (Lind et al., 2015).
• Quadratic Polyflow ODEs: Construction involves large-scale least squares to fit the linear closure of Lie derivatives, followed by simulation of a $2n$-dimensional linear ODE (Jungers et al., 2019).

5. Notable Applications and Case Studies

Degree-2 nonlinear approximation frameworks have been demonstrated across canonical approximations and dynamical systems:

Reference	Application Area	Example/Result
(Shen et al., 2019)	Universal function approximation	Squared exponent in best $N$-term error
(Vabishchevich, 2023)	Rational, exponential kernel sums	NNLS-based recovery for $x^{-\alpha}$, $\exp(-x^{\alpha})$
(He et al., 6 Dec 2025)	Data denoising, discontinuities	Global exponential convergence, quadratic manifold
(Lind et al., 2015)	2D function approximation (splines)	Besov space adaptivity for images/surface data
(Jungers et al., 2019)	Nonlinear ODE reduction	Logistic growth, Van der Pol oscillator

In piecewise smooth, discontinuous, or memory-kernel contexts, degree-2 nonlinear techniques yield error reductions not attainable by standard linear, polynomial, or degree-1 rational schemes.

6. Open Problems and Research Directions

Significant open questions and research frontiers include:

• Optimal Basis and Ring Selection: Developing provably optimal (or adaptive) selection of basis functions, ring partitions, or compositional atoms that balance expressivity and conditioning (He et al., 6 Dec 2025Lind et al., 2015).
• Index Function Compression and Learning: Efficient representations for index functions (e.g., $\zeta(x)$ in quadratic implicit schemes) that govern branch selection, possibly via advanced clustering, coding trees, or learning (He et al., 6 Dec 2025).
• Stability and Conditioning: Strategies to ensure numerical stability for polynomial root finding or when fitting high-degree function records, especially in the presence of noise (He et al., 6 Dec 2025).
• Extension to Higher Degrees and Multi-variate Settings: Generalizing the methodology to degree-$d$ and vector-valued or high-dimensional $x$; handling the manifold complexity and multivaluedness of the root structure (He et al., 6 Dec 2025).
• Rigorous Convergence Theory: Establishing comprehensive direct (Jackson) and inverse bounds in general domains or for functions with multiple jumps or singularities (Lind et al., 2015).

A plausible implication is that ongoing progress in these areas will further expand the algorithmic toolbox and theoretical underpinnings for nonlinear approximation, particularly for applications in data science, signal processing, and model reduction where capturing nonlinearity and abrupt transitions is critical.