---
title: Degree-2 Nonlinear Approximation
url: https://www.emergentmind.com/topics/degree-2-nonlinear-approximation-technique
type: topic
---

# Degree-2 Nonlinear Approximation

A degree-2 nonlinear approximation technique refers to methods in which the approximation of a function, signal, or dynamic system utilizes interactions or compositions involving second-degree (quadratic) nonlinearities. This concept traverses several subfields—including function approximation, neural networks, rational and quadratic representations, spline approximation, and the model reduction of nonlinear dynamical systems. A variety of rigorously analyzed frameworks realize degree-2 nonlinear approximations, yielding convergence, expressivity, or computational benefits over classical linear or degree-1 (affine) approaches.

## 1. Foundational Definitions and Models

Degree-2 nonlinear approximation involves constructing approximants in which quadratic nonlinear structure is explicitly modeled. Several disparate representations exist:

- **Compositional (Layered) Approximants**: The dictionary $\mathcal{D}_2$ comprises functions expressed as $T(x) = T^{(2)}(T^{(1)}(x))$, where each $T^{(i)}$ is typically a one-hidden-layer neural network or basis function. Linear combinations of $N$ such compositional atoms are employed to approximate a target $f$ in a chosen norm, minimizing the error $\varepsilon_{2, f}(N)$ [1902.10170].
- **Parameteric Nonlinear Terms**: Approximants $F(x) = \sum_{j=1}^{N} a_j \phi(x;\theta_j)$ use basis functions $\phi$ that are linear in $a_j$ but nonlinear (often rational or exponential) in $\theta_j$ [2301.05881].
- **Implicit Quadratic Manifold Approximations**: An algebraic variety $a(x) f^2(x) - b(x) f(x) - c(x) = 0$ is constructed, with $a$, $b$, and $c$ chosen as smooth functions (typically polynomials). The root of this quadratic in $f$ defines the approximation; a sign or index function $\zeta(x)$ selects the relevant local branch [2512.06246].
- **Quadratic Spline Approximation**: Functions on $\mathbb{R}^2$ are approximated by globally continuous, piecewise polynomial splines of degree 2 (i.e., $k=3$), supported on nonnested rings [1506.07510].
- **Quadratic Polyflow for Nonlinear ODEs**: Dynamical systems $\dot{x} = f(x)$ are approximated by a $2n$-dimensional linear system constructed to match the first two Lie derivatives of $f(x)$ along the flow [1902.04507].

These methods all embed a form of quadratic nonlinearity, either in the structure of the basis, the combinatorics of composition, or the underlying algebraic variety.

## 2. Structured Frameworks for Degree-2 Nonlinear Approximation

### 2.1. Compositional (Depth-2) Dictionaries

The degree-2 compositional dictionary $\mathcal{D}_2$ is central to approximation theory inspired by neural networks. Each atom $T \in \mathcal{D}_2$ is constructed as $T(x) = T^{(2)}(T^{(1)}(x))$ where each $T^{(i)}$ is typically realized as a one-hidden-layer ReLU network (or more generally, a continuous piecewise-linear function). The best $N$-term approximation error for a target $f$ is defined as:

$$
\varepsilon_{2,f}(N) = \min_{\substack{a_n \in \mathbb{R}, T_n \in \mathcal{D}_2}}\left\| f - \sum_{n=1}^N a_n T_n \right\|.
$$

This compositional construction enables the "squaring" of the effective piecewise linear resolution, enhancing the rate of convergence of the best $N$-term nonlinear approximation [1902.10170].

### 2.2. Two-Parameter Nonlinear Approximants via NNLS

Another widely-studied setting is the sum-of-atoms model:

$$
F(x) = \sum_{j=1}^N a_j \phi(x; \theta_j), \quad a_j \ge 0, \, \theta_j \in [c,d].
$$

Here, $\phi(x; \theta)$ is a nonlinear kernel (e.g., $\frac{1}{1+\theta x}$, $e^{-\theta x}$), and parameters $\theta_j$ are chosen adaptively. The weights $a_j$ are optimized via non-negative least squares (NNLS), while $\theta_j$ are sought, for example, using grid search and refinement strategies [2301.05881].

### 2.3. Quadratic Formula-based Implicit Representation

A recently introduced framework constructs an implicit quadratic relationship:

$$
a(x) f^2(x) - b(x) f(x) - c(x) = 0,
$$

with $a(x)$, $b(x)$, and $c(x)$ chosen (frequently via least squares) to minimize the approximation residual. The solution $f(x)$ is then reconstructed via the quadratic formula, with an index function $\zeta(x)$ selecting the relevant root at each $x$ [2512.06246].

### 2.4. Spline Approximation of Degree 2

The space $S(n,3)$ of piecewise quadratic ($k=3$) splines, defined on nonnested rings, yields a nonlinear degree-2 spline approximation paradigm for functions in $L^p(\Omega)$, combining local polynomial fits subject to global continuity and smoothness constraints [1506.07510].

### 2.5. Quadratic Polyflow Reduction for ODEs

Quadratic polyflow approximations for $\dot{x} = f(x)$ involve augmenting the state to $z = [x, f(x)]$ and seeking a linear ODE $\dot{z} = A z$ whose trajectories reproduce the first and second Lie derivatives of $f$. The matrix $A$ is selected to match these derivatives over a region of interest in state space [1902.04507].

## 3. Convergence Rates and Expressivity

Degree-2 nonlinear approximation techniques consistently demonstrate provable or numerically validated improvements in convergence rates or representational power relative to degree-1 methods.

- **Compositional improvement theorem** [1902.10170]: If the one-layer (degree-1) best $N$-term rate is $\varepsilon_{1,f}(N) = \mathcal{O}(N^{-\eta})$, the two-layer compositional analog satisfies $\varepsilon_{2,f}(N) = \mathcal{O}(N^{-2\eta})$. For $f\in \mathrm{Lip}(\alpha)$ on $[0,1]^d$, this yields $\varepsilon_{2, f}(N) = \mathcal{O}\left(N^{-2\alpha/d}\right)$.
- **Quadratic formula-based error bounds** [2512.06246]: The degree-2 (quadratic) implicit representation achieves global exponential convergence for functions exhibiting a single jump or discontinuity, surpassing both degree-0 (polynomial) and degree-1 (rational) approaches, where best $N$-term errors for the latter are algebraic or root-exponential at best.
- **Nonlinear spline approximation**: The best error in $L^p$-norm for $n$ rings in $S(n,3)$ decays as $\mathcal{O}(n^{-s/2})$ for $f$ in the Besov space $B_{\tau}^{s,3}$, where $s \in (0,2)$ [1506.07510].
- **NNLS-based sum-of-atoms** [2301.05881]: For analytic targets, the residual error typically decays exponentially with $N$; practical applications (such as rational approximation of fractional powers or exponential kernel sums) exhibit rapid convergence.
- **Polyflow quadratic ODE approximation**: Locally matches the second-order Taylor expansion ($O(t^3)$ error for $t$ in the Taylor radius) while introducing global exponential stability absent in standard Taylor truncations [1902.04507].

## 4. Algorithmic and Practical Considerations

Degree-2 nonlinear approximation techniques typically involve greater computational complexity or storage than their degree-1 counterparts, balanced by superior accuracy per degree of freedom.

- **Compositional Nets**: Layer-2 networks of width $O(N)$ require $O(N^2)$ parameters. This increases the cost per function evaluation but is manageable for $N$ up to a few thousand, particularly under parallelization [1902.10170].
- **NNLS Sum-of-Atoms**: The dominant cost is solving repeated NNLS subproblems over large design matrices ($n$ sample points by $l$ candidate parameters). Grid refinement improves accuracy at the expense of additional computation [2301.05881].
- **Quadratic Formula-based Least Squares**: Basis selection may employ greedy or rank-revealing QR strategies; numerical stability requires attention to normalization (e.g., ensuring $a(x)$ does not approach zero) and the economic encoding or denoising of the index function $\zeta(x)$ [2512.06246].
- **Spline Construction**: The nonlinear selection of optimal rings and polynomial fits is combinatorially challenging, typically requiring greedy search or adaptive partitioning; global continuity constraints must be enforced [1506.07510].
- **Quadratic Polyflow ODEs**: Construction involves large-scale least squares to fit the linear closure of Lie derivatives, followed by simulation of a $2n$-dimensional linear ODE [1902.04507].

## 5. Notable Applications and Case Studies

Degree-2 nonlinear approximation frameworks have been demonstrated across canonical approximations and dynamical systems:

| Reference        | Application Area                      | Example/Result                                    |
|------------------|--------------------------------------|---------------------------------------------------|
| [1902.10170]     | Universal function approximation     | Squared exponent in best $N$-term error           |
| [2301.05881]     | Rational, exponential kernel sums    | NNLS-based recovery for $x^{-\alpha}$, $\exp(-x^{\alpha})$  |
| [2512.06246]     | Data denoising, discontinuities      | Global exponential convergence, quadratic manifold |
| [1506.07510]     | 2D function approximation (splines)  | Besov space adaptivity for images/surface data     |
| [1902.04507]     | Nonlinear ODE reduction              | Logistic growth, Van der Pol oscillator            |

In piecewise smooth, discontinuous, or memory-kernel contexts, degree-2 nonlinear techniques yield error reductions not attainable by standard linear, polynomial, or degree-1 rational schemes.

## 6. Open Problems and Research Directions

Significant open questions and research frontiers include:

- **Optimal Basis and Ring Selection**: Developing provably optimal (or adaptive) selection of basis functions, ring partitions, or compositional atoms that balance expressivity and conditioning [2512.06246,1506.07510].
- **Index Function Compression and Learning**: Efficient representations for index functions (e.g., $\zeta(x)$ in quadratic implicit schemes) that govern branch selection, possibly via advanced clustering, coding trees, or learning [2512.06246].
- **Stability and Conditioning**: Strategies to ensure numerical stability for polynomial root finding or when fitting high-degree function records, especially in the presence of noise [2512.06246].
- **Extension to Higher Degrees and Multi-variate Settings**: Generalizing the methodology to degree-$d$ and vector-valued or high-dimensional $x$; handling the manifold complexity and multivaluedness of the root structure [2512.06246].
- **Rigorous Convergence Theory**: Establishing comprehensive direct (Jackson) and inverse bounds in general domains or for functions with multiple jumps or singularities [1506.07510].

A plausible implication is that ongoing progress in these areas will further expand the algorithmic toolbox and theoretical underpinnings for nonlinear approximation, particularly for applications in data science, signal processing, and model reduction where capturing nonlinearity and abrupt transitions is critical.

Source: https://www.emergentmind.com/topics/degree-2-nonlinear-approximation-technique