---
title: Minimum-Norm Interpolation Overview
url: https://www.emergentmind.com/topics/minimum-norm-interpolation
type: topic
---

# Minimum-Norm Interpolation Overview

Minimum-norm interpolation is a foundational concept in both computational mathematics and statistical learning theory: given data constraints, it selects among all possible interpolants the one minimizing a prescribed norm, often $\ell_2$, $\ell_1$, a Sobolev or RKHS norm, or a more general Banach-space norm. It underlies a vast range of problems, including regression in high-dimensional statistics, kernel methods in machine learning, underdetermined system parameterization, surface reconstruction, and functional analysis. The minimum-norm criterion introduces a strong implicit bias in overparameterized regimes, deeply influencing generalization, sparsity, stability, and operator theoretic properties.

## 1. Formal Frameworks for Minimum-Norm Interpolation

The abstract minimum-norm interpolation problem is as follows: let $\mathcal B$ be a Banach space of functions on a domain $\Omega$, equipped with a norm $\|\cdot\|_\mathcal{B}$, and let $L_1,\dots,L_N$ be bounded linear functionals (typically point evaluations: $L_j(f) = f(x_j)$). Given target data $y_j$, the minimum-norm interpolant solves
\[
\min_{f\in\mathcal B}\;\; \|f\|_\mathcal{B} \quad\text{subject to}\quad L_j(f) = y_j\ \forall j.
\]
A general representer theorem holds: under mild assumptions, minima exist and any minimizer $f^\star$ belongs to the finite-dimensional span of the evaluation functionals, i.e., $f^\star$ is determined by $\{L_j\}$ and $\mathcal B$ geometry [2006.14101].

### Explicit Representer Constructions

- **Hilbert space/RKHS**: $f^\star = \sum_j c_j K(\cdot, x_j)$, with $K$ the reproducing kernel and $c_j$ found by solving $K(x_i, x_j) c_j = y_i$.
- **Banach spaces**: similar constructions hold in reproducing kernel Banach spaces (RKBS), but the nonlinear geometry leads to nonlinear systems for the optimal coefficients. Duality maps, semi-inner products, and (for $\ell^1$ or $\ell^p$ norms) nonlinear homotopy or KKT approaches are required [2412.11473, 2006.14101].

### Finite-Dimensional Reductions

- In discrete settings or for structured problems (e.g., $\ell_1(\mathbb N)$), the minimum-norm interpolation reduces to small-scale convex programs or LPs, even when the original space is infinite-dimensional [2006.14101, 1306.5729].


## 2. Model Classes and Norm Choices

The practical and theoretical properties of minimum-norm interpolation are highly sensitive to the ambient geometry:

- **$\ell_2$ (Euclidean and Hilbert):** The minimum $\ell_2$-norm interpolant has closed-form in linear regression (pseudoinverse solution) and admits sharp generalization and stability analyses. When the system is underdetermined, solutions are typically non-sparse and exhibit strong benign overfitting only when the feature space dimension substantially exceeds sample size [2603.28956, 2008.03365, 2111.05987].
- **$\ell_1$ (Basis pursuit):** Promotes sparsity in the interpolant. The minimum $\ell_1$-norm interpolator arises in high-dimensional sparse regression and compressed sensing, with sharp risk analyses and phases of “multiple descent” in out-of-sample error as the overparameterization ratio grows [2111.05987, 2110.09502, 1306.5729].
- **Sobolev/Banach and RKHS norms:** Kernel interpolation, surface fitting, and high-order smooth approximations rely on minimization in Sobolev or reproducing kernel Hilbert (or Banach) space norms; the implicit bias favors spatial or frequency smoothness, and the geometry controls error and stability [1710.01419, 2008.03365, 2412.11473, 2507.08632, 2504.20617].
- **$L_p$ and $L_\infty$ norms:** For interpolating geometric data with curvature minimization criteria, $L_p$ and $L_\infty$ norms for $p>1$ lead to unique, regular spline or network solutions; as $p\to\infty$ the solution becomes piecewise quadratic and is characterized by a dual nonlinear system [1902.07264, 2212.11981].

## 3. Theoretical Guarantees: Error Bounds, Consistency, and Implicit Bias

### High-Dimensional Linear Regimes

- For isotropic Gaussian design $X \in \mathbb{R}^{n\times d}$, with $n\ll d$ and observations $y = X w^* + \varepsilon$:
    - **$\ell_2$-norm interpolator**: Prediction error scales as $O(\sigma^2 n/d)$; vanishing error requires $d/n\to\infty$ [2111.05987].
    - **Minimum $\ell_1$-norm interpolator**: If $w^*$ is $s$-sparse, error is $O(\sigma^2/\log(d/n))$ under $s \ll n/\log(d/n)$; consistency holds as $d/n\to\infty$ even at logarithmic rates [2111.05987]. In the moderately sparse regime, the excess risk exhibits multiple descent and ascent regions as a function of overparameterization due to geometric phase transitions in the $\ell_1$ ball [2110.09502].

### RKHS and Kernel Spaces

- **Convergence**: In the overparameterized limit, interpolants minimizing a weighted norm converge (in $L^q$) to the unique RKHS interpolant, pinned down by the data and kernel [2008.03365].
- **Generalization**: Deterministic and probabilistic bounds show error decay rates in terms of sample mesh-norm and kernel smoothness (Sobolev, spherical harmonics, NTK), but consistency in stronger Sobolev/Banach norms requires the true function to be sufficiently regular [2008.03365, 1710.01419].
- **Inconsistency in high smoothness**: For bounded kernels on low-dimensional domains, there exist sharp lower bounds (depending on kernel eigenvalue decay $\beta$ and embedding exponent $\alpha^*$) above which minimum-norm interpolation is statistically inconsistent; this creates a Sobolev-norm inconsistency threshold $s_0 = 3(\alpha^* - 1/\beta)$ [2504.20617].

### Neural Networks and Implicit Bias

- **Shallow ReLU networks**: With explicit weight decay $\lambda\to 0$ and width $m\to\infty$ at an appropriate scaling, empirical risk minimizers converge to the minimum Barron-norm interpolant [2311.06138].
- **Implicit bias**: Gradient descent tends to select solutions close to the minimum-norm interpolant under many initializations and architectures; however, the degree to which the true parameter norm is minimized can depend subtly on the optimizer, initialization scale, and explicit regularization [2311.06138].
- **Deep ReLU nets**: For $L_2$-norm minimum-norm interpolants in deep homogeneous ReLU nets, generalization and algorithmic stability occur when the network contains a low-rank “bottleneck” layer, with the low-rank bias arising from implicit regularization by gradient flow or weight decay [2602.13910].

### Geometric and Operator-Theoretic Properties

- **Minimum-norm projectors**: For affine interpolation on the Euclidean ball or convex bodies, the minimal projector norm is realized by interpolation at regular simplexes, with sharp constants (often $\theta_n(B)\sim \sqrt{n}$ for the ball) and is determined by geometric relations and Legendre polynomials [2405.01254, 2305.00687].
- **Minimum-norm in Banach spaces**: Uniform convexity (e.g., 2-uniform convexity) is necessary for sharp control of the structural bias and resulting generalization in overparameterized models, allowing precise non-Euclidean generalizations of the classical benign overfitting phenomenon [2603.28956].

## 4. Computational Approaches and Algorithmic Realizations

### Sparse Polynomial and Quadratic Interpolation

- Minimum-$\ell_1$-norm formulations enable recovery of sparse interpolants in underdetermined regimes using standard LP solvers. Guarantees can be established via tools from compressed sensing (e.g., restricted isometry properties), allowing recovery of polynomials or quadratic models from $O(s \log^4 n)$ points instead of $O(n^2)$ in the dense case [1306.5729].

### Sobolev and $H^2$ Minimum-Norm Updates

- For derivative-free optimization and adaptive model fitting, updating quadratic models via minimum $H^2$-norm difference (instead of the classical Frobenius norm) yields superior theoretical projection properties and empirical robustness, with all updates reducible to linear system solves via KKT conditions [2302.12017, 1710.01419].

### Interpolation Curve Networks and Surface Reconstruction

- Edge convex networks with minimum $L_p$-norm of curvature have unique and regular solutions for $1<p<\infty$, reducing to nonlinear equations in the parameters of the basis curves. The $L_\infty$ case has a weaker uniqueness property and solutions become piecewise quadratic with explicit multiplicative characterization [2212.11981, 1902.07264].
- Modern advances in point cloud geometry exploit minimum-norm kernel interpolation with mixed-dimensional bases for improved normal and curvature estimation from surface samples, typically reducing to block linear KKT systems [2507.08632].

## 5. Practical and Statistical Implications

- **Benign overfitting**: In many high-dimensional regimes, minimum-norm interpolation does not lead to overfitting; for Gaussian design and sufficiently high dimension/sparsity, prediction error vanishes [2111.05987, 2006.05942].
- **Failure of uniform convergence**: Classical uniform convergence over norm balls cannot explain generalization for minimum-norm interpolators; uniform convergence over the set of interpolators (i.e., predictors with zero empirical error) suffices, illuminating why low-norm plus perfect fit is critical for learning [2006.05942].
- **Transfer learning and covariate shift**: For overparameterized interpolators, finite-sample theory gives precise instance-wise risk bounds under distribution shift, classifying regimes where the shift is beneficial or malignant according to the spectral change in tail directions and level of overparameterization [2404.00522].
- **Operator norm minimization**: For optimal interpolation projectors, geometric functionals (e.g., Legendre polynomials, simplex-volume maximization) yield sharp lower and upper bounds for the minimal $C$-operator norm, with sharp asymptotic order in high dimension [2405.01254, 2305.00687].

## 6. Open Problems and Directions

Significant open questions remain regarding:

- Tight asymptotic constants in the high-dimensional regime for minimum-norm interpolators, especially in the proportional ($d/n\to\alpha$) setting [2111.05987].
- Extension beyond isotropic Gaussian design: sub-Gaussian, heavy-tailed, and dependent-feature regimes [2111.05987, 2603.28956].
- Robustness of minimum-norm interpolation under model misspecification or approximate sparsity [2111.05987].
- Numerical algorithms for $L_\infty$-minimum-norm network interpolation and Banach-space representer systems, especially in large-scale settings [2212.11981, 2006.14101].
- Connections between implicit bias, algorithmic stability, and network geometry in deep learning—with further formalizations of how minimum-norm criteria emerge under gradient-based training [2602.13910, 2311.06138].
- The extent of the phenomenon in fixed-dimension and non-asymptotic regimes, especially concerning the sharpness of inconsistency results in kernel and Banach space settings [2504.20617, 2603.28956].

## 7. Summary Table: Key Regimes and Results

| Problem/Norm        | Exact Error Rate                    | Sparsity/Overparam. Regime           | Consistency Criteria              | Principal Reference       |
|:--------------------|:------------------------------------|:-------------------------------------|:----------------------------------|:-------------------------|
| $\ell_2$ norm, isotropic Gaussian | $\mathbb{E}\|\widehat{y}-y^*\|^2/n \sim \sigma^2 (n/d)$ | $d\gg n$, no sparsity assumption     | $d/n \to \infty$                  | [2111.05987]             |
| $\ell_1$ norm, isotropic Gaussian | $\Theta(\sigma^2/ \log(d/n))$         | $s \ll n/ \log(d/n)$                 | $d/n \to \infty$, $s=O(1)$        | [2111.05987]             |
| RKHS norm, kernel    | Error plateaus at $O(h^k)$ ($L^\infty$) | $n$ large, mesh norm $h\to 0$        | Target in RKHS, dense sampling     | [2008.03365, 1710.01419] |
| RKHS norm, bounded kernel | Inconsistency in $[\mathcal{H}]^s$ for $s>s_0=3(\alpha^*-1/\beta)$ | Fixed $d$, $n\to\infty$             | $s<s_0$                          | [2504.20617]             |
| Deep ReLU (min $L_2$) | Stability holds under low-rank      | Layer bottleneck, $n\to\infty$       | Stable subnetwork + low-rank      | [2602.13910]             |


## References

- [2111.05987] "Tight bounds for minimum $\ell_1$-norm interpolation of noisy data"
- [2110.09502] "Minimum $\ell_{1}$-norm interpolators: Precise asymptotics and multiple descent"
- [2504.20617] "Sobolev norm inconsistency of kernel interpolation"
- [2603.28956] "Minimum Norm Interpolation via The Local Theory of Banach Spaces: The Role of $2$-Uniform Convexity"
- [2006.14101] "Representer Theorems in Banach Spaces: Minimum Norm Interpolation, Regularized Learning and Semi-Discrete Inverse Problems"
- [2507.08632] "Minimum-norm interpolation for unknown surface reconstruction"
- [2302.12017] "Least $H^2$ Norm Updating Quadratic Interpolation Model Function for Derivative-free Trust-region Algorithms"
- [1710.01419] "Minimum Sobolev norm interpolation of derivative data"
- [2311.06138] "Minimum norm interpolation by perceptra: Explicit regularization and implicit bias"
- [2602.13910] "Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks"
- [1306.5729] "Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization"
- [2212.11981] "Edge convex smooth interpolation curve networks with minimum $L_{\infty}$-norm of the second derivative"
- [1902.07264] "Interpolation of scattered data in $\mathbb{R}^3$ using minimum $L_p$-norm networks, $1<p<\infty$"
- [2008.03365] "Generalization error of minimum weighted norm and kernel interpolation"
- [2405.01254] "Optimal Lagrange Interpolation Projectors and Legendre Polynomials"
- [2305.00687] "The Minimum Norm of a Projector under Linear Interpolation on a Euclidean Ball"
- [2006.05942] "On Uniform Convergence and Low-Norm Interpolation Learning"
- [2412.11473] "Optimal interpolation in Hardy and Bergman spaces: a reproducing kernel Banach space approach"
- [2404.00522] "Minimum-Norm Interpolation Under Covariate Shift"

Source: https://www.emergentmind.com/topics/minimum-norm-interpolation