Normalized Weighted Least Squares (NWLS)

• NWLS is a function approximation strategy that uses Christoffel weights to stabilize least squares systems and achieve near-optimal recovery from finite samples.
• It employs a normalized weighted design matrix ensuring uniform row norms, minimal variance, and robust error bounds even in high-dimensional or irregular domains.
• NWLS is applied in uncertainty quantification, polynomial approximation, and numerical solutions for parametric and stochastic PDEs, offering proven stability and efficiency.

Normalised Weighted Least Squares (NWLS) is a methodology for function approximation in finite-dimensional spaces, achieving stable and near-optimal recovery from finite, possibly noisy samples. It leverages the Christoffel function and related sampling measures to guarantee well-conditioned least-squares systems and robust error bounds even in high-dimensional, irregular, or unbounded domains. The foundational principle is the normalization of the weighted design matrix, enabling uniform row norms and minimal variance in the empirical projection operator. NWLS underpins modern approaches in uncertainty quantification, polynomial approximation, and numerical solutions of parametric and stochastic PDEs (Cohen et al., 2016, Migliorati, 2019, Narayan et al., 2014).

1. Mathematical Formulation and Theoretical Framework

NWLS operates in a probability space $(X, \rho)$ where $d\rho$ is a probability measure on $X\subset \mathbb{R}^d$. Let $V_m\subset L^2(X, d\rho)$ be an $m$-dimensional space with an $L^2$-orthonormal basis $\{L_j\}_{j=1}^m$. The Christoffel function is defined as

$k_{m, \rho}(x) := \sum_{j=1}^m |L_j(x)|^2.$

NWLS specifies the weight and sampling measure as

$$w(x) = \frac{m}{k_{m, \rho}(x)}, \quad d\mu(x) = \frac{k_{m, \rho}(x)}{m} d\rho(x).$$

Given $n$ independent samples $\{x^i\}_{i=1}^n \sim \mu$, the weighted least-squares estimator $u_W\in V_m$ minimizes the weighted empirical risk:

$$u_W = \arg\min_{v\in V_m} \frac{1}{n}\sum_{i=1}^n w(x^i)\left(v(x^i)-u(x^i)\right)^2.$$

In matrix terms, with Gram matrix $$G_{jk} = \frac{1}{n}\sum_{i=1}^n w(x^i)L_j(x^i)L_k(x^i)$$ and right-hand side $d_j = \frac{1}{n}\sum_i w(x^i) u(x^i) L_j(x^i)$, the coefficients $v$ solve $Gv = d$ (Cohen et al., 2016, Narayan et al., 2014).

2. Christoffel Function and Sampling Measures

The Christoffel function $k_{m,\rho}$ plays a central role in sample weighting and measure design. Its reciprocal, normalized as $w(x) = m / k_{m, \rho}(x)$, ensures that the weighted design matrix $\sqrt{w(x)} L_j(x)$ has row norms equal to unity. Under the sampling measure $d\mu$, the Christoffel-weighted sum $k_{m,w}(x):=\sum_{j=1}^m w(x) |L_j(x)|^2=m$ is constant, minimizing the supremum $K_{m, w}$ and regularizing the least-squares system. For general bounded domains, the approximation space $V_n\subset L^2(\Omega, \mu)$ (with orthonormal basis $\{L_j\}$) yields $k_n(x)=\sum_{j=1}^n |L_j(x)|^2$, normalized via $w(x)=n/k_n(x)$ and corresponding sampling measure $d\sigma_n(x) = (k_n(x)/n) d\mu(x)$ (Migliorati, 2019).

A principal innovation is the prescription to sample from the measure induced by $k_{m,\rho}$ (or the pluripotential equilibrium measure in some settings), which yields stability and accuracy guarantees unattainable with naive Monte Carlo sampling from the orthogonality measure alone (Narayan et al., 2014).

3. Algorithmic Procedures

NWLS methodology can be summarized by the following steps:

1. Basis Construction: Identify an $L^2$-orthonormal basis $\{L_j\}$ (analytically or via discrete surrogate orthogonalization when unavailable in closed form, as for irregular domains) (Migliorati, 2019).
2. Christoffel Function Evaluation: Compute $k_{m,\rho}(x)$ for all evaluation points.
3. Sampling Measure: Draw sample points $\{x^i\}$ i.i.d. from $d\mu(x)=\frac{k_{m, \rho}(x)}{m}d\rho(x)$ or, for irregular domains, from an empirical surrogate measure.
4. Weight Calculation: Assign weights $w(x^i)=m/k_{m,\rho}(x^i)$.
5. Weighted Gram Matrix Assembly: Form $$G_{jk} = \frac1n\sum_{i=1}^n w(x^i)L_j(x^i)L_k(x^i)$$.
6. Empirical Right-hand Side: Assemble $d_j = \frac1n\sum_{i=1}^n w(x^i)u(x^i)L_j(x^i)$.
7. Solve Normal Equations: Compute coefficients $v$ via $Gv=d$.
8. Estimator Construction: Return $u_W(x) = \sum_{j=1}^m v_j L_j(x)$. In practice, truncation is used to enforce bounds on $|u_T(x)|$.

For irregular, non-tensor-product, or empirical domains, an additional Stage A constructs a discrete near-orthonormal basis via QR factorization of a Vandermonde-like matrix, at cost $O(\widetilde m n^2)$ with $\widetilde m$ scaling with the Christoffel function supremum $K_n$ (Migliorati, 2019).

4. Theoretical Guarantees: Stability and Error Bounds

NWLS provides high-probability stability and quasi-optimal error guarantees:

• Condition Number Bound: With probability at least $1-2n^{-r}$, if $m \leq \kappa(r) n / \ln n$, $\mathrm{cond}(G) \leq 3$ (Cohen et al., 2016).
• Error Convergence: The mean-square error of the clipped estimator $u_T$ satisfies

$$\mathbb{E}\|u-u_T\|^2_{L^2(d\rho)} \leq (1+\epsilon_n) e_m(u)^2 + 64 n^{-r},\;\; \epsilon_n \rightarrow 0 \text{ as } n\to\infty.$$

• Sample Complexity: In both regular and irregular domains, stable estimation requires $n \gtrsim m\ln m$ samples (or $m \gtrsim n\ln n$ for the number of samples fitting an $n$-dimensional space), a substantial efficiency improvement over classical unweighted least squares. The key analysis for irregular domains also shows that, provided $K_n \sim n$ or $n^2$, overall computational cost remains tractable up to $n$ in the thousands (Migliorati, 2019, Narayan et al., 2014).
• Bias Control: Exact sampling from the Christoffel-induced measure yields clean minimax-type error bounds, while sampling from approximate equilibrium measures risks persistent bias terms (Cohen et al., 2016).

5. Sampling Algorithms and Implementation in Multivariate and Irregular Domains

For multivariate problems with non-tensor-product measures $d\rho$, NWLS employs sequential conditional sampling:

• Each $x^k_1$ is drawn from a marginal depending on the basis; subsequent coordinates $x^k_q$ are sampled from conditional densities constructed from basis evaluations of prior coordinates.
• When the Christoffel function or equilibrium measure is defined only empirically (as for irregular domains), surrogate bases are constructed via QR-decomposition of evaluation matrices built on random samples (Migliorati, 2019).
• Univariate sampling within these algorithms may utilize rejection sampling (envelope method) or inverse transform sampling, both connected directly to the construction of the Christoffel-weighted density or its cumulative (Cohen et al., 2016).

This allows NWLS to be applied to domains with high or infinite dimensions and to polynomial spaces defined on unbounded supports (e.g., multivariate Hermite polynomials under Gaussian measures) (Cohen et al., 2016).

6. Connections, Comparison with Alternative Methods, and Extensions

NWLS generalizes and improves upon standard least-squares and Monte Carlo approaches:

• In standard least-squares, sampling from the orthogonality measure may yield extremely ill-conditioned Gram matrices when the Christoffel function spikes, resulting in poor accuracy unless a prohibitively large number of samples is used (Narayan et al., 2014).
• Sampling from the equilibrum measure or the Christoffel-induced measure renders the normalized rows of the design matrix, directly controlling the spectral properties of the Gram matrix and enabling efficient sample complexity independent of the details of $w$ or $V_m$ (Cohen et al., 2016).
• Recent alternative methods (e.g., Jakeman–Narayan–Zhou) propose sampling from fixed equilibrium measures with Christoffel weights. NWLS analysis demonstrates that only exact pairing of measure and weight achieves unbiased minimax error results (Cohen et al., 2016).
• NWLS extends to parametric and stochastic PDEs with infinite-dimensional parameter spaces by truncation to finite downward-closed index sets of cardinality $m$, with no modification to the overall methodology (Cohen et al., 2016).

A summary table of methodological positions:

Approach	Sampling Measure	Weights	Error/Condition Bounds
Standard Least Squares	Orthogonality measure $\rho$	Uniform	Potentially poor
NWLS (Christoffel)	Christoffel measure $\mu$	$m/k_{m,\rho}$	Uniform, minimax
Equilibrium measure + Christoffel	Equilibrium measure $\mu^*$	$m/k_{m,\rho}$	May introduce bias

7. Computational Complexity and Practical Aspects

The dominant computational cost in NWLS arises in basis orthogonalization (for irregular domains) and assembly of weighted Gram matrices:

• For regular domains with analytic bases, the cost is $O(m n)$ for Gram assembly and $O(n^3)$ for solving normal equations.
• For domains without analytic bases, QR factorization of a $\widetilde m \times n$ matrix is $O(\widetilde m n^2)$, with $\widetilde m$ scaling with the Christoffel supremum $K_n$.
• Evaluating basis functions at any point costs $O(n)$, and total costs remain $O(n^3\log n)$ provided $K_n$ grows mildly with $n$ (Migliorati, 2019).

Implementation is fully practical up to moderate or high dimensions ($n$ in the thousands), with complexity and stability tied directly to properties of the Christoffel function for the chosen approximation space and domain.

---

References:

• [Optimal weighted least-squares methods, (Cohen et al., 2016)]
• [Multivariate approximation of functions on irregular domains by weighted least-squares methods, (Migliorati, 2019)]
• [A Christoffel function weighted least squares algorithm for collocation approximations, (Narayan et al., 2014)]