---
title: 'NP-PIELM: Null-Space Projection Method'
url: https://www.emergentmind.com/topics/null-space-projected-pielm-np-pielm
type: topic
---

# NP-PIELM: Null-Space Projection Method

Null-Space Projected Physics-Informed Extreme Learning Machine (NP-PIELM) is a computational framework for enforcing exact linear constraints in the training of physics-informed extreme learning machines (PIELMs). NP-PIELM applies an algebraic projection in coefficient space to guarantee satisfaction of prescribed boundary or initial conditions at discrete collocation points. By exploiting the geometric structure of the coefficient manifold, the method transforms the original constrained optimization into an unconstrained least-squares problem in the null space of the boundary operator, removing the need for penalty weights, dual variables, and iterative constraint adjustment. The approach achieves single-shot training efficiency characteristic of extreme learning machines while ensuring that constraints are satisfied to machine precision [2601.10999].

## 1. PIELM Formulation and Traditional Constraint Enforcement

PIELM solves linear boundary value problems of the form
$$
\mathcal{L} u(x) = f(x), \quad x \in \Omega, \qquad \mathcal{B} u(x^b) = g(x^b), \quad x^b \in \partial\Omega,
$$
by positing a single-hidden-layer network ansatz:
$$
\hat{u}(x; \beta) = \sum_{k=1}^{N_\phi} \beta_k \sigma(w_k^\top x + b_k) = \phi(x)^\top \beta,
$$
where $\phi(x) \in \mathbb{R}^{N_\phi}$ is the fixed random feature vector, and $\beta \in \mathbb{R}^{N_\phi}$ are the output weights to be learned.

Standard PIELM enforces constraints via a penalty-based loss,
$$
J(\beta) = \|\Psi \beta - f\|_2^2 + \lambda \|B \beta - g\|_2^2,
$$
with $\Psi$, $B$, $f$, $g$ denoting the PDE and boundary collocation matrices and targets, and $\lambda>0$ a user-specified penalty. This formulation leads to only approximate satisfaction of the constraint $B\beta = g$, heavily dependent on $\lambda$. Poor selection of $\lambda$ yields either weak enforcement of constraints or disproportionate focus on them, often resulting in ill-conditioning and suboptimal PDE interior fits.

## 2. Admissible Coefficient Manifold and Null-Space Decomposition

To enforce the boundary constraints exactly, NP-PIELM characterizes the admissible set of coefficient vectors,
$$
\mathcal{M} = \{c \in \mathbb{R}^{N_\phi} \mid B c = d\},
$$
as an affine subspace. The fundamental theorem of linear algebra provides the direct sum:
$$
\mathbb{R}^{N_\phi} = \ker(B) \oplus \operatorname{range}(B^\top),
$$
which allows any admissible $c \in \mathcal{M}$ to be uniquely decomposed as
$$
c = c_p + n, \quad B c_p = d, \quad n \in \ker(B).
$$
Here, $c_p$ is a particular solution of $B c = d$ and variations within $\ker(B)$ do not affect the constraints. This geometric structure is central to NP-PIELM.

## 3. Translation-Invariant Parametrization and Projection

NP-PIELM constructs a translation-invariant parametrization for all feasible coefficients:
$$
c(w) = c_p + N w, \qquad w \in \mathbb{R}^m,
$$
where $N \in \mathbb{R}^{N_\phi \times m}$ is an orthonormal basis for $\ker(B)$. The minimal-norm particular solution is provided by $c_p = B^+ d$ with $B^+$ the Moore–Penrose pseudoinverse. The projector onto $\ker(B)$ is $P_{\mathcal{N}} = I - B^+ B$, and any orthonormal basis for $\operatorname{range}(P_{\mathcal{N}})$ may serve as $N$.

As $B N = 0$ by construction, the parametrization ensures $B c(w) = d$ for any $w$. This removes constraint handling from the optimization process entirely.

## 4. Reduction to an Unconstrained Least-Squares Problem

Within this parametrization, the original constrained residual minimization
$$
\min_c \|\Psi c - f\|_2^2 \quad \text{subject to} \quad Bc = d
$$
becomes an unconstrained problem in $w$:
$$
\min_{w \in \mathbb{R}^m} \| \tilde{\Psi} w - \tilde{f} \|_2^2, \qquad \tilde{\Psi} = \Psi N,\; \tilde{f} = f - \Psi c_p.
$$
The solution is obtained by solving the normal equations
$$
\tilde{\Psi}^\top \tilde{\Psi} w = \tilde{\Psi}^\top \tilde{f},
$$
or equivalently, in a single step via the pseudoinverse: $w^* = \tilde{\Psi}^+ \tilde{f}$, followed by reconstruction $c^* = c_p + N w^*$. This guarantees exact (up to numerical roundoff) enforcement of constraints at collocation points without penalty terms, dual variables, or iterative tuning.

## 5. Algorithmic Workflow

The NP-PIELM procedure proceeds as follows:

| Step | Description |
|------|-------------|
| 1    | Sample $N_x$ interior collocation points in $\Omega$ and $N_b$ boundary (or initial) points on $\partial\Omega$ |
| 2    | Construct matrices $\Psi \in \mathbb{R}^{N_x \times N_\phi}$ and $B \in \mathbb{R}^{N_b \times N_\phi}$ along with targets $f \in \mathbb{R}^{N_x}$, $d=g \in \mathbb{R}^{N_b}$ |
| 3    | Compute particular solution $c_p = B^+ d$ |
| 4    | Extract $N$, an orthonormal basis for $\ker(B)$ via SVD or rank-revealing QR |
| 5    | Form reduced system: $\tilde{\Psi} = \Psi N$, $\tilde{f} = f - \Psi c_p$ |
| 6    | Solve $\tilde{\Psi} w \approx \tilde{f}$ for $w^*$ by least-squares or pseudoinverse |
| 7    | Compute $c^* = c_p + N w^*$ for use in the output layer of the network |

## 6. Empirical Performance and Benchmarks

Benchmarking across various elliptic and parabolic PDEs demonstrates that NP-PIELM enforces linear constraints up to machine precision with training costs comparable to standard PIELMs. Representative problems and outcomes:

| Problem | $N_\phi$ | $N_x$ | $N_b$ | Max Error | Train Time (s) |
|---------|---------:|------:|------:|----------:|---------------:|
| 1D Conv.-Diff.-React. $([0,1]$, Dirichlet) | 501 | 1000 | 2 | $4.4 \times 10^{-9}$ | 0.14 |
| 1D Unsteady Adv.-Diff. (space-time) | 441 | 2025 | 135 | $\lesssim 4 \times 10^{-15}$ | 0.09 |
| 2D Poisson (mixed BC) | 256 | 900 | 120 | $\lesssim 5.6 \times 10^{-15}$ | 0.07 |
| 2D Unsteady Heat (complex domain) | 4693 | 7680 | 4384 | $\sim 10^{-10}$ | 19.4 |
| 2D Steady Stokes Flow | 1323 | 2500 | 401 | $10^{-14} \; (\vec{u})$, $10^{-10} \; (p)$ | - |

In all tests, constraints are met to machine epsilon and the interior residual is minimized to the best level achievable with the chosen random-feature basis [2601.10999].

## 7. Advantages, Caveats, and Theoretical Considerations

NP-PIELM offers strict constraint satisfaction at collocation points with no need for penalty weight tuning, a single linear algebra solve, domain- and geometry-agnostic enforcement, and improved conditioning relative to large-penalty formulations. This approach is tractable for moderate problem sizes and preserves the hallmark “single-shot” training efficiency of ELM techniques.

Limitations include the requirement that boundary or initial conditions be linear in $u$ (so that $B$ is linear), and the computational cost and memory implications for large $N_\phi$, especially if $N_b$ is also large. Constraint satisfaction is exact only at discrete collocation points; extension to continuous or weak constraint enforcement requires further development. Conditioning of the reduced system $\tilde{\Psi}$ may still pose challenges; regularization such as Tikhonov can mitigate this when necessary.

A plausible implication is that NP-PIELM is especially suited to PDEs with complex domains and a substantial number of constraints, provided these admit a tractable null-space basis construction. The framework reconciles the need for strictly satisfied data constraints with the efficiency and flexibility of random-feature models [2601.10999].

Source: https://www.emergentmind.com/topics/null-space-projected-pielm-np-pielm