---
title: 'Range–Null Space Decomposition: Theory & Applications'
url: https://www.emergentmind.com/topics/range-null-space-decomposition
type: topic
---

# Range–Null Space Decomposition: Theory & Applications

Range–Null Space Decomposition (RND), also written Range–Null Space Decomposition (RNSD), is the orthogonal separation induced by a linear operator between a component determined by the operator and a component invisible to it. For \(A\in\mathbb{R}^{m\times n}\), the Fundamental Theorem of Linear Algebra gives \(\mathbb{R}^n=\mathrm{Null}(A)\oplus \mathrm{Range}(A^\top)\) and \(\mathbb{R}^m=\mathrm{Null}(A^\top)\oplus \mathrm{Range}(A)\), so inverse problems can be reformulated as the superimposition of a measurement-consistent range-space term and a null-space term that carries unresolved degrees of freedom [2507.20731][2407.04634]. Recent work instantiates this structure in neural vocoding, super-resolution, spectral compressive imaging, diffusion-based restoration, arbitrary medical slice super-resolution, limited-angle CT, OFDM channel estimation, state-space realization, graph regularization, and graph unlearning [2507.20731][2211.13524][2305.09746][2212.00490][2606.26716][2409.13930][2607.03348][2508.02044].

## 1. Linear-algebraic foundations

Let \(A\in\mathbb{R}^{m\times n}\) be a linear operator mapping \(\mathbb{R}^n\to\mathbb{R}^m\). Its range space and null space are
$$
\mathrm{Range}(A)\triangleq \{ y \in \mathbb{R}^m \mid \exists x \in \mathbb{R}^n,\; y = A x \},
$$
$$
\mathrm{Null}(A)\triangleq \{ x \in \mathbb{R}^n \mid A x = 0 \}.
$$
For any \(y\in\mathbb{R}^m\), there is an orthogonal decomposition
$$
y = y_R + y_N,\quad y_R \in \mathrm{Range}(A),\; y_N \in \mathrm{Null}(A^\top),\quad \mathrm{Range}(A) \perp \mathrm{Null}(A^\top).
$$
Dually, for any \(x\in\mathbb{R}^n\),
$$
x = x_R + x_N,\quad x_R \in \mathrm{Range}(A^\top),\; x_N \in \mathrm{Null}(A),\quad \mathrm{Range}(A^\top) \perp \mathrm{Null}(A).
$$
These identities are the standard orthogonal decompositions used throughout recent RND formulations [2507.20731].

When \(A\) has full column rank, the orthogonal projector onto \(\mathrm{Range}(A)\) is
$$
P_{\mathrm{Range}(A)} = A \left(A^\top A\right)^{-1} A^\top,
$$
and the projector onto \(\mathrm{Null}(A^\top)\) is
$$
P_{\mathrm{Null}(A^\top)} = I_m - P_{\mathrm{Range}(A)}.
$$
When \(A\) has full row rank, one works in \(\mathbb{R}^n\) with
$$
P_{\mathrm{Range}(A^\top)} = A^\top \left(A A^\top\right)^{-1} A,\qquad
P_{\mathrm{Null}(A)} = I_n - P_{\mathrm{Range}(A^\top)}.
$$
These formulas underlie the use of pseudoinverse projectors in inverse problems, because they explicitly isolate what measurements determine from what measurements cannot determine [2507.20731].

A common specialization is a filter-bank transform. If \(M\in\mathbb{R}^{F_m\times F}\) maps a linear-frequency spectrogram \(s\in\mathbb{R}^{F\times T}\) to mel features \(m\in\mathbb{R}^{F_m\times T}\) by
$$
m = M s,
$$
with \(F_m < F\) and \(M\) full row rank, then each frame admits the decomposition
$$
s = s_R + s_N,\quad s_R \in \mathrm{Range}(M^\top),\; s_N \in \mathrm{Null}(M),
$$
where
$$
s_R = M^\top \left(M M^\top\right)^{-1} m,
$$
and
$$
P_{\mathrm{Null}(M)} = I_F - M^\top \left(M M^\top\right)^{-1} M.
$$
This is the template later used to interpret mel-to-linear spectrogram reconstruction as a range–null decomposition rather than as a monolithic regression problem [2507.20731].

## 2. Pseudoinverse, bases, and computational realizations

For the compressive model \(y = A x\) with \(A\in\mathbb{R}^{m\times n}\) and \(m<n\), the Moore–Penrose pseudoinverse \(A^\dagger\) yields the canonical decomposition
$$
x = A^{\dagger} A x + \left(I_n - A^{\dagger} A\right) x.
$$
Here \(A^\dagger A\) is the projector onto \(\mathrm{Range}(A^\top)\), while \(I_n-A^\dagger A\) projects onto \(\mathrm{Null}(A)\). Equivalently, every measurement-consistent solution can be written as
$$
\tilde{x} = A^{\dagger} y + \left(I - A^{\dagger} A\right)\hat{x},
$$
so the range-space term is the least-squares consistent component and the null-space term contains all remaining admissible freedom [2407.04634].

This formulation has a basis version. If \(Z\in\mathbb{R}^{n\times N}\) has orthonormal columns spanning \(\mathrm{Null}(A)\), then
$$
P_{\mathrm{Null}(A)} = Z Z^{\mathsf T},\qquad
P_{\mathrm{Range}(A^{\mathsf T})} = I - Z Z^{\mathsf T}.
$$
For a given \(x\in\mathbb{R}^n\),
$$
x_N = Z Z^{\mathsf T} x,\qquad x_R = x - x_N.
$$
This basis-centric view is useful when explicit null-space vectors are needed, rather than only implicit projector applications [2407.04634].

Large-scale null-space computation motivates algorithmic variants. "A randomized small-block Lanczos method for large-scale null space computations" shows that randomness can be utilized to allow for smaller block size \(d\) without sacrificing convergence or reliability, and that even \(d = 1\) becomes a safe choice by using a small random diagonal perturbation and a random initial guess [2407.04634]. The same paper states that a smaller block size combined with restarting and partial reorthogonalization results in reduced memory requirements and computational effort, and also allows for the incremental computation of the null space, without requiring a priori knowledge of the nullity [2407.04634].

A different algebraic realization appears in elimination theory. If \(A\) has rank \(r\), Gauss–Jordan elimination yields a reduced row echelon form
$$
Z=\begin{bmatrix} I_r & F \\ 0 & 0 \end{bmatrix}P,
$$
where \(F\) captures how the \(n-r\) dependent columns are linear combinations of the \(r\) pivot columns. In this view, \(F\) reveals bases for the row space and the null space, and yields the column–row factorization \(A=CR\) with \(R=[I_r\;F]P\) [2304.02659]. This does not replace pseudoinverse-based RND, but it shows that range/null structure can also be read directly from elimination.

## 3. Instantiation in neural vocoding

"Learning Neural Vocoder from Range-Null Space Decomposition" formulates mel-based neural vocoding as a time-frequency inverse problem in which the target linear-scale magnitude spectrogram is decomposed into a range-space part reconstructed by a fixed linear operator and a null-space part generated by a neural network [2507.20731]. The paper starts from mel formation
$$
\mathbf{X}^{mel} = \log\left(\mathcal{A} \, | \mathbf{S} | \right),
$$
with mel filter-bank matrix \(\mathcal{A}\in\mathbb{R}^{F_m\times F}\), and then removes the logarithm by defining
$$
\overline{\mathbf{X}^{mel}} = \exp\left(\mathbf{X}^{mel}\right) = \mathcal{A} \, | \mathbf{S} |.
$$
This matches the classical compressive model with \(A=\mathcal{A}\) and \(x=|S|\) [2507.20731].

The range-space estimate is
$$
|\tilde{\mathbf{S}}|_{\mathrm{range}} = \mathcal{A}^{\dagger} \, \overline{\mathbf{X}^{mel}},
$$
where \(\mathcal{A}^{\dagger} = \mathcal{A}^\top (\mathcal{A}\mathcal{A}^\top)^{-1}\). The null-space module predicts a magnitude residual and phase, and the residual is explicitly projected onto \(\mathrm{Null}(\mathcal{A})\):
$$
\mathbf{S}_N \equiv \left( \mathbf{I}_F - \mathcal{A}^{\dagger} \mathcal{A} \right)\, G(\cdot),
$$
so the final magnitude becomes
$$
|\tilde{\mathbf{S}}| = \mathbf{S}_R + \mathbf{S}_N,\qquad \mathbf{S}_R = \mathcal{A}^{\dagger} \overline{\mathbf{X}^{mel}}.
$$
The complex spectrogram is then
$$
\tilde{\mathbf{S}} = |\tilde{\mathbf{S}}| \, e^{j \tilde{\mathbf{\Phi}}}.
$$
This makes the range/null split explicit rather than implicit [2507.20731].

The paper gives three rationales. First, degradation consistency:
$$
\mathcal{A}\, |\tilde{\mathbf{S}}| = \overline{\mathbf{X}^{mel}} + 0,
$$
so the reconstruction preserves the information present in mel-space. Second, interpretability: the range-space part reflects what can be linearly “unfolded” from mel, whereas the null-space part represents exactly the lost components not recoverable by any linear inverse. Third, efficiency: the network only needs to model the null-space residuals and phase, rather than the entire spectrum [2507.20731].

An ablation shows that making \(\{\mathcal{A},\mathcal{A}^{\dagger}\}\) learnable harms orthogonality and performance: the null-space estimate becomes non-sparse and leaks into the range space. The paper presents this as evidence that a fixed, physically grounded projector is beneficial [2507.20731].

## 4. Dual-path architecture, training, and empirical behavior in RNDVoC

The proposed framework contains a Range-space Module (RSM) and a Null-space Module (NSM). The RSM is a fixed linear mel\(\to\)linear operation via \(\mathcal{A}^{\dagger}\). The NSM hierarchically encodes \( |\tilde{\mathbf{S}}|_{\mathrm{range}} \), models sub-band and sequential dependencies, decodes magnitude residuals and phase, and projects the magnitude residual into \(\mathrm{Null}(\mathcal{A})\) [2507.20731].

The Hierarchical Spectral Encoding Module splits the spectrum into \(I = 3\) spectral regions. The encoder uses Conv2d with stride, LayerNorm, and produces compact sub-band representations with channel size \(C = 256\). The spectral division follows a “from-fine-to-coarse” principle: lower and mid bands are finer to capture important features such as \(F0\) and harmonics, while higher bands are coarser [2507.20731].

The stack has \(B = 6\) Dual-Path Blocks. Each block contains a Cross-Band Module for sub-band interaction and a Narrow-Band Module with \(P = 2\) ConvNeXt v2 blocks operating per sub-band. The magnitude branch decodes sub-band outputs, applies \(\exp(\cdot)\) to enforce non-negativity, then applies the null-space projection. The phase branch outputs real and imaginary parts, and the phase is recovered via \(\mathrm{Atan2}(\mathrm{Im},\mathrm{Re})\) [2507.20731].

Training uses a reconstruction loss
$$
\mathcal{L}_{\mathrm{rec}} = \lambda_a \mathcal{L}_a + \lambda_p \mathcal{L}_p + \lambda_{ri} \mathcal{L}_{ri} + \lambda_{\mathrm{mel}} \mathcal{L}_{\mathrm{mel}} + \lambda_c \mathcal{L}_c,
$$
where \(\mathcal{L}_a\) is a log-amplitude loss, \(\mathcal{L}_p\) is a phase loss, \(\mathcal{L}_{ri}\) is a real/imaginary loss, \(\mathcal{L}_{\mathrm{mel}}\) is a mel-domain spectral loss, and \(\mathcal{L}_c\) is a consistency term. The paper also proposes an omnidirectional phase loss using a \(3\times 3\) convolutional operator to compute phase differentials in eight directions and instantaneous phase with fixed kernels \(\mathcal{K}\in\mathbb{R}^{9\times 3\times 3}\) [2507.20731]. Adversarial training uses hinge GAN with Multi-Period Discriminator and Multi-Resolution Spectrogram Discriminator, together with feature matching [2507.20731].

Optimization details are reported explicitly: LJSpeech at \(22.05\) kHz with split per VITS repository, LibriTTS at \(24\) kHz with train-clean-100/300 and train-other-500; FFT \(=1024\), Hann window \(=1024\), hop \(=256\); LJSpeech uses \(F_m=80, f_{\max}=8\) kHz and LibriTTS uses \(F_m=100, f_{\max}=12\) kHz; batch size \(=16\), segment \(=16384\), AdamW with \((\beta_1,\beta_2)=(0.8,0.99)\), initial learning rate \(=2\times 10^{-4}\), and \(1\)M generator and \(1\)M discriminator steps [2507.20731].

The proposed RNDVoC has \(\sim 3.14\)M parameters and \(34.10\) GMACs per \(5\) s audio. Inference speed is reported as CPU real-time factor \(\approx 14.96\times\) \((0.0669\) for \(5\) s\()\) and GPU \(\approx 233\times\) \((0.0043\) for \(5\) s\()\). Lightweight variants are also given: RNDVoC-Lite with \(0.71\)M parameters and \(9.54\) GMACs/5 s, and RNDVoC-UltraLite with \(0.08\)M parameters and \(1.66\) GMACs/5 s [2507.20731].

On LJSpeech, selected objective results are: RNDVoC \(3.14\)M params, \(34.10\) GMACs/5 s, M-STFT \(=0.9103\), PESQ \(=3.987\), MCD \(=2.0471\), Periodicity RMSE \(=0.0854\), V/UV F1 \(=0.9714\), Pitch RMSE \(=21.3183\), ViSQOL \(=4.8373\). BigVGAN-112M reports M-STFT \(=0.9001\), PESQ \(=4.107\), MCD \(=1.8769\), Periodicity RMSE \(=0.0838\), V/UV F1 \(=0.9716\), Pitch RMSE \(=20.6922\), ViSQOL \(=4.8699\). The paper states that RNDVoC achieves near–SOTA quality with only \(\approx 2.8\%\) of BigVGAN’s parameters and \(\approx 8.17\%\) computational cost, and \(\approx 10\times\) CPU speed-up [2507.20731].

On LibriTTS, selected objective results are: RNDVoC PESQ \(=4.226\), Periodicity RMSE \(=0.0742\), V/UV F1 \(=0.9698\), Pitch RMSE \(=23.9658\), ViSQOL \(=4.9154\). BigVGAN at \(1\)M steps reports PESQ \(=4.027\), Periodicity RMSE \(=0.1018\), V/UV F1 \(=0.9598\), and BigVGAN at \(5\)M steps reports PESQ \(=4.269\), Periodicity RMSE \(=0.0790\), V/UV F1 \(=0.9670\), ViSQOL \(=4.9632\). In MUSHRA tests on LibriTTS, RNDVoC scores \(80.74 \pm 0.99\) versus BigVGAN \(79.33 \pm 0.92\), statistically significant with \(p<0.05\) [2507.20731].

The ablations are structurally revealing. Removing omnidirectional phase loss degrades PESQ from \(3.987\) to \(3.892\) and MCD from \(2.047\) to \(2.214\). Removing RND further degrades PESQ from \(3.892\) to \(3.655\). Making \(\{\mathcal{A},\mathcal{A}^{\dagger}\}\) learnable yields PESQ \(3.645\) and MCD \(2.599\), while null-space estimates become dense and non-sparse. This supports the interpretation that explicit range/null factorization, not merely architectural scale, is responsible for the reported parameter–performance trade-off [2507.20731].

## 5. Imaging, reconstruction, and diffusion instantiations

In super-resolution, "GAN Prior based Null-Space Learning for Consistent Super-Resolution" uses a pooling-based decomposition in which average pooling \(P_{\downarrow}\) and replication upsampling \(P_{\uparrow}\) satisfy \(P_{\downarrow}(P_{\uparrow}(z)) = z\). The resulting projections are
$$
P_R = P_{\uparrow} P_{\downarrow},\qquad P_N = I - P_{\uparrow} P_{\downarrow},
$$
and the final consistent high-resolution image is
$$
\hat{\mathbf{x}} = \mathcal{P}_{\uparrow}(\mathbf{y}) + \hat{\mathbf{x}}_r - \mathcal{P}_{\uparrow}(\mathcal{P}_{\downarrow}(\hat{\mathbf{x}}_r)).
$$
The paper states that this analytically eliminates low-frequency inconsistencies, can replace pixel-wise losses, and speeds up the convergence of training up to \(2\sim 10\) times [2211.13524].

In spectral compressive imaging, "A Range-Null Space Decomposition Approach for Fast and Flexible Spectral Compressive Imaging" writes the CASSI reconstruction as
$$
\hat{x} = \Phi^+ y + (I - \Phi^+ \Phi) q,
$$
with \(\Phi^+ = \Phi^\top(\Phi\Phi^\top)^{-1}\) because \(\Phi\Phi^\top\) is diagonal and full rank. This produces a deterministic range-space component and a learned null-space component. The paper reports that RND-SAUNet has \(0.78\)M parameters, \(7.84\) GFLOPs, the lowest training memory \(6.50\) GB, and \(91\) FPS on Tesla V100 at \(256\times 256\) spatial resolution with \(N_\lambda=28\) [2305.09746].

In diffusion-based image restoration, "Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model" enforces exact data consistency at every reverse step by replacing the clean estimate \(x_{0|t}\) with
$$
\hat{x}_{0|t} = A^{\dagger} y + (I - A^{\dagger} A) x_{0|t}.
$$
The paper interprets arbitrary linear restoration tasks as fixing the range-space content and refining only the null-space during reverse diffusion. DDNM+ introduces a scaled correction
$$
\hat{x}_{0|t} = x_{0|t} - \Sigma_t A^{\dagger}(A x_{0|t} - y)
$$
for noisy restoration, together with adjusted sampling variance [2212.00490].

In arbitrary medical slice super-resolution, "Dual-Prior Guided Null-Space Learning with Mixture-of-Splines for Arbitrary Medical Slice Super-Resolution" uses a generalized inverse \(U\) satisfying \(D U = I\) and constructs
$$
\Pi_N \equiv I - U D,\qquad
V_{SR} = U(V_{LR}) + \Pi_N(\hat{V}_N).
$$
Because \(D\Pi_N=0\), the paper states that all learned details are confined to the null space and acquired slices are preserved with zero error. It reports gains such as \(+1.07\) dB over second best on Liver \(\times 2\), \(+0.46\) dB on Hepatic Vessels \(\times 7\), \(+0.62\) dB on IXI \(\times 5\), and real-data RPLHR-CT results of \(34.91\) dB PSNR and \(0.8925\) SSIM [2606.26716].

In limited-angle CT, "RN-SDEs: Limited-Angle CT Reconstruction with Residual Null-Space Diffusion Stochastic Differential Equations" rectifies the diffusion estimate by
$$
\tilde{x}_{0|t} = A^\dagger y + (I - A^\dagger A)x_{0|t},
$$
implemented with a learnable pseudo-inverse \(\hat{A}^\dagger_\eta\), and also proposes the residual form
$$
\tilde{x}_{0|t} = x_{0|t} - \Gamma_t \hat{A}^\dagger_\eta (A x_{0|t} - y).
$$
The paper reports that rectification reduces data-consistency error in mean and variance, and that RN-SDE requires \(\sim 66\) s per slice and \(593\) iterations on ChromSTEM versus \(\sim 4634\) s and \(1000\) iterations for DOLCE [2409.13930].

In OFDM channel estimation, "Diffusion-Based Noise-Adaptive Null-Space Channel Estimation for OFDM Systems" treats the DMRS mask as a diagonal operator \(\mathbf{M}\), with
$$
\mathbf{P}_{\mathcal{R}}=\mathbf{M}^{\dagger}\mathbf{M},\qquad
\mathbf{P}_{\mathcal{N}}=\mathbf{I}-\mathbf{M}^{\dagger}\mathbf{M}.
$$
The noiseless correction uses
$$
\hat{\mathbf{h}}_{0|t}=\mathbf{M}^\dagger\mathbf{y}+(\mathbf{I}-\mathbf{M}^\dagger\mathbf{M})\,\mathbf{h}_{0|t},
$$
while the noisy case uses a noise-adaptive posterior correction
$$
\hat{\mathbf{h}}_{0|t}=\mathbf{h}_{0|t}-\lambda_t\,\mathbf{M}^{\dagger}\left(\mathbf{M}\mathbf{h}_{0|t}-\mathbf{y}\right).
$$
The paper states that DANCE consistently achieves lower NMSE than MMSE, MATLAB nrChannelEstimator, DPS, and DMPS across SNR sweeps, DMRS configurations, Doppler shifts, and train-test mismatches [2607.03348].

## 6. Adjacent formulations, limitations, and conceptual boundaries

Several recent works use the same decomposition to refine only the invisible component rather than the full signal. "GSNR: Graph Smooth Null-Space Representation for Inverse Problems" parameterizes the null component as \(x = x_p + Bz\), constructs a null-restricted Laplacian \(T=P_n L P_n\), and regularizes only \(z\) or only the projected null signal. The paper states that this yields improved convergence, better coverage, and high predictability, and reports up to \(\approx 4.3\) dB over baseline formulations and up to \(\approx 1\) dB compared with end-to-end learned models in PSNR [2602.20328]. This suggests that null-only priors can alter conditioning without perturbing the range-space fit.

In system identification, "Range Space or Null Space: Least-Squares Methods for the Realization Problem" analyzes a Hankel matrix of Markov parameters and shows that the classical range-space-based method corresponds to a total least-squares solution, whereas the null-space-based method corresponds to an ordinary least-squares solution. The paper argues that the optimal realization is obtained through a weighted least-squares approach and provides consistency and asymptotic normality results [2505.19639]. Closely related, "Weighted Null Space Fitting (WNSF): A Link between The Prediction Error Method and Subspace Identification" states that SIM estimates the range space of the extended observability matrix via SVD, while WNSF estimates its null space, avoiding the need for singular value decomposition, and derives statistically optimal weighting for the null space fitting problem [2411.00506]. These works show that range- and null-space viewpoints are not restricted to image or audio inverse problems.

Graph-structured formulations also appear outside continuous inverse problems. "Graph Unlearning via Embedding Reconstruction -- A Range-Null Space Decomposition Approach" applies the decomposition to the linear transformation \(H=W^k\) in a GNN layer and constructs
$$
f_2^{\text{RND}}(i,j)=H^{\dagger}f_1(j,i)+(I-H^{\dagger}H)f_2(i,j),
$$
so that \(H f_2^{\text{RND}}(i,j)=f_1(j,i)\). The paper reports average running time \(1/40\) to \(1/88\) of retraining and membership-inference AUC values near \(0.5\) [2508.02044]. In a different direction, "On the null structure of bipartite graphs without cycles of length a multiple of 4" proves that, for \(C_{4k}\)-free bipartite graphs, the fundamental spaces of \(G\) are the direct sum of the fundamental spaces of canonical subgraphs \(C_N(G)\) and \(C_S(G)\), with \(C_N(G)\) nonsingular and \(C_S(G)\) carrying the full null space [1810.05802]. This is a graph-theoretic analogue of structural range/null separation.

A recurring limitation is operator mismatch. In the vocoder setting, if mel configuration at inference differs from training, \(\mathcal{A}^\dagger\) becomes inconsistent, harming range-space accuracy; the paper explicitly cites different \(F_m\), \(f_{\max}\), and window settings as failure modes [2507.20731]. In imaging and channel estimation, exact projection can be undesirable under noisy measurements, which motivates relaxed corrections in DDNM+, RN-SDEs, and DANCE rather than hard substitution of the measured range-space content [2212.00490][2409.13930][2607.03348]. A common misconception is that the null-space term is arbitrary. The cited methods instead constrain it through explicit projectors, null-only losses, diffusion priors, graph smoothness, or geometric continuity, so the null component is learnable but not unconstrained.

Another conceptual boundary concerns ambient space. For \(y\in\mathbb{R}^m\), the decomposition is between \(\mathrm{Range}(A)\) and \(\mathrm{Null}(A^\top)\); for \(x\in\mathbb{R}^n\), it is between \(\mathrm{Range}(A^\top)\) and \(\mathrm{Null}(A)\). Recent applications overwhelmingly work in the signal domain and therefore use \(A^\dagger A\) and \(I-A^\dagger A\), not \(A A^\dagger\). This distinction is operational rather than terminological: it determines which projector preserves measurements and which projector annihilates them. The broader literature collectively indicates that RND is best understood not as a single algorithm, but as a structural design principle for enforcing consistency, isolating irrecoverable degrees of freedom, and making explicit the split between known physics and learned priors [2507.20731][2407.04634][2602.20328].

Source: https://www.emergentmind.com/topics/range-null-space-decomposition