INPMAT: Iterative Null-Space Projection & Adaptive Thresholding

• INPMAT is an algorithmic framework for sparse recovery and low-rank matrix completion that alternates adaptive thresholding with null-space projection to ensure measurement consistency.
• The method dynamically adjusts the thresholding parameter at each iteration, eliminating the need for prior sparsity knowledge while accelerating convergence through feedback corrections.
• Empirical results demonstrate INPMAT’s superior performance in compressed sensing, matrix completion, and joint sparsity recovery, offering improved accuracy and reduced computational cost over classical methods.

Iterative Null-Space Projection with Adaptive Thresholding (INPMAT) is an algorithmic framework developed for sparse signal recovery and low-rank matrix completion. It leverages iterative alternation between thresholding (for support selection) and orthogonal null-space projection (for measurement consistency), and rigorously adapts thresholding parameters at each iteration. INPMAT and its variants are characterized by their ability to overcome the limitations of classical thresholding and projection algorithms by incorporating adaptive strategies that do not require prior knowledge of signal sparsity and by maintaining feasibility at each step. The approach has been extended to structured problems such as joint sparsity (MMV), matrix completion, and debiasing in neural representations, demonstrating superior empirical and theoretical performance relative to classical greedy and convex optimization-based methods (Esmaeili et al., 2016, Song et al., 2017, Han et al., 2020, Han et al., 2021).

1. Algorithmic Foundations

INPMAT aims to recover a sparse or low-rank structure from incomplete linear measurements. In compressed sensing, the core problem is to identify $x\in\mathbb{R}^N$ from measurements $y = A x + e$, where $A\in\mathbb{R}^{M\times N}$ ($M\ll N$), $x$ is $s$-sparse, and $e$ is noise. The key innovation of the method lies in its alternation between two stages:

• Thresholding (Support Selection): At each iteration $k$, a (hard) thresholding operator $H_{f(k)}(\cdot)$ selects the $f(k)$ largest entries (in magnitude) of the candidate signal, where $y = A x + e$0 is an adaptively growing function.
• Null-space Projection: The result is projected onto the null space of the measurement matrix using an orthogonal projector $y = A x + e$1, ensuring $y = A x + e$2 at every step (Song et al., 2017, Han et al., 2020).

The adaptive thresholding strategy is a central feature: $y = A x + e$3 may be linear, quadratic (e.g., $y = A x + e$4), or otherwise tailored to the problem structure. The feedback step, a least-squares correction within the selected support, re-allocates off-support energy and accelerates convergence.

2. Detailed Algorithmic Description

A generic INPMAT iteration proceeds as follows (Han et al., 2020, Han et al., 2021, Song et al., 2017):

1. Null-space Tuning:

$y = A x + e$5

where $y = A x + e$6 is the $y = A x + e$7-sparse feedback vector from the previous support selection step.

1. Adaptive Hard Thresholding and Feedback:
   • Form a proxy $y = A x + e$8.
   • Select $y = A x + e$9, the indices of the $A\in\mathbb{R}^{M\times N}$0 largest entries of $A\in\mathbb{R}^{M\times N}$1.
   • Compute the feedback:

   $A\in\mathbb{R}^{M\times N}$2

• Increment $A\in\mathbb{R}^{M\times N}$3.

1. Termination:
   • Stop if $A\in\mathbb{R}^{M\times N}$4 falls below a tolerance or after a maximum number of iterations (Han et al., 2020, Song et al., 2017).

For efficiency in large-scale settings, a suboptimal feedback implementation replaces explicit submatrix inversions by eigenvalue-based approximations (Song et al., 2017). The method generalizes to the matrix setting for completion, where singular value thresholding is used in lieu of hard thresholding.

3. Theoretical Properties

Convergence analysis of INPMAT is conducted in terms of the Restricted Isometry Property (RIP) and Preconditioned Restricted Isometry Property (P-RIP) constants of $A\in\mathbb{R}^{M\times N}$5 (Han et al., 2020, Song et al., 2017, Esmaeili et al., 2016):

• Contraction Guarantee: If $A\in\mathbb{R}^{M\times N}$6, where $A\in\mathbb{R}^{M\times N}$7 and $A\in\mathbb{R}^{M\times N}$8 are the preconditioned and standard RIP constants respectively, the sequence $A\in\mathbb{R}^{M\times N}$9 contracts linearly up to noise (Han et al., 2020).
• Support Refinement: Adaptive growth of $M\ll N$0 allows rapid inclusion of true support, resulting in faster contraction factors compared to non-adaptive schemes.
• No Prior Sparsity Required: The method does not require advance knowledge of $M\ll N$1; $M\ll N$2 grows adaptively until the residual is below threshold (Han et al., 2020, Song et al., 2017).
• Computational Complexity: Per-iteration cost is $M\ll N$3 (Han et al., 2020), significantly lower than that of canonical greedy or convex algorithms at early iterations.

4. Practical Applications and Extensions

INPMAT's framework has been successfully applied to a range of structured signal and data recovery problems:

• Sparse Recovery in Compressed Sensing: Demonstrates strong phase transition performance, reliably recovering $M\ll N$4-sparse vectors from $M\ll N$5 measurements with high probability, outperforming LASSO, Orthogonal Matching Pursuit, and related methods (Esmaeili et al., 2016).
• Low-Rank Matrix Completion: The MIMAT extension incorporates singular value thresholding and null-space projections, achieving minimal RMSE at lower sampling ratios than Soft-Impute, SVT, and SL0 (Esmaeili et al., 2016).
• Joint Sparsity Recovery in MMV Models: The OSNST+HT+$M\ll N$6–FB algorithm adapts INPMAT to Multiple Measurement Vectors (MMV), exploiting orthogonal subspace projections and functional feedbacks for temporally correlated signal ensembles (Han et al., 2021).
• Debiasing Neural Representations: Mechanistically analogous techniques iteratively remove linear signals of protected attributes from learned representations by projecting onto successive nullspaces; adaptive stopping mechanisms may be motivated by similar threshold-adaptation strategies (Ravfogel et al., 2020).

5. Empirical Performance and Comparative Results

Extensive numerical experiments corroborate the theoretical advantages of INPMAT:

• Sparse Recovery: INPMAT with $M\ll N$7 recovers up to $M\ll N$8 with $M\ll N$9 success rate for Gaussian signals. At higher sparsities, other algorithms degrade rapidly (Han et al., 2020).
• Signal-to-Noise Ratios: At 20% sampling and input SNR of $x$0, INPMAT achieves output SNR exceeding $x$1, outstripping LASSO, OMP, IMAT, CoSaMP, and SL0 (Esmaeili et al., 2016).
• Runtime: Suboptimal feedback variants achieve substantial runtime reductions (factor $x$2–$x$3) in large systems compared to classical NST+HT+FB or HTP, with negligible degradation in reconstruction accuracy (Song et al., 2017).
• Matrix Completion: In rank-10, $x$4 test matrices with high missing rates (up to $x$5), MIMAT attains RMSE $x$6 for far lower sampling than direct nuclear-norm approaches (Esmaeili et al., 2016).
• MMV/Joint Sparsity: OSNST+HT+$x$7–FB yields a superior balance of accuracy and computational efficiency versus SOMP, $x$8, SHTP, and TMSBL, and approaches the absolute support-recovery limit (Han et al., 2021).

6. Algorithmic Variants and Implementation Considerations

Several meaningful per-iteration and adaptive modifications have been developed:

Variant	Key Features	Reference
Suboptimal Feedback (AdptNST+HT+subOptFB)	Uses eigenvalue-based approximations to replace matrix inversions	(Song et al., 2017)
Functional Feedback (f-FB)	Feedback re-allocates tail energy to support, accelerating convergence	(Han et al., 2020, Han et al., 2021)
Matrix Completion (MIMAT)	Singular value thresholding, null-space projections	(Esmaeili et al., 2016)

The choice of $x$9 impacts both convergence rate and computational burden; quadratic or linear schedules are empirically favored. Parallelization and pre-computation of pseudoinverses or Gram matrices can further improve scalability.

7. Connections and Outlook

INPMAT unifies elements from greedy pursuit, iterative hard thresholding, and null-space correction schemes, while explicitly decoupling support selection from measurement consistency via adaptive regularization. Its methodological tenets have influenced advances in compressive learning, neural representation control, and large-scale convex optimization (Ravfogel et al., 2020, Esmaeili et al., 2016, Han et al., 2021, Han et al., 2020, Song et al., 2017).

A plausible implication is that similar adaptive, projection-based methods can be extended to broader classes of inverse problems and structured data analysis, including those with group, hierarchical, or manifold constraints. The relaxation of the need for prior sparsity knowledge and the integration of support-adaptive feedback mechanisms positions INPMAT and its variants as central tools for next-generation model-based signal processing.