Universal Sampling Discretization in Signal Processing

• Universal Sampling Discretization is a method that converts continuous or high-dimensional signals into discrete sparse representations for accurate reconstruction.
• It uses key metrics such as mutual coherence, RIP, and global 2-coherence to underpin its recovery guarantees in both noiseless and noisy environments.
• Greedy algorithms like OMP and its thresholded variants leverage these discretization principles to achieve scalable and efficient signal recovery.

Universal Sampling Discretization is a concept in signal processing and compressive sensing referring to the representation of continuous, high-dimensional, or analog signals using discrete samples in a fashion that enables accurate and robust recovery of structured or sparse signals. In modern frameworks, universal discretization typically centers around deterministic or random linear measurements, reduction to finite dictionaries or bases, and the development of algorithmic guarantees—often expressed through metrics such as coherence indices and restricted isometry constants. Recent progress has illuminated new structural parameters bridging different families of recovery guarantees and has refined known conditions for exact and stable signal reconstruction in underdetermined linear systems.

1. Formulation in Sparse Signal Recovery

Central to universal sampling discretization is the convexification of analog or continuous signal families into sparse vectors over discrete, often redundant dictionaries. Given a signal $x \in \mathbb{R}^d$ with $\|x\|_0 \leq s$ (i.e., $x$ is $s$-sparse), and a linear measurement matrix (dictionary) $\Phi \in \mathbb{R}^{n \times d}$ with normalized columns $\|\phi_i\|_2 = 1$, the measurement process yields $y = \Phi x + e$ with noise $\|e\|_2 \leq \epsilon$. The task then reduces to reconstructing $x$ given $y$ under generic conditions suitable for universal signal classes. This paradigm leverages sampling theorems which specify sufficient conditions on $\Phi$ (often randomly constructed or with universal deterministic structure) to ensure accurate reconstruction for all $s$-sparse signals.

2. Metrics Governing Universal Recovery: Coherence and RIP

Sampling discretization theory hinges on quantitative metrics capturing the suitability of $\Phi$ for universal recovery. Two central notions are the mutual coherence $M(\Phi)$ and the restricted isometry constant (RIC) $\delta_s$:

• Mutual Coherence: Defined as $$M(\Phi) = \max_{i \neq j} |\langle \phi_i, \phi_j \rangle|$$, characterizing the worst-case pairwise correlation between dictionary atoms.
• Restricted Isometry Constant (RIC): The smallest $\delta_s$ for which

$$(1-\delta_s)\|v\|_2^2 \leq \|\Phi v\|_2^2 \leq (1+\delta_s)\|v\|_2^2$$

holds for all $v$ with $\|v\|_0 \leq s$.

A third, recently introduced metric, global 2-coherence $\nu_k(\Phi)$ or $\mu_{2,k}(\Phi)$ (Yang et al., 2014, Yang et al., 2013), provides a bridge between the above:

$$\nu_k(\Phi) = \max_{i} \max_{\Lambda \subseteq [d]\setminus \{i\}, |\Lambda| \leq k} \left( \sum_{j \in \Lambda} \langle \phi_i, \phi_j \rangle^2 \right)^{1/2}$$

These parameters satisfy the sharp chain of inequalities:

$$M(\Phi) \leq \nu_{k-1}(\Phi) \leq \delta_k \leq \sqrt{k-1} \, \nu_{k-1}(\Phi) \leq (k-1) M(\Phi)$$

which tightly couple the effective “universality” of the sampling scheme to both local and global incoherence.

3. Greedy Reconstruction Algorithms and Universal Discretization

Sparse recovery from universal sampling leverages greedy algorithms such as Orthogonal Matching Pursuit (OMP), its weak/thresholded variants (WOMP, OMPT), and related methods. Standard OMP selects, at each iteration, the dictionary column most correlated with the residual; WOMP relaxes this by accepting any atom whose correlation exceeds a $\rho$-fraction of the optimal. OMPT introduces a threshold $\tau$ and accepts any atom with $|\langle r, \phi_i \rangle| \geq \tau \|r\|_2$. These algorithms crystallize the move from an analog selection principle to practical, universal, discretized procedures.

The exact recovery guarantees for such algorithms are universally quantified over all $k$-sparse $x$, provided the metric conditions (involving $M$, $\nu_k$, or $\delta_k$) are satisfied. For instance, for noiseless recovery via standard OMP, a sufficient universal bound is:

$\delta_k + \sqrt{k} \, \delta_{k+1} < 1$

which improves upon previously known thresholds (Yang et al., 2014). For OMPT,

$\delta_s + \sqrt{s} \, \mu_{2,s} < 1$

guarantees exact support recovery of all $s$-sparse signals (Yang et al., 2013).

4. Thresholding and Complexity Reduction

Universal sampling discretization also addresses the computational bottleneck of recovery algorithms. OMPT/thresholding methods, rather than scanning for the maximal inner product, select any index crossing a fixed threshold. This reduces computational cost from $O(sd)$ (standard OMP) to $O(s^2)$, since the expected number of inner products per iteration is approximately $1/\tau^2$, which can be set to $O(s)$ for appropriate $\tau$ (Yang et al., 2013). When the measurement matrix is highly incoherent, thresholding can select multiple atoms per iteration, further reducing the total number of steps required for recovery.

5. Stability and Generalization to Noisy and Infinite-Dimensional Settings

Universal discretization guarantees extend to the noisy case, assuming bounded noise and signal amplitudes. For example, with $y = \Phi x + e$, $\|e\|_2 \leq \epsilon$, and with proper choices of metric bounds and threshold parameters, algorithms such as OMPT recover $x$ with stability:

$$\| \hat{x} - x \|_2 \leq \frac{\epsilon}{\sqrt{1-\delta_s}}$$

and exact recovery of the support, provided

$\epsilon < C \cdot x_{\min}$

where $x_{\min}$ is the smallest nonzero in $x$ and $C$ is a function of the metrics (Yang et al., 2013).

A further extension considers generalized Hilbert spaces, where, assuming $f^\epsilon$ with $\|f - f^\epsilon\| \leq \epsilon$ and $f^\epsilon / C$ in the convex hull of $\{\pm \phi_i\}$, OMPT produces residuals of at most $\epsilon + \tau C$ in $O(s \log s)$ iterations for $\tau^2 \approx 1/s$ (Yang et al., 2013). This demonstrates the universality of the discretization and recovery principles beyond finite-dimensional $\ell_2$ settings.

6. Comparative Performance and Practical Implications

Numerical comparisons in (Yang et al., 2013) reveal that thresholded greedy approaches such as OMPT exhibit nearly identical exact recovery success rates to standard OMP for moderate sparsity levels, while significantly reducing the total computational cost. For example, for a concatenated identity and Fourier dictionary, with mutual coherence $M = 1/\sqrt{128}$ and threshold $\tau = \sqrt{M}$, OMPT matches OMP’s empirical phase transition but consumes only about $128$ inner-product evaluations per iteration regardless of sparsity, while OMP’s cost scales with $k$ and $d$.

A plausible implication is that universal discretization frameworks grounded in thresholded selection and exploiting global coherence metrics can yield scalable, robust algorithms applicable to broad classes of analog or high-dimensional signals.

7. Limitations and Open Questions

The guarantees supplied by universal sampling discretization principles are sufficient but in general not known to be necessary. Known results require normalized columns, exact sparsity, and uniform noise bounds, and do not address sharpness or optimality of the metric inequalities—the delineation of necessary and sufficient conditions remains an open question (Yang et al., 2014). There is no full theoretical characterization of universality for arbitrary deterministic matrices outside the specified coherence/RIP regimes. Empirical performance may exceed theoretical predictions for certain random matrices, suggesting room for further refinement of universal discretization guarantees.