---
title: Info-Theoretic Recovery Thresholds
url: https://www.emergentmind.com/topics/information-theoretic-recovery-thresholds
type: topic
---

# Info-Theoretic Recovery Thresholds

Information-theoretic recovery thresholds delineate the minimal requirements—specifically, sample sizes, measurement regimes, or SNR conditions—under which it is possible to reliably recover structured objects (such as supports, communities, permutations, or network structures) from noisy or incomplete data, regardless of computational complexity. These thresholds are central in signal processing, statistical learning, network science, and modern high-dimensional statistics, as they define fundamental phase transitions for inference problems. This article surveys canonical models, formal threshold criteria, core methodologies for proving sufficiency and necessity, and key regimes where information-theoretic and algorithmic recovery diverge, with emphasis on sparse signal recovery from heterogeneous data as developed in the mixed-quality setting [2605.10713], as well as representative theorems across structured models.

## 1. Canonical Models and Problem Settings

Information-theoretic thresholds arise in problems where an unknown combinatorial structure generates observable data corrupted by randomness, noise, or partial observation. Prototypical examples include:

- **Sparse vector recovery**: Identifying the support of a $k$-sparse vector $\beta^* \in \mathbb{R}^p$ from $n$ linear measurements $Y = X\beta^* + w$, often under i.i.d. Gaussian noise $w \sim \mathcal{N}(0,\sigma^2 I_n)$. Variants include *mixed-quality* settings, where measurements have heterogeneous noise variances [2605.10713], [0702301].
- **Community detection in Stochastic Block Models (SBM)**: Partitioning $n$ nodes into $r$ communities given observed edge weights or adjacency matrices, with information-theoretic thresholds in terms of Renyi divergence [1509.06418].
- **Graph alignment**: Recovering a hidden permutation between two correlated Erdős–Rényi graphs under specified correlation structure [1711.06783], [2406.05428].
- **Rank minimization**: Recovering a low-rank matrix $X_0\in\mathbb{R}^{n\times n}$ from linear projections, with thresholds for nuclear norm minimization established via geometric and information-theoretic criteria [1011.6326].
- **Recovery in structured networks**: Identification of parent sets in diffusion networks [1601.07932], edge recovery in dynamic models [2504.04748], and community detection in higher-order (hypergraph) SBM [2101.12369].

Each model defines a hypothesis class $\mathcal{H}$ (e.g., all supports of size $k$, all community assignments, all permutations) and an information channel $P_{\text{data}|\mathcal{H}}$. The threshold is the minimal value of a problem parameter (samples, SNR, etc.) above which recovery is possible in probability as $p,n \to \infty$.

## 2. Formal Threshold Criteria: Sufficient and Necessary Conditions

Thresholds are typically characterized in terms of two-sided bounds:

- **Sufficiency**: There exists a decoder (possibly of unbounded complexity) that achieves recovery with failure probability vanishing as $n,p\to\infty$ if the resources (e.g., samples, SNR) exceed a critical value.
- **Necessity**: For any decoder, failure probability is bounded away from zero (or tends to one, i.e., strong impossibility) if resources are below the critical value.

For sparse support recovery (mixed-quality measurements), key results from [2605.10713] are:

- Let $\beta^* \in \{0,1\}^p$ have exactly $s$ nonzeros, with $n_1$ high-quality (noise $\sigma_1^2$) and $n_2$ low-quality (noise $\sigma_2^2 > \sigma_1^2$) measurements.
- Critical (homogeneous) sample threshold: $n_\star = 2 s \log(p/s)$ for sparse regime ($s=o(p)$).

**Agnostic (unknown variances) sufficient condition**: For any fixed $\delta>0$, some $\epsilon>0$,
\[
n_1 \cdot \log\!\left(1 + \frac{\delta(2\sigma_2^2-\sigma_1^2)s}{2\sigma_2^4}\right)
+ n_2 \cdot \log\left(1 + \frac{\delta s}{2\sigma_2^2}\right)
\geq (1+\epsilon) n_\star
\implies \mathbb{P}(\text{support error} < 2\delta s) \to 1.
\]
**Informed (known variances/MLE) condition**:
\[
n_1 \cdot \log\left(1 + \frac{\delta s}{2\sigma_1^2}\right)
+ n_2 \cdot \log\left(1 + \frac{\delta s}{2\sigma_2^2}\right)
\geq (1+\epsilon) n_\star
\implies \mathbb{P}(\text{support error} < 2\delta s) \to 1.
\]
The thresholds for *exact* support recovery in the canonical homogeneous case are $n \gtrsim 2 s \log(p/s)$ at high SNR, with refined scaling at finite SNR [0702301].

For SBM [1509.06418], the sharp threshold is set by the order-1/2 Rényi divergence $I$ between within- and between-community edge distributions:
\[
n I / \log n > 1 \Longleftrightarrow \text{exact recovery possible}.
\]
In graph alignment [1711.06783], exact recovery in sparsely correlated ER graphs requires the intersection marginal $p_{11}$ to satisfy $p_{11} \gtrsim (\log n)/n$.

## 3. Quantitative Expressions: Price of Quality and Sample Trade-offs

A salient feature of mixed-quality measurement regimes is the introduction of *linear trade-offs* between high- and low-quality samples, formalized as the "Price of Quality":
\[
\alpha_1 n_1 + \alpha_2 n_2 \geq n_\star
\]
where $\alpha_i$ are coefficients depending on the noise variances and regime (see above). The *price of quality* $\gamma$ is the number of low-quality samples needed to replace one high-quality sample:
- **Agnostic (unknown variance):**
\[
\gamma_{\rm agnostic}(s,\sigma_1,\sigma_2) =
\frac{\log\left(1 + \frac{\delta(2\sigma_2^2-\sigma_1^2)s}{2\sigma_2^4}\right)}
{\log\left(1+\frac{\delta s}{2\sigma_2^2}\right)}
\leq 2
\]
- **Informed (variance known):**
\[
\gamma_{\rm informed}(s,\sigma_1,\sigma_2) =
\frac{\log\left(1 + \frac{\delta s}{2\sigma_1^2}\right)}{\log\left(1 + \frac{\delta s}{2\sigma_2^2}\right)}
\]
This captures that, for agnostic decoders, even very noisy measurements retain significant value (at most a factor 2 gap), whereas for an informed decoder, high-quality measurements can asymptotically substitute for arbitrarily many low-quality measurements.

## 4. Proof Techniques: Union Bounds, Large Deviations, and Fano's Inequality

Information-theoretic sufficiency is typically established via a *union bound* (over all incorrect hypotheses) coupled with Chernoff-type or moment-generating function (MGF) large deviation bounds for log-likelihood differences or loss gaps. The dominant error probability per incorrect model is suppressed exponentially in the number of measurements, counterbalanced by the (exponentially large) number of candidates. In [2605.10713], for each incorrect support at Hamming distance $2\delta s$, probability that it appears preferable to the truth is controlled as:
\[
\Pr\{\Delta(S)\leq 0\} \leq A^{-n_1/2} B^{-n_2/2},
\]
where $A, B$ are explicit SNR-dependent constants.

Necessity (impossibility) is typically produced via Fano's inequality, bounding the mutual information between the structured object and data, and showing that channel capacity is insufficient to distinguish among all possibilities. In the mixed-quality and related settings, the channel capacity per sample is replaced by expressions involving $\log(1 + \text{SNR})$ or generalizations for weighted/nonscalar data [0702301], [1509.06418].

## 5. Regimes and Phenomena: SNR, Partial Recovery, and Algorithmic Gaps

- **SNR Regimes**: At high SNR, all models collapse to information-theoretic minimal sample thresholds, i.e., $n \sim 2s\log(p/s)$. At low SNR or in highly heterogeneous regimes, informed decoders optimally drop low-quality data; agnostic decoders cannot but retain significant efficiency.
- **Partial Recovery**: Allowing a fraction $\alpha$ of incorrect entries reduces the sample complexity, with sharp thresholds still typically governed by $k\log(p/k)$-type scaling but with smaller leading constants [1501.07440], [1002.4458], [1901.10647], [2406.05428].
- **Statistical vs. Computational Gaps**: In certain models, notably planted dense cycles or submatrix localization, the information-theoretic threshold can be strictly below what is achievable efficiently—typically detected via low-degree polynomial barrier arguments [2402.00305], [2402.12515], [2101.12369].
- **Adaptive Sensing**: In group testing and 1-bit compressive sensing, adaptivity cannot reduce sample complexity, while in subsampling sparsity regimes, mild performance gains are possible [1402.5731].

## 6. Applications and Practical Implications

- **Budget allocation in experiments**: The derived price-of-quality criteria provide concrete design guidelines for allocating costly high-quality vs. cheap low-quality sensors, with rigorous backing for when dropping noisy data is information-theoretically justified [2605.10713].
- **Network structure learning**: For diffusion models or network dynamics, thresholds clarify the number of observed cascades or temporal trajectories needed (typically $\Omega(k\log p)$) for reliable parent-set determination [1601.07932], [2504.04748].
- **Matrix recovery**: Null space analysis and phase diagrams of nuclear norm minimization reveal explicit oversampling factors compared to degrees of freedom, quantifying the sharp transition to recoverability [1011.6326].
- **Quantum information**: In quantum error correction, information-theoretic recovery thresholds are characterized via mutual trace distance and coherent information criteria, with optimality of Petz and Schumacher–Westmoreland decoders established [2603.06520], [1504.04449].

## 7. Outlook and Open Directions

Contemporary research focuses on sharpening constant factors in threshold conditions, exploring finer-grained partial recovery phases, characterizing tight statistical-computational gaps in models with complex dependency structures (e.g., geometric random graphs, hypergraphs), and establishing robust analogs of these thresholds under additional or more general constraints (e.g., missing data, heterogeneous graphons, quantum channels) [2406.05428], [2207.11002], [2101.12369]. In many cases, algorithmic thresholds (for tractable decoding) exhibit order-wise or constant factor gaps from their information-theoretic counterparts, motivating ongoing research on efficient algorithms that close these gaps or prove them intrinsic.

Source: https://www.emergentmind.com/topics/information-theoretic-recovery-thresholds