---
title: Denoise-then-Retrieve Paradigm
url: https://www.emergentmind.com/topics/denoise-then-retrieve-paradigm
type: topic
---

# Denoise-then-Retrieve Paradigm

Searching arXiv for recent papers on the denoise-then-retrieve paradigm and adjacent formulations.
The denoise-then-retrieve paradigm is a family of workflows in which nuisance content is suppressed before, or as part of, a downstream retrieval operator. Across the supplied literature, “denoising” is not restricted to pixel smoothing: it can mean projecting an observation onto a learned noise manifold, filtering text-irrelevant video clips, compressing retrieved evidence into claim-relevant context, or using a denoiser-derived implicit prior during inverse reconstruction. The unifying principle is that retrieval quality degrades when the retriever is forced to operate on representations dominated by irrelevant, corrupted, or semantically mismatched information, and that performance can improve when the input to retrieval is first purified or selectively constrained [2508.11313].

## 1. Conceptual definition and scope

Within the supplied sources, the paradigm appears in both strict and hybrid forms. In its strict form, denoising is a distinct upstream stage that transforms the observation before a conventional retrieval or reconstruction step. The SPECT pipeline is an explicit example: a conditional GAN denoises sinograms, and only then are the cleaned sinograms reconstructed using Ordered Subset Expectation Maximization (OSEM) [2108.03903]. The video moment retrieval formulation is similarly explicit: text-irrelevant clips are filtered first, and conditional retrieval is then performed on purified multimodal representations [2508.11313].

A broader interpretation treats denoising as representation purification rather than signal restoration. In zero-shot composed image retrieval, Denoise-I2W first maps a reference image into a denoising pseudo-word token that excludes intention-irrelevant visual information, and retrieval is subsequently performed with a more faithful composed query representation [2410.17393]. In phase retrieval, the denoiser may operate as an implicit learned prior or as a regularizer within an iterative solver, so denoising and retrieval become interleaved rather than strictly sequential [2211.00884]; [2003.01792].

The supplied papers also show that the paradigm has neighboring but distinct forms. Decide Then Retrieve is described as a training-free framework that decides before retrieving, and more precisely as a hybrid that performs denoising both before retrieval and during retrieval selection rather than a pure denoise-then-retrieve pipeline [2601.03908]. Retrieve-Refine-Calibrate is even more clearly retrieve-then-denoise-then-verify: evidence is retrieved first, then refined to remove irrelevant information, and only after that is verification calibrated [2601.16555]. This suggests that the paradigm is best understood as a general design principle about controlling nuisance information, with different papers choosing different insertion points for the denoising operation.

## 2. Measurement-space denoising before reconstruction

A canonical formulation appears in imaging pipelines where retrieval means reconstructing a latent signal from noisy measurements. In SPECT imaging, the proposed method generates noisy sinograms from software phantoms by forward projection and Poisson corruption, trains a conditional GAN to map noisy sinograms to denoised sinograms using paired noisy/clean data, and reconstructs images with OSEM from either the original noisy sinograms or the GAN-denoised sinograms [2108.03903]. The retrieval stage is therefore not document ranking but tomographic recovery of the final image.

The architecture in that setting is a conditional GAN with an encoder-decoder generator and a discriminator trained to distinguish generated denoised sinograms from true clean sinograms. The generator is conditioned on the noisy sinogram, uses skip connections, and outputs a denoised sinogram; the discriminator has three convolutional layers and a final fully connected sigmoid output [2108.03903]. The paper reports denoising results on the Shepp-Logan-based evaluation for low, medium, and high noise, and compares standard OSEM reconstructions against OSEM applied to GAN-denoised sinograms.

| Setting | Metric summary | Reported values |
|---|---|---|
| Sinogram denoising | Low noise | MAPE 4.80%, MSE 0.0009, SSIM 0.975, PSNR 30.48 |
| Sinogram denoising | Medium noise | MAPE 5.83%, MSE 0.0010, SSIM 0.971, PSNR 29.91 |
| Sinogram denoising | High noise | MAPE 8.39%, MSE 0.0054, SSIM 0.930, PSNR 22.65 |

For the reconstruction stage, the paper reports that standard OSEM at low noise yields MSE 0.0082, SSIM 0.82, and PSNR 20.89, while the proposed denoise-then-retrieve pipeline yields MSE 0.0040, SSIM 0.89, and PSNR 23.94. At medium noise, the corresponding values are 0.0087/0.80/20.58 for standard OSEM and 0.0045/0.89/23.48 for the proposed method; at high noise, 0.0099/0.75/20.05 versus 0.0061/0.84/22.17 [2108.03903]. The authors state that the proposed method “significantly denoise[s] the sinograms” and “significantly improves the reconstructions.”

This formulation captures the most literal interpretation of the paradigm: improve the observation first, then apply the unchanged retrieval operator. A plausible implication is that such pipelines are especially attractive when a well-established downstream inverse method already exists and the principal bottleneck is observation quality rather than reconstruction architecture.

## 3. Representation denoising in multimodal retrieval

In text-driven Video Moment Retrieval, the denoise-then-retrieve formulation targets semantic noise rather than sensor noise. The paper argues that standard methods encode all video clips, including irrelevant ones, disrupting multimodal alignment and hindering optimization. DRNet operationalizes the alternative by first filtering text-irrelevant clips from videos and then retrieving the target moment using purified multimodal representations [2508.11313].

The first core module is Text-Conditioned Denoising (TCD). Video clips are encoded by concatenating CLIP and SlowFast features, text by the CLIP text encoder, and cross-attention injects query information into each video clip. A Context Interaction Operator built from bidirectional Mamba blocks then performs multimodal context integration. From the resulting contextualized features, dynamic kernels derived from pooled text features compute clip-wise text-clip alignment scores, which are thresholded with $\mu$ to form a binary noise mask. Applying this mask yields purified visual features $\hat{\boldsymbol f_v}$ [2508.11313].

The second module, Text-Reconstruction Feedback (TRF), provides auxiliary supervision. TRF reconstructs a sentence-level query embedding from the purified visual representation and aligns it with the pooled text embedding via a cosine-distance semantic consistency loss with $\lambda_t=2$. The paper explicitly frames this as preventing overly aggressive masking: if the purified video cannot reconstruct the text, the mask is probably too harsh or inaccurate [2508.11313]. Retrieval is then performed on the purified multimodal representation rather than on the raw clip sequence.

The reported results on QVHighlights test are 66.73 R1@0.5, 50.52 R1@0.7, 64.17 mAP@0.5, 45.79 mAP@0.75, and 43.73 mAP@Avg. On Charades-STA, DRNet reaches 60.86 R1@0.5 and 39.78 R1@0.7 with SlowFast+CLIP features, and 69.75 mIoU with VGG features; with VGG it also reports 59.03 R1@0.5 and 36.26 R1@0.7 [2508.11313]. The ablations attribute the largest drop to removing TCD, while removing TRF also hurts performance, and replacing Mamba-based CIOs with standard Transformers leads to a noticeable drop.

An analogous but distinct form appears in zero-shot composed image retrieval. Denoise-I2W addresses the mismatch that ordinary image-caption pre-training maps a reference image into a pseudo-word token that often carries intention-irrelevant content. Its pseudo triplet construction module automatically creates triples of pseudo-reference image, pseudo-manipulation text, and target image; its pseudo-composed mapping module then trains the mapping network to produce a denoising pseudo-word token that, when composed with the manipulation text, aligns with the target image [2410.17393]. The method is described as model-agnostic and annotation-free.

| Task | Baseline-to-Denoise-I2W change | Reported values |
|---|---|---|
| CIRR with Context-I2W | R@1/R@5/R@10/R@50 | 25.6/55.1/68.5/89.8 to 26.9/57.2/69.8/90.6 |
| ImageNet domain conversion | Avg. R@10/R@50 | 12.9/27.6 to 14.3/30.6 |
| COCO object composition | R@1/R@5/R@10 | 13.5/28.5/38.1 to 16.1/33.0/43.5 |

The paper further states that integrating Denoise-I2W with existing best models obtains consistent and significant performance boosts ranging from 1.45% to 4.17% without increasing inference costs [2410.17393]. In both DRNet and Denoise-I2W, denoising is a query-conditioned suppression of semantically irrelevant content before retrieval, not low-level restoration.

## 4. Denoiser priors in phase retrieval

Phase retrieval provides a more mathematical instance of the paradigm, where “retrieve” refers to recovering a signal from Fourier magnitude measurements. The supplied papers present two related formulations. Alternating Phase Langevin Sampling (APLS) treats the denoiser as an implicit prior and alternates between denoiser-guided posterior sampling and phase estimation [2211.00884]. “When deep denoising meets iterative phase retrieval” embeds a deep denoiser inside ADMM/HIO-style splitting via Regularization-by-Denoising (RED), creating hybrid denoise-then-retrieve and denoise-while-retrieve algorithms [2003.01792].

APLS starts from the phase retrieval model
$$
b = |Ax^*| + w,
$$
and for the Fourier case uses a noisy Fourier magnitude model with $4\times$ oversampled Fourier measurements. The outer loop updates the phase by $p_t \gets \mathrm{Ph}(Ax_{t-1})$, forms phase-augmented linearized measurements $x^c = p_t \odot b$, and calls a constrained Langevin posterior sampler with update
$$
x \gets x + h d_x + \gamma z,
$$
where
$$
d_x \gets (I - A^T A)D(x) + A^T(x^c - Ax).
$$
This is precisely described as a Denoise-then-Retrieve method: first denoise or sample using the implicit denoiser prior, then retrieve the phase-consistent estimate, and continue alternating [2211.00884].

The reported results emphasize robustness to distribution shift. On natural images at $\alpha=2$, HIO scores 18.72, prDeep 30.02, and APLS 29.34; at $\alpha=3$, 19.04, 28.62, and 28.69; at $\alpha=4$, 18.56, 28.05, and 27.67. On unnatural images at $\alpha=2$, HIO scores 19.75, prDeep 21.9, and APLS 22.32; at $\alpha=3$, 21.24, 24.08, and 25.5; at $\alpha=4$, 18.57, 22.29, and 22.44. On the Shenzhen chest X-ray dataset at $\alpha=3$, prDeep achieves average PSNR 33.68, while APLS reaches 35.04 [2211.00884].

The RED-based formulation begins from
$$
y^2 = M(x) + w = |Ax|^2 + w,
$$
and uses the RED regularizer
$$
R(x) = \frac{\lambda}{2}\langle x, x - D(x)\rangle.
$$
The paper proposes RED-ITA-F and RED-ITA-S, both ADMM-based hybrids that retain the physics of iterative phase retrieval while injecting a learned prior through DnCNN denoisers [2003.01792]. RED-ITA-F relaxes the Fourier measurement term; RED-ITA-S relaxes the oversampling or signal-consistency constraint. The paper explicitly states that with random initialization, Deep-ITA-F/S achieve the best PSNR and SSIM by large margins, and under random init and $\alpha=4$, Deep-ITA methods achieve around 61 dB PSNR overall versus 28 dB for DnCNN-ADMM and 24–34 dB for HIO/OSS/prDeep [2003.01792].

A related but more general denoising formulation appears in “Reconstructing the Noise Manifold for Image Denoising,” which argues that it is easier and more reliable to model the residual image than the full clean image space, and therefore better to constrain a denoiser to remove only information spanning the manifold of the residual image [2002.04147]. The paper does not perform retrieval itself, but the supplied details describe it as a denoise-then-retrieve-compatible denoiser because it is explicitly designed to preserve image content and structure while removing realistic sensor noise. Reported results include, on DnD, 49.90 dB/0.9861 RAW and 41.50 dB/0.9759 sRGB for the blind ResNet backbone, and 50.91 dB/0.9873 RAW and 42.11 dB/0.9775 sRGB for the non-blind ResNet backbone [2002.04147].

Taken together, these papers show that in inverse problems the paradigm can mean either a sequential denoise-then-solve recipe or an alternating algorithm in which denoising acts as a learned prior inside the retrieval loop.

## 5. Hybrid retrieval pipelines and denoising by selection or refinement

Not all systems in the supplied literature are pure denoise-then-retrieve methods, but several are closely related because they interpret denoising as filtering retrieval necessity or evidence quality. Decide Then Retrieve (DTR) changes the standard RAG order: the model first attempts a parametric answer, computes a generation uncertainty score
$$
u=-\frac{1}{T}\log P(\hat{a}\mid q),
$$
and triggers retrieval only when uncertainty exceeds a threshold such as $u=0.001, 0.005, 0.01$ [2601.03908]. The paper states that no-retrieval accuracy drops monotonically as uncertainty increases, validating uncertainty as a proxy for parametric correctness.

DTR then uses dual-path retrieval with adaptive information selection. One path retrieves with the original query embedding, the other with a pseudo-context generated by the prompt “Write a passage to answer this question,” and the candidate documents are scored jointly by
$$
s(d)=s_1(d,q)+s_2(d,p).
$$
The paper further gives a geometry-motivated rule
$$
s(d)=\cos(\theta_1+\theta_2)= s_1 s_2 - \sqrt{1-s_1^2}\sqrt{1-s_2^2}.
$$
The supplied description explicitly characterizes this as a hybrid rather than a pure denoise-then-retrieve method: DTR performs denoising or filtering at the decision stage by avoiding retrieval for low-uncertainty queries and at the retrieval stage by suppressing passages not jointly supported by both views [2601.03908].

Retrieve-Refine-Calibrate (RRC) positions the denoising step after retrieval. Instead of decomposing a claim into sub-claims, it extracts up to three entities with LUKE, retrieves evidence with BM25, refines the retrieved evidence into a concise claim-relevant paragraph $K^*$, and calibrates low-confidence predictions using thresholds $\theta = 0.90$ for HOVER and $\theta = 0.85$ for FEVEROUS-S [2601.16555]. The paper’s central claim is that decomposition can inject incorrect intermediate entities or relations, so the bottleneck is noisy intermediate representations rather than weak reasoning ability.

| Framework | Retrieval order | Denoising operation |
|---|---|---|
| DTR | Decide before retrieval | Uncertainty-guided triggering and adaptive evidence selection |
| RRC | Retrieve before denoising | Claim-specific evidence refinement and selective calibration |

The main results reported for RRC are Macro-F1 scores of 69.33 on HOVER 2-hop, 63.98 on HOVER 3-hop, 60.95 on HOVER 4-hop, and 72.55 on FEVEROUS-S [2601.16555]. Ablations show drops when removing the evidence refinement module, supporting the claim that retrieval noise is a major problem in complex claims. These systems broaden the paradigm: denoising can target the decision to retrieve, the retrieved evidence itself, or both.

## 6. Common principles, misconceptions, and boundaries

Across the supplied papers, several common principles recur. First, denoising is usually task-conditioned. DRNet’s mask is conditioned on text and suppresses text-irrelevant clips [2508.11313]. Denoise-I2W keeps only image information useful for the manipulation intent [2410.17393]. RRC refines evidence with respect to the original claim rather than using generic compression [2601.16555]. This suggests that the paradigm is not merely about reducing variance; it is about removing information that is irrelevant for a specific downstream retrieval objective.

Second, the denoising stage is often constrained so that useful content is not discarded. DRNet uses TRF to ensure purified video still reconstructs sentence semantics [2508.11313]. The noise-manifold denoiser argues that the generator should remove only information spanning the manifold of the residual image, which is presented as a way to reduce oversmoothing and preserve detail [2002.04147]. In phase retrieval, the denoiser is balanced against measurement consistency rather than used as a free-standing image translator [2211.00884]; [2003.01792].

Third, a common misconception is that denoise-then-retrieve always denotes a simple prefilter followed by a fixed retriever. The supplied literature contradicts that narrow reading. In APLS, denoising and retrieval alternate inside a single loop [2211.00884]. In RED-ITA, the denoiser is part of the optimization objective itself [2003.01792]. In DTR and RRC, denoising is implemented as decision filtering or evidence refinement rather than observation restoration [2601.03908]; [2601.16555].

A further boundary condition concerns evidentiary support. One supplied record, “Reason-Then-Retrieve for CoVR-R with Structured Edit Prompts and Dense-Sparse Fusion,” includes an abstract describing a reason-then-retrieve pipeline and reported retrieval metrics, but the accompanying details state that the provided content is only a LaTeX rebuttal or template document and therefore does not contain the method body or retrieval formulas [2606.02450]. From the supplied text alone, its relation to a denoise-then-retrieve interpretation cannot be specified beyond noting that it belongs to an adjacent family in which reasoning precedes retrieval.

Overall, the denoise-then-retrieve paradigm denotes a methodological commitment: retrieval should operate on purified inputs, purified representations, or purified evidence rather than on raw, noise-dominated signals. The supplied literature shows this commitment instantiated in medical imaging, phase retrieval, video moment retrieval, zero-shot composed image retrieval, adaptive retrieval-augmented generation, and fact-checking, with the exact locus of denoising varying by task and model class [2108.03903].

Source: https://www.emergentmind.com/topics/denoise-then-retrieve-paradigm