ReTiDe: Real-Time Denoise Systems
- Real-Time Denoise (ReTiDe) is a systems-level approach that integrates compact U-Net-style CNNs and hardware acceleration to achieve online, low-latency denoising.
- Key design patterns include wavelet-domain reformulation, compressed kernel prediction, and one-step residual refinement to meet fixed inference budgets.
- The framework is applied across motion-picture processing, thermal infrared robotics, medical imaging, and speech enhancement, delivering significant efficiency gains.
Real-Time Denoise (ReTiDe) designates, in its narrow sense, an open-source, blind, hardware-accelerated image denoising system that quantizes a compact U-Net-style CNN to INT8 and deploys it on AMD Deep Learning Processor Unit-based Alveo FPGAs for motion-picture workflows (Li et al., 4 Oct 2025). In the surrounding literature, the same expression is also used more loosely to describe the objective of real-time denoising under strict latency, power, and integration constraints across thermal infrared robotics, Monte Carlo rendering, low-dose CT, live video, and speech enhancement (Rhee et al., 18 Jun 2026, Wang et al., 16 May 2026, Rota et al., 3 Mar 2026). The unifying technical problem is not denoising alone, but denoising that remains online-ready, numerically stable, and deployment-compatible when noise is domain-specific and inference budgets are fixed at the frame, slice, or hop level.
1. Terminology, scope, and reported operating regimes
A common misconception is that ReTiDe denotes a single denoising architecture. The supplied literature does not support that reading. It names a DPU-based FPGA system explicitly as ReTiDe (Li et al., 4 Oct 2025), whereas several other works are described as advancing the goal of real-time denoising without adopting the acronym as a formal method name, or are mapped to it only conceptually in later technical syntheses (Rhee et al., 18 Jun 2026, Fan et al., 2022). This suggests that ReTiDe is best understood as a systems-level regime defined by latency, throughput, and deployment constraints rather than by one canonical model family.
Across domains, the reported operating points vary, but they are all anchored in online or low-latency execution. Thermal infrared denoising with TIDY is reported at approximately $34.3$ FPS at (Rhee et al., 18 Jun 2026). RDDM reports about $15$–$20$ ms to denoise a single low-dose CT slice (Wang et al., 16 May 2026). PocketDVDNet reports $0.017$ s per $480$p frame on an RTX 4090 (Morris et al., 23 Jan 2026). TVF operates causally on non-overlapping $1024$-sample frames at $48$ kHz, corresponding to approximately $21$ ms algorithmic latency (Rota et al., 3 Mar 2026). TRU-Net reports 0 ms lookahead and per-frame processing times of 1 ms on Intel i5-5257U and 2 ms on Intel i7-6700HQ (Choi et al., 2021).
| Domain | Representative system | Reported real-time characteristic |
|---|---|---|
| Motion-picture processing | ReTiDe (Li et al., 4 Oct 2025) | 3,746.1 GOPS; 203.59 GOPS/W; 0.004 s inference |
| Thermal infrared robotics | TIDY (Rhee et al., 18 Jun 2026) | 34.3 FPS at 3 |
| Low-dose CT | RDDM (Wang et al., 16 May 2026) | 15–20 ms per 4 slice |
| Real camera video | PocketDVDNet (Morris et al., 23 Jan 2026) | 0.017 s per 480p frame |
| Speech enhancement | TVF (Rota et al., 3 Mar 2026) | 5 ms algorithmic latency |
These figures indicate that “real-time” is modality-relative. In imaging systems it is typically expressed as FPS, per-frame latency, or GOPS/W; in speech systems it is expressed as lookahead, hop size, and per-frame compute time; in medical imaging it is often expressed as one-step slice-wise inference.
2. Recurring design patterns in ReTiDe-style systems
Despite the diversity of domains, several recurrent architectural patterns appear. One is the relocation of computational burden away from runtime. TIDY reformulates thermal infrared denoising in the wavelet domain, so that the encoder performs a Haar DWT at 6, the backbone operates on 7, and the decoder reconstructs with IDWT (Rhee et al., 18 Jun 2026). Because level-8 DWT maps 9 to $15$0, the paper estimates an approximately $15$1 speedup and reports empirical GFLOPS reduction from approximately $15$2 to approximately $15$3 with DWT+FiLM at $15$4 (Rhee et al., 18 Jun 2026).
A second pattern is compressed prediction followed by cheap deterministic reconstruction. In real-time Monte Carlo denoising, the weight-sharing kernel prediction network predicts a compact single-channel encoding of the per-pixel kernel map rather than directly predicting all $15$5 kernel weights, and reconstructs the kernel through unfolding and softmax normalization (Fan et al., 2022). The reconstructed kernel weights are
$15$6
and filtering is then
$15$7
This reduces network output throughput and supports $15$8 ms or $15$9 ms inference at $20$0, depending on whether the encoder uses three or six RepVGG blocks (Fan et al., 2022).
A third pattern is the absorption of iterative refinement into training. RDDM explicitly contrasts itself with diffusion and flow baselines by incorporating multi-step distribution evolution into the training dynamics through a residual drifting field, thereby enabling one-step denoising (Wang et al., 16 May 2026). In residual space, the conditional drift is
$20$1
with kernelized attraction and repulsion terms computed over real and generated residual mini-batches. The test-time rule is correspondingly simple: $20$2 This is a distinctive ReTiDe pattern: expensive distributional dynamics are internalized during optimization rather than executed during inference (Wang et al., 16 May 2026).
A fourth pattern is explicit low-latency inductive bias. TVF constrains enhancement to a differentiable $20$3-band IIR filter cascade whose coefficients are predicted frame-by-frame by a lightweight neural backbone (Rota et al., 3 Mar 2026). The time-varying IIR recurrence is
$20$4
TRU-Net similarly hard-codes causality by using frequency-axis convolutions, a frequency-axis GRU within frame, and a uni-directional time-axis GRU across frames, with $20$5 ms lookahead (Choi et al., 2021). In both cases, interpretability and bounded latency are treated as first-order design constraints rather than secondary deployment concerns.
3. Thermal infrared and video denoising
In thermal infrared robotics, TIDY is presented as a wavelet-domain, robotics-ready denoiser trained on real clean-noisy TIR pairs from SCaN-TIR, a dataset of $20$6k aligned pairs at $20$7 captured with two adjacent FLIR A65 cameras (Rhee et al., 18 Jun 2026). The paper identifies two dominant degradation modes in uncooled microbolometer imagery: stochastic additive noise and fixed-pattern noise expressed as stripe artifacts. Its central claim is that wavelets disentangle low-frequency scene structure from directional high-frequency detail, making stochastic noise and row/column stripe artifacts more accessible to targeted suppression (Rhee et al., 18 Jun 2026).
The method combines a pixel-domain reconstruction term with two wavelet-domain penalties. Wavelet Entropy is designed to suppress stochastic Gaussian noise, and Wavelet Directional Stripe Index penalizes directional stripe coherence. The WDSI terms are defined as
$20$8
and averaged across scales. On SCaN-TIR, adding $20$9 yields significant gains, with the paper reporting PSNR up to 0 and SSIM 1, while runtime remains approximately 2 FPS on RTX 4090 (Rhee et al., 18 Jun 2026). The same work reports downstream gains for thermal inertial odometry and zero-shot monocular depth estimation, indicating that real-time denoising is treated as a robotics systems component rather than as an isolated restoration benchmark (Rhee et al., 18 Jun 2026).
For real camera video, PocketDVDNet illustrates a different route to ReTiDe: aggressive compression plus domain-adapted distillation (Morris et al., 23 Jan 2026). Starting from FastDVDNet, the authors induce sparsity with OBProxSG, perform structured channel pruning, retrain a teacher on a five-component sRGB camera noise model, and distill a compact student that removes explicit noise-map inputs (Morris et al., 23 Jan 2026). The model size drops from 3M to 4M parameters, a 5 reduction, and runtime improves from 6 s to 7 s per 8p frame on RTX 4090 (Morris et al., 23 Jan 2026). The reported realistic noise model includes signal-dependent shot noise, signal-independent read noise, quantization noise, banding noise, and periodic interference, which is significant because many earlier video denoisers were trained under simpler noise assumptions.
A related receiver-side pipeline for online conferencing uses a hybrid scheduling strategy rather than a single unified model (Bisht et al., 2023). Keyframes, or one-fifth of the frames, are denoised with HI-GAN, while the remaining four-fifths are processed by FastDVDnet; a Noise Detector gates the denoise path and can bypass clean frames directly to render (Bisht et al., 2023). The paper locates the module between decode and render on the receiver, after the jitter buffer and decoder but before display, and emphasizes post-decode deployment for low-latency conferencing systems (Bisht et al., 2023). This suggests that ReTiDe, in video communication, often refers as much to scheduling and systems placement as to neural architecture.
4. Rendering, path tracing, and scientific visualization
Real-time rendering denoising contributes several of the most explicit efficiency–quality trade-off formulations in the ReTiDe literature. The weight-sharing kernel prediction network for Monte Carlo denoising targets very low spp path-traced inputs, including 9-spp frames, under strict time budgets (Fan et al., 2022). Its decoder is parameter-free and highly parallelizable, and the method reports quality comparable to direct kernel-prediction baselines while roughly halving denoising time for $0.017$0-spp inputs (Fan et al., 2022). Compared with a neural bilateral grid denoiser, it reports heavier network capacity but lighter reconstruction, with $0.017$1 ms for the six-block variant and $0.017$2 ms for the three-block variant at $0.017$3 on RTX 2080 Ti (Fan et al., 2022).
A different approach is real-time controllable denoising, where the critical systems contribution is post-inference editability rather than raw denoising speed (Zhang et al., 2023). RCD replaces the last layer of an existing CNN denoiser so that it outputs multiple noise maps in a single inference, then applies a network-free Noise Decorrelation process and reconstructs the final output by interpolation. The variance law
$0.017$4
gives an explicit control rule for denoising strength. The paper reports $0.017$5 s for $0.017$6 edit operations on GTX 1080Ti, or approximately $0.017$7 ms per edit, after one forward pass (Zhang et al., 2023). This is a distinct branch of ReTiDe logic: runtime responsiveness can refer to interactive control as well as to initial inference.
Path-traced video pipelines also introduce stateful sampling-denoising co-design. The RL-based stateful neural adaptive sampling and denoising framework trains a sampling importance network with reinforcement learning, keeps all per-pixel samples rather than averaging them, updates a $0.017$8-channel spatiotemporal latent, and denoises from that latent (Scardigli et al., 2023). At $0.017$9 spp, it reports $480$0 dB PSNR at $480$1 ms network inference time, and at $480$2 spp it surpasses the best baseline at $480$3 spp while reducing rendering time by a factor of $480$4 (Scardigli et al., 2023). The paper’s relevance to ReTiDe lies in the fact that denoising quality is no longer separable from sample allocation under tight budgets.
Scientific visualization with volumetric path tracing poses an additional complication: noisy G-buffers invalidate many surface-oriented denoisers (Iglesias-Guitian et al., 2021). The proposed real-time denoiser replaces geometry guidance with a temporally stable radiometric feature and applies weighted recursive least squares per pixel, followed by a spatial blend (Iglesias-Guitian et al., 2021). With $480$5, $480$6, and $480$7, the denoiser is reported at approximately $480$8 ms at $480$9p, independent of scene or volume complexity, while path tracing at $1024$0 spp takes approximately $1024$1 ms on NVIDIA GTX 1070 (Iglesias-Guitian et al., 2021). The method is training-free, which further broadens the meaning of ReTiDe: not all real-time denoisers in this literature are learned at inference time or even learned at all.
5. Medical and audio instantiations
Low-dose CT denoising demonstrates that ReTiDe principles are not confined to natural images. RDDM is explicitly motivated by the impracticality of multi-step generative inference for real-time use and replaces that with a residual-driven drifting model that performs one-step denoising (Wang et al., 16 May 2026). Three variants are reported. RDDM-Fine uses $1024$2 and $1024$3; RDDM-Balanced uses $1024$4 and $1024$5; RDDM-Smooth uses $1024$6 and $1024$7 (Wang et al., 16 May 2026). On the quarter-dose Mayo test set, RDDM-Fine achieves PSNR $1024$8 dB, SSIM $1024$9, and FID $48$0, while RDDM-Smooth achieves the best PSNR $48$1 dB and SSIM $48$2 (Wang et al., 16 May 2026). The inference cost is reported as approximately $48$3–$48$4 ms per $48$5 slice on a single NVIDIA H100, with one function evaluation (Wang et al., 16 May 2026).
Speech enhancement supplies a parallel but technically distinct lineage of real-time denoising. TVF combines a lightweight backbone with a differentiable $48$6-band IIR filter cascade predicted frame-by-frame in real time (Rota et al., 3 Mar 2026). The model has approximately $48$7M parameters and processes causal, non-overlapping $48$8-sample frames at $48$9 kHz (Rota et al., 3 Mar 2026). In evaluation on the Valentini-Botinhao dataset, TVF is reported as best among the compared models on PESQ, POLQA, MOS-Noise, and MOS-Overall, while DFNet3 remains strongest on SI-SDR and LSD (Rota et al., 3 Mar 2026). This establishes a recurrent trade-off in ReTiDe systems: explicit DSP structure can improve interpretability and perceptual behavior even when a less constrained model wins on some signal-domain metrics.
TRU-Net advances a different speech ReTiDe path by pushing model size and latency downward (Choi et al., 2021). It uses a Tiny Recurrent U-Net with $21$0M parameters, an INT8 footprint of $21$1 MB, and a phase-aware $21$2-sigmoid mask for simultaneous denoising and dereverberation (Choi et al., 2021). The magnitude component of the mask is
$21$3
with phase recovered through a cosine-law construction. The model reports $21$4 ms lookahead and end-to-end per-frame times of $21$5 ms or $21$6 ms on commodity CPUs, with competitive DNS and WHAMR performance (Choi et al., 2021). An earlier hearing-aid-oriented system likewise frames real-time denoising as Wiener-gain prediction under an $21$7 ms latency budget, using a $21$8-band hearing-aid-grade polyphase filter bank, $21$9 ms lookahead, and approximately 00 ms of analysis/synthesis filter-bank delay (Aubreville et al., 2018). Together these works show that, in audio, ReTiDe is tightly linked to causality, bounded buffering, and explicit control of algorithmic delay.
6. ReTiDe as an FPGA motion-picture system
The paper that formalizes the acronym treats ReTiDe as a production-oriented denoising service for motion-picture processing on data-centre FPGAs (Li et al., 4 Oct 2025). The system quantizes a compact U-Net-style encoder–decoder to INT8 through post-training quantization followed by quantization-aware training, compiles the result with AMD Vitis AI to an xmodel, and executes it on AMD DPU hardware on an Alveo U50 card (Li et al., 4 Oct 2025). The network is symmetric, derived from the CycleGAN generator, uses six downsampling stages and matched transposed-convolution upsampling stages, concatenative skip connections, LeakyReLU with 01 in the down path, and ReLU in the up path, while omitting residual connections, normalization, and dropout (Li et al., 4 Oct 2025).
The deployment model is explicitly systems-centric. ReTiDe is served through a client–server interface so that denoising can be invoked remotely from existing workflows, including a modified NUKE MLClient2 plugin with enlarged message buffers to support 02K-capable transfers (Li et al., 4 Oct 2025). Pre-processing segments and batches incoming images or video, VART/XRT handles FPGA execution, and outputs are stitched before being returned to the client (Li et al., 4 Oct 2025). The paper emphasizes that the service architecture allows denoising to be inserted into post-production tools and potentially codec pipelines without disrupting artist tooling.
Its principal quantitative claim is efficiency. On the reported benchmark, the Alveo U50 configuration reaches 03 GOPS and 04 GOPS/W, which the paper states is 05 higher throughput and 06 higher energy efficiency than prior FPGA denoising accelerators (Li et al., 4 Oct 2025). In a cited latency comparison, measured inference time is 07 s for ReTiDe versus 08 s for L-DnCNN (Li et al., 4 Oct 2025). Quality degradation after quantization is reported as negligible in PSNR/SSIM terms, with QAT recovering most of the PTQ loss and, at high noise on BSD68, ReTiDe(Q) surpassing L-DnCNN (Li et al., 4 Oct 2025).
The benchmarks are still image denoising benchmarks rather than end-to-end cinema or codec evaluations. Training uses DIV2K and LSDIR with additive Gaussian noise 09, and evaluation includes BSD68, Urban100, Set12, and BSD100 (Li et al., 4 Oct 2025). This matters for interpretation. The paper’s strongest contribution is not a new denoising prior for real camera or codec artifacts; it is a hardware/software serving stack showing that compact blind denoisers can be quantized, deployed on DPU-based FPGAs, and integrated into real production workflows with a favorable energy profile (Li et al., 4 Oct 2025).
7. Limitations, open questions, and conceptual significance
The literature also constrains what ReTiDe can currently mean. The FPGA ReTiDe paper is spatial-only and does not include temporal modeling; it also notes that quantization error is most visible at low noise levels and suggests hybrid or mixed precision as future work (Li et al., 4 Oct 2025). TIDY reports that performance can further benefit from larger real paired datasets and that incorporating temporal consistency may improve video stability and accuracy (Rhee et al., 18 Jun 2026). PocketDVDNet notes that in extreme low light larger models can recover more fine detail than the compact student (Morris et al., 23 Jan 2026). RDDM does not evaluate robustness across dose levels, scanners, and protocols beyond the reported quarter-dose Mayo setting (Wang et al., 16 May 2026).
Rendering-oriented systems expose a different set of unresolved issues. The Monte Carlo kernel-prediction work still finds very large kernels difficult to encode compactly (Fan et al., 2022). The RL-based adaptive sampling framework inherits the usual challenges of very sparse indirect effects and large disocclusions (Scardigli et al., 2023). Material-agnostic pre-shading denoising with neural integral operators reports real-time execution and easy integration with TAA, but the provided summary does not include explicit latency or FPS numbers, and the method remains limited to isotropic BSDFs without depth of field or motion blur (Schied et al., 23 Jul 2025). In volumetric path tracing, reprojection based on the first real collision can mis-handle motion ambiguities in highly heterogeneous media (Iglesias-Guitian et al., 2021).
Audio systems reveal an additional tension between interpretability and unconstrained performance. TVF’s linear, time-domain IIR restriction improves transparency and bounded behavior, but DFNet3 remains better on SI-SDR and LSD in the reported comparison (Rota et al., 3 Mar 2026). TRU-Net attains very small on-device footprint, but reverberant conditions remain harder than non-reverberant ones, as reflected in lower reported scores on the DNS reverberant set (Choi et al., 2021). This suggests that ReTiDe should not be equated automatically with maximal restoration quality; its defining feature is quality under operational constraint.
Taken together, the literature supports a precise but plural understanding of ReTiDe. It is, formally, a named FPGA denoising service for energy-efficient motion-picture processing (Li et al., 4 Oct 2025). It is also, in a broader and increasingly cross-domain sense, a design program in which denoising is re-engineered around online deployment: wavelet-domain reduction for TIR robotics, compact kernel reconstruction for path tracing, one-step residual generation for LDCT, compressed temporal models for video, and causal recurrent or differentiable-filter pipelines for speech (Rhee et al., 18 Jun 2026, Fan et al., 2022, Wang et al., 16 May 2026, Morris et al., 23 Jan 2026, Rota et al., 3 Mar 2026). The central research question is therefore not merely how to denoise, but how to do so in the exact computational, temporal, and systems context in which the restored signal must immediately be used.