PPORLD-EDNetLDCT: RL-Based Adaptive LDCT Denoising

• The paper introduces an adaptive RL-controlled low-dose CT denoising framework that integrates a UNet-like EDNet with PPO to optimize image quality via PSNR and SSIM rewards.
• It formulates denoising as a sequential decision process where the RL agent selects actions like single/multiple passes, fine-tuning, or skipping, ensuring efficient training and lightweight inference.
• Ablation studies across clinical datasets show that the method outperforms fixed denoising approaches and conventional iterative and supervised models in both quantitative metrics and structural preservation.

Searching arXiv for the named method and closely related LDCT denoising/reconstruction papers. arXiv search query: "PPORLD-EDNetLDCT low-dose CT PPO reinforcement learning denoising" PPORLD-EDNetLDCT is an adaptive low-dose CT denoising framework that couples a compact encoder–decoder network with a Proximal Policy Optimization reinforcement learning agent. It formulates denoising as a sequential decision process in which the agent learns, from image-quality feedback, how to apply the denoiser, whether to use iterative passes, when to skip processing, and, during training, when to fine-tune the denoiser itself. The method is presented as a reinforcement learning-based approach with Encoder-Decoder for LDCT, trained in a custom Gymnasium environment and evaluated on the Low Dose CT Image and Projection dataset, the NIH–AAPM–Mayo Clinic Low Dose CT Challenge, and a COVID-19 LDCT classification setting (Sutradhar et al., 3 Sep 2025).

1. Conceptual definition and problem setting

PPORLD-EDNetLDCT addresses the standard low-dose CT trade-off: reduced radiation exposure is accompanied by increased quantum noise and streak artifacts that blur edges and obscure small structures. The paper positions this problem against two conventional families of methods. Iterative reconstruction can be effective but is described as computationally intensive, dependent on careful regularization tuning, and prone to oversmoothing fine details. Supervised deep learning with fixed objectives is described as learning static denoising mappings that may generalize poorly across dose levels, scanners, and anatomical regions, while also tending to prioritize pixel fidelity over structural fidelity (Sutradhar et al., 3 Sep 2025).

The central methodological claim is that reinforcement learning can adapt denoising behavior on a per-image basis. Rather than learning only a fixed image-to-image mapping, the framework learns a denoising policy through interaction with an environment that returns rewards computed from PSNR and SSIM. In this formulation, EDNet remains the image-restoration backbone, but the RL agent controls how EDNet is used during training. This suggests a division of labor in which CNN-based restoration handles representation learning, while PPO governs adaptive control over denoising intensity and refinement strategy (Sutradhar et al., 3 Sep 2025).

A common misunderstanding is that the reinforcement learning controller must remain active at deployment. The paper states the opposite: during inference, EDNet executes a single, fixed forward pass, and the training-only actions that fine-tune EDNet weights or adjust PPO hyperparameters are disabled. In that sense, adaptation is concentrated in training, while deployment remains lightweight (Sutradhar et al., 3 Sep 2025).

2. Reinforcement learning formulation

The method is specified as a Markov decision process over LDCT slices. The state $s_t$ is the current CT image representation at time $t$; during training it is the current, potentially partially denoised slice $I_t$, normalized to $[0,1]$ and resized to $512 \times 512$. The action space is discrete and consists of five controls: apply EDNet once, apply EDNet multiple times with iterative passes $K \in \{2,3\}$, fine-tune EDNet weights on-the-fly through a single gradient descent step using MSE to ground truth, skip processing, or adjust PPO learning parameters to stabilize learning when reward variance indicates instability. The transition $s_{t+1}$ is obtained by executing the selected action, so the environment is nonstationary during training because one action explicitly changes EDNet parameters (Sutradhar et al., 3 Sep 2025).

The reward is computed directly from image quality by comparing the current denoised image $I_d$ to the available high-dose target $I_{hd}$. The paper uses

$$\text{PSNR} = 10 \log_{10}\!\left(\frac{\text{MAX}^2}{\text{MSE}}\right), \qquad \text{MSE} = \frac{1}{HW}\sum_{i=1}^{H}\sum_{j=1}^{W}\big(I_d(i,j)-I_{hd}(i,j)\big)^2,$$

together with the standard SSIM expression, and defines the scalar reward as

$t$0

NaNs in either metric are set to zero. The reward therefore combines a pixel-domain fidelity term and a structural-perceptual term, with clipping used as an explicit stabilization device (Sutradhar et al., 3 Sep 2025).

Policy learning follows the standard PPO decomposition. The policy ratio is

$t$1

and optimization uses the clipped surrogate objective, a value-function term, and an entropy bonus. Advantage estimation is performed with GAE using the temporal-difference residual

$t$2

and

$t$3

The stated hyperparameters are $t$4, $t$5, and PPO clip parameter $t$6; the PPO optimizer is AdamW with learning rate $t$7 (Sutradhar et al., 3 Sep 2025).

3. EDNet architecture and the control interface

EDNet is described as a UNet-like encoder–decoder with three downsampling stages, symmetric upsampling, ReLU nonlinearities, batch normalization, skip connections, and a final $t$8 output convolution. The encoder uses $t$9 convolutions with 64 filters in stage 1, 128 filters after downsampling in stage 2, and 256 filters after further downsampling in stage 3. The bottleneck has 512 filters at $I_t$0. The decoder contains three transposed-convolution upsampling blocks, halving channels at each stage from $I_t$1, concatenating skip features from the encoder, and producing the denoised image through the final output layer (Sutradhar et al., 3 Sep 2025).

The paper emphasizes that PPO does not replace gradient-based learning; rather, it governs EDNet usage. In operational terms, the agent decides when EDNet should be applied once, when it should be applied multiple times, when processing should be skipped to avoid over-denoising, and, during training only, when a single MSE-based fine-tuning step should be taken on EDNet weights. When the fine-tune action is selected, EDNet minimizes

$I_t$2

The overall training is described conceptually as combining PPO policy learning and supervised fine-tuning,

$I_t$3

with the important caveat that $I_t$4 updates the policy/value network and $I_t$5 updates EDNet only when the agent explicitly selects the fine-tune action (Sutradhar et al., 3 Sep 2025).

The PPO agent itself is an MLP whose input matches the flattened state representation, with two hidden layers of 256 and 128 units, ReLU activations, and a softmax output over five actions. The value head follows the standard PPO setup. EDNet is optimized with AdamW at learning rate $I_t$6, and the reported training schedule is 200 epochs on the TCIA LDCT data, with slices normalized to $I_t$7 and resized to $I_t$8 (Sutradhar et al., 3 Sep 2025).

4. Data, evaluation protocol, and reported results

The paper reports experiments on three datasets with distinct roles. The Low Dose CT Image and Projection dataset from TCIA contains 100 clinical chest CT scans, split 80/20 with no patient overlap; slices are normalized to $I_t$9 and resized to $[0,1]$0, and the projection data are explicitly stated to be unused. The NIH–AAPM–Mayo Clinic Low Dose CT Challenge contributes 30 contrast-enhanced abdominal scans with full-dose acquisition at $[0,1]$1 and quarter-dose simulation at $[0,1]$2. A COVID-19 LDCT classification dataset with 160 scans, including 56 normal and 104 COVID-19 cases, is used to assess downstream diagnostic utility (Sutradhar et al., 3 Sep 2025).

Dataset	Role	Reported performance
Low Dose CT Image and Projection (TCIA)	LDCT denoising	PSNR 41.87, SSIM 0.9814, RMSE 0.00236
NIH–AAPM–Mayo Clinic Low Dose CT Challenge	Cross-dataset comparison	PSNR 41.52, SSIM 0.9723, RMSE 0.0051
COVID-19 LDCT dataset	Downstream classification validation	Accuracy 94%; 4% higher than denoising without RL-based denoising

Against comparative methods, PPORLD-EDNetLDCT is reported to exceed PWLS-ULTRA, RED-CNN, DP-ResNet, and attention-guided U-Net in PSNR and RMSE while also achieving the best SSIM on TCIA. It is also reported to edge out SIST on the same dataset, with PSNR 41.87 versus 41.80, SSIM 0.9814 versus 0.916, and RMSE 0.00236 versus 0.00246. Qualitatively, HU-difference maps are described as showing noise removal with minimal structural distortion, although localized HU increases are noted as warranting inspection (Sutradhar et al., 3 Sep 2025).

The ablation study is central to the paper’s claim that adaptive control, rather than EDNet alone, drives performance. Removing PPO and using EDNet-only single-pass denoising reduces performance to PSNR 29.52 and SSIM 0.8427. Fixed multi-pass denoising without PPO reaches PSNR 31.08. Eliminating reward clipping yields PSNR 25.13 despite SSIM 0.9602, which the paper interprets as instability and oversmoothing or metric imbalance. No GAE gives PSNR 30.41. Restricting the action space to “apply once/multiple” reaches PSNR 36.92. Removing skip connections increases RMSE to 0.00342 and lowers SSIM to 0.9281. Wilcoxon signed-rank tests against the full model yield $[0,1]$3 across ablations, with most $[0,1]$4 (Sutradhar et al., 3 Sep 2025).

Efficiency is reported separately for training and inference. The full model requires approximately 320 s per epoch for training and approximately 46 ms per slice at inference. The paper attributes training overhead to RL, while noting that all variants retain nearly constant inference because deployment uses only a single EDNet forward pass (Sutradhar et al., 3 Sep 2025).

5. Position within the LDCT literature

Within LDCT methodology, PPORLD-EDNetLDCT occupies the post-reconstruction, image-domain denoising regime, but distinguishes itself by adaptive policy learning rather than a fixed restoration map. This differs from the dual-domain W-net approach that explicitly combines Fourier-space and image-space U-nets and uses the central slice theorem to motivate frequency-domain processing without raw sinogram access. That method reports FI and IF dual-domain cascades outperforming single-domain U-nets on the AAPM Low-dose CT Grand Challenge, with FI reaching SSIM $[0,1]$5, PSNR $[0,1]$6 dB, and NRMSE $[0,1]$7 (Chung et al., 2020).

It also differs from model-based or unrolled reconstruction frameworks that keep explicit data-fidelity operators in the loop. PFBS-AIR unrolls proximal forward–backward splitting with analytical-reconstruction preconditioning and reports superiority over AR, IR, and FBPConvNet across simulated dose levels, for example at $[0,1]$8 with PSNR $[0,1]$9 and SSIM $512 \times 512$0 (Ding et al., 2019). ELDA, similarly, is framed as a provably convergent learned inexact descent algorithm with a non-local regularizer and reports PSNR $512 \times 512$1 and SSIM $512 \times 512$2 at the 2.5% dose level on AAPM-Mayo, using only 19 layers (Zhang et al., 2021). These comparisons indicate that PPORLD-EDNetLDCT belongs to a distinct design family: adaptive RL-controlled denoising rather than explicit physics-unrolled reconstruction.

Nomenclature around the acronym space is potentially confusing. “LDNet” in "Lesion-aware Dynamic Kernel for Polyp Segmentation" refers to an endoscopic polyp segmentation architecture with dynamic kernels, lesion-aware cross-attention, and efficient self-attention; it is not an LDCT denoising framework (Zhang et al., 2023). Likewise, the peer-to-peer federated continual learning method icP2P-FL addresses LDCT denoising across institutions without a central server, uses a modified ResNet rather than an EDNet-like backbone, and explicitly states that the term “PPORLD-EDNetLDCT” does not appear in that paper (Wang et al., 2023).

6. Limitations, safety considerations, and future directions

The paper identifies several limitations. Generalization to other scanners and protocols, including brain CT, dual-energy CT, and vendors beyond Siemens, remains to be validated. The reward depends on PSNR and SSIM, which the paper states may not fully reflect clinical quality. Occasional oversmoothing may occur if the reward becomes imbalanced. These constraints are especially relevant because the method optimizes image-quality surrogates rather than explicitly task-aware clinical endpoints (Sutradhar et al., 3 Sep 2025).

Safety is discussed through HU-difference visualization. The paper states that structural deviation is minimal overall, but also notes that red HU increases should be inspected to avoid masking subtle pathology. A plausible implication is that quantitative fidelity alone is insufficient for deployment decisions, particularly when denoising is intended to support lesion detection or subtle parenchymal assessment (Sutradhar et al., 3 Sep 2025).

The proposed future directions are explicitly broader than simple architectural scaling. They include enhanced reward shaping with lesion-aware terms, radiomics-guided metrics, or multi-task segmentation or classification rewards; domain adaptation and few-shot transfer across scanners; unsupervised RL and hybrid self-supervised PPO; transformers or attention-integrated EDNet; and multi-agent coordination for slice consistency across volumes. In the current formulation, PPORLD-EDNetLDCT is therefore best understood as an adaptive training framework for LDCT denoising whose primary novelty lies in using PPO to learn when and how a compact EDNet should be deployed, rather than as a replacement for physics-based reconstruction itself (Sutradhar et al., 3 Sep 2025).