Patch-based Diffusion Inverse Solver (PaDIS)

• The paper introduces PaDIS as a patch-based diffusion inverse solver that learns local patch priors with positional encoding to approximate global image scores.
• It integrates the learned patch prior into various diffusion inverse solvers, demonstrating superior performance in CT, MRI, and 3D reconstruction tasks.
• PaDIS achieves enhanced data and memory efficiency, outperforming whole-image models especially in settings with limited training data.

Patch-based Diffusion Inverse Solver (PaDIS) is a patch-based, position-aware diffusion inverse solver that learns a diffusion prior on local image patches and uses that prior to solve inverse problems without requiring whole-image diffusion training. In the formulation introduced in “Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problems” (Hu et al., 2024), the global score function of an image is approximated from patch scores together with positional encoding, and the resulting prior is inserted into diffusion inverse solvers such as DPS, Langevin dynamics, predictor-corrector sampling, and VE-DDNM. Subsequent work extended the same basic idea to generalized MRI priors and patch-wise inference on MRI restoration tasks (Roy et al., 25 Jan 2025), to mismatched-distribution inverse problems with self-supervision and small-data fine-tuning (Hu et al., 2024), to complex-valued multi-coil MRI reconstruction (Sanda et al., 25 Sep 2025), and to fully 3D CT reconstruction with local 3D patches, positional encoding, and downsampled global context (Yang et al., 20 Dec 2025).

1. Definition and problem setting

PaDIS operates in the standard inverse-problem setting

$y = \mathcal{A}(x) + n,$

where $x$ is the unknown image, $\mathcal{A}$ is a known forward operator, and $n$ is noise. In Bayesian form, reconstruction is based on

$$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$

with the diffusion model supplying the prior term $\log p(x)$ (Hu et al., 2024).

The original PaDIS paper targets CT reconstruction, deblurring, and super-resolution, including sparse-view CT and natural-image restoration, and was motivated by the observation that full-image diffusion training is computationally expensive and data-hungry, especially for high-dimensional or high-resolution data such as 3D images (Hu et al., 2024). The method therefore learns the image prior from patches rather than whole images, while still constructing a prior for the entire image.

A central feature of PaDIS is that it is a prior module rather than a task-specific end-to-end reconstructor. The learned patch prior can be plugged into different diffusion inverse solvers, and the same unconditional prior can be reused across multiple inverse problems (Hu et al., 2024). Later MRI work makes the same point in a different form: shifted-grid patch-based inference can be independently applied to multiple inverse solvers such as DPS or DiffPIR, regardless of training on whole images or on patches (Roy et al., 25 Jan 2025).

A plausible implication is that PaDIS belongs simultaneously to the patch-based generative-model literature and to plug-and-play or score-based inverse reconstruction. That interpretation is explicit in later work: patch-based diffusion models are treated as generalized priors inserted into iterative reconstruction frameworks, with data consistency handled by the forward model and the learned prior handled through denoising or score evaluation (Roy et al., 25 Jan 2025).

2. Patch factorization, positional encoding, and global score construction

The core probabilistic construction in PaDIS is a patch-factorized approximation to the image distribution. For an $N \times N$ image $x_0$, the image is zero padded so that it can be covered by a $(k+1)\times(k+1)$ grid of non-overlapping patches of size $P\times P$, with

$x$0

For each offset pair $x$1, the padded image induces patches $x$2 and a border region $x$3, and the image distribution is modeled as

$x$4

From this factorization, the full-image score is written as

$x$5

where the global score is the sum of border and patch scores (Hu et al., 2024).

Patch training alone would lose global spatial semantics, so PaDIS adds positional encoding. In the original 2D formulation, normalized $x$6- and $x$7-coordinate arrays are extracted together with each patch and concatenated with the patch channels before being fed to the UNet. This makes the patch-level score position-aware rather than location-agnostic (Hu et al., 2024). Later variants preserve the same principle while changing the representation: the mismatched-distribution method adds a 2D positional array of x-coordinates scaled to $x$8 (Hu et al., 2024), and the fully 3D CT extension adds three coordinate channels corresponding to normalized $x$9-, $\mathcal{A}$0-, and $\mathcal{A}$1-coordinates (Yang et al., 20 Dec 2025).

The original PaDIS implementation also uses random tilings of the padded image during sampling. Different offset choices shift patch boundaries across iterations, which reduces persistent blocking artifacts and makes the aggregated score behave more like a full-image prior (Hu et al., 2024). In subsequent MRI work, this same idea is referred to as shifted-grid inference: the image is divided into overlapping patches, processed patchwise, and reassembled through averaging or weighted aggregation, with multiple shifted grids used to smooth grid-based artifacts (Roy et al., 25 Jan 2025).

This suggests that PaDIS is best understood not as independent patch generation, but as a Monte Carlo approximation to a global prior whose local factors are patch distributions endowed with position information.

3. Diffusion training and inverse-solver integration

The diffusion backbone in the original PaDIS paper is trained with denoising score matching in a denoiser parameterization. For noisy input $\mathcal{A}$2 at noise level $\mathcal{A}$3, the network $\mathcal{A}$4 is trained using

$\mathcal{A}$5

and Tweedie’s formula gives the score estimate

$\mathcal{A}$6

Training is carried out on patches, often with multiple patch sizes, rather than on whole images (Hu et al., 2024).

At inference time, PaDIS is inserted into a diffusion inverse solver by replacing a whole-image score evaluation with an aggregated patch score evaluation. In the original algorithm, at diffusion step $\mathcal{A}$7, one randomly selects a tiling, extracts all patches, evaluates the denoiser on each patch, converts denoiser outputs to patch scores, aggregates them into a whole-image score using Equation (2), and then performs a data-fidelity correction followed by a diffusion update (Hu et al., 2024).

In DPS-style reconstruction, this yields an update of the form

$\mathcal{A}$8

for the data step, followed by a Langevin or reverse-diffusion step driven by the patch-aggregated score (Hu et al., 2024). Later MRI work spells out the same structure for both DPS and DiffPIR: patch extraction, per-patch score or denoising evaluation, aggregation to a full-image score or denoised image, then data-consistency enforcement through the forward model (Roy et al., 25 Jan 2025).

The main algorithmic variants described across the literature are summarized below.

Framework	Prior action	Data-consistency mechanism
PaDIS with VE-DPS	Patch score aggregation	Gradient step on measurement loss
Patch-based DiffPIR	Patch denoising aggregation	Quadratic data step / proximal update
OOD PaDIS	Patch denoiser + CG	CG plus self-supervised or fine-tuned adaptation

A distinct but related development concerns posterior approximation for diffusion inverse solvers with non-linear operators. “Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solvers” argues that posterior means can lie outside the support of the image distribution and that consistency-model posterior samples are preferable for non-linear operators (Xu et al., 2024). A plausible implication is that patch-based inverse solvers using non-linear patch operators could benefit from posterior-sample approximations rather than posterior means, although that extension is presented as guidance rather than as part of the original PaDIS formulation.

4. Efficiency, data efficiency, and performance

The main empirical claim of the original PaDIS work is that patch-based diffusion priors are substantially more data-efficient and memory-efficient than whole-image priors while remaining competitive or superior in reconstruction quality (Hu et al., 2024). On sparse-view CT with 20 views, the reported averages are:

Method	PSNR / SSIM
Baseline (FBP)	24.93 / 0.595
ADMM-TV	26.82 / 0.724
Whole-image diffusion	32.84 / 0.835
PaDIS (Ours)	33.57 / 0.854

For the more ill-posed 8-view CT setting, the same table reports 25.74 / 0.706 for whole-image diffusion and 29.48 / 0.767 for PaDIS (Hu et al., 2024). On CelebA-HQ deblurring, the reported scores are 30.19 / 0.853 for whole-image diffusion and 30.80 / 0.870 for PaDIS; on $\mathcal{A}$9 super-resolution they are 29.17 / 0.827 and 29.47 / 0.846, respectively (Hu et al., 2024).

The data-efficiency study in the same paper is especially central to the PaDIS concept. For CT reconstruction with 20 views, as training-set size decreases from 2304 to 144 slices, PaDIS remains relatively stable, whereas whole-image diffusion degrades more sharply. At 144 images, PaDIS reports 32.28 PSNR and 0.841 SSIM, compared with 29.12 PSNR and 0.804 SSIM for whole-image diffusion (Hu et al., 2024). The interpretation offered in the paper is straightforward: each full image yields many training patches, so the effective number of training examples is much larger.

Memory reduction is also explicit in the broader patch-diffusion literature. “Memory Efficient Diffusion Probabilistic Models via Patch-based Generation” reports that when an entire image is divided into $n$0 patches, maximum memory consumption can be reduced by half while maintaining comparable image quality (Arakawa et al., 2023). The original PaDIS paper similarly reports that patch-based models reach best results after about 12 hours of training, whereas whole-image models require about 24–36 hours, and that inference time remains similar because patch evaluations can be parallelized (Hu et al., 2024).

The MRI feasibility study extends these observations to generalized MRI priors. For knee denoising with DPS and a whole-image-trained prior, the reported results are 27.88 ± 1.09 PSNR and 0.19 ± 0.02 LPIPS for $n$1 whole-image inference, 27.77 ± 1.07 and 0.19 ± 0.03 for $n$2 patch inference, and 27.67 ± 1.05 and 0.19 ± 0.03 for $n$3 patch inference, indicating only minor degradation under patch-based evaluation (Roy et al., 25 Jan 2025). The same study reports approximately 25% GPU memory reduction when moving from $n$4 whole-image inference to $n$5 patch inference in DiffPIR denoising (Roy et al., 25 Jan 2025).

5. MRI, mismatched-distribution, and 3D extensions

A major line of post-2024 work generalizes PaDIS from 2D natural and CT images to more demanding medical-imaging settings.

In “Investigating the Feasibility of Patch-based Inference for Generalized Diffusion Priors in Inverse Problems for Medical Images,” a single generalized MRI diffusion prior is trained on approximately 289k slices spanning multiple anatomies and field strengths, and then evaluated under whole-image and patch-based inference for denoising and super-resolution on knee and brain MRI (Roy et al., 25 Jan 2025). The central result is that whole-image-trained models can be safely used patchwise when shifted-grid inference and reflection padding are applied, while patch-trained models are slightly more resilient to varying patch sizes at test time (Roy et al., 25 Jan 2025).

In “Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems,” the patch-based prior is used in two distinct OOD regimes: a single-measurement regime with no in-distribution training images, and a small-dataset regime with only a handful of in-distribution images (Hu et al., 2024). The paper uses a patch-based diffusion prior trained on patches only, combines it with conjugate-gradient data consistency, and in the single-measurement regime adds a self-supervised loss

$n$6

In the main single-measurement table, patch self-supervision exceeds whole-image self-supervision across CT, deblurring, and super-resolution; for example, on CT with 60 views the reported values are 40.47 PSNR and 0.957 SSIM for the whole-image method, versus 41.45 PSNR and 0.969 SSIM for the patch-based method (Hu et al., 2024). The same paper argues that whole-image diffusion models are prone to memorization and overfitting under small-data fine-tuning, whereas patch-based models remain more stable because even a single image contains many patches (Hu et al., 2024).

The MRI-specific clinical extension is “Patch-Based Diffusion for Data-Efficient, Radiologist-Preferred MRI Reconstruction,” which adapts PaDIS to complex-valued, multi-coil MRI with forward model

$n$7

Complex images are represented as two real channels, and a position-aware patch prior is integrated into VE-DPS for $n$8 undersampled FastMRI brain reconstruction (Sanda et al., 25 Sep 2025). Across all contrasts at $n$9, the reported results for 25 training slices are 32.97 PSNR, 0.847 SSIM, and 0.143 NRMSE for PaDIS-MRI versus 30.51, 0.817, and 0.185 for the whole-image FastMRI-EDM baseline (Sanda et al., 25 Sep 2025). In a blinded three-radiologist study, PaDIS-MRI reconstructions were chosen as diagnostically superior in 55 of 60 cases, or 91.7% overall (Sanda et al., 25 Sep 2025).

The 3D generalization is “Local Patches Meet Global Context: Scalable 3D Diffusion Priors for Computed Tomography Reconstruction,” which introduces a fully 3D patch-based prior with downsampled 3D global context $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$0 and 3D positional encodings (Yang et al., 20 Dec 2025). Its probabilistic formulation augments the patch prior to

$$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$1

and uses a 3D UNet on local 3D patches plus global context. On sparse-view CT, the method reports PSNR values such as 38.56 dB at 20 views on LIDC $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$2, outperforming FBP, ADMM-TV, FBP-UNet, DDS, DiffusionBlend, and Blend+FT; it also reports reconstruction of $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$3 volumes in about 20 minutes on a single NVIDIA A100 (Yang et al., 20 Dec 2025).

6. Artifact control, limitations, and broader methodological context

Patch-wise diffusion introduces characteristic artifact modes. The original PaDIS paper notes that random tilings are needed because fixed patch boundaries would otherwise induce visible block artifacts (Hu et al., 2024). The MRI feasibility study isolates a more specific failure mode: when generalized priors are evaluated patchwise with zero padding, sharp foreground-to-background discontinuities at patch borders can be amplified by the diffusion prior and iterative solver, producing visible vertical and horizontal lines near image borders (Roy et al., 25 Jan 2025). The proposed remedy is simple but consequential: reflection padding rather than zero padding, combined with shifted-grid overlap and averaging (Roy et al., 25 Jan 2025).

Patch size remains a structural trade-off throughout the literature. In the original PaDIS CT study, a patch size of $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$4 yielded the best reported CT 20-view result, while both very small patches and whole-image training performed worse (Hu et al., 2024). In the 3D CT extension, smaller patches reduce training and sampling cost but degrade prior quality; for LIDC, the paper reports PSNR 33.06 for patch size $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$5, 31.74 for $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$6, and 27.87 for $$\log p(x \mid y) \propto \log p(x) + \log p(y \mid x),$$7 (Yang et al., 20 Dec 2025). The medical-MRI feasibility paper similarly reports a plateau in memory savings when patches become too small, together with the risk of weaker global context (Roy et al., 25 Jan 2025).

Another recurrent limitation is the tension between local realism and long-range coherence. The mismatched-distribution paper states this directly: patch-based factorization can neglect long-range dependencies, and global consistency is enforced only through CG and overlapping patch aggregation (Hu et al., 2024). The 3D CT work addresses the same issue by conditioning local patches on a downsampled global volume and positional encodings (Yang et al., 20 Dec 2025). This suggests that the historical trajectory of PaDIS is toward progressively richer mechanisms for reintroducing global structure without abandoning patch-wise efficiency.

A related methodological discussion appears in “DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing,” which argues that, in many diffusion inverse solvers, the likelihood term dominates while the prior mainly provides initialization (Chen et al., 21 Nov 2025). A plausible implication is that PaDIS can be interpreted within a broader decoupled framework: diffusion supplies a realistic patch prior or initialization manifold, while reconstruction quality is strongly shaped by explicit data-consistency optimization. That interpretation is compatible with both the plug-and-play MRI work (Roy et al., 25 Jan 2025) and the OOD self-supervised method (Hu et al., 2024), even though PaDIS itself remains a patch-prior construction rather than a decoupling principle.

In its mature form, PaDIS denotes a family of inverse solvers built on three commitments: local patch modeling, explicit spatial awareness, and modular integration with physics-based data consistency. Across 2D CT, natural-image restoration, generalized MRI priors, complex-valued multi-coil MRI, OOD inverse problems, and scalable 3D CT, the recurring empirical conclusion is that patch-based diffusion priors are especially effective when memory is limited, training data are scarce, or training and test distributions do not align perfectly (Hu et al., 2024).