Patch-Based Diffusion Framework (TDiff)

• TDiff is a patch-based diffusion strategy that replaces full-image denoising with localized patch operations to improve memory and data efficiency.
• The framework employs overlapping patches, positional encoding, and staged refinement to maintain global consistency while reducing artifacts.
• It demonstrates enhanced performance in inverse problems and high-resolution restoration by balancing local priors with efficient aggregation methods.

Searching arXiv for the specified paper and closely related patch-based diffusion work. Searching arXiv for (Roy et al., 25 Jan 2025). Patch-based diffusion frameworks are a family of diffusion-model formulations in which training, inference, or both are performed on localized image patches rather than on full images. In inverse problems and restoration, these methods use patchwise denoising to approximate or instantiate an image prior while reducing memory demands, increasing data efficiency, and enabling operation on high-resolution or otherwise difficult inputs. In the cited literature, the label “TDiff” has been used in more than one way: as a patch-based thermal restoration framework in “TDiff: Thermal Plug-And-Play Prior with Patch-Based Diffusion” (Dashpute et al., 7 Oct 2025), and, in the CutDiffusion context, as a name for a two-stage patch-wise diffusion extrapolation process (Lin et al., 2024). Across these usages, the common technical core is local diffusion computation on patches followed by a mechanism for preserving global consistency.

1. Conceptual scope and definition

Patch-based diffusion departs from whole-image diffusion by replacing a single global denoising pass with patch-local denoising, typically accompanied by overlap, aggregation, positional conditioning, or staged refinement. The motivation stated repeatedly across the literature is that whole-image diffusion is often computationally and memory intensive, especially for high-resolution data and inverse problems (Roy et al., 25 Jan 2025, Hu et al., 2024).

A canonical inverse-problem formulation appears in the MRI literature as

$$\hat{x} = \arg\max_x \frac{1}{2} \|y-Hx\|^2 + \lambda \phi(x),$$

where $y$ is the observed measurement, $H$ is the degradation or forward operator, and $\phi(x)$ is a learned prior supplied by the diffusion process (Roy et al., 25 Jan 2025). Patch-based diffusion frameworks retain this plug-and-play structure, but the denoiser or score model is applied to image patches $x_p$, with full-image restoration produced by aggregation. A representative patchwise denoising relation is

$$\hat{x}_{p, t-1} = \mathrm{Denoiser}_\theta(x_{p, t}, \sigma_t),$$

followed by weighted averaging or overlap-tile recombination (Roy et al., 25 Jan 2025).

This suggests that “TDiff” is best understood not as a single architecture, but as a patch-centric diffusion design pattern: divide the image or latent into patches, denoise locally, and enforce global coherence by positional information, overlap, shifted grids, blending windows, or staged reconstruction.

2. Core mechanics of patch-based diffusion

The basic mechanics vary by application, but several recurring components are explicit in the literature. First, training may be performed on random crops or fixed-size patches instead of whole images. In the generalized MRI study, the compared regimes are whole image training on $256 \times 256$ images and random patch training on $128 \times 128$ patches, without added padding (Roy et al., 25 Jan 2025). In PaDIS, image patches are randomly extracted from possibly zero-padded images and augmented with positional encoding in the form of two normalized coordinate arrays in $[-1,1]$ (Hu et al., 2024). In Patch Diffusion, the denoiser is conditioned on patch location through two additional coordinate channels and patch size is randomized during training (Wang et al., 2023).

Second, inference commonly uses overlapping extraction and aggregation. The MRI feasibility study evaluates shifted-grid inference, following PaDIS, where overlapping patches are extracted on a grid that is shifted across multiple passes and the outputs are aggregated to form the final restored image (Roy et al., 25 Jan 2025). TDiff for thermal restoration divides the image into overlapping patches, denoises patches independently at each diffusion timestep, multiplies each output by a smooth spatial window, and reconstructs the image by summing or averaging the overlapping weighted patches (Dashpute et al., 7 Oct 2025). The adverse-weather restoration framework similarly performs denoising on overlapping patches and averages per-pixel noise estimates across overlaps during every denoising step, yielding a global smoothed noise map for size-agnostic restoration (Özdenizci et al., 2022).

Third, several works explicitly address how a full-image score or prior is assembled from patchwise quantities. PaDIS defines a position-aware patch-based diffusion inverse solver in which the score function of the whole image is obtained through scores of patches and their positional encoding (Hu et al., 2024). Its derivation expresses the full-image score as a sum over patch-level score functions and a border term: $$\nabla \log p(x) = \sum_{i,j=1}^{M} \left[ s_{i,j,B}(x_{i,j,B}) + \sum_{r=1}^{(k+1)^2} s_{i,j,r}(x_{i,j,r}) \right].$$ A related out-of-distribution inverse-problem study uses a patch-based diffusion prior that learns the image distribution solely from patches and approximates the underlying distribution as a product over overlapping patches extracted from a zero-padded image (Hu et al., 2024).

3. Training regimes, positional information, and patch priors

A central technical issue in patch-based diffusion is the loss of global context. The literature addresses this with positional information, diversified patch sizes, and occasional access to global views.

Patch Diffusion introduces a conditional score function at the patch level in which patch location in the original image is included as additional coordinate channels, while patch size is randomized and diversified during training (Wang et al., 2023). Its patch-wise loss is written as

$y$0

and the corresponding score function is conditioned on patch location and size. The same work states that a small fraction of training steps are performed on full-size images, with stochastic scheduling at $y$1 reported as an optimal trade-off between speed and quality (Wang et al., 2023).

PaDIS uses position-aware patch training with normalized $y$2 and $y$3 coordinate channels and varying patch sizes to help the network learn cross-region dependencies and accelerate training (Hu et al., 2024). This framework is intended as a black-box prior for inverse solvers, so the patch model is not tied to a single restoration task. In the generalized MRI study, shifted-grid patch-based inference does not require positional embeddings or anatomy-specific priors, which marks a different design choice: generalization is sought through aggregation strategy and a diverse training set rather than explicit spatial conditioning (Roy et al., 25 Jan 2025).

Older memory-efficiency work on patch-by-patch generation approached the context problem differently. “Memory Efficient Diffusion Probabilistic Models via Patch-based Generation” uses position-wise conditioning with one-hot patch identifiers and a Global Content Conditioning mechanism obtained by average-pooling the whole image to patch size and concatenating it with each patch (Arakawa et al., 2023). The paper states that when an entire image is divided into $y$4 patches, maximum memory consumption can be reduced by half while maintaining comparable image quality (Arakawa et al., 2023).

4. Inference, aggregation, and artifact suppression

Patchwise diffusion inference introduces boundary discontinuities that do not arise in whole-image evaluation, and a substantial portion of the literature is devoted to avoiding seams, blocking, or inconsistent structure.

In MRI plug-and-play inference, a specific artifact source is zero-padding at patch boundaries. The generalized MRI study reports that transition artifacts, such as horizontal and vertical lines at foreground-background edges, appear under zero-padding and are especially pronounced in MRI with diverse anatomical content. The paper states that switching to reflection-padding removes these foreground-to-background transition artifacts (Roy et al., 25 Jan 2025). It further reports that aggregation reduces visible grid or blocking artifacts.

TDiff for thermal imaging uses overlap and smooth spatial windowing to suppress seams. During inference, the current image estimate $y$5 is split into overlapping patches, each patch is denoised independently, each patch output is multiplied by a smooth spatial window, and the full image is reconstructed by weighted summation or averaging: $y$6 The patch division, processing, and recombination are performed at every diffusion step, not only at the end (Dashpute et al., 7 Oct 2025).

The adverse-weather restoration work uses a related but more explicitly global mechanism: all per-pixel noise estimates from overlapping patches are accumulated and averaged to obtain a global smoothed noise map, and the reverse denoising update is then performed on the entire image (Özdenizci et al., 2022). This is the basis for its size-agnostic behavior.

A distinct inference design appears in CutDiffusion, which splits patch-wise extrapolation into an initial comprehensive structure denoising phase and a subsequent specific detail refinement phase (Lin et al., 2024). Stage 1 operates on non-overlapping patches and uses pixel-level interaction, randomly permuting corresponding pixels among patches at each denoising step to promote structural consistency. Stage 2 reassembles a higher-resolution latent and performs standard overlapping patch denoising with averaging on overlapping regions. The reported intent is to reduce the number of patches and mitigate content repetition while maintaining low GPU memory (Lin et al., 2024).

5. Plug-and-play inverse problems and medical imaging

Patch-based diffusion has been particularly active in inverse problems, where the local prior can be inserted into iterative solvers. PaDIS is explicitly framed as a Patch-based position-aware Diffusion Inverse Solver that can be plugged into different diffusion inverse solvers and used for CT reconstruction, deblurring, and superresolution in both natural and medical image domains (Hu et al., 2024). The paper states that PaDIS outperforms previous diffusion inverse solvers trained on entire image priors in the case of limited training data, thereby emphasizing data efficiency (Hu et al., 2024).

The generalized MRI feasibility study examines whether patches can replace whole-image training and inference for a generalized diffusion prior on MRI images. The diffusion prior is trained on a diverse MRI dataset of approximately 289,000 images from various anatomies and scanners, producing a generalized model applicable across body regions (Roy et al., 25 Jan 2025). The reported findings are that patch-based training and inference yield comparable performance, measured by PSNR and LPIPS, to whole-image training and inference across evaluated plug-and-play solvers, tasks, and datasets. The paper further states that PSNR and LPIPS differences are typically less than $y$7 dB PSNR and less than $y$8 LPIPS, and that patch-wise inference reduces memory consumption by up to 25% when moving from $y$9 to $H$0 patches, with diminishing returns below that threshold (Roy et al., 25 Jan 2025).

The out-of-distribution inverse-problem study sharpens a different advantage of patch priors. It investigates scenarios in which the training and test distributions are mismatched and reports that whole-image models are prone to memorization and overfitting, whereas a patch-based model can avoid these issues (Hu et al., 2024). In the single-measurement setting, the method updates the patch-based score network with a self-supervised loss

$H$1

alternating denoising, data-consistency enforcement, and network refinement (Hu et al., 2024). The paper reports, for example, 33.77 dB PSNR on 20-view CT for the patch-based self-supervised method versus 33.19 for a whole-image model, and 30.34 for deblurring versus 29.50 (Hu et al., 2024).

A later MRI reconstruction study extends PaDIS to complex-valued, multi-coil MRI reconstruction and compares it with a whole-image diffusion baseline (Sanda et al., 25 Sep 2025). It reports that models trained on as few as 25 k-space images outperform the whole-image baseline on PSNR, SSIM, NRMSE, pixel-level uncertainty, cross-contrast generalization, and robustness to severe k-space undersampling, and that in a blinded study with three radiologists the patch-based reconstructions were chosen as diagnostically superior in 91.7% of cases (Sanda et al., 25 Sep 2025). A plausible implication is that patch-based diffusion priors are not only computationally attractive but also clinically consequential in data-scarce regimes.

6. Extensions beyond classical restoration

Patch-based diffusion is not limited to generic restoration or inverse imaging. The same local-denoising principle has been adapted to adversarial defense, high-resolution video interpolation, hybrid-resolution serving, and large-scale 3D completion.

DIFFender uses a unified, text-guided diffusion model to localize and restore adversarial patches (Kang et al., 2023, Wei et al., 2024). Its central empirical observation is the Adversarial Anomaly Perception phenomenon: multiple denoising runs of the same adversarial image produce consistent outputs in the background but differ significantly in adversarial patch regions. The estimated patch mask is obtained from repeated denoising with two prompts and binarization of the average absolute difference: $H$2 The same diffusion model is then used for inpainting-based restoration (Kang et al., 2023).

HiFI applies a patch-based cascaded pixel diffusion model to high-resolution frame interpolation (Hur et al., 2024). Rather than using separate models at progressively higher resolutions, it uses a single model that always performs diffusion at the same resolution and upsamples by processing overlapping patches of the inputs and the prior solution. The paper states that this drastically reduces memory usage at inference, allows a single model to solve both frame interpolation and spatial up-sampling, and achieves state-of-the-art or comparable performance on Vimeo, Xiph, X-Test, and SEPE-8K (Hur et al., 2024).

PATCHEDSERVE moves patching from model design to systems design (Sun et al., 16 Jan 2025). It is a patch management framework for SLO-optimized hybrid-resolution diffusion serving that splits requests of different resolutions into common-size patches, batches them together, and uses patch-level cache reuse. The paper reports 30.1% higher SLO satisfaction compared to a state-of-the-art serving system while not hurting image quality (Sun et al., 16 Jan 2025).

PatchScene extends the paradigm to voxelized 3D LiDAR scene completion (Xu et al., 2 Jun 2026). It breaks the global voxel space into overlapping local patches, performs diffusion denoising within each patch, and uses confidence-guided spatio-temporal fusion and an annular-flow diffusion strategy to propagate information from near-range to far-range regions. The paper reports state-of-the-art performance on SemanticKITTI and generalization from 20 m training ranges to 50 m scenes without retraining (Xu et al., 2 Jun 2026).

7. Reported advantages, trade-offs, and interpretive issues

Across the cited work, three advantages recur. The first is memory efficiency. Patch-based evaluation reduces the size of the tensor processed at each denoising step, which is the most direct explanation for the reported memory savings in MRI inference (Roy et al., 25 Jan 2025), patch-by-patch image generation (Arakawa et al., 2023), and high-resolution interpolation (Hur et al., 2024). The second is data efficiency. Because each image yields many patches, patch-based training can be effective in limited-data regimes, a point made explicitly in Patch Diffusion (Wang et al., 2023), PaDIS (Hu et al., 2024), out-of-distribution inverse problems (Hu et al., 2024), and MRI reconstruction (Sanda et al., 25 Sep 2025). The third is scalability to arbitrary or very large resolutions, enabled by tiling, overlap, or fixed-resolution recursion (Özdenizci et al., 2022, Hur et al., 2024).

The main trade-offs are equally consistent. Very small patches can cause a slight drop in performance (Roy et al., 25 Jan 2025), larger patch counts can worsen image quality and expose boundary inconsistencies (Arakawa et al., 2023), and improper padding or naive tiling leads to visible artifacts (Roy et al., 25 Jan 2025, Dashpute et al., 7 Oct 2025). Several papers therefore treat patch aggregation not as a peripheral engineering choice but as a core modeling decision.

A common misconception is that patch-based diffusion necessarily sacrifices global structure. The literature does not support such a universal claim. Some methods explicitly reintroduce global information through coordinate channels or partial full-image training (Wang et al., 2023), some through score aggregation over shifted grids (Hu et al., 2024), some through smooth overlap and averaging (Özdenizci et al., 2022, Dashpute et al., 7 Oct 2025), and some through staged structure-first then detail-refinement inference (Lin et al., 2024). Another misconception is that patching is only an inference-time trick. In fact, patch-based diffusion has been used for training-only acceleration, training-and-inference priors, and system-level serving optimization (Wang et al., 2023, Hu et al., 2024, Sun et al., 16 Jan 2025).

Within this landscape, “TDiff” denotes a patch-based diffusion viewpoint centered on local priors and patchwise denoising. In its thermal-imaging instantiation, TDiff is a plug-and-play prior trained on $H$3 and $H$4 thermal patches, evaluated on denoising, super-resolution, and deblurring, and combined with back-projection and least-squares guidance during inverse updates (Dashpute et al., 7 Oct 2025). More broadly, the surrounding literature suggests that patch-based diffusion has become a general strategy for reconciling strong diffusion priors with limited memory, limited data, and mismatched test conditions across 2D, video, medical, adversarial, and 3D domains.