From Sparse X-rays to 3D CT: Training-Free Reconstruction with Diffusion Priors

Abstract: Solving 3D medical inverse problems typically requires training dedicated supervised models for each specific task and measurement setting. To break this dependency, we present TF-PRDiT: a training-free conditional sampling framework that converts a frozen voxel-level 3D Diffusion Transformer prior into a versatile inverse medical problem solver. Building on the posterior-sampling view of diffusion inverse solvers, TF-PRDiT enforces measurement consistency during sampling via a task-specific forward operator rather than updating model weights, enabling a single pretrained prior to be reused across diverse conditional settings. Our method combines a predictor-corrector sampler with likelihood-based guidance on the denoised prediction, providing stable data-fidelity correction while preserving the underlying 3D anatomical prior. We highlight our framework's capability on the challenging task of X-ray-to-CT reconstruction by integrating a differentiable DRR projector to allow gradients to propagate directly from projection space back to voxels without any retraining. Experiments on LIDC-IDRI demonstrate that TF-PRDiT achieves strong reconstruction quality and uniquely scales to an arbitrary number of input X-rays (1-12) under a unified model, with performance improving consistently as additional views are provided. Beyond X-ray-to-CT, we show that simply swapping the forward operator extends the same frozen model to 3D super-resolution, volumetric infilling, and deblurring without any task-specific retraining, demonstrating that a single 3D diffusion prior can serve as a universal solver for volumetric medical inverse problems.

• The paper introduces TF-PRDiT, a training-free predictor–corrector sampler that combines a frozen voxel-level 3D diffusion prior with differentiable forward operators for sparse X-ray-to-CT reconstruction and other volumetric inverse problems.
• TF-PRDiT achieves 29.06 dB PSNR and 0.767 SSIM from two X-ray views, surpasses supervised baselines in the biplanar setting, and improves steadily to 34.61 dB PSNR and 0.880 SSIM with 12 views.
• The same prior transfers to 3D super-resolution, infilling, and deblurring by changing only the measurement operator, although iterative inference, operator accuracy, and validation beyond simulated chest CT remain important limitations.

Overview

This paper introduces TF-PRDiT, a training-free conditional sampling framework that converts a frozen, voxel-level 3D Diffusion Transformer prior into a general-purpose solver for volumetric medical inverse problems. The central claim is that a single pretrained unconditional prior over chest CT volumes can be reused across heterogeneous tasks—sparse X-ray-to-CT reconstruction, 3D super-resolution, volumetric infilling, and deblurring—without any task-specific retraining, by imposing measurement consistency during inference through a differentiable forward operator $\mathcal{A}$. The work builds on the posterior-sampling view of diffusion inverse solvers, most notably Diffusion Posterior Sampling (DPS), but instantiates it in native voxel space rather than in 2D images or compressed latent representations (Chung et al., 2022, Zhang et al., 2023).

Method

The problem is posed as $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$, where $\mathbf{x}$ is a 3D volume and $\mathcal{A}$ is a known, differentiable forward operator. For X-ray-to-CT, $\mathcal{A}$ is a differentiable digitally reconstructed radiograph (DRR) projector implemented with DiffDRR, so that gradients flow directly from projection space back to voxels [gopalakrishnan2022fast]. Swapping $\mathcal{A}$ for downsampling, masking, or blurring operators yields the other restoration tasks, and the number of X-ray views $M$ becomes a runtime input: each additional view simply contributes another projection-space residual $$\|\mathcal{P}_{g_i}(\hat{\mathbf{x}}_0)-\mathbf{y}_i\|_2^2$$ to the guidance loss.

The frozen prior is a pixel-level residual diffusion Transformer (PRDiT) trained unconditionally on LIDC-IDRI at $128^3$ resolution, jointly predicting noise and the clean-volume estimate $(\hat{\boldsymbol{\epsilon}}, \hat{\mathbf{x}}_0)$. The authors argue that voxel-space generation matters here because latent compression can attenuate fine anatomical boundaries. The sampler is a predictor–corrector scheme under the cosine-sine parameterization, with three components that constitute the paper's methodological contributions:

• $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$0-step predictor with variance-preserving corrector. The predictor strides $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$1 timesteps using the prior-induced direction $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$2; a corrector then rescales and re-noises to restore the marginal variance at the target timestep, preventing drift when $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$3. Larger $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$4 promotes broader stochastic exploration, which the authors find beneficial for severely ill-posed projection-to-volume reconstruction.
• Likelihood guidance on the denoised estimate. Following the DPS approximation, the Jacobian $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$5 is omitted for tractability, and gradients of the data-consistency loss are backpropagated only through $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$6 while the denoiser remains fixed. The paper is explicit that it introduces no new posterior-guidance principle; the contribution is the adaptation to native 3D volumes with differentiable volumetric operators.
• Cosine-decay guidance schedule. The guidance scale $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$7 is large at high noise to shape global structure and decays to zero as $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$8, avoiding the under-correction/over-correction trade-off of fixed guidance.

X-ray-to-CT results

Experiments use LIDC-IDRI with the X2CT-GAN split (916 train / 102 test), CT intensities clamped to $$\mathbf{y} = \mathcal{A}(\mathbf{x}) + \boldsymbol{\xi}$$9 HU, and projections generated by DiffDRR. Metrics are computed on full volumes rather than selected slices, with mean ± standard deviation over three seeds.

Views	MSE ($\mathbf{x}$0)	PSNR (dB)	SSIM	SNR (dB)
1	22.51 ± 1.46	21.34 ± 0.30	0.509 ± 0.012	6.01 ± 0.30
2	3.65 ± 0.12	29.06 ± 0.12	0.767 ± 0.002	13.72 ± 0.12
4	1.96 ± 0.02	31.71 ± 0.06	0.824	16.37
6	1.39 ± 0.02	33.20 ± 0.07	0.852	17.86
8	1.19 ± 0.01	33.89 ± 0.05	0.865	18.55
12	1.02 ± 0.02	34.61 ± 0.07	0.880	19.27

In the biplanar setting, TF-PRDiT achieves 29.06 dB PSNR and 0.767 SSIM, surpassing supervised baselines including PerX2CT, DiffuX2CT, and DX2CT—improving over DX2CT by 0.70 dB PSNR and 0.004 SSIM [kyung2023perspective, liu2024diffux2ct, jeong2025dx2ct]. This is a notable result: a training-free sampler outperforms supervised models trained specifically for the two-view configuration. In the single-view setting, by contrast, TF-PRDiT is only competitive with, and slightly behind, X2CT-GAN (21.34 vs. 21.75–22.13 dB PSNR), and the paper acknowledges that the single-view problem is highly under-constrained. The comparison with published numbers carries a caveat the authors state plainly: differences in projection generation, preprocessing, and metric implementations prevent a strictly controlled comparison against results copied from prior papers.

The most distinctive empirical finding is monotonic scaling from 1 to 12 views under one frozen model, with the largest jump at 2 views, where the second projection resolves depth ambiguity. This behavior is structurally impossible for supervised X-ray-to-CT architectures whose input channels and conditioning are fixed at training time.

Generalization and ablations

By replacing only $\mathbf{x}$1, the same frozen prior solves $\mathbf{x}$2/$\mathbf{x}$3/$\mathbf{x}$4 super-resolution (e.g., 38.12 dB at $\mathbf{x}$5 with $\mathbf{x}$6), volumetric infilling, and deblurring. Ablations show task-dependent optimal predictor strides: $\mathbf{x}$7 is best for X-ray-to-CT (3.65 MSE, 29.06 dB), reflecting the value of broader exploration under severe ambiguity, while $\mathbf{x}$8 minimizes MSE for super-resolution (0.42 vs. 0.46 at $\mathbf{x}$9). Cosine-decay guidance consistently outperforms fixed guidance on both infilling and 2-view reconstruction, with the ablation attributing the gain to strong early structural correction and reduced late-stage over-correction.

Limitations and open questions

The paper concedes several limitations directly. Inference is slower than feed-forward supervised models, since each reconstruction requires iterative sampling with per-step backpropagation through $\mathcal{A}$0. Reconstruction quality depends on the accuracy of the forward operator: unmodelled acquisition geometry or corruption can produce biased guidance even with a strong prior. The enhanced PRDiT normalization changes (affine-free RMSNorm on the clean-image branch) are not ablated individually, so the authors do not attribute reconstruction gains to either normalization change in isolation. Validation is confined to chest CT on LIDC-IDRI with simulated DiffDRR projections; broader validation across anatomies, scanners, and clinical acquisition settings remains future work. Open questions include whether the DPS-style Jacobian approximation remains reliable at even fewer views or with realistic noise and scatter models, and whether inference cost can be reduced without sacrificing the exploration benefit of large predictor strides.

Conclusion

TF-PRDiT demonstrates that a frozen voxel-level 3D diffusion prior, combined with differentiable measurement guidance, a $\mathcal{A}$1-step predictor–corrector sampler, and cosine-decayed likelihood guidance, can match or exceed supervised baselines on biplanar X-ray-to-CT reconstruction and scale continuously across 1–12 views, while transferring to super-resolution, infilling, and deblurring with only a change of forward operator. Its practical significance lies in decoupling the generative prior from the measurement model, though its clinical applicability depends on forward-operator fidelity, inference cost, and validation beyond simulated single-dataset conditions.