Exact Posterior Score Estimation for Solving Linear Inverse Problems

Abstract: Diffusion and flow-based models learn powerful data priors by training a denoiser to reverse Gaussian corruption. To use this prior to solve a linear inverse problem, one needs to sample from the posterior, but the score that the prior provides is the unconditional score, not the posterior score. Existing methods either steer a fixed pretrained denoiser with approximate measurement-matching corrections, or train a conditional restoration model that abandons the denoising structure of the prior. We derive the exact posterior score in closed form for linear Gaussian inverse problems under general Gaussian interpolants, and show that posterior sampling reduces to a denoising problem at an operator-dependent shifted pivot under an anisotropic noise covariance. We turn this identity into Exact Posterior Score (EPS), a denoising training objective that preserves the input/output structure of standard pretraining and can therefore be trained from scratch or fine-tuned from a pretrained denoiser. At inference, EPS uses the same sampler as the underlying backbone, with no likelihood gradients or projections. We evaluate EPS on five linear inverse problems across FFHQ and ImageNet, where it outperforms training-free and training-based baselines on fidelity, perceptual, and distributional metrics, while using roughly an order of magnitude fewer denoiser evaluations than gradient-based posterior samplers.

• The paper introduces Exact Posterior Score (EPS) with a closed-form derivation under a linear Gaussian setting, enabling precise posterior sampling using diffusion models.
• The paper reformulates the denoising process by computing an operator-aware pivot that integrates measurement data, resulting in improved efficiency and performance.
• The paper demonstrates empirical advantages over prior methods in various inverse tasks, achieving faster sampling and enhanced fidelity on key performance metrics.

Exact Posterior Score Estimation for Solving Linear Inverse Problems

Problem Setting and Existing Methods

Linear inverse problems, where one seeks to reconstruct $x_0$ from noisy linear measurements $y = A x_0 + \eta$, are central in computational imaging. Key applications include compressive sensing, inpainting, super-resolution, and deblurring. The inherent ill-posedness for non-invertible or ill-conditioned $A$ renders the correct target not a single estimate but the posterior $p(x_0|y)$.

Diffusion and flow-based generative models provide strong off-the-shelf priors for such problems, but vanilla application yields unconditional samples. Sampling the posterior $p(x_0|y)$ using these models necessitates access to the posterior score $\nabla_{x_t} \log p(x_t|y)$ during diffusion reversal, rather than the unconditional score $\nabla_{x_t} \log p(x_t)$. Existing approaches fall into two broad categories:

• Training-free methods: These use a fixed (pretrained) denoiser and introduce measurement-matching updates, e.g., Diffusion Posterior Sampling (DPS), DDNM, $\Pi$GDM, often with corrections derived via projected denoised estimates or empirical measurement loss gradients.
• Training-based methods: These re-train or fine-tune models explicitly conditioned on $y$, such as conditional diffusion models (e.g., Palette), bridge-based models, or posterior distillation frameworks.

However, training-free approaches are inherently approximate (the guidance merely approximates the true posterior score), while generic training-based approaches lack inductive bias toward the true posterior denoising geometry and do not leverage the operator structure of the forward model.

Theoretical Contribution: Closed-Form Posterior Score

The core theoretical result is the derivation of the exact form of the posterior score and posterior denoiser under a linear Gaussian setting:

Given the diffusion interpolant $x_t = \alpha_t x_0 + \beta_t \epsilon$, and $y = A x_0 + \eta$0, both $y = A x_0 + \eta$1 and $y = A x_0 + \eta$2 Gaussian, the conditional density $y = A x_0 + \eta$3 remains Gaussian (by marginalization). The authors derive:

$y = A x_0 + \eta$4

where:

• $y = A x_0 + \eta$5
• $y = A x_0 + \eta$6
• $y = A x_0 + \eta$7 is a Tweedie-type denoiser: the MMSE estimator of $y = A x_0 + \eta$8 given observation $y = A x_0 + \eta$9 corrupted under anisotropic covariance $A$0.

Posterior sampling thus reduces to denoising at an operator-dependent pivot $A$1 and anisotropic uncertainty $A$2, with the core insight that the measurement $A$3 manifests not as additional conditioning, but as a shift and reweighting in the denoising geometry.

EPS: Training and Inference

The authors formulate Exact Posterior Score (EPS), a denoising regression objective that is fully aligned with the structure of standard diffusion model pretraining, with the exception that the model input is $A$4 (not $A$5) and is noised according to $A$6 (not isotropic noise).

EPS can be fine-tuned from a pretrained unconditional denoiser or trained from scratch. At inference, sampling is performed with the exact same sampler as used for the backbone, substituting the EPS denoiser which takes as input the pivot and measurement.

• Training: The denoiser $A$7 minimizes expected squared error to $A$8 given $A$9 across corrupted data and measurement noise, with the anisotropic geometry determined uniquely by $p(x_0|y)$0 and $p(x_0|y)$1.
• Sampling: Each diffusion step queries $p(x_0|y)$2; no gradients or projections on the likelihood are needed.

A crucial technical contribution is the realization that the required anisotropic noising can be simulated using typical isotropic noise via the closed-form pivot construction, vastly simplifying implementation.

Critical Comparison with Prior Approaches

The closed-form EPS construction clarifies the limitation of all prior training-free methods: they approximate the exact posterior denoiser $p(x_0|y)$3 with the unconditional denoiser $p(x_0|y)$4, possibly augmented by correction terms. This introduces inevitable bias unless $p(x_0|y)$5 is isotropic or in high-noise limits.

Furthermore, popular conditional or bridge-based methods (e.g., Palette, InDI, I2SB) train models exposed only to $p(x_0|y)$6 pairs, not to the analytic pivot or covariance derived from the operator. Thus, these networks cannot fully leverage the structure that the exact posterior provides; they must learn operator effects end-to-end, which is inefficient and less principled.

Empirical Evaluation and Numerical Results

EPS is evaluated on five canonical linear inverse problems (random inpainting, box inpainting, $p(x_0|y)$7 super-resolution, Gaussian deblurring, motion deblurring) across two datasets: FFHQ and ImageNet. Results are compared to major baselines in both training-free (DPS, DAPS, DDNM, $p(x_0|y)$8GDM, MPGD) and training-based (Palette) families.

Highlights from the empirical results:

• Metric dominance: EPS matches or exceeds all baselines on pointwise (PSNR, SSIM), perceptual (LPIPS, FID), and distributional (CRPS, MMD) metrics, often by substantial margins.
• Sampling efficiency: EPS achieves its asymptotic performance in ∼20 sampling steps (NFE), while all baselines require 5–10x more steps and still underperform.
• Posterior mean at high noise: A single high-noise evaluation—direct Tweedie at $p(x_0|y)$9—gives the posterior mean $p(x_0|y)$0, which is MMSE-optimal for distortion metrics but, as expected, trades off distributional realism.
• Operator-agnostic efficiency: The computational overhead of the structured pivot solve is negligible (<1 ms per step), and the full model is as fast as the unconditional backbone at inference.

Ablations confirm:

• Conditioning on the structured pivot, not $p(x_0|y)$1, is critical.
• The benefit of EPS over Palette arises wholly from this input alignment—Palette cannot match distributional or distortion metrics given matched capacity and training regime.
• Fine-tuning EPS from a pretrained denoiser converges in a small fraction of the iterations needed for full retraining.

Theoretical and Practical Implications

EPS clarifies the mathematical structure of posterior sampling with diffusion models in the linear Gaussian case, pinpointing exactly which guidance terms are necessary for exact posterior recovery—both theoretically and algorithmically. This provides several immediate implications:

• Posterior uncertainty: EPS yields properly calibrated samples, permitting uncertainty quantification and downstream Bayesian decision-making.
• Efficient fine-tuning: Since only the input geometry changes (not the output or target distribution), EPS enables rapid adaptation across tasks with minimal computational cost.
• Sampling modularity: EPS is compatible with any diffusion backbone and sampler, requiring no modification to architecture or inference code except the analytic input transformation and potentially light fine-tuning.
• Foundation for extensions: While the current derivation holds exactly only for linear Gaussian operators, it establishes an architectural inductive bias (posterior-aligned pivoting and anisotropic denoising) likely beneficial for more general inverse or likelihood-based imaging models.

Future Directions

• Extension to nonlinear and non-Gaussian operators: The closed-form posterior score is unique to the linear Gaussian case. Adapting EPS principles (e.g., analytic computation of a pivot) in settings with nonlinear $p(x_0|y)$2 or non-Gaussian $p(x_0|y)$3 remains an open challenge.
• Integration with latent diffusion: Solving pixel-space inverse problems with latent diffusion backbones will require careful handling of the nonlinearity induced by the decoder.
• Amortization and meta-learning: Given the negligible cost of fine-tuning and the generality of the pivot construction, amortized or multi-task versions of EPS could support broad classes of inverse operators with minimal additional storage or computation.

Conclusion

This work establishes a theoretically exact, practically efficient methodology for posterior sampling in linear inverse problems using generative diffusion priors. By recognizing that the posterior score reduces to operator-aware denoising at a measurement-induced pivot, EPS simultaneously inherits the strengths of pretrained diffusion models and achieves exact Bayesian calibration with minimal computational overhead, outperforming prior training-free and conditional approaches across fidelity, perception, and uncertainty metrics (2606.17048).