DEFT: Doob’s h-transform Fine-Tuning

• The paper introduces DEFT, a diffusion-model adaptation framework that learns a generalized h-transform correction to convert an unconditional reverse process into a conditional one without retraining the full model.
• DEFT maintains efficiency by training only a small auxiliary network—around 4%–9% of the backbone—thereby reducing compute costs while achieving state-of-the-art performance on inverse problems.
• Empirical results across imaging, CT reconstruction, and protein design demonstrate DEFT’s superior performance on perceptual metrics and lower sampling cost compared to traditional methods.

Searching arXiv for DEFT and closely related Doob h-transform diffusion adaptation papers. Searching arXiv for "DEFT diffusion Doob h-transform" and related follow-up work. Doob’s $h$-transform efficient fine-tuning (DEFT) is a diffusion-model adaptation framework that reinterprets conditional generation as conditioning an unconditional reverse diffusion process and then learns only the resulting generalized $h$-transform correction, rather than retraining the full backbone. In its original formulation, DEFT keeps a pretrained unconditional diffusion or score model fixed and trains a small auxiliary network $h_t^\phi$ so that the conditional reverse dynamics are realized as the unconditional dynamics plus a learned additive correction. The method was introduced to exploit large, expensively trained unconditional diffusion models for inverse problems and other conditional generation tasks, while unifying conditional training and conditional sampling under Doob’s $h$-transform (Denker et al., 2024).

1. Problem setting and conceptual definition

DEFT is motivated by a standard posterior-sampling setup in which a pretrained unconditional model approximates the prior score

$$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$

while the downstream task requires sampling from $p(x_0\mid y)$, or equivalently from the posterior score $\nabla_{x_t}\log p_t(x_t\mid y)$ (Denker et al., 2024). Inverse problems supply the canonical examples: inpainting, super-resolution, CT reconstruction, and motif-constrained protein generation all require conditional sampling, but end-to-end conditional retraining is expensive, and inference-time guidance methods are often hyperparameter-sensitive or computationally heavy.

The central DEFT claim is that the missing conditional information is not an arbitrary task-specific residual. It is the generalized $h$-transform term induced by conditioning the reverse diffusion process. This makes DEFT conceptually different from ordinary parameter-efficient fine-tuning: instead of partially modifying the unconditional backbone, it keeps that backbone unchanged and learns the time-dependent correction needed to convert the unconditional reverse process into the correct conditional one (Denker et al., 2024).

This framing also gives DEFT a precise efficiency target. The expensive pretrained unconditional generator remains intact, and only a small conditional module is optimized. In the original imaging and protein experiments, the trainable network is repeatedly described as much smaller than the base model, often about $4\%$–$9\%$ of the backbone in the major experiments, with about $h$0–$h$1 in the CT setting (Denker et al., 2024). A practical implication is that DEFT can be used even when only forward-pass access to the unconditional model is available.

2. Doob’s $h$2-transform and the generalized conditional reverse process

The mathematical basis of DEFT is the reverse-time SDE formulation of diffusion models. The unconditional forward process is written as

$h$3

and the unconditional reverse process as

$h$4

If one conditions on a terminal event $h$5, the reverse process remains an SDE but acquires an extra drift term,

$h$6

where the added gradient is the classical Doob $h$7-transform correction (Denker et al., 2024).

DEFT uses the generalized version appropriate for noisy observations and posterior sampling. Given an observation model $h$8, the conditional reverse process becomes

$h$9

with

$h_t^\phi$0

This yields the exact posterior-score decomposition

$h_t^\phi$1

which is the formal version of “prior score plus guidance” (Denker et al., 2024).

The original DEFT paper also gives a path-space representation. If $h_t^\phi$2 is the path measure of the unconditional SDE, then the conditional path measure satisfies

$h_t^\phi$3

From this viewpoint, denoising score matching, stochastic control, and Schrödinger-bridge-style formulations are equivalent representations of the same $h_t^\phi$4-transformed conditional process. This equivalence is one of DEFT’s main theoretical unifications: conditional sampling and conditional training target the same object, but parameterize it differently (Denker et al., 2024).

3. Training objective, learned correction, and inference mechanism

DEFT learns a network $h_t^\phi$5 whose optimal value is the generalized $h_t^\phi$6-transform score correction,

$h_t^\phi$7

The core score-matching objective is

$h_t^\phi$8

Its minimizer is exactly the desired conditional correction (Denker et al., 2024).

For DDPM-style $h_t^\phi$9-parameterization, this becomes

$h$0

with

$h$1

The pretrained unconditional model $h$2 is frozen. Only $h$3 is updated. A consequential implementation detail is that the objective needs evaluation of the pretrained model but not differentiation through it (Denker et al., 2024).

The learned correction also admits a Tweedie-estimate interpretation. If $h$4 is the unconditional denoised estimate, then the conditional estimate is

$h$5

Thus DEFT may be viewed as learning how to correct the unconditional denoiser into a conditional denoiser (Denker et al., 2024).

The practical parameterization used in the original paper is

$h$6

For imaging, $h$7 is an attention U-Net and $h$8 is a small 3-layer MLP with SiLU activations; the final layer of $h$9 and most of $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$0 are zero-initialized so that training starts close to the unconditional sampler (Denker et al., 2024).

At inference, DEFT simply replaces the unconditional $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$1-prediction by the corrected one,

$$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$2

where $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$3 and $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$4. The resulting $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$5 is then used in the standard DDIM or DDPM reverse update. In operational terms, the backbone remains an unconditional prior model and the $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$6-network supplies the conditional score correction (Denker et al., 2024).

4. Position relative to guidance, conditional training, and adjacent $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$7-transform methods

A major contribution of DEFT is interpretive rather than purely algorithmic. The framework recasts several existing diffusion-conditioning strategies as approximations, special cases, or alternative parameterizations of the same generalized $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$8-transform (Denker et al., 2024).

Reconstruction-guidance methods such as DPS are interpreted as online approximations to $$s_t^{\theta^*}(x_t) \approx \nabla_{x_t} \log p_t(x_t),$$9. In that view, guidance-scale tuning acts as an approximation parameter, and the computational burden comes from repeatedly differentiating through the large unconditional model during sampling. DEFT moves that computation offline into a learned correction network, eliminating test-time backpropagation through the base model and avoiding a guidance-scale schedule (Denker et al., 2024).

Methods that directly train the full conditional score, such as conditional denoising estimators, target $p(x_0\mid y)$0 itself. DEFT instead learns only the additive residual

$p(x_0\mid y)$1

while reusing the unconditional score already stored in the base network. This is why DEFT is parameter-efficient rather than full conditional retraining (Denker et al., 2024).

Subsequent literature has developed several closely related $p(x_0\mid y)$2-transform variants. “Iterative Importance Fine-tuning of Diffusion Models” treats DEFT as prior work and extends it to a self-supervised regime in which target samples are unavailable. It iteratively generates trajectories, reweights them with path-space importance weights, resamples, and refits the $p(x_0\mid y)$3-transform via a supervised score-matching loss, thereby yielding an iterative, importance-weighted DEFT-like procedure (Denker et al., 6 Feb 2025). “Training-Free Adaptation of Diffusion Models via Doob’s $p(x_0\mid y)$4-Transform” instead computes the Doob correction online by Monte Carlo during sampling and performs no parameter updates at all, making it an inference-time rather than fine-tuning variant (Zhu et al., 18 Feb 2026). “Inference-Time Alignment for Diffusion Models via Doob’s Matching” likewise keeps the base score frozen but learns an auxiliary $p(x_0\mid y)$5-head by gradient-penalized regression, with non-asymptotic guarantees for both guidance estimation and terminal-sample convergence (Chang et al., 10 Jan 2026).

These developments suggest a broader research program around the same decomposition: $p(x_0\mid y)$6 A plausible implication is that DEFT functions as the parameter-efficient branch of this family, whereas DOIT and $p(x_0\mid y)$7-Edit occupy the training-free inference-time branch (Zhu et al., 18 Feb 2026).

5. Empirical results in imaging, medical reconstruction, and protein design

In the original DEFT experiments, the method is evaluated on natural-image inverse problems, CT reconstruction, and protein motif scaffolding, with the explicit claim that it is much faster than existing baselines while achieving state-of-the-art performance across a variety of linear and non-linear benchmarks (Denker et al., 2024).

On ImageNet $p(x_0\mid y)$8 inverse problems, DEFT uses a pretrained unconditional model of about $p(x_0\mid y)$9–$\nabla_{x_t}\log p_t(x_t\mid y)$0M parameters and a trainable correction network of about $\nabla_{x_t}\log p_t(x_t\mid y)$1M parameters. In inpainting, DEFT reports PSNR $\nabla_{x_t}\log p_t(x_t\mid y)$2, SSIM $\nabla_{x_t}\log p_t(x_t\mid y)$3, KID $\nabla_{x_t}\log p_t(x_t\mid y)$4, LPIPS $\nabla_{x_t}\log p_t(x_t\mid y)$5, and top-1 $\nabla_{x_t}\log p_t(x_t\mid y)$6, while the total timing is $\nabla_{x_t}\log p_t(x_t\mid y)$7 hours including training and $\nabla_{x_t}\log p_t(x_t\mid y)$8 seconds per sample. The paper notes that RED-diff is stronger on distortion metrics in this task, whereas DEFT is best on perceptual metrics (Denker et al., 2024).

On super-resolution, DEFT again sacrifices some PSNR but attains the best perceptual metrics, with KID $\nabla_{x_t}\log p_t(x_t\mid y)$9, LPIPS $h$0, and top-1 $h$1. On HDR reconstruction, it reports PSNR $h$2, SSIM $h$3, and KID $h$4, outperforming the cited baselines. On phase retrieval, the reported values are PSNR $h$5, SSIM $h$6, and KID $h$7. On nonlinear deblurring, DEFT reports PSNR $h$8, SSIM $h$9, KID $4\%$0, LPIPS $4\%$1, and top-1 $4\%$2, with timing around $4\%$3 hours total and $4\%$4–$4\%$5 seconds per sample (Denker et al., 2024).

In CT, DEFT is compared against DPS and RED-diff on AAPM 2016 and LoDoPab-CT. On AAPM, DEFT reports PSNR $4\%$6, SSIM $4\%$7, and time $4\%$8s; on LoDoPab-CT, it reports PSNR $4\%$9, SSIM $9\%$0, and time $9\%$1s. The paper emphasizes that DEFT uses 100 DDIM steps while the baselines use 1000, and describes the medical-imaging results as strong reconstruction performance with substantially lower sampling cost (Denker et al., 2024).

In protein motif scaffolding, DEFT fine-tunes a $9\%$2k-parameter module on top of a pretrained unconditional Genie model with 4.1M parameters. Reported success rates are $9\%$3 for DEFT and $9\%$4 for DPS, with DEFT solving $9\%$5 tasks and DPS solving $9\%$6. An amortized conditional model still performs better, at $9\%$7, but it uses a larger model and longer training. The paper therefore positions DEFT as the more efficient conditional adaptation scheme rather than the highest-capacity fully amortized one (Denker et al., 2024).

6. Later instantiations, limitations, and nomenclature

A direct application-specific instantiation is DEFT-VTON, which adapts the DEFT principle to virtual try-on. There, the pretrained autoencoder and score diffusion U-Net are frozen, and only the h-transform network is trained. The paper reports that baseline PEFT trains about $9\%$8–$9\%$9 of the backbone, whereas DEFT trains only about $h$00. It further adds an adaptive consistency loss to reduce denoising steps, reporting that the consistency-enhanced system can work with as few as 15 denoising steps, that quality metrics stabilize after about 12 steps in an ablation, and that adding the consistency loss speeds up VTO inference by up to $h$01. On VITON-HD, the 25-step DSM-only DEFT variant reports $h$02 SSIM, $h$03 LPIPS, $h$04 FID, and $h$05 KID, while the 15-step consistency-enhanced model reports $h$06 SSIM, $h$07 LPIPS, $h$08 FID, and $h$09 KID (Xu et al., 16 Sep 2025).

DEFT’s main practical limitation in the original formulation is that it requires a small paired fine-tuning dataset for the target task. This is the central tradeoff relative to zero-shot guidance methods such as DPS or $h$10GDM. Another limitation, made explicit in DEFT-VTON, is dependence on the pretrained unconditional model: because the trainable $h$11-transform network is only a small fraction of the backbone, artifacts, biases, and failure modes of the base prior can propagate into the adapted model (Denker et al., 2024).

The acronym also requires disambiguation. In later literature, “DEFT” may refer to unrelated methods. “Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives” defines DEFT as Dynamic Entropy Fine-Tuning and explicitly does not discuss Doob’s $h$12-transform (Wang et al., 11 Feb 2026). “Device-Edge Cooperative Fine-Tuning of Foundation Models as a 6G Service” uses DEFT to mean device-edge fine-tuning, again without any Doob-transform content (Wu et al., 2023). Within diffusion-model adaptation, however, DEFT specifically denotes the frozen-backbone, learned-$h$13-correction framework introduced for conditional generation in inverse problems (Denker et al., 2024).

Taken together, the DEFT line establishes a particular design pattern for diffusion adaptation: preserve the pretrained unconditional prior, learn only the generalized $h$14-transform correction, and realize conditional generation by adding that correction to the reverse denoising dynamics. Subsequent work suggests that this pattern extends both toward self-supervised fine-tuning and toward training-free inference-time control, but the defining DEFT identity remains the same: $h$15