---
title: 'DiffImpute: Diffusion Imputation Method'
url: https://www.emergentmind.com/topics/diffimpute
type: topic
---

# DiffImpute: Diffusion Imputation Method

DiffImpute denotes a diffusion-model-based approach to missing-value imputation, and more specifically the DDPM-based tabular imputer introduced in “DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Model” [2403.13863]. In that original formulation, a denoising diffusion probabilistic model is trained on complete tabular datasets and then used to reconstruct missing entries while preserving observed values through mask-aware sampling, with additional mechanisms such as Harmonization and a refined non-Markovian sampler [2403.13863]. Subsequent work uses the term in a broader sense for conditional, score-based, residual, EM-based, MNAR-aware, cross-domain, and uncertainty-aware imputers across tabular, time-series, image, spatiotemporal, and microbiome settings [2210.17128][2405.20690][2605.25439]. This suggests a broader editorial usage in which “DiffImpute” refers not only to one named method but to a family of diffusion-based imputation systems.

## 1. Definition and historical placement

The original DiffImpute method addresses tabular data imputation under Missing Completely At Random (MCAR) and Missing At Random (MAR) by training a DDPM on complete rows and then conditioning the reverse process on observed entries at inference time [2403.13863]. Its design premise is that a generative model of complete rows can produce credible imputations for missing entries “without undermining the authenticity of the existing data,” and that the reverse trajectory can be made more coherent through Harmonization and a refined non-Markovian sampling process [2403.13863].

A closely related but earlier tabular line is “Conditional Score-based Diffusion Models for Tabular data” (TabCSDI), which adapts CSDI-style conditional diffusion to tabular vectors and explicitly studies one-hot encoding, analog bits encoding, and feature tokenization for mixed numerical and categorical variables [2210.17128]. Whereas DiffImpute trains on complete tabular datasets and conditions at inference through mask-aware sampling, TabCSDI directly learns the conditional distribution \(p_\theta(\mathbf{x}_0^{ta} \mid \mathbf{x}_0^{co})\) for target versus conditional features [2210.17128].

Later work broadened the methodological scope. DiffPuter combines an unconditional score-based diffusion model with an Expectation-Maximization algorithm so that diffusion training corresponds to the M-step and conditional sampling to the E-step [2405.20690]. PRDIM extends diffusion imputation to Missing Not at Random (MNAR) by jointly modeling data and missing mask, \(p_{\theta,\phi}(X,M)=p_\theta(X)p_\phi(M\mid X)\), through a diffusion backbone, a pattern recognizer, and an EM loop [2605.25439]. MissHDD argues that a single diffusion process is insufficient for heterogeneous tabular data and separates continuous DDIM-based denoising from a discrete latent-path diffusion channel [2511.14543]. Diff-Joint further generalizes the problem by treating some `na` values as meaningfully missing and therefore not necessarily imputable, learning a joint diffusion model over data and a latent missingness mask [2606.05073].

The same broad pattern appears outside conventional tabular imputation. DSDI, DiffImp, RDPI, and LSSDM use diffusion for probabilistic time-series or spatiotemporal imputation, each altering the denoiser, conditioning pathway, or latent target [2501.06585][2410.13338][2412.12642][2409.08917]. CD\(^2\)-TSI adds cross-domain adaptation, while DepMicroDiff adapts conditional diffusion to sparse microbiome abundance tables with a Dependency-Aware Transformer and LLM-conditioned metadata [2506.12412][2507.23676].

## 2. Core probabilistic formulation

Across the literature, DiffImpute-style methods share the same generative backbone: a forward diffusion process that gradually corrupts data and a learned reverse process that reconstructs missing content conditioned on observed content. In the original DiffImpute, the DDPM forward chain is
\[
q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) = \mathcal{N}\Bigl(\mathbf{x}_t;\, \sqrt{\alpha_t}\mathbf{x}_{t-1},\, (1 - \alpha_t)\mathbf{I}\Bigr),
\]
with closed form
\[
q(\mathbf{x}_t \mid \mathbf{x}_0)
= \mathcal{N}\Bigl(\mathbf{x}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{x}_0,\; (1 - \bar{\alpha}_t)\mathbf{I}\Bigr),
\]
and a reverse model
\[
p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t) = \mathcal{N}\bigl(\mathbf{x}_{t-1};\, \boldsymbol{\mu}_\theta(\mathbf{x}_t, t),\, \sigma_t^2 \mathbf{I}\bigr),
\]
parameterized through a denoising network \(f_\theta(\mathbf{x}_t,t)\) that predicts the Gaussian noise [2403.13863]. The training objective is a smooth L1 noise-prediction loss, while TabCSDI uses the corresponding conditional diffusion loss for target features given conditional features [2210.17128].

For imputation, the central object is the conditional distribution over missing entries given observed ones. In DiffImpute this is enforced procedurally: the observed region is re-noised from the actual observed data at every reverse step, the unknown region is denoised by the model, and both are merged by the binary mask [2403.13863]. TabCSDI formalizes the same logic by splitting each example into a conditional part \(\mathbf{x}^{co}\) and a target part \(\mathbf{x}^{ta}\), learning \(p_\theta(\mathbf{x}_0^{ta} \mid \mathbf{x}_0^{co})\) directly [2210.17128].

DiffPuter makes the latent-variable structure explicit. Let \(\mathbf{x}=(\mathbf{x}^{\rm obs},\mathbf{x}^{\rm mis})\). Its forward perturbation is
\[
\mathbf{x}_t = \mathbf{x}_0 + \sigma(t)\,\boldsymbol{\varepsilon},\quad \boldsymbol{\varepsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}),
\]
and the reverse-time SDE is
\[
d\mathbf{x}_t = -2\,\dot{\sigma}(t)\,\sigma(t)\,\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t)\,dt + \sqrt{2\,\dot{\sigma}(t)\,\sigma(t)}\,d\boldsymbol{\omega}_t.
\]
The score-matching loss upper-bounds the negative log-likelihood of the data density, which is why the diffusion training step functions as an approximate M-step, while the tailored conditional sampler functions as an E-step [2405.20690].

PRDIM modifies this standard probabilistic picture to handle non-ignorable missingness. With \(X^{\text{obs}}=X\odot M\) and \(X^{\text{mis}}=X\odot(\mathbf{1}-M)\), it maximizes the observed-data likelihood
\[
p_{\theta,\phi}(X^{\text{obs}}, M)
=
\int p_\theta(X)\,p_\phi(M\mid X)\,dX^{\text{mis}},
\]
under an ELBO and an EM procedure [2605.25439]. This changes the target from learning \(p_\theta(X)\) alone to learning the joint \(p_{\theta,\phi}(X,M)\), which is essential under MNAR because \(p_\phi(M\mid X)\) depends on unobserved values [2605.25439].

Diff-Joint generalizes the same principle in a different direction. It augments the domain with an explicit `na` state and models a joint diffusion state \(s=(x,c)\), where \(c\in\{0,1\}^d\) is a latent mask of meaningfully missing entries. The forward SDE and denoising score matching are then applied to the concatenated representation of data and mask, so the model learns \(p(x,c)\) rather than only \(p(x)\) [2606.05073].

## 3. Conditioning mechanisms and denoising architectures

A defining feature of DiffImpute systems is how conditioning on observed information is injected into the denoiser. The original DiffImpute uses a mask-aware reverse trajectory in which known entries are repeatedly replaced by noised versions of the actual observed values, while unknown entries follow the learned reverse update [2403.13863]. To support this in tabular data, it introduces four tabular denoising networks—MLP, ResNet, Transformer, and a tabular U-Net—and a Time Step Tokenizer that produces learned scale-and-shift embeddings for each feature dimension [2403.13863]. In its reported experiments, the Transformer denoiser is the strongest variant [2403.13863].

TabCSDI adapts CSDI from time series to feature-wise tabular vectors by removing temporal architecture and using a transformer plus MLP over features, while systematically studying one-hot encoding, analog bits encoding, and feature tokenization for mixed data types [2210.17128]. Feature tokenization is particularly relevant because it places numerical and categorical features into a common embedding space of fixed dimension, which proved especially effective for multi-category variables on Census [2210.17128].

Time-series and spatiotemporal variants emphasize different backbones. DSDI couples a conditional DDPM-style imputer with a boundary-enhancing weight-reducing injection strategy and a multi-scale temporal S4-based U-Net, targeting long contiguous gaps and long-range dependencies [2501.06585]. DiffImp replaces Transformer or CNN denoisers with a Bidirectional Attention Mamba block and a Channel Mamba Block, giving linear-time sequence modeling with explicit bidirectional temporal encoding and inter-variable coupling [2410.13338]. RDPI uses a two-stage structure in which a deterministic imputer first predicts a coarse reconstruction, and a conditional diffusion model then refines residuals rather than raw values; its denoiser combines 1D convolution, temporal self-attention, graph neural networks, and spatial self-attention [2412.12642].

Other variants modify the conditioning substrate itself. LSSDM first learns a latent distribution via a variational graph autoencoder, reconstructs coarse missing values, and then refines them with a conditional diffusion model in data space, effectively coupling a latent prior with diffusion refinement [2409.08917]. CD\(^2\)-TSI adds shared side information and domain-specific denoising networks for source and target domains, alongside FFT-based FMixup priors and output-level cross-domain consistency alignment [2506.12412]. DepMicroDiff places conditional diffusion in a VAE latent space and supplies context through both observed latent values and BERT-encoded patient metadata; its Dependency-Aware Transformer uses a binary dependency matrix derived from directional relations and mutual information to constrain attention among taxa [2507.23676].

The original DiffImpute also introduces two sampling-time mechanisms that became conceptually influential. Harmonization repeatedly re-noises and re-denoises intermediate samples to improve coherence between observed and imputed regions, and Impute-DDIM replaces the full Markovian chain with a refined non-Markovian sampler that remains compatible with Harmonization while reducing inference cost [2403.13863].

## 4. Missingness regimes, latent masks, and selective imputation

One major axis of variation within DiffImpute research is the treatment of the missingness mechanism itself. The original DiffImpute is explicitly positioned for MCAR and MAR, and does not model the mask generatively [2403.13863]. TabCSDI likewise focuses on conditional diffusion over missing features without an explicit missingness model [2210.17128]. DiffPuter is agnostic at the level of the diffusion objective and is evaluated under MCAR, MAR, and MNAR, but its probabilistic target remains the full-data density \(p_\theta(\mathbf{x})\) and the conditional \(p_\theta(\mathbf{x}^{\rm mis}\mid \mathbf{x}^{\rm obs})\) derived from it [2405.20690].

PRDIM changes this by treating MNAR as a first-class generative object. Its pattern recognizer \(D_\phi(X)\approx[p_\phi(M_d=1\mid X)]_{d=1}^D\) parameterizes \(p_\phi(M\mid X)\) through a Bernoulli factorization and binary cross-entropy loss, and its gradient is injected into the reverse update:
\[
X_{t-1} = \sqrt{\bar{\alpha}_{t-1}\left(\hat{X}_0 - \frac{1-\bar{\alpha}_t}{\sqrt{\bar{\alpha}_t}\nabla_{X_t}\mathcal{L}_\text{PR}(M,\hat{X}_0, D_\phi)\right) + \sqrt{1-\bar{\alpha}_{t-1}}\ \varepsilon.
\]
This means the reverse score is guided not only by data likelihood but also by mask likelihood, making the imputation consistent with the learned missing pattern under MNAR [2605.25439].

Diff-Joint addresses a different conceptual limitation: not every missing entry should be imputed. It distinguishes “meaningfully missing” entries, where `na` is a semantically valid state, from observation-induced missing values that should be recovered [2606.05073]. For each missing entry, it estimates both an imputed value and a meaningful-missingness label \(\hat c_{i,j}\). The update rule combines uncertainty clustering and majority vote over sampled latent masks:
\[
\widehat c_{i,j}^{(t+1)} =
\mathbbm 1\{i\in\mathcal C_{1,j}^{(t+1)}\}
\lor
\operatorname{mode}\{c_{i,j}^{(t,k)}\}_{k=1}^K.
\]
Entries labeled as meaningful missing are preserved as `na`, whereas only those labeled ordinary missing are imputed [2606.05073].

MissHDD reframes the problem from the perspective of feature geometry rather than missingness semantics. It argues that conditional consistency and information collapse arise because conventional Gaussian diffusion treats heterogeneous variables as though they lived on a common Euclidean manifold [2511.14543]. Its remedy is a continuous DDIM-based channel for numerical variables and a loopholing-based simplex-preserving channel for categorical variables. For the latter, the forward kernel is
\[
q(\mathbf{p}_{t}^{(j)} \mid \mathbf{p}_{t-1}^{(j)}) = (1-\beta_t)\,\mathbf{p}_{t-1}^{(j)}+\beta_t \mathbf{u}^{(j)},
\]
which keeps every categorical state on the simplex at every time step [2511.14543].

Cross-domain imputation introduces yet another missingness-related complication: incompleteness hinders adaptation because the target domain is both shifted and missing. CD\(^2\)-TSI treats this as a joint data, model, and algorithm problem, using FMixup to fill only original missing values with low-frequency cross-domain spectral priors and CDCA to align source and target denoisers only when their outputs are moderately similar [2506.12412].

## 5. Representative variants, empirical performance, and applications

The published record presents DiffImpute as a high-performing but heterogeneous family. The table summarizes representative variants and their reported differentiating mechanisms.

| Method | Core mechanism | Setting |
|---|---|---|
| DiffImpute [2403.13863] | DDPM on complete tabular data; Harmonization; Impute-DDIM; MLP/ResNet/Transformer/U-Net denoisers | Tabular, MCAR/MAR |
| TabCSDI [2210.17128] | Conditional score-based diffusion with one-hot, analog bits, and feature tokenization | Tabular, mixed numerical/categorical |
| DiffPuter [2405.20690] | Unconditional score model with EM; M-step as diffusion training; E-step as conditional sampling | In-sample and out-of-sample tabular imputation |
| PRDIM [2605.25439] | Diffusion backbone plus pattern recognizer \(p_\phi(M\mid X)\); ELBO and EM under MNAR | Time series, images, tabular |
| MissHDD [2511.14543] | Continuous DDIM channel plus loopholing discrete diffusion channel | Heterogeneous tabular, MCAR/MAR/MNAR |
| Diff-Joint [2606.05073] | Joint diffusion over data and latent meaningful-missingness mask | Selective tabular imputation |

The original DiffImpute reports that, “when paired with the Transformer as the denoising network, it consistently outperforms its competitors, boasting an average ranking of 1.7 and the most minimal standard deviation,” while “the next best method lags with a ranking of 2.8 and a standard deviation of 0.9” [2403.13863]. DiffPuter reports, in its abstract, “an average improvement of 6.94% in MAE and 4.78% in RMSE compared to the most competitive existing method” across ten diverse datasets [2405.20690]. MissHDD reports higher imputation accuracy, more stable sampling trajectories, and improved robustness across MCAR, MAR, and MNAR compared with existing diffusion-based and classical methods [2511.14543].

Time-series and spatiotemporal variants also report strong gains. DSDI states that it outperforms existing imputation methods and attributes the gains to boundary-aware AR injection and a multi-scale S4-based U-Net; for example, on ETT Point MAE it reports \(0.108\pm0.001\) versus \(0.117\pm0.002\) for MTSCI, and on AQI Simulated MAE \(0.139\pm0.001\) versus \(0.146\pm0.001\) for MTSCI [2501.06585]. DiffImp reports substantially improved probabilistic calibration on Electricity, with CRPS-sum \(1.51\cdot10^{-2}\) versus \(1.96\cdot10^{-2}\) for SSSD and \(2.14\cdot10^{-2}\) for CSDI, and faster sampling than both CSDI and SSSD on the reported benchmarks [2410.13338]. RDPI reports reduced sampling cost and strong MSE improvements on air-quality datasets; on AQI36 in-sample it reports MAE 7.98 and MSE 238.25, compared with MAE 9.41 and MSE 361.28 for MIDM and MAE 9.60 and MSE 372.49 for CSDI [2412.12642].

The family has also expanded into modality-specific applications. LSSDM reports best MAE and CRPS among the compared generative baselines on AQI-36, PhysioNet 2012, and PeMS-BAY, coupling a variational graph autoencoder with conditional diffusion refinement [2409.08917]. DepMicroDiff reports microbiome reconstruction gains on TCGA cancer datasets, reaching Pearson correlation up to 0.712 and cosine similarity up to 0.812 while lowering RMSE and MAE across multiple cancer types [2507.23676]. ImDiffusion uses diffusion-based time-series imputation as an anomaly-detection primitive and reports a 11.4% increase in detection F1 score in a Microsoft production system compared to the legacy approach [2307.00754].

Several papers emphasize that improvements are especially pronounced when the modeling assumption matches the pathology of the data. PRDIM reports that its gains are particularly large on real MNAR masks and smaller under MAR or MCAR, where the pattern recognizer becomes less informative [2605.25439]. Diff-Joint reports substantially better token-level recovery accuracy and downstream task performance when missingness itself carries signal, precisely because it can preserve semantically meaningful `na` values instead of forcing a regular-value reconstruction [2606.05073].

## 6. Limitations and open directions

Despite strong empirical results, the literature identifies persistent limitations. The original DiffImpute requires complete training data, treats all features as continuous after preprocessing, and still incurs a substantial sampling cost even with Impute-DDIM [2403.13863]. DiffPuter notes that each EM iteration retrains or further trains a diffusion model, which is more expensive than shallow predictive models or simple statistical imputers, and it does not prove global convergence of the outer EM loop [2405.20690].

Categorical and mixed-type data remain a recurring difficulty. TabCSDI shows that categorical embedding choices matter substantially, especially for multi-category features, implying that naïve continuous relaxations are often inadequate [2210.17128]. MissHDD makes this criticism explicit, arguing that Gaussian diffusion over heterogeneous tables produces semantic drift for categorical variables and instability for numerical ones, and that structure-aware dual-channel diffusion is therefore necessary [2511.14543]. PRDIM notes a different modeling limitation: its pattern recognizer assumes conditionally independent Bernoulli indicators given \(X\), while real missingness can be block-level or hierarchical [2605.25439].

Computation is another common bottleneck. PRDIM requires gradients of the pattern recognizer loss with respect to the diffusion state at every reverse step, which “roughly doubles or more the cost vs unguided diffusion” [2605.25439]. DSDI, DiffImp, RDPI, and DepMicroDiff all retain the generic diffusion burden of repeated denoising steps, even when specialized samplers or efficient backbones reduce that burden [2501.06585][2410.13338][2412.12642][2507.23676].

Several open directions recur across papers. One is explicit treatment of the missingness mechanism: PRDIM addresses MNAR, but most DiffImpute variants still assume ignorable or unspecified missingness [2605.25439]. Another is selective imputation: Diff-Joint argues that some missing entries should remain missing, not be filled, and operationalizes that distinction through joint diffusion over data and latent mask [2606.05073]. A third is domain and modality adaptation: CD\(^2\)-TSI extends diffusion imputation to domain shift, while DepMicroDiff uses LLM-conditioned metadata and dependency-aware attention to incorporate structured context [2506.12412][2507.23676].

Taken together, these developments indicate that DiffImpute has evolved from a DDPM-based tabular imputer into a broader research program centered on conditional generative reconstruction under incomplete observation. The main unresolved questions concern how best to combine calibrated uncertainty, efficient sampling, explicit missingness modeling, heterogeneous variable geometry, and semantically meaningful absence within a unified imputation framework.

Source: https://www.emergentmind.com/topics/diffimpute