---
title: Conditional Bernoulli Diffusion Models
url: https://www.emergentmind.com/topics/conditional-bernoulli-diffusion-model
type: topic
---

# Conditional Bernoulli Diffusion Models

Searching arXiv for recent papers on conditional Bernoulli diffusion models and related discrete diffusion work.
Use the arXiv search tool to look up: "Conditional Bernoulli diffusion model binary segmentation BerDiff UMBD CBDiff GraphGUIDE CDGraph"
Searching arXiv: conditional Bernoulli diffusion model binary segmentation BerDiff UMBD CBDiff GraphGUIDE CDGraph
Conditional Bernoulli diffusion models are conditional discrete diffusion models defined on binary random variables, typically masks, residuals, adjacency bits, or node-condition indicators. Their central design choice is to replace Gaussian corruption with Bernoulli corruption so that the forward chain remains on the native support $\{0,1\}^d$, while the reverse chain is conditioned on exogenous information such as an input image, a coarse prediction, an uncertainty map, semantic features, or structural constraints. In recent literature, this family has been instantiated for medical image segmentation, image forgery localization, camouflaged object detection refinement, thin-structure CT segmentation, controllable graph generation, and dual-conditional social graph synthesis [2304.04429] [2510.19597] [2506.10712] [2507.12985] [2302.03790] [2311.01729].

## 1. Binary-state diffusion formalism

A common abstraction uses a clean binary target $x_0 \in \{0,1\}^d$, latent binary states $x_t \in \{0,1\}^d$ for $t=1,\dots,T$, and a conditioning variable $c$. The forward process is a Markov chain that progressively randomizes $x_0$, while the reverse model learns $p_\theta(x_{t-1}\mid x_t,c)$. What distinguishes the Bernoulli family from Gaussian DDPMs is that every step preserves binary support exactly rather than perturbing the target in a continuous space.

The literature uses several closely related forward kernels. BerDiff adopts a Bernoulli mixing kernel toward a symmetric prior,
$$
q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),
$$
with closed-form marginal
$$
q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),
$$
where $\alpha_t=1-\beta_t$ and $\bar{\alpha}_t=\prod_{\tau=1}^t \alpha_\tau$ [2304.04429]. CBDiff expresses the same binary setting in two-class one-hot form and writes the stepwise corruption as
$$
q(X_t \mid X_{t-1})=\mathrm{BernCat}\!\left(\Pi_t=\frac{\beta_t}{2}\mathbf{1}+\alpha_t X_{t-1}\right),
$$
which drives the mask toward the uniform base $\frac12\mathbf{1}$ [2510.19597].

Other works use explicit bit-flip chains. For thin-structure segmentation, the per-voxel transition is
$$
q(x_t^i=s \mid x_{t-1}^i=r)=
\begin{cases}
1-\beta_t,& s=r,\\
\beta_t,& s\neq r,
\end{cases}
$$
with marginal
$$
q(x_t^i=1\mid x_0^i)=\frac{1-\bar{\alpha}_t}{2}+\bar{\alpha}_t x_0^i,
$$
where now $\alpha_t=1-2\beta_t$ [2507.12985]. GraphGUIDE generalizes this idea to graph edges and defines three kernels—bit-flip, bit-one, and bit-zero—so that the terminal distribution can be, respectively, an Erdős–Rényi graph with $p=1/2$, the complete graph, or the empty graph [2302.03790].

A more strongly conditional variant appears in UMBD. Instead of diffusing toward $\frac12$, UMBD defines
$$
q(x_t \mid x_0,\pi)=\mathrm{Bern}\!\left(\bar{\alpha}_t x_0 + (1-\bar{\alpha}_t)\pi\right),
$$
where the base parameter is $\pi=U\odot M_c$, with $M_c$ a coarse prediction and $U$ an uncertainty mask. In that formulation, corruption is anchored to a mask-dependent prior rather than a generic symmetric Bernoulli base [2506.10712]. This suggests that “conditional Bernoulli diffusion” is not a single kernel but a family of binary diffusion processes differing mainly in the choice of terminal prior and the way conditioning enters the chain.

## 2. Reverse process, posterior structure, and learning targets

A central reason these models are tractable is that the binary forward chains admit closed-form posteriors. BerDiff derives
$$
q(y_{t-1}\mid y_t,y_0)=\mathrm{Bernoulli}(y_{t-1};\theta_{\mathrm{post}}(y_t,y_0)),
$$
where $\theta_{\mathrm{post}}$ is obtained by a two-channel normalization over the binary state [2304.04429]. CBDiff analogously computes
$$
q(X_{t-1}=k\mid X_t,X_0)
=
\frac{\pi_{t-1}(k\mid X_0)\,\ell_t(k\mid X_t)}
{\sum_{j\in\{0,1\}}\pi_{t-1}(j\mid X_0)\,\ell_t(j\mid X_t)},
$$
so that the reverse model can use the exact discrete posterior in the same role played by the Gaussian posterior in DDPMs [2510.19597].

The dominant reverse parameterization is predictor-of-$x_0$. In CBDiff, the network predicts the clean-mask distribution $\hat{\Pi}_0=\phi_\theta(X_t,Y,N,F,t)$ and analytically maps it to $\hat{\Pi}_{t-1}$:
$$
\hat{\Pi}_{t-1}
=
\mathrm{Normalize}\!\left(
\left[\frac{1-\bar{\alpha}_{t-1}}{2}\mathbf{1}+\bar{\alpha}_{t-1}\hat{\Pi}_0\right]
\odot
\left[\frac{1-\alpha_t}{2}\mathbf{1}+\alpha_t X_t\right]
\right).
$$
This makes the output space time-invariant and stabilizes training [2510.19597].

BerDiff and UMBD instead predict Bernoulli “noise.” BerDiff reconstructs the clean mask by
$$
\hat{\mu}(y_t,t,x)=\theta_{\mathrm{post}}\!\left(y_t,\left|y_t-\hat{\epsilon}(y_t,t,x)\right|\right),
$$
with $|\,\cdot\,|$ equal to XOR for binary variables [2304.04429]. UMBD follows the same posterior-parameterization principle:
$$
\hat{x}_0=\left|x_t-\hat{\epsilon}(x_t,t,U\odot M_c)\right|,
\qquad
p_\theta(x_{t-1}\mid x_t,c,t)=\mathrm{Bern}\!\left(\phi_{\mathrm{post}}(x_t,\hat{x}_0,U\odot M_c)\right),
$$
where the posterior mean is computed analytically from the forward chain [2506.10712]. GraphGUIDE uses the same plug-in strategy for graph edges: the network predicts $\hat{x}_0$, and the sampler replaces $x_0$ in the exact Bernoulli posterior $q(x_{t-1}\mid x_t,x_0)$ with $\hat{x}_0$ [2302.03790].

Training objectives are discrete ELBO surrogates adapted to the chosen parameterization. CBDiff uses KL between true and predicted posteriors for intermediate steps and cross-entropy at $t=1$ [2510.19597]. BerDiff optimizes
$$
L_{\mathrm{Total}}=L_{\mathrm{KL}}+\lambda_{\mathrm{BCE}}L_{\mathrm{BCE}},
$$
with $\lambda_{\mathrm{BCE}}=1$ in the experiments [2304.04429]. UMBD adds supervised refinement losses on the reconstructed mask,
$$
L_{\mathrm{Diff}}=L_{\mathrm{KL}}+L_{w\_\mathrm{IoU}}+L_{w\_\mathrm{BCE}},
$$
while the thin-structure model trains on per-timestep negative log-likelihood of $x_{t-1}$ under a Bernoulli reverse kernel [2506.10712] [2507.12985]. CDGraph further augments KL and reconstruction terms with a classifier loss encoding mutual dependency of dual conditions [2311.01729].

## 3. Modes of conditionality

The conditioning variable in a conditional Bernoulli diffusion model is not standardized; different applications inject it through different operators and at different points of the chain. In medical image segmentation, BerDiff conditions the reverse model on the input image $x$ using a conditional U-Net that ingests $x$, the noised mask $y_t$, and the time embedding $t$ [2304.04429]. In image forgery localization, CBDiff conditions on the forged image $Y$, a forensic noise-residual map $N$ from Noiseprint++, and multi-scale DINO features $F=\{f_i\}$ [2510.19597].

CBDiff’s main conditioning module is Time-Step Cross-Attention (TSCAttention). In the deepest three UNet blocks, UNet features $h$ attend to DINO features $f$ after explicit time modulation,
$$
(a_t,b_t)=\mathrm{MLP}(e_t),\qquad \gamma_t(f)=a_t\odot f+b_t,
$$
followed by
$$
Q=W_Qh,\quad K=W_K\gamma_t(f),\quad V=W_V\gamma_t(f),\quad
\mathrm{TSCAttn}(h,f,t)=\mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt d}\right)V.
$$
The design is intended to make semantic guidance sensitive to denoising stage and noise level [2510.19597].

UMBD introduces a different conditioning regime in which uncertainty is itself the gate that decides where diffusion should occur. The model defines $x_0=U\odot M_{GT}$ and $\pi=U\odot M_c$, so diffusion is selectively applied to residual regions with poor segmentation quality rather than to the entire mask. The uncertainty map $U$ is estimated at inference by HUQNet, which combines a Bayesian branch, entropy of the coarse mask, a residual attention module, and cross-attention fusion to produce $\hat{U}$ [2506.10712]. This suggests a structurally conditional view of diffusion: the chain is not merely conditioned on context, but spatially modulated by uncertainty.

In graph generation, conditionality becomes explicitly rule-based or label-guided. GraphGUIDE performs conditional generation through hard masks or projections on edges at every reverse step. Two binary masks, $M^{\mathrm{keep}}$ and $M^{\mathrm{ban}}$, overwrite Bernoulli edge probabilities so that selected edges must be present or absent, enabling full control over arbitrary structural properties without predefined labels [2302.03790]. CDGraph instead conditions jointly on two specified node conditions and models their co-evolution with the adjacency matrix. Its reverse decomposition couples social contagion,
$$
p_\theta(X_{c_i}^{(t-1)}\mid X_{c_i}^{(t)},E^{(t)}),
$$
and social homophily,
$$
p_\theta(E^{(t-1)}\mid E^{(t)},X_{c_i}^{(t)},X_{c_j}^{(t)}),
$$
then adds hierarchical classifier guidance to favor satisfaction of both conditions [2311.01729].

## 4. Sampling, diversity, and uncertainty quantification

A defining feature of these models is that they are generative over binary structures rather than merely discriminative predictors. Sampling typically starts from a terminal Bernoulli prior—uniform in BerDiff and CBDiff, problem-specific in UMBD, or kernel-dependent in GraphGUIDE—and then iteratively applies stochastic reverse transitions. In CBDiff, diversity arises from random initialization $X_T\sim\mathrm{BernCat}(\tfrac12\mathbf 1)$, stochastic sampling at each reverse step, and optional different random seeds or temperature-like rescaling of $\hat{\Pi}_{t-1}$ [2510.19597]. BerDiff similarly attributes diversity to randomness in $y_T$, in $\hat{\epsilon}_t$, and in resampling intermediate latents, and interprets the sample mean as a saliency map highlighting consistent ROI [2304.04429].

Several papers make uncertainty a first-class output. CBDiff explicitly produces $S$ masks $\{\hat{X}_0^{(s)}\}$ and derives confidence maps either from the mean $\bar{\Pi}_0=\frac{1}{S}\sum_s \hat{\Pi}_0^{(s)}$ or from vote ratios and entropy-like agreement statistics [2510.19597]. In the thin-structure model, $S=200$ sampled masks are aggregated into a voxelwise consensus probability
$$
P(x=1)\approx \frac{1}{S}\sum_i m_i(p),
$$
which then drives a correction energy involving unary consensus terms and pairwise terms based on position proximity, consensus-level similarity, and gradient-direction similarity [2507.12985]. UMBD places uncertainty one stage earlier: instead of quantifying ambiguity only after sampling, it uses uncertainty estimates to decide where generative refinement should be applied [2506.10712].

Acceleration is usually achieved by sub-sequence or DDIM-style sampling. BerDiff uses a DDIM-inspired non-Markovian update so that 10 uniformly spaced timesteps can significantly speed up inference [2304.04429]. UMBD trains with $T=1000$ but uses DDIM sub-sequence sampling with $T=10$ for inference and also evaluates down to $T=3$ with minimal loss of quality [2506.10712]. GraphGUIDE, although defined with $T=1000$, treats control as an intervention on the reverse chain and can switch between stochastic sampling and deterministic thresholding depending on whether exact structural satisfaction or stochasticity is preferred [2302.03790].

A common misconception is that multiple outputs indicate unstable prediction. In this literature, multiplicity is intentional: the sampled set is used to represent annotation ambiguity, spatial uncertainty, or multiple plausible binary structures rather than to provide a single point estimate.

## 5. Principal application domains and reported behavior

The strongest concentration of work is in dense binary prediction. BerDiff addresses medical image segmentation with inherent ambiguity and reports superior diversity and accuracy relative to Gaussian diffusion baselines and other generative models [2304.04429]. CBDiff adapts the framework to image forgery localization, where the target is a sparse tampering mask and uncertainty-aware forensic output is operationally relevant [2510.19597]. UMBD uses conditional Bernoulli diffusion as a post-processing refiner for camouflaged object detection, restricting diffusion to uncertain residual regions [2506.10712]. A related thin-structure framework applies Bernoulli diffusion to orbital bone segmentation and uses consensus-driven correction to repair disconnected or under-segmented regions [2507.12985].

Beyond segmentation, the same principles extend to binary graphs. GraphGUIDE models graph edges as Bernoulli variables and demonstrates conditional control over cliques, disjoint communities, and ring structures through reverse-step masking [2302.03790]. CDGraph expands the setting to dual-conditional social graph synthesis, where both node-condition indicators and adjacency bits are diffused and jointly denoised under homophily, contagion, and classifier guidance [2311.01729].

| Model | Domain | Reported result |
|---|---|---|
| BerDiff | Medical image segmentation | Dice 89.7 on BRATS 2021; GED=0.238±0.01 and HM-IoU=0.596±0.00 on LIDC-IDRI |
| CBDiff | Image forgery localization | Protocol 1 average F1≈0.824 and AUC≈0.971; CBDiff-large reaches Ave F1≈0.628 and Ave AUC≈0.883 under Protocol 2 |
| UMBD | Camouflaged object detection refinement | Average gains of 5.5% in MAE and 3.2% in weighted F-measure |
| Thin-structure model | Orbital medial wall and floor segmentation | Medial wall: DSC 86.31, Recall 87.83, Precision 85.38; floor: DSC 91.36, Recall 93.24, Precision 90.08 |
| GraphGUIDE | Conditional graph generation | 100% success in the reported conditional generation demonstrations |
| CDGraph | Dual-conditional social graph synthesis | Validity 1.000 on multiple datasets and correlation regimes in representative settings |

Across these applications, reported ablations repeatedly favor Bernoulli over Gaussian corruption for binary targets. CBDiff reports an ablation with Ave F1 $0.696$ for Bernoulli noise versus $0.608$ for Gaussian noise [2510.19597]. BerDiff reports, at 21,000 iterations on LIDC-IDRI, GED(16) $=0.252$ versus $0.671$ and HM-IoU(16) $=0.575$ versus $0.020$ for Bernoulli and Gaussian respectively [2304.04429]. UMBD likewise states that replacing Bernoulli with Gaussian corruption deteriorates performance [2506.10712]. These results do not establish a universal theorem, but they do show a consistent empirical pattern in binary-structured output spaces.

## 6. Relation to discrete diffusion, limitations, and open directions

Conditional Bernoulli diffusion models are best understood as the binary specialization of discrete diffusion. CBDiff explicitly situates its corruption as the two-class specialization of multinomial diffusion and argmax flows, emphasizing that the binary case admits simple closed-form posteriors and a straightforward predictor-of-$X_0$ parameterization [2510.19597]. UMBD similarly contrasts Bernoulli with categorical diffusion by noting that Bernoulli is appropriate for binary foreground/background tasks, whereas multi-class segmentation would require a categorical extension [2506.10712]. In graph generation, CDGraph is presented as following the discrete diffusion paradigm of D3PM and DiGress, specialized to Bernoulli node and edge variables [2311.01729].

The principal limitations are also consistent across domains. Iterative reverse sampling remains more expensive than feedforward discriminative models. CBDiff-large, at approximately $140$M parameters, is heavier than many discriminative baselines and incurs longer inference due to iterative sampling, although CBDiff-small at approximately $40$M reduces memory and runtime [2510.19597]. BerDiff reports inference at approximately $0.4$ s per image, about $10\times$ slower than a traditional U-Net [2304.04429]. UMBD’s refinement quality depends on the accuracy of $\hat{U}$; over- or under-estimation of uncertainty can cause missed residuals or perturb correct pixels [2506.10712]. CBDiff notes failure modes for highly subtle forgeries under severe domain shift and for extremely small tampered areas [2510.19597]. The thin-structure model remains tailored to binary targets and would require categorical diffusion or constrained independent Bernoullis for multi-class use [2507.12985].

Open directions named in the literature are correspondingly concrete. CBDiff proposes multimodal conditioning with metadata or text, lightweight distillations for edge deployment, and improved uncertainty calibration [2510.19597]. UMBD suggests categorical diffusion for multi-class segmentation, improved posterior parameterizations, alternative samplers such as advanced DPM solvers, and temporal aggregation across steps [2506.10712]. Thin-structure work points to 3D diffusion, anatomical priors, and adaptive sampling in high-uncertainty regions [2507.12985]. Graph settings raise separate scalability questions because dense adjacency diffusion is naturally $O(N^2)$ and the current experiments focus on ego-graphs with up to $N=100$ nodes [2311.01729] [2302.03790].

Taken together, these developments define a coherent model class: diffusion models whose stochastic process, likelihood surrogate, and conditional interfaces are all aligned with binary output structure. The practical significance of the class lies not merely in replacing Gaussian noise with Bernoulli noise, but in making binary support, exact discrete posteriors, conditional control, and sample-based uncertainty available within a single generative framework.

Source: https://www.emergentmind.com/topics/conditional-bernoulli-diffusion-model