Conditional Bernoulli Diffusion Models

• Conditional Bernoulli diffusion models are generative frameworks defined on binary data that preserve discrete support by using Bernoulli noise instead of Gaussian noise.
• They replace continuous corruption with binary processes, enabling closed-form posteriors and precise conditioning in reverse steps for tasks like segmentation and graph synthesis.
• These models deliver improved performance in medical imaging, forgery localization, and graph generation by quantifying uncertainty and leveraging specialized conditioning strategies.

Searching arXiv for papers on conditional Bernoulli diffusion models and related discrete diffusion work. Use the arXiv search tool to look up: "Conditional Bernoulli diffusion model binary segmentation BerDiff UMBD CBDiff GraphGUIDE CDGraph" Searching conditional Bernoulli diffusion model binary segmentation BerDiff UMBD CBDiff GraphGUIDE CDGraph Conditional Bernoulli diffusion models are conditional discrete diffusion models defined on binary random variables, typically masks, residuals, adjacency bits, or node-condition indicators. Their central design choice is to replace Gaussian corruption with Bernoulli corruption so that the forward chain remains on the native support $\{0,1\}^d$, while the reverse chain is conditioned on exogenous information such as an input image, a coarse prediction, an uncertainty map, semantic features, or structural constraints. In recent literature, this family has been instantiated for medical image segmentation, image forgery localization, camouflaged object detection refinement, thin-structure CT segmentation, controllable graph generation, and dual-conditional social graph synthesis (Chen et al., 2023, Lei et al., 22 Oct 2025, Shen et al., 12 Jun 2025, An et al., 17 Jul 2025, Tseng et al., 2023, Tsai et al., 2023).

1. Binary-state diffusion formalism

A common abstraction uses a clean binary target $x_0 \in \{0,1\}^d$, latent binary states $x_t \in \{0,1\}^d$ for $t=1,\dots,T$, and a conditioning variable $c$. The forward process is a Markov chain that progressively randomizes $x_0$, while the reverse model learns $p_\theta(x_{t-1}\mid x_t,c)$. What distinguishes the Bernoulli family from Gaussian DDPMs is that every step preserves binary support exactly rather than perturbing the target in a continuous space.

The literature uses several closely related forward kernels. BerDiff adopts a Bernoulli mixing kernel toward a symmetric prior,

$$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$

with closed-form marginal

$$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$

where $\alpha_t=1-\beta_t$ and $x_0 \in \{0,1\}^d$0 (Chen et al., 2023). CBDiff expresses the same binary setting in two-class one-hot form and writes the stepwise corruption as

$x_0 \in \{0,1\}^d$1

which drives the mask toward the uniform base $x_0 \in \{0,1\}^d$2 (Lei et al., 22 Oct 2025).

Other works use explicit bit-flip chains. For thin-structure segmentation, the per-voxel transition is

$x_0 \in \{0,1\}^d$3

with marginal

$x_0 \in \{0,1\}^d$4

where now $x_0 \in \{0,1\}^d$5 (An et al., 17 Jul 2025). GraphGUIDE generalizes this idea to graph edges and defines three kernels—bit-flip, bit-one, and bit-zero—so that the terminal distribution can be, respectively, an Erdős–Rényi graph with $x_0 \in \{0,1\}^d$6, the complete graph, or the empty graph (Tseng et al., 2023).

A more strongly conditional variant appears in UMBD. Instead of diffusing toward $x_0 \in \{0,1\}^d$7, UMBD defines

$x_0 \in \{0,1\}^d$8

where the base parameter is $x_0 \in \{0,1\}^d$9, with $x_t \in \{0,1\}^d$0 a coarse prediction and $x_t \in \{0,1\}^d$1 an uncertainty mask. In that formulation, corruption is anchored to a mask-dependent prior rather than a generic symmetric Bernoulli base (Shen et al., 12 Jun 2025). This suggests that “conditional Bernoulli diffusion” is not a single kernel but a family of binary diffusion processes differing mainly in the choice of terminal prior and the way conditioning enters the chain.

2. Reverse process, posterior structure, and learning targets

A central reason these models are tractable is that the binary forward chains admit closed-form posteriors. BerDiff derives

$x_t \in \{0,1\}^d$2

where $x_t \in \{0,1\}^d$3 is obtained by a two-channel normalization over the binary state (Chen et al., 2023). CBDiff analogously computes

$x_t \in \{0,1\}^d$4

so that the reverse model can use the exact discrete posterior in the same role played by the Gaussian posterior in DDPMs (Lei et al., 22 Oct 2025).

The dominant reverse parameterization is predictor-of-$x_t \in \{0,1\}^d$5. In CBDiff, the network predicts the clean-mask distribution $x_t \in \{0,1\}^d$6 and analytically maps it to $x_t \in \{0,1\}^d$7:

$x_t \in \{0,1\}^d$8

This makes the output space time-invariant and stabilizes training (Lei et al., 22 Oct 2025).

BerDiff and UMBD instead predict Bernoulli “noise.” BerDiff reconstructs the clean mask by

$x_t \in \{0,1\}^d$9

with $t=1,\dots,T$0 equal to XOR for binary variables (Chen et al., 2023). UMBD follows the same posterior-parameterization principle:

$t=1,\dots,T$1

where the posterior mean is computed analytically from the forward chain (Shen et al., 12 Jun 2025). GraphGUIDE uses the same plug-in strategy for graph edges: the network predicts $t=1,\dots,T$2, and the sampler replaces $t=1,\dots,T$3 in the exact Bernoulli posterior $t=1,\dots,T$4 with $t=1,\dots,T$5 (Tseng et al., 2023).

Training objectives are discrete ELBO surrogates adapted to the chosen parameterization. CBDiff uses KL between true and predicted posteriors for intermediate steps and cross-entropy at $t=1,\dots,T$6 (Lei et al., 22 Oct 2025). BerDiff optimizes

$t=1,\dots,T$7

with $t=1,\dots,T$8 in the experiments (Chen et al., 2023). UMBD adds supervised refinement losses on the reconstructed mask,

$t=1,\dots,T$9

while the thin-structure model trains on per-timestep negative log-likelihood of $c$0 under a Bernoulli reverse kernel (Shen et al., 12 Jun 2025, An et al., 17 Jul 2025). CDGraph further augments KL and reconstruction terms with a classifier loss encoding mutual dependency of dual conditions (Tsai et al., 2023).

3. Modes of conditionality

The conditioning variable in a conditional Bernoulli diffusion model is not standardized; different applications inject it through different operators and at different points of the chain. In medical image segmentation, BerDiff conditions the reverse model on the input image $c$1 using a conditional U-Net that ingests $c$2, the noised mask $c$3, and the time embedding $c$4 (Chen et al., 2023). In image forgery localization, CBDiff conditions on the forged image $c$5, a forensic noise-residual map $c$6 from Noiseprint++, and multi-scale DINO features $c$7 (Lei et al., 22 Oct 2025).

CBDiff’s main conditioning module is Time-Step Cross-Attention (TSCAttention). In the deepest three UNet blocks, UNet features $c$8 attend to DINO features $c$9 after explicit time modulation,

$x_0$0

followed by

$x_0$1

The design is intended to make semantic guidance sensitive to denoising stage and noise level (Lei et al., 22 Oct 2025).

UMBD introduces a different conditioning regime in which uncertainty is itself the gate that decides where diffusion should occur. The model defines $x_0$2 and $x_0$3, so diffusion is selectively applied to residual regions with poor segmentation quality rather than to the entire mask. The uncertainty map $x_0$4 is estimated at inference by HUQNet, which combines a Bayesian branch, entropy of the coarse mask, a residual attention module, and cross-attention fusion to produce $x_0$5 (Shen et al., 12 Jun 2025). This suggests a structurally conditional view of diffusion: the chain is not merely conditioned on context, but spatially modulated by uncertainty.

In graph generation, conditionality becomes explicitly rule-based or label-guided. GraphGUIDE performs conditional generation through hard masks or projections on edges at every reverse step. Two binary masks, $x_0$6 and $x_0$7, overwrite Bernoulli edge probabilities so that selected edges must be present or absent, enabling full control over arbitrary structural properties without predefined labels (Tseng et al., 2023). CDGraph instead conditions jointly on two specified node conditions and models their co-evolution with the adjacency matrix. Its reverse decomposition couples social contagion,

$x_0$8

and social homophily,

$x_0$9

then adds hierarchical classifier guidance to favor satisfaction of both conditions (Tsai et al., 2023).

4. Sampling, diversity, and uncertainty quantification

A defining feature of these models is that they are generative over binary structures rather than merely discriminative predictors. Sampling typically starts from a terminal Bernoulli prior—uniform in BerDiff and CBDiff, problem-specific in UMBD, or kernel-dependent in GraphGUIDE—and then iteratively applies stochastic reverse transitions. In CBDiff, diversity arises from random initialization $p_\theta(x_{t-1}\mid x_t,c)$0, stochastic sampling at each reverse step, and optional different random seeds or temperature-like rescaling of $p_\theta(x_{t-1}\mid x_t,c)$1 (Lei et al., 22 Oct 2025). BerDiff similarly attributes diversity to randomness in $p_\theta(x_{t-1}\mid x_t,c)$2, in $p_\theta(x_{t-1}\mid x_t,c)$3, and in resampling intermediate latents, and interprets the sample mean as a saliency map highlighting consistent ROI (Chen et al., 2023).

Several papers make uncertainty a first-class output. CBDiff explicitly produces $p_\theta(x_{t-1}\mid x_t,c)$4 masks $p_\theta(x_{t-1}\mid x_t,c)$5 and derives confidence maps either from the mean $p_\theta(x_{t-1}\mid x_t,c)$6 or from vote ratios and entropy-like agreement statistics (Lei et al., 22 Oct 2025). In the thin-structure model, $p_\theta(x_{t-1}\mid x_t,c)$7 sampled masks are aggregated into a voxelwise consensus probability

$p_\theta(x_{t-1}\mid x_t,c)$8

which then drives a correction energy involving unary consensus terms and pairwise terms based on position proximity, consensus-level similarity, and gradient-direction similarity (An et al., 17 Jul 2025). UMBD places uncertainty one stage earlier: instead of quantifying ambiguity only after sampling, it uses uncertainty estimates to decide where generative refinement should be applied (Shen et al., 12 Jun 2025).

Acceleration is usually achieved by sub-sequence or DDIM-style sampling. BerDiff uses a DDIM-inspired non-Markovian update so that 10 uniformly spaced timesteps can significantly speed up inference (Chen et al., 2023). UMBD trains with $p_\theta(x_{t-1}\mid x_t,c)$9 but uses DDIM sub-sequence sampling with $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$0 for inference and also evaluates down to $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$1 with minimal loss of quality (Shen et al., 12 Jun 2025). GraphGUIDE, although defined with $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$2, treats control as an intervention on the reverse chain and can switch between stochastic sampling and deterministic thresholding depending on whether exact structural satisfaction or stochasticity is preferred (Tseng et al., 2023).

A common misconception is that multiple outputs indicate unstable prediction. In this literature, multiplicity is intentional: the sampled set is used to represent annotation ambiguity, spatial uncertainty, or multiple plausible binary structures rather than to provide a single point estimate.

5. Principal application domains and reported behavior

The strongest concentration of work is in dense binary prediction. BerDiff addresses medical image segmentation with inherent ambiguity and reports superior diversity and accuracy relative to Gaussian diffusion baselines and other generative models (Chen et al., 2023). CBDiff adapts the framework to image forgery localization, where the target is a sparse tampering mask and uncertainty-aware forensic output is operationally relevant (Lei et al., 22 Oct 2025). UMBD uses conditional Bernoulli diffusion as a post-processing refiner for camouflaged object detection, restricting diffusion to uncertain residual regions (Shen et al., 12 Jun 2025). A related thin-structure framework applies Bernoulli diffusion to orbital bone segmentation and uses consensus-driven correction to repair disconnected or under-segmented regions (An et al., 17 Jul 2025).

Beyond segmentation, the same principles extend to binary graphs. GraphGUIDE models graph edges as Bernoulli variables and demonstrates conditional control over cliques, disjoint communities, and ring structures through reverse-step masking (Tseng et al., 2023). CDGraph expands the setting to dual-conditional social graph synthesis, where both node-condition indicators and adjacency bits are diffused and jointly denoised under homophily, contagion, and classifier guidance (Tsai et al., 2023).

Model	Domain	Reported result
BerDiff	Medical image segmentation	Dice 89.7 on BRATS 2021; GED=0.238±0.01 and HM-IoU=0.596±0.00 on LIDC-IDRI
CBDiff	Image forgery localization	Protocol 1 average F1≈0.824 and AUC≈0.971; CBDiff-large reaches Ave F1≈0.628 and Ave AUC≈0.883 under Protocol 2
UMBD	Camouflaged object detection refinement	Average gains of 5.5% in MAE and 3.2% in weighted F-measure
Thin-structure model	Orbital medial wall and floor segmentation	Medial wall: DSC 86.31, Recall 87.83, Precision 85.38; floor: DSC 91.36, Recall 93.24, Precision 90.08
GraphGUIDE	Conditional graph generation	100% success in the reported conditional generation demonstrations
CDGraph	Dual-conditional social graph synthesis	Validity 1.000 on multiple datasets and correlation regimes in representative settings

Across these applications, reported ablations repeatedly favor Bernoulli over Gaussian corruption for binary targets. CBDiff reports an ablation with Ave F1 $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$3 for Bernoulli noise versus $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$4 for Gaussian noise (Lei et al., 22 Oct 2025). BerDiff reports, at 21,000 iterations on LIDC-IDRI, GED(16) $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$5 versus $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$6 and HM-IoU(16) $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$7 versus $$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$8 for Bernoulli and Gaussian respectively (Chen et al., 2023). UMBD likewise states that replacing Bernoulli with Gaussian corruption deteriorates performance (Shen et al., 12 Jun 2025). These results do not establish a universal theorem, but they do show a consistent empirical pattern in binary-structured output spaces.

6. Relation to discrete diffusion, limitations, and open directions

Conditional Bernoulli diffusion models are best understood as the binary specialization of discrete diffusion. CBDiff explicitly situates its corruption as the two-class specialization of multinomial diffusion and argmax flows, emphasizing that the binary case admits simple closed-form posteriors and a straightforward predictor-of-$$q(y_t \mid y_{t-1}) := \mathrm{Bernoulli}\!\left((1-\beta_t)y_{t-1} + \frac{\beta_t}{2}\right),$$9 parameterization (Lei et al., 22 Oct 2025). UMBD similarly contrasts Bernoulli with categorical diffusion by noting that Bernoulli is appropriate for binary foreground/background tasks, whereas multi-class segmentation would require a categorical extension (Shen et al., 12 Jun 2025). In graph generation, CDGraph is presented as following the discrete diffusion paradigm of D3PM and DiGress, specialized to Bernoulli node and edge variables (Tsai et al., 2023).

The principal limitations are also consistent across domains. Iterative reverse sampling remains more expensive than feedforward discriminative models. CBDiff-large, at approximately $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$0M parameters, is heavier than many discriminative baselines and incurs longer inference due to iterative sampling, although CBDiff-small at approximately $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$1M reduces memory and runtime (Lei et al., 22 Oct 2025). BerDiff reports inference at approximately $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$2 s per image, about $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$3 slower than a traditional U-Net (Chen et al., 2023). UMBD’s refinement quality depends on the accuracy of $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$4; over- or under-estimation of uncertainty can cause missed residuals or perturb correct pixels (Shen et al., 12 Jun 2025). CBDiff notes failure modes for highly subtle forgeries under severe domain shift and for extremely small tampered areas (Lei et al., 22 Oct 2025). The thin-structure model remains tailored to binary targets and would require categorical diffusion or constrained independent Bernoullis for multi-class use (An et al., 17 Jul 2025).

Open directions named in the literature are correspondingly concrete. CBDiff proposes multimodal conditioning with metadata or text, lightweight distillations for edge deployment, and improved uncertainty calibration (Lei et al., 22 Oct 2025). UMBD suggests categorical diffusion for multi-class segmentation, improved posterior parameterizations, alternative samplers such as advanced DPM solvers, and temporal aggregation across steps (Shen et al., 12 Jun 2025). Thin-structure work points to 3D diffusion, anatomical priors, and adaptive sampling in high-uncertainty regions (An et al., 17 Jul 2025). Graph settings raise separate scalability questions because dense adjacency diffusion is naturally $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$5 and the current experiments focus on ego-graphs with up to $$q(y_t \mid y_0)=\mathrm{Bernoulli}\!\left(\bar{\alpha}_t y_0 + \frac{1-\bar{\alpha}_t}{2}\right),$$6 nodes (Tsai et al., 2023, Tseng et al., 2023).

Taken together, these developments define a coherent model class: diffusion models whose stochastic process, likelihood surrogate, and conditional interfaces are all aligned with binary output structure. The practical significance of the class lies not merely in replacing Gaussian noise with Bernoulli noise, but in making binary support, exact discrete posteriors, conditional control, and sample-based uncertainty available within a single generative framework.