---
title: 'JD3P: Joint Discrete Denoising Diffusion'
url: https://www.emergentmind.com/topics/joint-discrete-denoising-diffusion-process-jd3p
type: topic
---

# JD3P: Joint Discrete Denoising Diffusion

The Joint Discrete Denoising Diffusion Process (JD3P) is a family of probabilistic generative models that extend denoising diffusion processes to handle structured data comprising both discrete and continuous variables, or multiple interdependent discrete modalities. JD3P provides a unified, theoretically grounded, and highly practical framework for stochastic generation and modeling in domains requiring the synthesis or reconstruction of heterogeneous symbolic, parametric, and spatial data. Key innovations in JD3P include forward and reverse noising/denoising processes tailored to both categorical and continuous domains, permutation-invariant modeling, and mechanisms for joint multimodal co-refinement, as exemplified in state-of-the-art systems for computer-aided design (CAD) sketch generation [2507.11579], unified vision-language-action modeling [2511.01718], and discrete Markov frameworks [2205.14987].

## 1. Mathematical Foundations of JD3P

JD3P generalizes denoising diffusion probabilistic models (DDPMs) to the joint generation of sequence elements where each primitive is characterized by a tuple of continuous and discrete components. In the canonical SketchDNN formulation [2507.11579], a CAD primitive is defined as $x_i = (b_i, c_i, p_i)$ with $b_i$ a binary construction-aid flag, $c_i$ a categorical class label (e.g., Line, Circle, Arc, Point), and $p_i$ a vector of class-dependent continuous parameters. The forward diffusion process consists of two independent Markov chains:

- **Continuous channel:** Each $p_t$ is diffused through a Gaussian process:
  $$
  q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) = \mathcal{N}\left(\mathbf{x}_t; \sqrt{\alpha_t}\,\mathbf{x}_{t-1},\,\beta_t I\right).
  $$
  The closed-form marginal permits direct sampling of noise levels at arbitrary time steps.

- **Discrete channel (Gaussian-Softmax diffusion):** Discrete variables are diffused by adding Gaussian noise to the logit space of the one-hot encoding and projecting to the probability simplex via softmax at each step:
  $$
  \mathbf{u}_t = \sqrt{\alpha_t} \log(\mathbf{y}_{t-1}) + \sqrt{1-\alpha_t}\,\varepsilon,\quad \varepsilon \sim \mathcal{N}(0, I),
  $$
  $$
  \mathbf{y}_t = \mathrm{softmax}(\mathbf{u}_t).
  $$
  As $t\to T$, the distribution approaches the uniform distribution on the simplex.

The model parameters are optimized by minimizing a loss combining mean-squared error (MSE) for continuous variables and cross-entropy for categorical variables, corresponding to a variational lower bound (ELBO) with distinct continuous and discrete terms.

## 2. Joint Modeling of Multimodal and Heterogeneous Data

JD3P's formalism naturally supports data where each element or modality follows distinctive statistical laws. In SketchDNN [2507.11579], the challenge of heterogeneous parameterizations is solved by embedding all primitive types in a unified vector and gating each block via the predicted class probabilities (“superposition”). This superposition enables continuous gradients even for mixed-symbolic representations and improves the robustness and flexibility of learned models. For CAD sketches as unordered sets, forward and reverse chains are constructed to factorize over primitives, and transformer-based denoisers are implemented without positional encodings to enforce permutation invariance.

In vision-language-action applications [2511.01718], JD3P is employed to jointly denoise actions and visual predictions by concatenating future vision and action tokens in a single discrete diffusion trajectory. The approach extends to tokenized multimodal sequences, where language, vision, and action blocks share the same transformer backbone under a hybrid attention mask structure.

## 3. Forward and Reverse Processes

JD3P’s forward (noising) processes are adapted to the underlying data: Gaussian for continuous variables and stochastic masking or noise-injection for discrete/categorical variables. Reverse processes are learned via deep neural networks, predicting either the denoised initial (clean) state or categorical probabilities. The analytic reverse step for Gaussian-Softmax diffusion matches the posterior of the forward process, ensuring consistent joint denoising.

**SketchDNN reverse for continuous:**
$$
\mathbf{x}_{t-1} = \mu_\theta(\mathbf{x}_t, t) + \sigma_t \varepsilon,
$$
with $\mu_\theta$ and $\sigma_t$ derived from the forward kernel and the network’s denoised prediction.

**SketchDNN reverse for discrete:**
$$
\mathbf{u}_{t-1} = \frac{\sqrt{\alpha_t}(1-\bar\alpha_{t-1})\log(\mathbf{y}_t) + \sqrt{\bar\alpha_{t-1}(1-\alpha_t)}\log(\hat{\mathbf{y}_0^\theta})}{1-\bar\alpha_t} + \sigma_t \varepsilon,
$$
followed by softmax projection.

In discrete-only settings [2511.01718], the forward process replaces tokens with a mask symbol independently and reverses via conditional categorical logits. The loss is computed over masked positions via cross-entropy.

## 4. Permutation Invariance and Structured Prediction

JD3P advances permutation-invariant modeling by constructing Markov chains and neural denoisers that do not depend on primitive order. In CAD sketch modeling [2507.11579], both forward and reverse chains are factorized across primitives, and transformers are stripped of positional encodings to maintain the necessary permutation equivariance. Empirical ablation studies confirm that this design decision is crucial: adding positional encodings slightly degrades performance, while removing compositional superposition (i.e., Gaussian-Softmax diffusion) causes substantial loss in both fidelity and likelihood.

## 5. Practical Implementation and Algorithmic Schemes

JD3P models are trained via stochastic gradient descent using mini-batches and randomly sampled timesteps. Both SketchDNN and UD-VLA provide explicit pseudocode outlining forward diffusion, denoising network calls, computation of MSE and cross-entropy losses, and parameter update steps.

Sampling proceeds by initializing variables at the maximally noisy state and performing denoising steps iteratively. For multimodal discrete domains [2511.01718], inference-time techniques include prefix key–value caching, adaptive mask schedules, tempered sampling, and sub-vocabulary masking to enforce modality coherence during generation.

## 6. Empirical Results and Performance Benchmarks

On the SketchGraphs CAD dataset, JD3P delivers state-of-the-art results in both likelihood and sample quality: negative log-likelihood is reduced from 84.80 to 81.33 bits/sketch, and Fréchet Inception Distance from 16.04 to 7.80 relative to autoregressive baselines [2507.11579]. Ablations establish the critical advantage of Gaussian-Softmax superposition and permutation-invariant denoising.

For unified vision-language-action systems [2511.01718], JD3P yields faster inference (4× faster than autoregressive baselines) and higher scores on embodied robotics benchmarks, with improvements in both average trajectory lengths and task success rates over competing architectures.

## 7. Extensions and Related Frameworks

The general framework of JD3P is extensible to a variety of domains, including fully discrete spaces [2205.14987], where continuous-time Markov chain (CTMC) analogues provide efficient training, sharp theoretical guarantees (explicit ELBOs and TV distance bounds), and powerful tau-leaping samplers.

Latent Discrete Diffusion Models (LDDMs) [2510.18114] further extend the concept by coupling discrete diffusion of tokens with continuous latent channels, capturing inter-token dependencies that are otherwise lost with factorized masked denoisers. This highlights JD3P’s flexibility in supporting both joint and sequential coupling of modalities, with empirical improvements in unconditional language modeling and structured prediction.

In summary, JD3P frameworks unify discrete and continuous diffusion trajectories, enabling joint, permutation-invariant, and cross-modal generative modeling with provable training objectives and scalable sampling algorithms. Their effectiveness is established across domains such as CAD synthesis, embodied vision-language-action, and generic discrete data generation.

Source: https://www.emergentmind.com/topics/joint-discrete-denoising-diffusion-process-jd3p