Unified Continuous Generative Models

• Unified Continuous Generative Models (UCGM) are frameworks that integrate continuous diffusion, flow-matching, and autoencoding to unify generation, reconstruction, and representation learning.
• They apply a unified training regime using encoding, noising, denoising, and decoding processes across diverse modalities like images, text, and proteins.
• UCGMs offer flexible, scalable architectures that facilitate controlled data manipulation, editing, and improved performance in tasks such as image synthesis and protein fitness prediction.

Unified Continuous Generative Models (UCGM) are a class of generative modeling frameworks that extend the principles of continuous-time generative processes—specifically diffusion, flow-matching, and optimal transport—to provide a unified approach to generation, reconstruction, and representation learning across diverse modalities and tasks. UCGMs form a technical foundation that subsumes and connects diffusion probabilistic models, continuous normalizing flows, transformer-based autoencoders, and hybrid methods, and are implemented to target both high-dimensional structured data (images, text, audio, proteins, molecules) and even infinite-dimensional function spaces. Central to modern UCGMs is the ability to couple encoding/decoding with continuous-latent diffusion or flow, supporting both high-fidelity sample generation and accurate, controllable representations, all under compatible training and inference procedures.

1. Core Principles and Formal Structure

At the heart of UCGM architectures is the parameterization of a continuous-time stochastic process, typically indexed by $t \in [0,1]$ or $t\in\{1,\dots,T\}$. Generation proceeds by transforming simple reference samples (Gaussian or uniform noise) into target-data forms through a learned, often reversible, sequence of mappings involving both latent encoders and decoders. In the case of generalized encoding-decoding diffusion probabilistic models (EDDPMs) (Liu et al., 29 Feb 2024), the chain is:

• Encoding: For each sample $x_0 \in \mathbb R^d$, an encoder $\mathcal{E}_\lambda$ produces a lower-dimensional latent $z_1\in\mathbb R^m$ with

$$q_\lambda(z_1|x_0) = \mathcal{N}(z_1; \mathcal{E}_\lambda(x_0), \beta_0 I)$$

• Forward Noising: Latents $z_1$ are repeatedly noised via

$$q(z_t|z_{t-1}) = \mathcal{N}(z_t; \sqrt{1-\beta_t}z_{t-1}, \beta_t I),\quad t=2,\ldots,T$$

• Reverse Denoising: A parameterized denoiser $p_\theta(z_{t-1}|z_t)$ performs

$$p_\theta(z_{t-1}|z_t) = \mathcal{N}(z_{t-1}; \mu_\theta(z_t, t), \Sigma_\theta(z_t,t))$$

• Decoding: At $t=1$, a decoder $\mathcal{D}_\phi$ reconstructs

$$p_\phi(x_0|z_1) = \mathcal{N}(x_0; \mathcal{D}_\phi(z_1), \sigma^2 I)$$

for continuous data, or categorical for discrete data.

Training objectives are derived as negative variational bounds on the marginal likelihood:

$$\mathcal{L}(\lambda, \phi, \theta) = \mathbb{E}_{x_0}[-\log p_{\phi, \theta}(x_0)] \leq \mathbb{E}_{q} \left[ -\log p(z_T) - \sum_{t=1}^T \log \frac{p_\theta(z_{t-1}|z_t)}{q(z_t|z_{t-1})} \right]$$

with explicit KL and reconstruction terms partitioned according to structure (diffusion, alignment, and decoder loss) (Liu et al., 29 Feb 2024).

Alternative UCGM formulations generalize to continuous flow networks (Lahlou et al., 2023), wavelet-based function-space maps (Alberti et al., 2022), and implicit transport via gradient flows and McKean-Vlasov equations (Gao et al., 2020).

2. Unified Training and Inference Methodologies

UCGMs are notable for their flexible training procedures that encompass multi-step (diffusion/flow-matching) regimes as well as few-step (consistency) paradigms via a continuous interpolation parameter $\lambda\in[0,1]$. For example, the UCGM-T objective (Sun et al., 12 May 2025) constructs noisy/clean interpolants

$$x_t = \alpha(t) z + \gamma(t) x,\quad z_t = \hat{\alpha}(t) z + \hat{\gamma}(t) x$$

and trains a neural predictor $F_\theta(x_t, t)$ to regress $z_t$, with a multi-step MSE or a consistency-based loss depending on $\lambda$. The corresponding UCGM-S sampler adapts to both ODE (flow-matching) and SDE (diffusion) regimes, directly recovering standard score-based and consistency-based sampling as special cases.

These models are often instantiated in latent space via variational autoencoders, with the generative process acting either on the full data or a spatially/temporally compressed representation. Self-conditioning, task-aware conditioning, and hybrid encoder-decoder objectives are used to integrate discriminative learning (contrastive self-distillation) and unified multi-task capabilities (Xiang et al., 16 May 2025, Wang et al., 11 Aug 2025).

Empirical training details include importance sampling in time, adaptive mixing for self-boosting, use of exponential moving averages, and mixed-precision computation for efficiency and stability. Table 1 summarizes case-paper FIDs for UCGM variants (Sun et al., 12 May 2025):

Model	Steps	FID (ImageNet 256x256)
UCGM-T ($\lambda=0$)	20	1.30
UCGM-T ($\lambda=1$)	2	1.42
UCGM-S (REPA-E-XL)	40	1.06

3. Representation, Reconstruction, and Modality Fusion

UCGMs explicitly unify generation, reconstruction, and representation learning within a single framework. After joint optimization, the architecture supports:

• Generation: Sample from prior, denoise through learned flows, decode to data space.
• Reconstruction: Encode data to latents, decode through trained decoder.
• Representation: Extract latent encoding as a compact summary suitable for downstream regression/classification.
• Editing and Interpolation: Operations in latent space enable controlled manipulation, semantic morphing, and attribute-based editing (Liu et al., 29 Feb 2024).

Modal-agnostic architecture is achieved by selecting modality-appropriate encoders and decoders (e.g., UNet, Transformer, ResNet for images; BERT/GPT for text; light Transformers for protein sequences), preserving the same diffusion/flow inner loop and overall loss structure (Liu et al., 29 Feb 2024, Wang et al., 11 Aug 2025).

A key empirical result is translation to protein fitness prediction, where DiLED yields lower mean squared error in fitness regression than VAE or ReLSO, and maintains or betters cross-entropy for reconstruction (Liu et al., 29 Feb 2024).

4. Theoretical Foundations and Model Variants

The theoretical underpinning spans three foundations:

• Optimal Transport/Gradient Flows: UCGMs can be viewed as discretized approximations to Monge-Ampère flows (optimal transport), or as McKean-Vlasov evolutions, with continuous mappings realized by ODE integration (Gao et al., 2020).
• Flow-Matching and Consistency Models: Generalized flow-matching and symmetric objectives, as in SymmFlow (Caetano et al., 12 Jun 2025), enforce bidirectional consistency across semantic and data dimensions in a Neural ODE, allowing one training loss to unify image synthesis, segmentation, and classification.
• Continuous Generative Flow Networks: Amortized variational inference in continuous and hybrid domains (GFlowNets) relies on flow-matching and trajectory-balance conditions specified in measure-theoretic terms, extending UCGM framework to probabilistic inference and structure learning (Lahlou et al., 2023).

Unified loss formulations span KL, trajectory balance, Bregman divergences, and contrastive/projection-based losses.

5. Applications Across Domains

UCGMs have demonstrated practical effectiveness in a broad spectrum of fields:

• Vision: State-of-the-art image synthesis (FID=1.30 on ImageNet 256x256 with 20 steps), semantic segmentation, and image classification via SymmFlow (Caetano et al., 12 Jun 2025, Sun et al., 12 May 2025).
• Text and Proteins: BLEU and perplexity improvements on reconstruction and fluency over prior latent/auto-regressive models on text datasets; protein fitness prediction and conditional generation outperform Gaussian/discrete diffusion baselines (Liu et al., 29 Feb 2024).
• Speech: UniFlow covers enhancement, extraction, echo cancellation, and source separation via a single VAE+Diffusion Transformer latent model, training on task-id conditional signals across all speech front-end tasks with optimal trade-offs between quality and real-time factor (Wang et al., 11 Aug 2025).
• Chemistry/Materials: ADiT achieves valid and unique molecules/materials generation, unifying molecules (QM9) and crystals (MP20) with shared all-atom representation and latent diffusion, scaling up to 450M-parameter DiT for improved validity and uniqueness (Joshi et al., 5 Mar 2025).
• Function Spaces: CGNNs produce invertible, stable generative maps in $L^2(\mathbb R)$ using wavelet MRAs, and theory is extended to neural field and operator-based UCGMs (Alberti et al., 2022).

A sample of reported metrics in application-specific domains:

Task	Model	Metric	Value
Text (Yelp, BLEU)	DiLED	Reconstruction BLEU	92.1%
Images (CelebA, FID@T=50)	DiLED	FID	Outperforms LDM/DDIM
Proteins (Gifford)	DiLED	MSE/Pearson (fitness)	0.211/0.844
Speech Enh. (OVRL)	UniFlow (FM)	Speech OVRL (no reverb)	3.45
Molecules (QM9, validity)	ADiT	Validity	97.43%

6. Limitations and Future Directions

Despite their breadth, UCGMs have limitations:

• Scalability: Large DiT backbones (e.g., 1.7B parameters in UniFlow) impose memory/latency constraints for on-device or real-time use (Wang et al., 11 Aug 2025).
• Latent Compatibility: Unified latent spaces may be suboptimal for highly divergent tasks, possibly requiring conditional adaptation or hybrid latent coding (Wang et al., 11 Aug 2025).
• Resolution Constraints: In multi-task/image-semantic flows, latent-space architectures can limit output fidelity (observed in SymmFlow segmentation resolution) (Caetano et al., 12 Jun 2025).

Proposed extensions include distillation/pruning for edge deployment, hierarchical latent spaces, discrete-continuous hybridization, adaptation to novel modalities, and theoretical advances in flow-matching for infinite-dimensional function spaces (Alberti et al., 2022).

A plausible implication is the continued unification of generative, discriminative, and structured prediction within a common continuous generative backbone, with architectures adapting via modular conditioning and the fusion of encoder/decoder and flow objectives.

7. Representative Algorithms and Model Comparison

UCGMs unify and subsume a wide range of previously siloed generative models. The following table summarizes representative UCGM instantiations and their core methodologies:

Framework	Key Mechanism	Domain(s)
EDDPM/DiLED	Encoder-Decoder Diffusion	Images, Text, Proteins (Liu et al., 29 Feb 2024)
UCGM-T/S	Unified Loss & Sampler (λ-tuning)	Images, Latents (Sun et al., 12 May 2025)
SymmFlow	Symmetric Flow-Matching ODE	Images/Seg/Cls (Caetano et al., 12 Jun 2025)
UniFlow	VAE + Conditional DiT	Speech Tasks (Wang et al., 11 Aug 2025)
ADiT	Diffusion Transformer (All-Atom)	Molecules/Materials(Joshi et al., 5 Mar 2025)
CGNN	Multiresolution Wavelet Layers	Functions/Signals (Alberti et al., 2022)

These frameworks validate the central claim that a unified continuous generative model can integrate generation, reconstruction, and representation in a principled, modality-independent, and scalable fashion.