ParaUni: Unified Multimodal Generation

• ParaUni is a unified multimodal framework that extracts hierarchical features from vision-language models using learnable query tokens in parallel.
• Its Layer Integration Module (LIM) fuses outputs from all VLM layers with reinforcement-driven, layerwise adjustments to optimize reward metrics.
• The design enables modular flexibility and efficient updates, enhancing image fidelity, composition, and semantic alignment.

ParaUni is a unified multimodal generative framework that integrates hierarchical features extracted in parallel from vision-LLMs (VLMs) and fuses them via a modular Layer Integration Module (LIM) to drive a frozen diffusion decoder. Distinct from prior methods that rely on features from a single VLM layer or cascaded fusion, ParaUni addresses the challenge of representation heterogeneity and reward alignment by employing reinforcement-driven, layerwise dynamic adjustment to optimize the use of low- to high-level information across all VLM layers. This approach delivers improved visual generation, modular flexibility, and principled reward-based optimization in unified multimodal systems (Tan et al., 5 Dec 2025).

1. Model Architecture and Workflow

The ParaUni pipeline accepts either image or text prompts. Input is processed by a frozen autoregressive transformer-based VLM. To extract comprehensive representations, $N$ learnable query tokens $q_l$ are inserted into each of the $L$ transformer layers of the VLM. For each layer $l$, query tokens $q_l$ cross-attend to that layer's hidden states, producing a feature set $f_l$. All $L$ feature sets $\{ f_l \}_{l=1}^L$ are then processed simultaneously by the shared, lightweight LIM—composed of one or two self-attention layers and LayerNorm (typically with $d \approx 1024$)—yielding a fused representation $c$. This output serves as the cross-attention context for a frozen diffusion decoder (e.g., DiT), which performs denoising to generate the final image.

The VLM and diffusion model parameters remain frozen; only the LIM parameters are trained. This strict separation allows for efficient model updates and the ability to interchange VLM or diffusion modules without global retraining. Conditioning the diffusion model solely through LIM ensures a unified computational graph while maintaining modularity between understanding and generation.

2. Parallel Feature Extraction and Layer Integration Module (LIM)

For each layer $i$, $n$ learnable query tokens $q_i \in \mathbb{R}^{n \times d}$ are used to extract $f_i$ via cross-attention. These outputs, $\{ f_i \}$, capture distinct visual-semantic abstractions at various network depths. Unlike sequential/cascaded fusing, all layer extractions occur in parallel.

Feature fusion proceeds as follows: each $f_i$ is first processed by a shared Transformer block $f_\theta$ and LayerNorm $\mathrm{LN}$: $$c_i = \mathrm{LN}(f_\theta(f_i)),\qquad \forall i=1\dots L$$ Fused context is produced by uniform averaging, with fusion weights $w_i = 1/L$ by default: $c_\mathrm{fused} = \frac{1}{L} \sum_{i=1}^L c_i$ More generally, learnable fusion weights $w_i$ (subject to $\sum_i w_i = 1$ and $w_i \geq 0$) may be used: $F_\mathrm{fused} = \sum_{i=1}^L w_i\, c_i$ This design enables efficient GPU batching and storage of $L$ sets of $n \times d$ features, which are subsequently reduced to $n \times d$ after fusion. Only the lightweight LIM is trained, making adaptation efficient.

3. Layer-wise Reward Sensitivity and Reinforcement-driven Adjustment

Empirical analysis confirms distinct reward sensitivities across VLM layers:

• Shallow layers: Enhance low-level texture fidelity
• Mid-level layers: Influence aesthetic (score $R_a$) and preference (Pickscore $R_p$) metrics
• Deep layers: Drive semantic alignment as measured by the CLIP score $R_c$

Reward signals are formally defined:

• $$R_c(x) = \mathrm{CLIP}(\text{prompt}, x) \in [-1, 1]$$, higher values indicate stronger semantic alignment
• $R_a(x) = \mathrm{aesthetic\_model}(x) \in [0,12]$, higher is more aesthetically pleasing
• $R_p(x) = \mathrm{pickscore\_model}(x) \in [0,1]$, higher is more human-preferred

ParaUni treats the diffusion generator as a policy $\pi_\theta$ over images $x$, maximizing $J(\theta) = \mathbb{E}_{x \sim \pi_\theta} [R(x)]$ using Flow-GRPO to update diffusion-side parameters.

Layer-wise Dynamic Adjustment Mechanism (LDAM) introduces targeted stochastic perturbations. For a given reward $R^k$ (with $k \in \{c, a, p\}$), if the reward plateaus or degrades or the layer's gradient norm $g$ spikes, Gaussian noise $\epsilon \sim \mathcal{N}(0, I)$ is injected into the corresponding $c_i$ during training:

• $c_i \leftarrow c_i\, (1 + \gamma \epsilon)$ $\gamma$ is scaled so that the perturbation magnitude is $O(1\%)$, and a “cooling-off” period prevents instabilities. Only layers most sensitive to the monitored reward receive perturbations, pushing the model out of local minima in reward space.

4. Performance Evaluation and Ablation Analysis

ParaUni demonstrates robust improvements over prior unified and diffusion baselines. Key benchmarks include:

Model/Conditioning	GenEval	DPG-Bench
ParaUni (no RL)	0.87	83.45
Best prior unified method	0.86	83.08
Diffusion baselines	0.55–0.74	—

Multi-stage RL (Aesthetic $\rightarrow$ Pickscore $\rightarrow$ CLIP) further increases targeted rewards, with CLIP score rising by approximately 1.2%.

Ablation studies establish the necessity of each architectural component:

• Omitting any category of layer (shallow/mid/deep) during conditioning degrades GenEval by as much as 0.14.
• Combining features from only the last VLM layer reduces GenEval from 0.87 to 0.82.
• Eliding LayerNorm from LIM reduces GenEval by 4 points; removing the shared Transformer block is more detrimental.
• Disabling GradNorm or reward-degradation guidance in LDAM diminishes targeted RL-stage gains by 30–50%.

Qualitative analyses show ParaUni achieves finer texture (shallow information), improved composition (mid-level), and more accurate prompt alignment (deep-level) compared to single-layer baselines, excelling on demanding multi-object prompts.

5. Modularity, Applications, and Extension Strategies

ParaUni’s architectural decoupling and modular fusing entail notable practical consequences:

• Text-to-image: Enhanced detail and semantics facilitate creative and commercial generation.
• Image-to-image and editing: The VLM’s frozen weights enable transfer to applications such as style transfer, in-painting, and guided editing via conditioning swaps.
• Human-in-the-loop preference alignment: LDAM enables integration of novel reward signals (e.g., for style branding or safety) by pinpointing responsive VLM layers.

Potential extensions include:

• Multimodal fusion by adding query heads for additional modalities (audio, video), reusing the LIM.
• Training fusion weights $w_i$ to adaptively emphasize layers for various tasks rather than using uniform weights.
• Continuous RL via differentiable, layer-wise reward-weighting policies: $w_i \leftarrow w_i + \alpha \nabla_{w_i} R^k$
• Scaling to larger VLMs and high-resolution diffusion decoders (e.g., for 4K generation).

6. Significance and Outlook

ParaUni operationalizes comprehensive, parallel interaction with VLM representations and leverages layer-specific reinforcement-driven guidance for unified multimodal generation tasks. By fusing learnable queries from all VLM layers via an efficient LIM and governing adaptation through LDAM, ParaUni achieves simultaneous improvements in fidelity, preference alignment, and modularity. It provides a foundation for extensible, reward-driven architecture in multimodal generation, supporting repeated advances in data efficiency and cross-domain flexibility (Tan et al., 5 Dec 2025).