Dream2Image: Multimodal Image Synthesis

• Dream2Image is a multimodal framework that transforms neural, textual, or visual conditions into images using structured generative pipelines.
• It integrates techniques such as brain decoding, multi-view synthesis, layered compositing, and restoration to enhance image fidelity and downstream usability.
• Applications span fMRI-based dream decoding, 3D reconstruction, and customized image editing, emphasizing interpretability and control in synthesis.

Dream2Image denotes a research pattern in which latent, neural, textual, or visual conditions are transformed into images through structured generative pipelines. In the cited literature, the term appears in two closely related senses. First, it names an explicit multimodal resource—an open EEG, dream-text, and AI-image dataset designed for dream decoding research (Bellec, 3 Oct 2025). Second, it functions as a broader systems concept: a family of architectures that map brain activity, reference images, edit instructions, structural conditions, degraded inputs, or multi-view prompts into images, layered composites, or image sets suitable for 3D reconstruction (Xia et al., 2023). Taken together, these works frame Dream2Image as an umbrella for multimodal image synthesis and reconstruction rather than a single canonical model.

1. Terminological scope and research landscape

Across recent work, Dream2Image spans brain decoding, conditional generation, customization, editing, restoration, and super-resolution. The common denominator is not a particular backbone, but a conditional mapping in which the input is more structured than unconstrained text and the output is an image or image-like representation suitable for downstream use.

Work	Primary conditioning	Primary output
DREAM (Xia et al., 2023)	fMRI	reconstructed image
Dream2Image dataset (Bellec, 3 Oct 2025)	pre-awakening EEG, dream reports	AI-generated dream image
MultiImageDream (Kim et al., 2024)	one or more reference images + text	multi-view images and 3D-ready guidance
DreamLayer (Huang et al., 17 Mar 2025)	text decomposed into layer prompts	coherent transparent layers + composite
DreamO (Mou et al., 23 Apr 2025)	multiple reference images + placeholders + text	customized image
DreamOmni (Xia et al., 2024)	text, images, masks, drag cues, maps	generation and editing outputs
DreamClear (Ai et al., 2024)	degraded image + reference + text	restored photorealistic image
DreamSR (Dong et al., 15 May 2026)	LR image + global and local prompts	ultra-high-resolution SR output
DreamSync (Sun et al., 2023)	text with VLM feedback during training	better aligned T2I generations

This distribution of tasks indicates that Dream2Image has become a convenient label for multimodal image reconstruction systems in which image synthesis is coupled to intermediate control variables, external evaluators, or structured latent factors. A plausible implication is that the term now describes a design philosophy—multimodal conditioning plus image synthesis—more than a single benchmark problem.

2. Brain-activity-to-image decoding

In brain decoding, DREAM formalizes a canonical Dream2Image pipeline as a two-step inverse problem from neural measurements to intermediate cues and then to image synthesis (Xia et al., 2023). Given an fMRI measurement vector, DREAM first predicts three visual factors,

$\text{fMRI} \mapsto \{\hat S, \hat C, \hat D\},$

where $\hat S$ is a semantic embedding, $\hat C$ a color field, and $\hat D$ a depth map; it then uses these to guide a pretrained text-to-image diffusion model,

$\{\hat S,\hat C,\hat D\} \mapsto \hat I.$

The architecture mirrors the organization of the human visual system. The Reverse Visual Association Cortex (R-VAC) maps fMRI to a CLIP-compatible semantic tensor of shape $77 \times 768$, while the Reverse PKM (R-PKM) maps fMRI to RGBD, from which depth and a spatial color palette are derived. Guided reconstruction is performed with Stable Diffusion plus T2I-Adapter modules for color and depth, with composition weights $\omega_c$ and $\omega_d$ controlling the contribution of each cue.

DREAM is trained on the NSD dataset with subject-specific models for sub01, sub02, sub05, and sub07. The paper reports ~8,859 training images per subject with ~24,980 fMRI response trials and 982 test images with ~2,770 fMRI trials, using the “nsdgeneral” ROI. Its quantitative evaluation uses PixCorr, SSIM, AlexNet(2), AlexNet(5), InceptionV3, CLIP image embedding similarity, EffNet-B1, and SwAV metrics, and it additionally evaluates depth with AbsRel, SqRel, RMSE, and RMSE log, plus color with CD and STRESS. For subject 1, it reports CLIP $\approx 95\%$ and Inception $\approx 94.8\%$ (Xia et al., 2023).

The explicit Dream2Image dataset extends this line from fMRI-based viewed-image reconstruction to sleep EEG aligned with dream reports (Bellec, 3 Oct 2025). It contains 38 participants, ~31.105 hours of sleep EEG, and 129 dream samples. Each sample includes up to four pre-awakening EEG windows—T-15s, T-30s, T-60s, and T-total up to 120 seconds—plus a verbatim transcript, a one-sentence description, and a 512-pixel AI-generated image. The EEG is harmonized to 17 channels, resampled to 400 Hz, stored as $\hat S$0 arrays, and z-score normalized per channel. The retained channels are $\hat S$1.

A central misconception is that the dataset’s images are direct neural reconstructions. They are not. The paper states that each image is generated from the dream report through a multi-stage semantic extraction, prompt creation, neuropsychological validation, DALL·E 3 generation, and fidelity filtering loop, with only images scoring at least 3 on a 0–5 fidelity scale retained (Bellec, 3 Oct 2025). By contrast, DREAM reconstructs viewed images directly from fMRI using learned neural decoders and diffusion guidance (Xia et al., 2023).

3. Multi-view and layered image formation

A second major Dream2Image branch treats images themselves as prompts for further image or 3D generation. MultiImageDream extends ImageDream’s single-image-prompted multi-view diffusion to multiple image prompts without retraining the backbone (Kim et al., 2024). ImageDream itself augments MVDream, which generates four orthogonal views—front, back, left, and right—from text—by adding a local controller and a pixel controller. The local controller injects CLIP features into cross-attention layers; the pixel controller injects the image latent into 3D dense attention so that generated views attend to the reference image at pixel level. MultiImageDream generalizes both controllers to $\hat S$2 images by concatenating resampled CLIP tokens and stacking per-image latents as extra views: $\hat S$3 and the 3D attention tensor changes from $\hat S$4 to $\hat S$5. The backbone weights remain unchanged, and 3D generation proceeds through SDS-based NeRF optimization. The paper reports that optimization takes about 1 hour with only a slight overhead for multi-image prompts. On synthesized multi-view outputs, a baseline 1-ImageDream configuration with QIS 27.10, CLIP(TX) 32.39, and CLIP(IM) 85.69 is improved by several 2-image variants, including QIS 28.95 and CLIP(IM) 87.10 for pixel(f) + local(fb) (Kim et al., 2024).

DreamLayer addresses a different decomposition: not viewpoint, but transparent scene layers (Huang et al., 17 Mar 2025). A $\hat S$6-layer image contains a background, multiple foreground RGBA layers, and a global composited layer. DreamLayer generates all of them simultaneously from text by adding three mechanisms to Stable Diffusion v1.5: Context-Aware Cross-Attention (CACA), Layer-Shared Self-Attention (LSSA), and Information Retained Harmonization (IRH). CACA derives object-specific global context maps from the global layer’s cross-attention and aligns foreground-layer attention maps with them through $\hat S$7 and $\hat S$8. LSSA concatenates latent sequences across layers while keeping per-layer queries, so each layer attends to all others without collapsing semantics. IRH fuses foreground and background latents in latent space during late denoising, allowing the model to synthesize shadows and other interaction effects rather than merely stacking RGBA layers in pixel space.

DreamLayer is trained on a ~400k-sample synthetic multi-layer dataset, with 408,187 total images, 1453 classes, and a layer distribution of ~305k two-layer, ~87k three-layer, and ~15k four-layer samples (Huang et al., 17 Mar 2025). On 2-layer composite images, the paper reports AES 7.013, CLIP 34.835, and FID 50.761 for DreamLayer, compared with AES 6.930, CLIP 34.678, and FID 53.950 for SD v1.5, and AES 6.522, CLIP 32.466, and FID 63.481 for LayerDiffusion. Its user study reports 71.34% preference for multi-layer quality and 83.03% for background quality over LayerDiffusion.

These two systems expose complementary Dream2Image decompositions. MultiImageDream makes viewpoint structure explicit; DreamLayer makes alpha-compositional structure explicit. This suggests that Dream2Image methods increasingly externalize latent factors that earlier T2I systems left implicit.

4. Unified customization and editing frameworks

DreamO and DreamOmni both pursue a unified Dream2Image engine, but they operationalize unification differently. DreamO uses Flux-1.0-dev as a DiT backbone with Flow Matching, freezing the base model and training LoRA modules plus condition-related embeddings (Mou et al., 23 Apr 2025). The noising process is

$\hat S$9

and the model predicts the velocity $\hat C$0. All condition images—identity, subject, clothing, style, or other references—are encoded by the Flux VAE into the same latent token space as the noisy image. DreamO then adds a learnable condition embedding, an index embedding per condition group, and RoPE positional encoding, so heterogeneous inputs become a single transformer sequence.

Two auxiliary mechanisms make this unified tokenization usable. The feature routing constraint supervises condition-to-image attention maps with target masks: $\hat C$1 The placeholder strategy inserts tokens such as $\hat C$2 into the text prompt and binds each condition image to its designated placeholder via $\hat C$3. DreamO trains progressively in three stages: 20K iterations on Subject200K and two-column data, 90K iterations on the full multi-task mixture, and 3K iterations of quality alignment on 40K Flux-generated images with 95% reference-token dropout. The supported task set includes identity, subject appearance, multi-subject composition, virtual try-on, and two forms of style transfer (Mou et al., 23 Apr 2025).

DreamOmni uses a different unification strategy centered on a VLM encoder, Qwen2-VL 7B, plus a latent DiT with Rectified Flow (Xia et al., 2024). Its training objective is

$\hat C$4

with $\hat C$5. Conditioning $\hat C$6 is extracted from the penultimate layer of Qwen2-VL and can include text prompts, source images, masks, drag tuples, structural maps, or reference images. Latent tokens and VLM tokens undergo joint self-attention, followed by separate FFNs for the two modalities. DreamOmni also adopts UNet-style long connections between early and late features; the paper reports that these long connections yield 4× faster convergence than a vanilla DiT under matched settings.

The task coverage is broader than in most prior single-backbone systems: pure T2I, inpainting, outpainting, instruction-based editing, drag-based editing, image-conditioned generation from canny/depth/segmentation maps, subject-driven generation, and segmentation/detection-like editing (Xia et al., 2024). Training uses 125M real T2I images—103M from a filtered LAION subset and 22M annotated with InternVL2—plus about 68M synthetic samples. The resolution curriculum spans 377K iterations at 256×256, 189K at 512×512, and 140K at 1024×1024 on 64× A100 GPUs. On GenEval, DreamOmni reports overall 0.70, counting 0.65, and position 0.34, matching SD3-Medium overall and improving on its counting and position scores. On inpainting and outpainting, it reports FID 0.8371 and 1.6926, respectively, versus 1.3522 and 2.9179 for SD-Inpainting (Xia et al., 2024).

5. Alignment, restoration, and ultra-high-resolution synthesis

DreamSync shifts the Dream2Image problem from conditioning design to training-time alignment (Sun et al., 2023). It treats image generation as a self-training loop—Sample, Evaluate, Filter, Finetune—in which a T2I model generates $\hat C$7 candidates per prompt, a VQA model derived from the TIFA framework scores semantic faithfulness, VILA scores aesthetics, and the model is LoRA-finetuned on the best filtered samples. For SDXL, the paper uses $\hat C$8, $\hat C$9, LoRA rank $\hat D$0, 10,000 prompts per iteration, 8 images per prompt, and 2500 training steps per round. It reports improvements of +1.7 on TIFA Mean, +3.7 on TIFA Absolute, +2.9 on DSG1K, and +3.4 on VILA aesthetic for SDXL, with the resulting model reaching TIFA Mean 85.2, TIFA Absolute 49.2, DSG1K 86.3, and VILA 64.3 (Sun et al., 2023).

DreamClear addresses Dream2Image as restoration from degraded inputs to photorealistic outputs (Ai et al., 2024). Its first component, GenIR, constructs a privacy-safe synthetic IR dataset through three stages: image-text pair construction, dual-prompt based fine-tuning, and data generation and filtering. GenIR produces ~1,000,000 HQ images at resolution $\hat D$1, cropped or resized to $\hat D$2 for training. DreamClear itself is a PixArt-$\hat D$3-based DiT restoration model with a dual-branch architecture: an LQ branch, a reference branch derived from SwinIR, and an MLLM text branch using LLaVA. ControlFormer provides transformer-native control, and the Mixture of Adaptive Modulator (MoAM) uses a degradation map and $\hat D$4 experts for token-wise adaptive modulation. The model is trained on 32× A100 GPUs for 7 days with AdamW at $\hat D$5 and sampled with iDDPM for 50 steps at CFG scale 4.5. On synthetic benchmarks it reports DISTS 0.1637 and FID 20.61; on real benchmarks it achieves NIQE 4.4381 on RealSR, 4.6295 on DRealSR, and 3.5556 on RealLQ250, while a user study reports more than 45% of total votes and top choice for ~80% of images (Ai et al., 2024).

DreamSR extends this restoration logic to ultra-high-resolution SR under patch-wise diffusion inference (Dong et al., 15 May 2026). Built on FLUX.1-dev, it uses a dual-branch MM-ControlNet in which the pre-trained DiT receives global prompts while the ControlNet receives local patch-level prompts, thereby addressing over-generation caused by global-prompt misalignment on local patches. Its two-stage inference consists of one-step degradation removal with a Restoration Acceleration LoRA, followed by 16 steps of texture generation from an intermediate noisy state $\hat D$6, for 17 total steps. The paper reports training on 580k general and 120k face HQ images with text descriptions, using 16 NVIDIA H20 GPUs, AdamW at $\hat D$7, and 512×512 crops under a Receptive-Field Enhancement strategy. At 2560×1440 resolution on a single H20 GPU, DreamSR reports 86 seconds for diffusion. Qualitatively and quantitatively, it is described as state-of-the-art or near SOTA on no-reference metrics across RealLQ250, RealLR200, and RealDeg (Dong et al., 15 May 2026).

These systems show that Dream2Image is not limited to first-pass generation. It also encompasses alignment loops, restoration modules, and SR stages that refine or correct images after initial synthesis.

6. Data regimes, limitations, and future directions

A recurring structural issue is data scarcity at the point where the condition becomes most informative. In neural decoding, DREAM remains subject-specific, and the explicit Dream2Image EEG dataset contains only 129 samples from 38 participants, with variability in dream recall and strong inter-subject variability (Xia et al., 2023). The dataset paper explicitly warns that its images are approximate visual reconstructions derived from text rather than literal neural readouts, and it lists small sample size and recall variability as limitations (Bellec, 3 Oct 2025).

In image-prompted and multi-view systems, conditioning often exceeds what the backbone was trained to exploit. MultiImageDream adds multi-image prompting only at test time, without explicit multi-image training; its quantitative gains on 3D metrics are modest, evaluation covers only 39 prompts, and the prompts used in the paper are generated rather than real multi-view captures (Kim et al., 2024). DreamLayer currently supports up to four layers, depends on synthetic decomposition with detection, matting, and inpainting, and remains a 2D design without explicit 3D structure (Huang et al., 17 Mar 2025).

Unified customization and editing systems inherit their own trade-offs. DreamO’s spatial control is mediated by routing and placeholders rather than a general geometric controller, so a plausible implication is that layout precision remains weaker than in explicitly map-conditioned architectures (Mou et al., 23 Apr 2025). DreamOmni reports strong unified performance, but large-angle rotations in drag editing remain challenging and can produce shape distortions (Xia et al., 2024).

Restoration-oriented Dream2Image variants are constrained by compute, patch design, and prompt quality. DreamClear requires large-scale synthetic curation plus heavy DiT training; DreamSR explicitly notes computational cost and the difficulty of generating meaningful patch-level prompts at inference (Ai et al., 2024). DreamSync depends on the base model’s ability to produce at least some faithful candidates per prompt; if the base model rarely reaches the correct mode, the self-training loop has limited bootstrap signal (Sun et al., 2023).

Taken together, these works suggest three converging directions. First, Dream2Image systems are moving toward unified multimodal conditioning, whether through explicit token routing, VLM-derived conditioning spaces, or structured controllers. Second, synthetic data generation is becoming a core infrastructure layer rather than a supplement, as seen in DreamLayer’s 400k multi-layer corpus, DreamOmni’s large synthetic editing sets, and DreamClear’s 1M-image GenIR dataset (Huang et al., 17 Mar 2025). Third, the field is shifting from single-output generation toward controllable intermediate representations—semantic, depth, color, view, layer, degradation, or patch context—that make image synthesis more interpretable, more editable, and more amenable to downstream 3D or neuroscientific use.