FlowScene: Style-Consistent Indoor Scene Generation with Multimodal Graph Rectified Flow
Abstract: Scene generation has extensive industrial applications, demanding both high realism and precise control over geometry and appearance. Language-driven retrieval methods compose plausible scenes from a large object database, but overlook object-level control and often fail to enforce scene-level style coherence. Graph-based formulations offer higher controllability over objects and inform holistic consistency by explicitly modeling relations, yet existing methods struggle to produce high-fidelity textured results, thereby limiting their practical utility. We present FlowScene, a tri-branch scene generative model conditioned on multimodal graphs that collaboratively generates scene layouts, object shapes, and object textures. At its core lies a tight-coupled rectified flow model that exchanges object information during generation, enabling collaborative reasoning across the graph. This enables fine-grained control of objects' shapes, textures, and relations while enforcing scene-level style coherence across structure and appearance. Extensive experiments show that FlowScene outperforms both language-conditioned and graph-conditioned baselines in terms of generation realism, style consistency, and alignment with human preferences.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
FlowScene: Making Realistic, Style‑Matching 3D Rooms from Simple Descriptions
Overview: What is this paper about?
This paper introduces FlowScene, a computer system that can build complete 3D indoor rooms—like bedrooms, living rooms, and dining rooms—that look realistic and share a consistent style. It can take different kinds of input, such as text (e.g., “a wooden dining table with four matching chairs”), pictures of objects, and information about how objects relate to each other (e.g., “the chair is next to the table”). FlowScene uses all this to create a scene where the objects fit together in both layout and look.
Think of it like a smart room designer: you tell it what you want, maybe show a few example images, and it builds a 3D room with furniture that matches and looks right together.
Goals and Questions
In simple terms, the paper asks:
- Can we make 3D rooms that look real and are easy to control (where you can say exactly what objects to add and how they are arranged)?
- Can we make sure everything in the room matches the same style (for example, all modern or all vintage, same materials and colors)?
- Can we combine different inputs—text, images, and relationships between objects—to guide the design?
- Can we do this quickly and more faithfully than older methods?
How FlowScene Works (Explained Simply)
The big idea: a “scene graph” as a blueprint
- Imagine planning a room using cards connected by strings:
- Each card is an object (bed, table, lamp).
- Each string shows a relationship (“next to,” “in front of,” “same style as”).
- Each object can also hold words (text descriptions) and pictures (example images).
- This connected map is called a “multimodal scene graph.” “Multimodal” just means it can include both words and images.
Three teams working together
FlowScene builds a room in three coordinated steps, like three teams that keep talking to each other:
- Layout team: decides where each object goes and how it’s rotated (the room’s “floor plan”).
- Shape team: designs the 3D shapes of each object (what the bed, chair, or table actually looks like in 3D).
- Texture team: paints and textures each object (colors, materials like wood or fabric, patterns).
These teams share information so the final room is consistent in both placement and style.
How the system learns: “rectified flow” (a fast, steady cleanup)
- Many AI generators start with random noise and slowly “clean it up” into an image or 3D object.
- Rectified flow is a way of doing this clean‑up in a straighter, more direct route, so it needs fewer steps and runs faster.
- FlowScene uses this approach for all three teams to turn noise into a well‑designed layout, shapes, and textures.
How objects “talk” to each other
- There’s a special module (the “InfoExchangeUnit”) that lets objects share information through the scene graph while they are being generated.
- Example: If you say “all chairs match the table’s style,” the chairs and the table pass style hints back and forth so the chairs end up matching the table in both shape and texture.
Making shapes and textures efficient
- Shapes are stored in a compressed form using a 3D codebook (a kind of “zip file” for shapes) to save memory and speed things up.
- Textures are built on top of the shape, using features extracted from multiple views, so the painted look matches the object’s geometry.
What They Tested and Found
The team trained FlowScene on a large 3D furniture dataset (3D‑FRONT and its graph version, SG‑FRONT) and compared it to several other systems that either:
- retrieve existing models based on text, or
- generate 3D scenes from graphs.
Key results show that FlowScene:
- Makes more realistic scenes:
- It scored better on measures like FID and KID, which check how close generated scenes look to real ones.
- Produces better objects:
- Objects like beds, lamps, and nightstands had shapes closer to real furniture and covered more of the real variety.
- Follows instructions more accurately:
- It matched text prompts and object relationships better than the others.
- Keeps style consistent:
- Chairs matched tables; sofas matched coffee tables; colors and materials were coherent across the whole scene.
- Human testers preferred FlowScene’s style consistency and overall look.
- Runs faster:
- Thanks to its rectified flow design and the way the three teams share information, it generated scenes more quickly than previous graph-based methods.
Why This Matters
- For designers and architects: It can quickly turn sketches, text ideas, and reference images into a high‑quality 3D room that looks consistent and realistic.
- For VR/AR and games: It speeds up content creation by auto‑generating believable indoor spaces with coherent style.
- For robotics and simulation: It provides realistic, structured scenes that follow object relationships, helpful for training robots or testing indoor navigation.
- For education and creativity: It lowers the barrier to designing 3D spaces—you can describe what you want in simple language and get a styled, controlled result.
In short, FlowScene shows that combining a “graph of objects and relationships” with a fast, collaborative generation method can produce better, more consistent 3D rooms that follow your instructions—faster than before.
Practical Applications
Overview
FlowScene introduces a tri-branch, graph-conditioned rectified flow model that jointly generates scene layout, object shapes, and object textures from a multimodal scene graph (text and/or images per object plus explicit inter-object relations). Its core “InfoExchangeUnit” enables node-to-node information exchange during denoising, resulting in per-object control, relation compliance, and scene-level style consistency. Empirically, FlowScene outperforms language-only and graph-based baselines in realism, controllability, and human preference—while being significantly faster.
Below are practical applications arising from these findings and methods.
Immediate Applications
These are deployable now with modest engineering effort and domain integration.
- Interior design ideation and client visualization (Architecture/Engineering/Construction; Real Estate; Advertising/Marketing)
- Rapidly generate style-consistent room options that honor spatial constraints and user-specified relations (e.g., “bed next to window,” “chairs in same style as table”).
- Potential tools/workflows:
- Unity/Unreal/Blender plugins for “scene-on-demand.”
- Revit/SketchUp add-ons to produce conceptual layouts and dressed scenes for client review.
- Real-estate virtual staging pipelines that keep consistent style across furniture sets.
- Assumptions/dependencies: Residential indoor bias from 3D-FRONT; alignment to real-world dimensions and code constraints requires calibration; dataset/IP licensing for textures and assets.
- E-commerce visual merchandising and bundle recommendation (Retail/E-commerce; Advertising/Marketing)
- Auto-generate product bundles and virtual showrooms with consistent aesthetics for catalog pages, PDPs, and 3D viewers.
- Potential tools/workflows:
- “Style-consistent set builder” integrating SKU metadata and rendering pipelines.
- A/B testing pipeline generating scene variants to optimize engagement.
- Assumptions/dependencies: Integration with PIM/DAM systems; brand style libraries; ensuring product geometry/texture fidelity; licensing for generated content.
- Level-blockout and set dressing for games/VR (Gaming; XR/Metaverse; Media/Entertainment)
- Generate coherent indoor levels with consistent look-and-feel and controllable layout/relations, speeding up environment production.
- Potential tools/workflows:
- Unity/Unreal editor tool that ingests a node-graph or short prompt and outputs a playable, navigable indoor space.
- “Style-lock” toggles in creative tools to enforce cross-object style.
- Assumptions/dependencies: Navmesh baking, collision meshes, and game-engine material conversion; potential domain shift to non-residential themes.
- Synthetic data generation for 3D perception and VLM/LLM evaluation (Academia; Software/ML Platform)
- Produce labeled, relation-aware 3D scenes to augment datasets for detection, segmentation, layout estimation, and scene-graph prediction.
- Potential tools/workflows:
- “Scene-as-a-Service” API for dataset augmentation with programmatic control over object mix and relations.
- Benchmark harness using FPVScore-style prompts to evaluate instruction adherence and relational compliance.
- Assumptions/dependencies: Data diversity still limited to home interiors; need careful distribution matching to reduce bias.
- Robotics simulation environments for indoor navigation/manipulation (Robotics)
- Generate relation-compliant indoor layouts for training and testing policies (e.g., “mug on table,” “cabinet to the left of stove”) with style coherence to reduce spurious cues.
- Potential tools/workflows:
- Habitat/Isaac Sim/Gazebo scenario generator plugin exporting URDF/OBJ/GLB with bounding boxes and textures.
- Domain randomization recipes that vary style while preserving task-critical relations.
- Assumptions/dependencies: Physics/material properties not modeled by FlowScene; requires simulator-side physical parameters and affordances; sim-to-real validation remains essential.
- Interactive co-creation tools via GUI + LLM/VLM graph parsing (Software; Creative Tools; Education)
- Users select objects/relations in a node editor or type a sentence; LLM/VLM produces a multimodal graph; FlowScene returns a textured 3D scene.
- Potential tools/workflows:
- Web-based node-graph editor with drag-and-drop object types and “same style as” edges.
- Studio assistant that iteratively refines scenes via conversational prompts.
- Assumptions/dependencies: Reliability of LLM/VLM scene-graph parsing; user-in-the-loop validation of relations and scales.
- Privacy-preserving content creation and analytics (Policy/Compliance; Advertising/Marketing)
- Replace real-home imagery with realistic synthetic scenes for testing layout analytics or showcasing products without capturing personal data.
- Potential tools/workflows:
- Synthetic dataset factories for privacy-critical markets (EU/health).
- Assumptions/dependencies: Governance for synthetic realism vs. disclosure; watermarking of generated scenes.
- Teaching and training aids (Education; Academia)
- Teach spatial reasoning, interior design principles, or scene graphs by interactively generating examples and counter-examples of constraints and styles.
- Potential tools/workflows:
- Classroom sandbox that visualizes how graph edges affect layout, geometry, and texture.
- Assumptions/dependencies: Needs guardrails for age-appropriate and culturally diverse content.
Long-Term Applications
These require further research, scaling, domain adaptation, or tighter system integration.
- End-to-end design assistants producing manufacturable CAD/BIM (Architecture/Engineering/Construction)
- From natural language and sketches to code-compliant, costed, and schedulable models (IFC/BIM), with traceability from constraints to final design.
- Potential tools/workflows:
- BIM-integrated “Constraint-to-CAD” pipeline: FlowScene for concept, followed by parametric CAD conversion, code checking, and optimization (cost/energy).
- Assumptions/dependencies: Large, diverse datasets beyond 3D-FRONT; integration with building codes, materials, MEP; accurate scale/material semantics and physics.
- Real-time AR home planning and style-consistent shopping (Retail/E-commerce; XR; Daily Life)
- On-device inference that understands current room scans and proposes consistent arrangements and purchasable bundles in AR.
- Potential tools/workflows:
- Mobile app that fuses LiDAR/photogrammetry with a multimodal graph; “one-tap restyle” preserving constraints.
- Assumptions/dependencies: Edge acceleration of rectified flow; robust scene-graph extraction from noisy scans; product catalog alignment.
- Safety-critical robotics training and evaluation (Robotics; Policy/Safety)
- Closed-loop generation of hard scenarios (clutter, occlusions, edge-case relations), grounded in physics and affordances, to certify domestic robots.
- Potential tools/workflows:
- “Scenario curriculum engine” coupling FlowScene (static geometry/texture) with physics/material/affordance layers and task generators.
- Assumptions/dependencies: High-fidelity physical simulation, contact/friction/materials; sim-to-real transfer protocols and standards.
- Healthcare and assisted living environment design (Healthcare; Public Sector)
- Simulation and optimization of patient rooms and home modifications (fall risk reduction, accessibility), with style-consistent options that meet clinical protocols.
- Potential tools/workflows:
- Clinician-facing configurators connected to ergonomic and safety models; VR therapy spaces tailored to patient needs.
- Assumptions/dependencies: Domain adaptation to clinical standards; validation with human factors research; liability and regulatory approvals.
- Digital twins of facilities with “generative interiors” (Smart Buildings; IoT; Real Estate)
- Maintain up-to-date, style-consistent digital twins that reflect changing occupancy and furniture layouts, aiding space planning and sensor placement.
- Potential tools/workflows:
- Twin platforms that convert sensor logs and partial scans into updated multimodal graphs, then regenerate interiors at scale.
- Assumptions/dependencies: Fusion of sparse telemetry with scene-graph inference; enterprise-scale asset management; versioning/governance.
- AAA-grade procedural content generation with player-adaptive style (Gaming; XR/Metaverse)
- Live, coherent indoor worlds tailored to player preferences and gameplay, with consistent art direction across sessions.
- Potential tools/workflows:
- “Style policy” controllers that condition FlowScene on evolving player profiles and narrative constraints.
- Assumptions/dependencies: Low-latency inference; multi-agent content governance to prevent degenerate or biased outputs.
- Cross-domain expansion beyond homes (Retail stores, offices, hospitals, warehouses) and into 4D (spatio-temporal) scenes (Multiple sectors)
- Extend multimodal graphs and exchange units to public/commercial interiors; model dynamic elements and time-varying relations for simulation.
- Potential tools/workflows:
- 4D FlowScene with temporal exchange units supporting schedules, flows, and occupancy.
- Assumptions/dependencies: New datasets with diverse categories/relations; temporal annotations; broader evaluation protocols.
- Standards, audits, and governance for synthetic 3D data (Policy; Standards Bodies; Academia)
- Methods and metrics (e.g., FPVScore extensions) to certify instruction adherence, relation compliance, and style consistency; guidance on bias, IP, and disclosure.
- Potential tools/workflows:
- Open benchmarks and audit suites for graph-conditioned 3D generators; content provenance tooling.
- Assumptions/dependencies: Multistakeholder coordination; legal clarity on generative 3D IP; watermarking standards.
- Marketplaces and APIs for “style-consistent asset packs” (Software Platforms; Creative Ecosystems)
- Curated, generatively expanded asset libraries where new objects inherit a collection’s style without manual texturing.
- Potential tools/workflows:
- “Style tokens” or graph-pattern presets; commercial APIs for batch generation with SLAs.
- Assumptions/dependencies: Rights management for generated assets; interoperability (GLTF/USD/IFC); quality gating.
Notes on Feasibility and Dependencies
- Data and domain shift: Current training on 3D-FRONT skews to residential interiors; cross-domain deployment will require fine-tuning on domain-specific datasets and new relation ontologies.
- Parsing reliability: Upstream LLM/VLM errors in building the multimodal graph can degrade outputs; user-in-the-loop validation or constrained UIs are recommended.
- Physics and semantics: FlowScene ensures relational and stylistic coherence but not physical simulation (materials, mass, friction). Robotics and safety-critical uses need simulator-side physics and affordance annotations.
- Scale and measurement: Bounding boxes are normalized; accurate real-world units and tolerances require calibration and integration with CAD/BIM standards (IFC).
- Compute and latency: Few-step rectified flow improves speed, but edge/mobile AR will need optimization, distillation, or cloud offloading.
- IP and licensing: Respect dataset licenses; clarify rights for generated textures/meshes; consider watermarking and disclosure for synthetic media.
- Bias and representation: Style propagation may amplify aesthetic biases from training data; content filters and diversity controls are necessary in consumer and policy-facing applications.
Glossary
- 1-Nearest Neighbor Accuracy (1-NNA): A statistical measure used to assess how closely generated samples match the distribution of real samples by nearest-neighbor classification accuracy; lower is better for generative evaluation. "and 1-Nearest Neighbor Accuracy (1-NNA)~\cite{yang2019pointflow}."
- 3D-FRONT: A large-scale dataset of indoor scenes with furniture layouts used for training and evaluation of 3D scene generation methods. "We train FlowScene on 3D-FRONT~\cite{fu20213d} with SG-FRONT~\cite{zhai2024commonscenes}."
- 3D-FUTURE: A dataset of CAD furniture models often used for mesh retrieval and scene composition in 3D synthesis. "from 3D-FUTURE~\cite{fu20213d_future}."
- AdamW: An optimizer that decouples weight decay from gradient-based updates, commonly used to train deep models. "independently optimized using AdamW with an initial learning rate of 1e-4"
- Autoregressive generators: Generative models that produce outputs sequentially by conditioning each step on previously generated elements. "works span autoregressive generators~\cite{wang2021sceneformer,paschalidou2021atiss,zhao2024roomdesigner}"
- CLIP: A vision–LLM that provides joint text and image embeddings for conditioning and evaluation. "CLIP~\cite{radford2021learning}"
- CLIPScore: A text–image similarity metric based on CLIP embeddings that evaluates how well generated visuals adhere to textual instructions. "we report CLIPScore measuring the adherence between top-down renderings and user instructions"
- Coverage (COV): A diversity metric indicating the proportion of real samples covered by the set of generated samples (higher is better). "using MMD (), COV, and 1-NNA metrics"
- DINOv2: A self-supervised vision transformer providing robust image features, used here to extract multiview object features. "DINOv2 encoder~\cite{oquab2023dinov2}"
- Egocentric views: First-person camera viewpoints used to evaluate spatial and semantic adherence in generated 3D scenes. "FPVScore on multiple egocentric views"
- FID_CLIP: A variant of FID computed in a CLIP embedding space to assess distribution similarity between generated and real images. "FID, , and KID"
- Flow matching: A training framework for generative models that learns a time-dependent vector field transporting a prior to data along prescribed paths. "Rectified flow and flow matching~\cite{liu2022flow} have emerged as a strong alternative to diffusion-based generators"
- FPVScore: A perceptual ranking metric using multiple first-person views and prompts to assess prompt adherence, layout, quality, and style consistency. "FPVScore on multiple egocentric views"
- Fréchet Inception Distance (FID): A standard metric that compares distributions of real and generated images via Inception features; lower indicates closer match. "Fréchet Inception Distance (FID)~\cite{heusel2017gans}"
- Graph Convolutional Network (triplet-GCN): A graph neural network variant that jointly processes subject, predicate, and object features for message passing and aggregation. "triplet Graph Convolutional Network (triplet-GCN)~\cite{johnson2018image}"
- InfoExchangeUnit: A graph-conditioned module that fuses node features with current denoising states to enable inter-object information exchange during generation. "we adapt the triplet-GCN~\eqref{eq:triplet-gcn} to an InfoExchangeUnit"
- Kernel Inception Distance (KID): A metric measuring similarity between real and generated image distributions via MMD in Inception feature space; lower is better. "Kernel Inception Distance (KID)~\cite{binkowski2018demystifying}"
- LayoutExchangeUnit: A specialized InfoExchangeUnit that exchanges and enforces global layout constraints during the layout denoising process. "The LayoutExchangeUnit iteratively applies temporal layout constraints to the generation process"
- LogNormal(1,1): A log-normal sampling schedule for the time variable in rectified flow training, shaping the interpolation between data and noise. "t is sampled from a LogNormal(1,1) derived schedule"
- Multimodal scene graph: A scene graph where each node aggregates textual and/or visual features in addition to category embeddings for richer conditioning. "a multimodal scene graph is introduced by~\cite{yang2025mmgdreamer}"
- ODE (Ordinary Differential Equation): A continuous-time formulation used for deterministic sampling by integrating a learned velocity field in rectified flow models. "integrate the reverse-time ODE "
- Rectified flow: A generative modeling approach that learns straight-line velocity fields to transport noise to data with few-step deterministic ODE sampling. "Rectified flow and flow matching~\cite{liu2022flow} have emerged as a strong alternative to diffusion-based generators"
- Scene graph: A graph-structured representation of scenes with object nodes and relational edges capturing spatial/semantic relationships. "Scene graphs provide a symbolic representation of a scene as a graph with object nodes and directed edges encoding inter-object relations."
- SG-FRONT: A dataset providing 3D scenes with annotated scene graphs used for training and evaluation. "with SG-FRONT~\cite{zhai2024commonscenes}"
- ShapeExchangeUnit: A specialized InfoExchangeUnit for exchanging shape-related information among objects to ensure consistent geometry during generation. "specialized into a ShapeExchangeUnit"
- Sparse flow transformer: A flow-transformer architecture adapted to sparse data representations (e.g., voxel grids) for efficient denoising. "the texture branch employs a sparse flow transformer"
- TextureExchangeUnit: A specialized InfoExchangeUnit that exchanges texture information among objects to promote cross-object appearance consistency. "the TextureExchangeUnit exchanges texture information among nodes"
- VLMs (Vision–LLMs): Models jointly trained on images and text used for parsing or conditioning multimodal inputs. "modern LLMs or VLMs as graph constructors"
- VQ-VAE (Vector-Quantized Variational Autoencoder): An autoencoder with a discrete codebook that compresses high-dimensional data into compact latent tokens for efficient generation. "We use a shape VQ-VAE ~\cite{van2017neural}"
- Voxelization: The process of converting 3D object geometry into a grid of voxels (3D pixels) for structured processing and encoding. "voxelizing objects into a sparse structure "