Function2Scene: 3D Layout from Functional Specs

• Function2Scene is a framework that generates 3D indoor scene layouts from natural-language functional briefs rather than object-centric prompts.
• It parses input into personas, activities, and customized design constraints to tailor room designs to specific occupant needs.
• The system employs an iterative check-and-repair loop using numeric, LLM, and VLM tools to ensure spatial validity and functional performance.

Searching arXiv for Function2Scene and closely related papers to ground the article in current literature. Function2Scene is a framework for generating 3D indoor scene layouts from functional specifications rather than object-centric prompts. In this formulation, the input is a natural-language design brief describing intended occupants, activities, and needs, and the output is a room layout whose organization is evaluated by how well it supports human use. The term is also useful in a broader research sense for systems that map functional intent into structured scene representations, but the name most directly refers to the 2026 framework “Function2Scene: 3D Indoor Scene Layout from Functional Specifications” (Wang et al., 29 May 2026). That work reframes text-driven indoor scene synthesis away from enumerating furniture and toward deriving customized design constraints, generating an initial layout, and iteratively repairing it through tool-augmented evaluation (Wang et al., 29 May 2026).

1. Conceptual framing and problem definition

Function2Scene begins from the observation that most text-driven 3D indoor scene synthesis methods are object-centric: they ask what furniture should appear in a room, but not how the room should support its occupants’ activities and physical requirements (Wang et al., 29 May 2026). The framework instead treats interior layout generation as a functionality-first problem. A brief such as “a bedroom for a couple where one partner reads late while the other sleeps early” is interpreted not merely as a request for a bed and related furnishings, but as a specification involving multiple occupants, temporal routines, glare concerns, circulation, reachability, privacy, and possibly distinct activity zones (Wang et al., 29 May 2026).

In the formal system introduced in 2026, the input is a functional specification, defined as a natural-language design brief describing who will use a room and what they need to do there (Wang et al., 29 May 2026). The system parses this brief into personas and activities, derives a tailored set of functional design constraints, generates an initial room layout, and then refines that layout through an iterative check-and-repair loop that combines geometric measurements, LLM-based contextual reasoning, and VLM-based visual assessment (Wang et al., 29 May 2026). The target output is therefore not simply a plausible furniture arrangement, but a layout that better satisfies room-specific functional requirements.

This orientation distinguishes Function2Scene from adjacent strands of scene synthesis. Graph-conditioned scene generators such as FlowScene focus on generating layout, shape, and texture from multimodal scene graphs, with controllability and style consistency as central goals (Yang et al., 20 Mar 2026). Functional scene-graph predictors such as OpenFunGraph focus on recovering object nodes, interactive elements, and functional relations from posed RGB-D sequences in real environments (Zhang et al., 24 Mar 2025). Function2Scene differs by using functional intent as the primary conditioning signal for layout generation itself, rather than treating function as a downstream annotation or a side constraint.

2. Input representation: personas, activities, and customized constraints

A defining feature of Function2Scene is that it does not directly ask a LLM to generate a final scene from the raw brief (Wang et al., 29 May 2026). Instead, the first stage extracts three intermediate structures: Personas, Activities, and Functional design constraints (Wang et al., 29 May 2026). Personas encode who the occupants are and what special needs they have; activities encode what is done in the room; and the constraint set is drawn from a taxonomy of 17 criteria spanning Spatial, Ergonomic, Activity, and Environmental categories (Wang et al., 29 May 2026).

The system produces a parsed scene description and a set of functional constraints (Wang et al., 29 May 2026). The parsed scene description is an LLM-friendly reformulation of the original brief into a clearer, more structured form resembling conventional text-to-scene prompts, but enriched with functional intent (Wang et al., 29 May 2026). The critical technical point is that the constraints are customized to the specific personas and activities in the brief rather than applied as a fixed universal rulebook (Wang et al., 29 May 2026). This implies that the same room type may induce different constraint subsets depending on occupant needs.

The taxonomy of 17 criteria is central to the framework’s representation of function. Spatial criteria include S1 Geometry validity, S2 Boundary attachment, S3 Spatial relationships, S4 Scale and proportion, and S5 Visual composition (Wang et al., 29 May 2026). Ergonomic criteria include E1 Circulation, E2 Interaction clearance, E3 Reachability, and E4 Body fit and posture (Wang et al., 29 May 2026). Activity criteria include A1 Activity zone, A2 Sightlines and privacy, A3 Workflow sequencing, and A4 Multi-activity compatibility (Wang et al., 29 May 2026). Environmental criteria include N1 Natural light access, N2 Glare prevention, N3 Acoustic separation, and N4 Ventilation and thermal comfort (Wang et al., 29 May 2026).

This representation marks a substantive shift from scene generation conditioned on object lists or coarse relation graphs. A plausible implication is that Function2Scene treats design intent as a structured constraint system rather than as a latent prompt prior. That distinguishes it both from one-shot LLM layout synthesis and from scene-graph-based generation, where relations are typically about object co-occurrence, placement, or style rather than human-centered usability (Yang et al., 20 Mar 2026).

3. Layout representation and initialization pipeline

Function2Scene organizes generation into two main stages: Initialization and Constraints-based evaluation and refinement (Wang et al., 29 May 2026). The initialization stage itself contains three components: parsing the functional brief, room structure generation, and furniture initialization (Wang et al., 29 May 2026).

The room shell is generated in a JSON-based DSL whose room_structure explicitly encodes walls, floor, ceiling, doors, and windows (Wang et al., 29 May 2026). The paper specifies a right-handed coordinate system with $+X$ east, $-X$ west, $+Y$ up, and $+Z$ south (Wang et al., 29 May 2026). Each element includes a location centroid, dimensions = [width, height, depth] in meters, and a facing angle in degrees (Wang et al., 29 May 2026). Openings may appear as standalone entries or as holes in wall objects (Wang et al., 29 May 2026). The DSL is machine-checkable and human-readable/editable, which is essential for the subsequent verification loop (Wang et al., 29 May 2026).

After the empty room is created and optionally verified by a human, the system generates an initial furniture arrangement (Wang et al., 29 May 2026). The authors explicitly note that LLM-only generation at this stage often yields overlaps, awkward adjacencies, and placements that are physically plausible yet functionally unusable (Wang et al., 29 May 2026). This motivates the refinement stage as an essential rather than optional component of the framework.

The initialization strategy is related in spirit to prior language-driven indoor generation systems such as Holodeck and LayoutVLM, which also use structured prompting and scene representations (Wang et al., 29 May 2026). Function2Scene, however, differs in that the initial layout is treated as provisional. The system assumes that direct LLM generation is insufficiently grounded, so it defers final authority to the later tool-augmented evaluation process (Wang et al., 29 May 2026).

4. Constraint evaluation and the check-and-repair loop

The core of Function2Scene is the check-and-repair loop, which performs iterative evaluation and refinement of the generated layout (Wang et al., 29 May 2026). Rather than solving all constraints jointly in one optimization problem, the system evaluates constraints sequentially in priority order, using six priority tiers: T1 foundational spatial validity, T2 layout plausibility, T3 activity support, T4 ergonomic fit, T5 medium-priority visual/sequence/environment effects, and T6 lower-priority environmental refinements (Wang et al., 29 May 2026). Lower tiers are addressed only after higher tiers have been satisfied (Wang et al., 29 May 2026).

For each constraint, the LLM interprets the constraint in the context of the current layout, selects the relevant tool or tools, receives measurements or judgments, decides whether the constraint is satisfied, proposes a local repair if necessary, applies the repair, and then re-evaluates the affected constraint before moving on (Wang et al., 29 May 2026). The emphasis on local adjustments is important: typical repairs include translating objects, rotating objects, moving objects out of paths or articulation zones, adjusting cluster placements, relocating screens, desks, or storage, and in some cases resizing by replacement rather than in-place scaling (Wang et al., 29 May 2026). At the end, Tier 1 spatial constraints are rechecked to ensure that later changes have not broken fundamental geometry (Wang et al., 29 May 2026).

The evaluation machinery is heterogeneous by design. The paper groups tools into three classes: Numeric/geometric tools, LLM-based reasoning tools, and VLM-based visual assessment tools (Wang et al., 29 May 2026). Numeric and geometric tools include boundary_check(), bbox_collision(), contact_check(), wall_angle_check(), object_exist(), object_info(), size_ratio(), pathfinding(), path_width(), articulation_zone(), chair_clearance(), free_floor_area(), total_path_length(), window_obs_ratio(), screen_window_info(), zone_distance(), vent_obs_ratio(), and distance_check() (Wang et al., 29 May 2026). LLM-based reasoning tools include size_check(), reach_check(), posture_check(), object_in_zone(), activity_support_check(), inbetween_check(), workflow_check(), multi_activity_check(), glare_check(), and acoustic_check() (Wang et al., 29 May 2026). The VLM-based layer consists of visual_balance_check() (Wang et al., 29 May 2026).

Several criteria are given operational form. Circulation is checked using A* pathfinding over the floor plane via pathfinding(start, end, scene, resolution=0.05); path_width() then measures minimum clearance along that path (Wang et al., 29 May 2026). Floor contact passes for floor objects if $Z = 0$, otherwise the object is snapped to the floor plane (Wang et al., 29 May 2026). Wall attachment passes for wall-mounted objects if the angle to the nearest wall is within $5^\circ$, otherwise rotation and translation are applied to achieve flush placement (Wang et al., 29 May 2026). Interaction clearance is measured with functions such as articulation_zone(obj, scene) and chair_clearance(chair, table, scene) (Wang et al., 29 May 2026). Activity zones use free_floor_area(zone, scene) to compute unoccupied area and then rely on LLM interpretation to decide whether the pose required by the activity can fit (Wang et al., 29 May 2026). Environmental conditions such as glare and acoustics combine measurements with contextual reasoning, for example via screen_window_info() plus glare_check(), or zone_distance() plus acoustic_check() (Wang et al., 29 May 2026).

This tool-diverse loop is one of the framework’s main methodological claims. The authors argue that purely self-reflective iterative refinement without grounded measurements can degrade layout quality, and their ablations support that view (Wang et al., 29 May 2026). More broadly, this makes Function2Scene an example of a tool-augmented design agent rather than a direct text-to-layout generator.

5. Experimental setting, baselines, and reported results

Function2Scene is evaluated on 30 real interior-design cases curated from Architectural Digest (Wang et al., 29 May 2026). These cases span 10 room types, including bedrooms, kitchens, living rooms, dining rooms, studios/ateliers, home library, guestroom, nursery, great room, and mezzanine (Wang et al., 29 May 2026). They also cover 30 distinct personas, including examples such as a retired couple, a chef, a drag queen, a child with autism, and a YouTuber (Wang et al., 29 May 2026). The diversity is intended to test whether the system can adapt constraints to different lifestyles and functional needs.

The baselines are three recent LLM-based indoor scene synthesis systems: Holodeck, iDesign, and LayoutVLM (Wang et al., 29 May 2026). These baselines are evaluated under both the original functional prompt and the paper’s parsed scene description, which allows the study to separate the effect of prompt reformulation from the effect of the Function2Scene pipeline itself (Wang et al., 29 May 2026).

The main evaluation is a two-alternative forced-choice (2AFC) perceptual study on Prolific (Wang et al., 29 May 2026). The setup involved 30 participants recruited initially, with screening and attention checks; each participant evaluated 30 comparisons, each comparison presented the room brief and two rendered layouts in randomized order, and participants were asked to choose the more functional layout while prioritizing structural validity first and then functional criteria (Wang et al., 29 May 2026). The final reported analysis retained 32 valid participants from 45 submissions, after filtering failures and duplicates (Wang et al., 29 May 2026). The main metric is pairwise preference rate, defined as the percentage of trials in which participants preferred Function2Scene over a baseline or ablation (Wang et al., 29 May 2026).

The headline result is an aggregate 94.3% pairwise preference for Function2Scene (Wang et al., 29 May 2026). The paper reports 92.2% and 88.9% against Holodeck under the functional-prompt and parsed-prompt conditions respectively; 94.4% and 98.9% against iDesign; and 96.7% and 94.4% against LayoutVLM (Wang et al., 29 May 2026). The parsed-prompt condition often improves the baselines, particularly iDesign, which suggests that functional structuring of prompts is beneficial even without the full Function2Scene loop (Wang et al., 29 May 2026). However, the framework’s strongest gains are attributed to the grounded iterative refinement rather than prompt engineering alone (Wang et al., 29 May 2026).

Ablation results reinforce this interpretation. Reported ablations include functional prompt without iterative update and without tools: 83.3%, parsed prompt without iterative update and without tools: 83.3%, functional prompt with iterative update but without tools: 78.9%, and parsed prompt with iterative update but without tools: 80.0% (Wang et al., 29 May 2026). The key empirical finding is stated directly: iterative refinement without grounded evaluation tools can actually hurt (Wang et al., 29 May 2026). This result is important because it shows that self-correction alone is not enough; the loop must be anchored in measurable or visually assessable criteria.

6. Relation to adjacent Function2Scene-style research

Although the term “Function2Scene” most precisely denotes the 2026 layout framework, it also names a broader orientation in scene understanding and generation: moving from describing what is present and where it is, toward representing what the scene enables and how it should behave for downstream use.

One nearby line of work is functional scene graph prediction. OpenFunGraph predicts functional 3D scene graphs from posed RGB-D sequences in real indoor spaces, with object nodes, interactive element nodes, and directed functional edges from interactive elements to the objects they affect (Zhang et al., 24 Mar 2025). Relationships are categorized as local or remote, such as a door handle opening a door or a switch controlling a ceiling light (Zhang et al., 24 Mar 2025). The system uses foundation models for node detection, multiview description generation, and sequential reasoning over local and remote functional relations, and it reports strong gains over adapted Open3DSG and ConceptGraph baselines on SceneFun3D and FunGraph3D (Zhang et al., 24 Mar 2025). In relation to Function2Scene, this work addresses functional understanding of existing environments rather than synthesis of layouts from design briefs. A plausible implication is that such graph representations could serve as a downstream semantic layer for validating whether generated layouts expose the intended controls and affordances.

Another neighboring line is multimodal graph-conditioned scene generation. FlowScene uses a multimodal scene graph and a tri-branch rectified-flow generator to synthesize layout, shape, and texture for indoor scenes while maintaining style consistency (Yang et al., 20 Mar 2026). Nodes may contain category, text, and image features, and branch-specific exchange units perform node-wise information exchange during denoising so that objects are generated collaboratively rather than independently (Yang et al., 20 Mar 2026). This offers strong controllability and scene-level appearance coherence. Compared with Function2Scene, FlowScene is more comprehensive in geometric and visual generation, but its conditioning remains graph-based rather than explicitly persona-and-activity-based. Function2Scene is therefore narrower in output scope, focusing on 3D indoor layout, but deeper in human-centered functional specification.

A third adjacent strand concerns functionally addressable scene representations for editing. DM-NeRF augments NeRF with an object field that assigns each 3D point an object identity code, learned from 2D supervision, and then uses an inverse query algorithm to manipulate selected objects in 3D while handling collisions and occlusions (Wang et al., 2022). Through a Function2Scene lens, DM-NeRF turns a scene into a queryable function over 3D space, enabling object-wise edits after reconstruction (Wang et al., 2022). This differs from Function2Scene’s design-agent paradigm, but both share a concern with moving from passive representation to actionable, structured control.

At a broader scale, Sat2Scene demonstrates a related transition from image translation to scene synthesis. Given a satellite image, it generates a sparse 3D urban scene representation that supports arbitrary-view rendering, using 3D diffusion on point-based geometry and neural rendering for view-consistent outputs (Li et al., 2024). Sat2Scene is not function-driven in the interior-design sense, but it exemplifies the broader “input specification to reusable scene” pattern that Function2Scene also instantiates (Li et al., 2024).

Taken together, these works suggest that “Function2Scene” can denote a family of methods in which the conditioning signal is richer than object labels alone. In some cases the signal is a functional brief (Wang et al., 29 May 2026), in others a graph of interactive affordances (Zhang et al., 24 Mar 2025), a multimodal relation graph (Yang et al., 20 Mar 2026), or an object-aware implicit field for manipulation (Wang et al., 2022). The common thread is that scene generation or representation is guided by structured intent rather than by isolated image prompts.

7. Significance, limitations, and likely research directions

The significance of Function2Scene lies in its argument that scene synthesis should be judged by how well a space supports human use, not only by how plausible its objects look (Wang et al., 29 May 2026). The framework therefore bridges classical rule-based layout design and modern foundation-model pipelines: LLMs are not treated as one-shot layout generators but as interpreters of design briefs, customizers of constraints, and agents within an iterative design loop (Wang et al., 29 May 2026). The reported preference results indicate that this functionality-first framing yields layouts that human evaluators consider more suitable to the stated briefs than those produced by recent object-centric baselines (Wang et al., 29 May 2026).

The paper is also explicit about its limitations. It requires detailed functional specifications, whereas real users often begin with vague goals and may need a conversational upstream interface to elicit relevant constraints (Wang et al., 29 May 2026). Its tooling remains somewhat coarse, relying on simple geometric checks, LLM judgments, and VLM top-down assessments; the authors suggest that richer tools such as embodied simulation with articulated models, physically accurate lighting estimation, acoustic simulation, and more semantic DSLs for distance and dimension requirements would improve robustness (Wang et al., 29 May 2026). The system also assumes a fixed architectural shell, so it does not co-optimize room shape, openings, partitions, and furnishing as a unified problem (Wang et al., 29 May 2026). Finally, because several constraints rely on LLM semantic reasoning, the system remains vulnerable to incorrect context interpretation, faulty reachability or posture judgments, and inaccurate repair suggestions, even though tool grounding mitigates some of this risk (Wang et al., 29 May 2026).

These limitations indicate several research directions. One is integration with functional scene understanding systems such as OpenFunGraph, which could provide explicit representations of interactive elements and control relations for both validation and robotic downstream use (Zhang et al., 24 Mar 2025). Another is coupling functionality-first layout synthesis with higher-fidelity multimodal generation systems such as FlowScene, potentially combining persona-and-activity constraints with graph-coupled layout, geometry, and texture generation (Yang et al., 20 Mar 2026). A further direction is to embed more physically grounded simulation into the loop, replacing or supplementing heuristic checks with embodied measurements of navigation, visibility, reach, acoustics, or thermal comfort. This suggests that Function2Scene is less a final solution than a redefinition of the optimization target in indoor scene synthesis.

In that sense, Function2Scene marks a shift from object placement to use-oriented design computation. Rather than asking whether a generated room contains plausible furniture, it asks whether the arrangement supports occupants, routines, ergonomics, and environmental comfort as specified in the brief (Wang et al., 29 May 2026). That reframing is its primary contribution to the literature on scene generation and functional scene understanding.