Code-as-Room: A Framework for Room Representation

Updated 5 July 2026

Code-as-Room is a paradigm that represents indoor environments as explicit programmatic states, enabling controllable scene synthesis and editing.
It employs diverse formalisms—from tensors and bounding boxes to DSLs and executable code—to capture spatial and interactive properties.
Multiple pipelines integrate generation, compilation, and localized modifications to support robust evaluation and dynamic scene editing.

Searching arXiv for the cited works on Code-as-Room and closely related room-as-code representations. Code-as-Room is a research paradigm in which a room, indoor environment, or room-like artifact is represented as an explicit programmatic state rather than only as a rendered image or static mesh. Across the literature, this state takes multiple forms: a holistic scene code for indoor layout synthesis, a semantic proxy room of bounding boxes, a Python-like visual program, HTML and Web Components for WebXR escape rooms, executable Blender programs for articulated assets, RDF-like symbolic tuples for a memory environment, a JSON-based indoor DSL, and, in software visualization, a room metaphor in which source code is spatialized as enterable rooms (Fang et al., 2023, Schult et al., 2023, Kim et al., 21 Jun 2025, Yang et al., 18 May 2026, Wang et al., 19 May 2026, O'Connor, 2024, Kim et al., 2022, Chen et al., 12 Dec 2025, Khaloo et al., 2017). The unifying idea is that room structure, object identity, relations, constraints, and edit operations become explicit and inspectable, enabling controllable generation, localized modification, execution, and evaluation.

1. Scope and historical breadth

The term does not denote a single canonical representation. In "Ctrl-Room" (Fang et al., 2023), a room is a tensor of per-object attributes used for text-to-3D generation and editing. In "ControlRoom3D" (Schult et al., 2023), a room is a semantic proxy room composed of semantic 3D bounding boxes plus a style prompt. In "Programmable-Room" (Kim et al., 21 Jun 2025), a room is specified by an LLM-written Python-like visual program. In "Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis" (Yang et al., 18 May 2026), it is executable Blender Python code synthesized from a top-down image under a structured execution harness. In "SceneCode" (Wang et al., 19 May 2026), it becomes an executable world program with articulated objects and simulator-ready SDF export. In "RoomPilot" (Chen et al., 12 Dec 2025), the shared representation is IDSL, an Indoor Domain-Specific Language. Outside 3D scene synthesis, "ESCape the ClassRoom" (O'Connor, 2024) uses web code to define rooms, puzzles, and state transitions, "A Machine With Human-Like Memory Systems" (Kim et al., 2022) encodes room events and queries as RDF-like tuples, and "Code Park" (Khaloo et al., 2017) maps each software class to a navigable room.

Work	Room-as-code representation	Domain
Ctrl-Room (Fang et al., 2023)	Holistic scene code $x_0 \in \mathbb{R}^{N\times D}$	Text-to-3D room meshes
ControlRoom3D (Schult et al., 2023)	Semantic proxy room $M=\{B_k\}$	Controlled room mesh generation
ESCape the ClassRoom (O'Connor, 2024)	HTML, A-Frame entities, Web Components, JS state machines	WebXR educational escape rooms
Programmable-Room (Kim et al., 21 Jun 2025)	Python-like visual program over generation/editing modules	Interactive textured room generation
RoomPilot (Chen et al., 12 Dec 2025)	JSON-based IDSL	Multimodal indoor scene synthesis
SceneCode (Wang et al., 19 May 2026)	Executable Blender Python programs plus registry	Editable articulated indoor worlds
The Room (Kim et al., 2022)	RDF-like symbolic tuples and memory state	Gym-compatible memory environment
Code Park (Khaloo et al., 2017)	One class → one room with syntax-aware wallpaper	3D code visualization

This breadth matters because it separates Code-as-Room from narrower notions such as procedural layout generation or text-to-image prompting. Some systems treat code as the room’s latent state, some as the room’s executable construction procedure, and some as the interaction logic governing what can happen inside the room (Fang et al., 2023, O'Connor, 2024, Wang et al., 19 May 2026). A plausible implication is that Code-as-Room is best understood as a family of explicit scene representations rather than a single file format or algorithm.

2. Representational forms and formal structure

A central line of work encodes a room as an object-wise structured state. Ctrl-Room defines a room scene $S$ with $m$ walls and $n$ furniture items as a set of $N=m+n$ object rows,

$x_0=\{o_i\}_{i=1}^N,\qquad o_i=[c_i,l_i,s_i,r_i],$

with $l_i\in\mathbb{R}^3$ for center location, $s_i\in\mathbb{R}^3$ for size, $r_i\in\mathbb{R}$ for orientation, and $M=\{B_k\}$ 0 for a one-hot class label (Fang et al., 2023). The representation explicitly includes walls, doors, and windows, not only furniture. The “main direction” is defined as the normal of the largest wall, and object rotations are measured as angles between object front directions and that main direction. The code is a flat $M=\{B_k\}$ 1 matrix, so relations are learned implicitly by the diffusion model rather than encoded as explicit graph constraints (Fang et al., 2023).

ControlRoom3D uses a different but equally explicit formulation. Its semantic proxy room is

$M=\{B_k\}$ 2

where $M=\{B_k\}$ 3 is box center, $M=\{B_k\}$ 4 is box size, $M=\{B_k\}$ 5 is semantic class id, and $M=\{B_k\}$ 6 is instance id (Schult et al., 2023). Walls, floor, ceiling, doors, and windows are included as regular boxes, and for planar structural classes the paper sets $M=\{B_k\}$ 7, making them hard geometric constraints during depth alignment (Schult et al., 2023). This representation is less object-internal than SceneCode’s part-wise programs, but more geometrically explicit than text-only descriptions.

Programmatic systems push the representation from declarative layout toward executable construction. Programmable-Room stores intermediate variables such as room corner coordinates $M=\{B_k\}$ 8, panorama layout $M=\{B_k\}$ 9, depth $S$ 0, semantic map $S$ 1, panorama texture $S$ 2, and furniture CSS-like layouts, all produced and transformed by module calls written in a Python-like script (Kim et al., 21 Jun 2025). SceneCode formalizes the contract between room-level planning and object generation with

$S$ 3

where $S$ 4 is object category, $S$ 5 textual description, $S$ 6 target dimensions, $S$ 7 style context, $S$ 8 placement transform, and $S$ 9 support relation (Wang et al., 19 May 2026). It then lifts each request to an ObjectPlan

$m$ 0

with semantic parts, primitive types, local poses, materials, symmetry tags, and movability flags (Wang et al., 19 May 2026).

Symbolic and state-machine variants are equally explicit, but emphasize logic rather than geometry. ESCape the ClassRoom models an escape experience as a room graph $m$ 1 and global state

$m$ 2

with room location, puzzle states, lock states, inventory, and remaining time (O'Connor, 2024). The Room formalizes its environment as a POMDP whose observation is an RDF-like event tuple and query pair, and whose action is a location answer (Kim et al., 2022). RoomPilot’s IDSL is hierarchical, spanning building, room, and object levels with geometry, relation graphs, DOFs, and interaction attributes (Chen et al., 12 Dec 2025). Taken together, these systems show that Code-as-Room can mean flat tensors, sets of boxes, graph states, typed memory entries, or full executable programs.

3. Generation, compilation, and execution pipelines

A defining property of Code-as-Room systems is that generation proceeds through explicit intermediate stages. Ctrl-Room separates layout modeling from appearance modeling. Its Layout Generation Stage trains a text-conditional diffusion model over the scene code, using a 1D U-Net denoiser $m$ 3 with self-attention and cross-attention, and its Appearance Generation Stage uses a fine-tuned ControlNet-Segmentation model to synthesize a panoramic RGB image conditioned on the projected semantic layout and text prompt (Fang et al., 2023). The layout diffusion objective is

$m$ 4

and the sampled code is instantiated as oriented bounding boxes before panorama synthesis and mesh reconstruction (Fang et al., 2023).

ControlRoom3D also uses a multi-stage pipeline, but starts from a user-defined 3D semantic proxy room. It rasterizes semantic, instance, and near/far depth maps with PyTorch3D, generates a panorama by splitting it into eight perspective tiles with 90° FOV and 45° rotational offsets, enforces cross-tile consistency with Correspondence-Aware Attention, predicts metric depth with ZoeDepth, aligns depth with the proxy room through a geometry alignment loss, reconstructs a watertight mesh via Poisson surface reconstruction, and completes missing regions by iterative view synthesis and integration (Schult et al., 2023). The key geometric penalty is the near/far depth consistency term $m$ 5, complemented by a surface-normal term $m$ 6 (Schult et al., 2023).

Programmable-Room makes the room-generation pipeline itself explicit as code. GPT-4 writes a Python-like program over modules such as GenShape, GenLayout, GenDepth, GenSemantic, GenTexture, GenEmptyRoom, GenFurniture, and Merge (Kim et al., 21 Jun 2025). Its panorama generator, PRIG, conditions jointly on text and three visual prompts—layout, depth, and semantic map—using multi-scale feature injection and feature denormalization, and augments the latent diffusion loss with a BiLSTM 1D layout loss,

$m$ 7

to improve structural fidelity (Kim et al., 21 Jun 2025).

The 2026 "Code-as-Room" framework generalizes this staged idea to top-down-image-conditioned synthesis. It defines a five-phase, ten-stage pipeline: spatial semantic analysis, object-centric scene graph construction, layout code generation with visual feedback, layout augmentation for walls and salient minors, layout-grounded object description, object geometry replacement, material assignment, texture assignment, and lighting/render setup with deterministic post-hoc corrections (Yang et al., 18 May 2026). A structured execution harness enforces termination, cross-stage memory maintains typed artifacts, and a critique–sanitize–revise loop with $m$ 8 stabilizes layout generation (Yang et al., 18 May 2026).

SceneCode and RoomPilot shift from image-conditioned generation toward compilation. SceneCode uses a room-level planner–designer–critic loop to infer a structured house layout, emit AssetRequests, route each request to one of five code-generation strategies plus a ThinCover template, synthesize part-wise Blender Python programs, validate them through execution-guided repair and refine loops, compile outputs into SDF, and register all artifacts in a persistent scene-state registry (Wang et al., 19 May 2026). RoomPilot parses text or CAD into IDSL, then refines the state with a dual-channel energy,

$m$ 9

combined with progressive rule activation and operator scheduling, before instantiating walls, openings, retrieved assets, or category-specific procedural generators (Chen et al., 12 Dec 2025). This suggests a continuum from scene code as a conditioning interface to scene code as an executable, optimizer-facing specification.

4. Editing, interaction, and state persistence

Code-as-Room is closely tied to editability because room attributes are exposed as addressable fields. In Ctrl-Room, object-level edits correspond directly to field updates in $n$ 0: move updates $n$ 1, resize updates $n$ 2, rotate updates $n$ 3, and replace updates $n$ 4 with optional changes to size and orientation (Fang et al., 2023). The modified code is reprojected to a semantic panorama and propagated through a mask-guided editing module. The paper defines source, target, and inpainting masks, then combines inpainting with a DIFT-based feature consistency loss,

$n$ 5

to preserve object identity during moved or resized edits without edit-specific retraining (Fang et al., 2023).

Programmable-Room exposes editing at the module level. Because the LLM-generated program binds intermediate variables, an edit such as changing room width or replacing a furniture item triggers only the affected modules: geometry edits recompute shape, layout, depth, and empty room; texture edits re-run panorama synthesis; furniture edits rewrite the CSS-like layout and reload the relevant assets (Kim et al., 21 Jun 2025). SceneCode offers analogous locality through executable object programs and registry versioning: changing drawer count, hinge side, or joint limits modifies only the corresponding program and metadata, after which only that object’s artifacts are recompiled (Wang et al., 19 May 2026). RoomPilot similarly permits edits by changing IDSL fields, rule weights, counts, styles, or specific placements, followed by partial re-optimization (Chen et al., 12 Dec 2025).

In ESCape the ClassRoom, editability and interaction are centered on room logic rather than geometry. Rooms, puzzles, doors, triggers, narrative panels, and timers are authored in HTML as A-Frame entities and Web Components, then orchestrated by JavaScript state machines and DOM events such as game-state-event and game-state-updated (O'Connor, 2024). Unlocking a door corresponds to a state transition together with GLTF visibility changes and dynamic navmesh updates. Because the global state machine is stored on the a-scene, state persists across rooms and any entity can react to updates (O'Connor, 2024).

Symbolic interaction appears again in The Room and Code Park. The Room supports hybrid intelligence because humans and machines can read and write the same RDF-like symbolic structures in episodic and semantic memory stores, and multi-agent collaboration combines the most recent episodic answer or strongest semantic answer across agents (Kim et al., 2022). Code Park makes code itself explorable as room interiors: users move between a bird’s-eye overview and first-person room mode, click syntax-highlighted wall content, invoke go-to definition through animated transitions, and rely on room placement and class-name roofs for wayfinding (Khaloo et al., 2017). In all of these systems, the room is not merely displayed; it is a mutable state surface.

5. Evaluation regimes and reported empirical properties

The evaluation literature is heterogeneous because different Code-as-Room systems target different outputs: panoramas, meshes, executable scripts, physically interactive assets, WebXR experiences, or code-understanding workflows. Ctrl-Room reports panorama FID, CLIP Score, Inception Score, and time per $n$ 6 panorama, as well as mesh-based render metrics and a user study on Perceptual Quality and 3D Structure Completeness (Fang et al., 2023). On Structured3D, Ctrl-Room achieves panorama FID 21.02, CLIP Score 21.58, Inception Score 3.429, and time 61.1 s, compared with Text2Light at FID 56.22 and 81.56 s and MVDiffusion at FID 34.76 and 208.5 s; in the mesh user study it reports PQ 3.89 and 3DS 3.746, both the best among the compared methods (Fang et al., 2023).

ControlRoom3D emphasizes global plausibility, structure completeness, and perceptual quality. Its averaged metrics include CLIP Score 28.4, Inception Score 5.75, 3D structure completeness 4.19, layout plausibility 4.54, and perceptual quality 4.07, outperforming the MVDiffusion-like and Text2Room baselines in layout plausibility and user-study metrics (Schult et al., 2023). Its ablations attribute improvements to the proxy room, geometry alignment, panorama generation, and mesh completion, with the full system reaching PA 4.76, 3DS 4.22, and PQ 4.23 in user study scores (Schult et al., 2023).

Programmable-Room evaluates both image generation and mesh quality. For panorama generation, PR with BiLSTM reports FID 65.68 and KID 0.02354, improving over PR without BiLSTM at FID 84.89 and KID 0.03811, and over PanFusion at FID 72.66 and KID 0.03363 (Kim et al., 21 Jun 2025). For 3D mesh generation, PR reports PQ 3.57, 3DS 3.82, and 154.61 s, compared with Text2Room at PQ 2.68, 3DS 2.39, and 5179.08 s, Holodeck at PQ 2.52 and 180.00 s, and SceneScape at PQ 2.18 and 9300.00 s (Kim et al., 21 Jun 2025).

The 2026 top-down-image Code-as-Room benchmark focuses on visual understanding, spatial reasoning, code generation, and scene quality. Gemini3-Flash with CaR reaches Object Recall 58.9%, Functional Accuracy 88.42%, SelfOverlap 2.57%, Layout IoU 72.0%, Rotation Accuracy 93.5%, Support Accuracy 93.5%, Agent Completion 100%, and Blender Execution Rate 100%; Gemini3.1-pro with CaR reaches Layout IoU 73.2%, Spatial Relation 79.8%, Agent Completion 100%, and Execution Rate 95.5% (Yang et al., 18 May 2026). The memory ablation reports that removing cross-stage memory reduces Object Recall from 55.5% to 48.2%, Layout IoU from 73.2% to 58.0%, and Rotation Accuracy from 93.6% to 88.4% (Yang et al., 18 May 2026).

SceneCode and RoomPilot evaluate controllability and physical usability more directly. SceneCode reports CNT 79.4%, ATR 74.0%, NAV 100.0%, COL 11.3%, and OOB 0.4% at scene level, while object-level metrics show PBR 0.6066, MAT 1.5738, NME 0.0000, faces 6013.64, vertices 4945.07, and UV islands 22.03, all reported against an image-to-3D baseline (Wang et al., 19 May 2026). RoomPilot reports Layout Fidelity 0.58, CSRcount 0.89, CSRrel 0.90 for the full model, zero out-of-boundary count, zero collisions across categories, and user-study scores of Visual Quality 4.10, Layout Reasonableness 4.31, and Physical Plausibility 4.42 (Chen et al., 12 Dec 2025).

The non-3D variants use different evidence. ESCape the ClassRoom does not report frame-rate or load-time metrics and does not present a new user study, but documents WebXR delivery tested on Meta Quest 2 and 3 and emphasizes installation-free deployment (O'Connor, 2024). The Room compares cumulative reward and success rate for handcrafted memory policies, and reports that two agents with 16 memories each outperform one agent with 32 memories, with no statistical tests reported for the multi-agent figure (Kim et al., 2022). Code Park provides two user studies: in the first, it reports that participants rated it easier to get familiar with than Visual Studio and significantly better for becoming familiar with the codebase structure, while Visual Studio remained faster for several targeted tasks; in forced-choice results, fun was 0 versus 28 in favor of Code Park, and preference for learning a codebase was 7 versus 21 in favor of Code Park (Khaloo et al., 2017).

6. Limitations, misconceptions, and research directions

A common misconception is that Code-as-Room implies hard-coded constraints and guaranteed physical correctness. Several systems explicitly reject that interpretation. Ctrl-Room states that explicit analytic constraints such as collision and no-overlap are not hard-coded, and that consistency is learned from Structured3D priors and further supported by loop-consistent panoramic sampling (Fang et al., 2023). ControlRoom3D likewise notes that its published method achieves plausibility through proxy-conditioned diffusion and depth alignment rather than an explicit layout loss (Schult et al., 2023). Programmable-Room states that furniture placement relies on LLM heuristics and that no explicit collision detection, snapping, or physics support is described (Kim et al., 21 Jun 2025). By contrast, RoomPilot makes structural and semantic constraints explicit through energy terms and typed relations, and SceneCode compiles articulated assets into SDF with collision proxies and inertial properties (Chen et al., 12 Dec 2025, Wang et al., 19 May 2026). The literature therefore spans implicit-prior systems and explicit-rule systems.

A second misconception is that code-based rooms are necessarily photoreal or necessarily efficient. SceneCode reports an average wall-clock time of 7:25:36 per evaluation run and average token cost of $21.73, with roughly 62% devoted to object construction and 38% to agent planning (Wang et al., 19 May 2026). The 2026 image-conditioned Code-as-Room framework is currently optimized for top-down images and notes difficulty with arbitrary-view inputs, highly detailed small objects, and layout drift under excessive feedback iterations (Yang et al., 18 May 2026). SceneCode notes that primitive-based programs can look less photoreal than retrieved assets (Wang et al., 19 May 2026). ESCape the ClassRoom identifies accessibility as a current issue needing work and does not yet implement multiplayer or teacher observers (O'Connor, 2024). The Room notes brittleness to non-commonsense placements and future needs around trust and conflict resolution in human collaboration (Kim et al., 2022). Code Park notes limited language support, modest codebase scales, and the latency cost of 3D traversal (Khaloo et al., 2017).

The reported future directions are correspondingly diverse. Ctrl-Room’s scene code is presented as amenable to future rule-based validators and a scene DSL with operations such as add(o), remove(i), move(i,\Delta l), and replace(i,c') (Fang et al., 2023). ESCape the ClassRoom proposes multiplayer, adaptive puzzles, formative assessment, and continued accessibility work (O'Connor, 2024). Programmable-Room points toward multi-room hierarchical programs, stronger geometric constraints, explicit opening geometry, collision-aware furniture optimization, seam-aware texturing, and richer export and editing APIs (Kim et al., 21 Jun 2025). SceneCode identifies neural texturing, parallel object generation, richer friction and damping models, and expansion beyond indoor environments (Wang et al., 19 May 2026). RoomPilot proposes richer behavior schemas, learned constraint solvers, real-time IDSL editing, and physics-aware multi-agent interaction (Chen et al., 12 Dec 2025). A plausible implication is that the paradigm is converging toward hybrid systems in which declarative scene code, executable generation, and explicit optimization coexist rather than compete.

In that sense, Code-as-Room is less a single method than a representational commitment: room structure and room behavior are surfaced as code-level objects, relations, and transformations. Whether the concrete substrate is a tensor, a set of boxes, a JSON DSL, an HTML scene graph, a symbolic POMDP state, or a collection of Blender programs, the research program consistently treats explicit representation as the basis for controllability, editability, reproducibility, and downstream execution (Fang et al., 2023, Schult et al., 2023, Kim et al., 21 Jun 2025, Yang et al., 18 May 2026, Wang et al., 19 May 2026, Chen et al., 12 Dec 2025).