---
title: 'Code-as-Room: A Framework for Room Representation'
url: https://www.emergentmind.com/topics/code-as-room
type: topic
---

# Code-as-Room: A Framework for Room Representation

Searching arXiv for the cited works on Code-as-Room and closely related room-as-code representations.
Code-as-Room is a research paradigm in which a room, indoor environment, or room-like artifact is represented as an explicit programmatic state rather than only as a rendered image or static mesh. Across the literature, this state takes multiple forms: a holistic scene code for indoor layout synthesis, a semantic proxy room of bounding boxes, a Python-like visual program, HTML and Web Components for WebXR escape rooms, executable Blender programs for articulated assets, RDF-like symbolic tuples for a memory environment, a JSON-based indoor DSL, and, in software visualization, a room metaphor in which source code is spatialized as enterable rooms [2310.03602] [2312.05208] [2506.17707] [2605.18451] [2605.19587] [2408.08273] [2204.01611] [2512.11234] [1708.02174]. The unifying idea is that room structure, object identity, relations, constraints, and edit operations become explicit and inspectable, enabling controllable generation, localized modification, execution, and evaluation.

## 1. Scope and historical breadth

The term does not denote a single canonical representation. In "Ctrl-Room" [2310.03602], a room is a tensor of per-object attributes used for text-to-3D generation and editing. In "ControlRoom3D" [2312.05208], a room is a semantic proxy room composed of semantic 3D bounding boxes plus a style prompt. In "Programmable-Room" [2506.17707], a room is specified by an LLM-written Python-like visual program. In "Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis" [2605.18451], it is executable Blender Python code synthesized from a top-down image under a structured execution harness. In "SceneCode" [2605.19587], it becomes an executable world program with articulated objects and simulator-ready SDF export. In "RoomPilot" [2512.11234], the shared representation is IDSL, an Indoor Domain-Specific Language. Outside 3D scene synthesis, "ESCape the ClassRoom" [2408.08273] uses web code to define rooms, puzzles, and state transitions, "A Machine With Human-Like Memory Systems" [2204.01611] encodes room events and queries as RDF-like tuples, and "Code Park" [1708.02174] maps each software class to a navigable room.

| Work | Room-as-code representation | Domain |
|---|---|---|
| Ctrl-Room [2310.03602] | Holistic scene code $x_0 \in \mathbb{R}^{N\times D}$ | Text-to-3D room meshes |
| ControlRoom3D [2312.05208] | Semantic proxy room $M=\{B_k\}$ | Controlled room mesh generation |
| ESCape the ClassRoom [2408.08273] | HTML, A-Frame entities, Web Components, JS state machines | WebXR educational escape rooms |
| Programmable-Room [2506.17707] | Python-like visual program over generation/editing modules | Interactive textured room generation |
| RoomPilot [2512.11234] | JSON-based IDSL | Multimodal indoor scene synthesis |
| SceneCode [2605.19587] | Executable Blender Python programs plus registry | Editable articulated indoor worlds |
| The Room [2204.01611] | RDF-like symbolic tuples and memory state | Gym-compatible memory environment |
| Code Park [1708.02174] | One class → one room with syntax-aware wallpaper | 3D code visualization |

This breadth matters because it separates Code-as-Room from narrower notions such as procedural layout generation or text-to-image prompting. Some systems treat code as the room’s latent state, some as the room’s executable construction procedure, and some as the interaction logic governing what can happen inside the room [2310.03602] [2408.08273] [2605.19587]. A plausible implication is that Code-as-Room is best understood as a family of explicit scene representations rather than a single file format or algorithm.

## 2. Representational forms and formal structure

A central line of work encodes a room as an object-wise structured state. Ctrl-Room defines a room scene $S$ with $m$ walls and $n$ furniture items as a set of $N=m+n$ object rows,
$$
x_0=\{o_i\}_{i=1}^N,\qquad o_i=[c_i,l_i,s_i,r_i],
$$
with $l_i\in\mathbb{R}^3$ for center location, $s_i\in\mathbb{R}^3$ for size, $r_i\in\mathbb{R}$ for orientation, and $c_i\in\mathbb{R}^C$ for a one-hot class label [2310.03602]. The representation explicitly includes walls, doors, and windows, not only furniture. The “main direction” is defined as the normal of the largest wall, and object rotations are measured as angles between object front directions and that main direction. The code is a flat $N\times D$ matrix, so relations are learned implicitly by the diffusion model rather than encoded as explicit graph constraints [2310.03602].

ControlRoom3D uses a different but equally explicit formulation. Its semantic proxy room is
$$
M=\{B_k\},\qquad B_k=(p,s,c,i),
$$
where $p\in\mathbb{R}^3$ is box center, $s\in\mathbb{R}^3$ is box size, $c$ is semantic class id, and $i$ is instance id [2312.05208]. Walls, floor, ceiling, doors, and windows are included as regular boxes, and for planar structural classes the paper sets $D_n=D_f$, making them hard geometric constraints during depth alignment [2312.05208]. This representation is less object-internal than SceneCode’s part-wise programs, but more geometrically explicit than text-only descriptions.

Programmatic systems push the representation from declarative layout toward executable construction. Programmable-Room stores intermediate variables such as room corner coordinates $S$, panorama layout $L$, depth $D$, semantic map $M$, panorama texture $G$, and furniture CSS-like layouts, all produced and transformed by module calls written in a Python-like script [2506.17707]. SceneCode formalizes the contract between room-level planning and object generation with
$$
A=(c,\tau,d,\sigma,T,\rho),
$$
where $c$ is object category, $\tau$ textual description, $d\in\mathbb{R}^3$ target dimensions, $\sigma$ style context, $T\in SE(3)$ placement transform, and $\rho$ support relation [2605.19587]. It then lifts each request to an ObjectPlan
$$
P=\{(p_i,s_i,T_i^{loc},m_i,\pi_i,\mu_i)\}_{i=1}^{N},
$$
with semantic parts, primitive types, local poses, materials, symmetry tags, and movability flags [2605.19587].

Symbolic and state-machine variants are equally explicit, but emphasize logic rather than geometry. ESCape the ClassRoom models an escape experience as a room graph $G=(V,E)$ and global state
$$
S_t=\langle loc_t, status_t^P, status_t^L, inv_t, \tau_t\rangle,
$$
with room location, puzzle states, lock states, inventory, and remaining time [2408.08273]. The Room formalizes its environment as a POMDP whose observation is an RDF-like event tuple and query pair, and whose action is a location answer [2204.01611]. RoomPilot’s IDSL is hierarchical, spanning building, room, and object levels with geometry, relation graphs, DOFs, and interaction attributes [2512.11234]. Taken together, these systems show that Code-as-Room can mean flat tensors, sets of boxes, graph states, typed memory entries, or full executable programs.

## 3. Generation, compilation, and execution pipelines

A defining property of Code-as-Room systems is that generation proceeds through explicit intermediate stages. Ctrl-Room separates layout modeling from appearance modeling. Its Layout Generation Stage trains a text-conditional diffusion model over the scene code, using a 1D U-Net denoiser $\epsilon_\theta(x_t,t,y)$ with self-attention and cross-attention, and its Appearance Generation Stage uses a fine-tuned ControlNet-Segmentation model to synthesize a panoramic RGB image conditioned on the projected semantic layout and text prompt [2310.03602]. The layout diffusion objective is
$$
L=\mathbb{E}_{x_0,t,y,\epsilon}\big[\|\epsilon-\epsilon_\theta(x_t,t,y)\|^2\big],
$$
and the sampled code is instantiated as oriented bounding boxes before panorama synthesis and mesh reconstruction [2310.03602].

ControlRoom3D also uses a multi-stage pipeline, but starts from a user-defined 3D semantic proxy room. It rasterizes semantic, instance, and near/far depth maps with PyTorch3D, generates a panorama by splitting it into eight perspective tiles with 90° FOV and 45° rotational offsets, enforces cross-tile consistency with Correspondence-Aware Attention, predicts metric depth with ZoeDepth, aligns depth with the proxy room through a geometry alignment loss, reconstructs a watertight mesh via Poisson surface reconstruction, and completes missing regions by iterative view synthesis and integration [2312.05208]. The key geometric penalty is the near/far depth consistency term $L_d$, complemented by a surface-normal term $L_n$ [2312.05208].

Programmable-Room makes the room-generation pipeline itself explicit as code. GPT-4 writes a Python-like program over modules such as GenShape, GenLayout, GenDepth, GenSemantic, GenTexture, GenEmptyRoom, GenFurniture, and Merge [2506.17707]. Its panorama generator, PRIG, conditions jointly on text and three visual prompts—layout, depth, and semantic map—using multi-scale feature injection and feature denormalization, and augments the latent diffusion loss with a BiLSTM 1D layout loss,
$$
\mathcal{L}=\lambda_{latent}\mathcal{L}_{latent}+\lambda_{BiLSTM}\mathcal{L}_{BiLSTM},
$$
to improve structural fidelity [2506.17707].

The 2026 "Code-as-Room" framework generalizes this staged idea to top-down-image-conditioned synthesis. It defines a five-phase, ten-stage pipeline: spatial semantic analysis, object-centric scene graph construction, layout code generation with visual feedback, layout augmentation for walls and salient minors, layout-grounded object description, object geometry replacement, material assignment, texture assignment, and lighting/render setup with deterministic post-hoc corrections [2605.18451]. A structured execution harness enforces termination, cross-stage memory maintains typed artifacts, and a critique–sanitize–revise loop with $T_{max}=5$ stabilizes layout generation [2605.18451].

SceneCode and RoomPilot shift from image-conditioned generation toward compilation. SceneCode uses a room-level planner–designer–critic loop to infer a structured house layout, emit AssetRequests, route each request to one of five code-generation strategies plus a ThinCover template, synthesize part-wise Blender Python programs, validate them through execution-guided repair and refine loops, compile outputs into SDF, and register all artifacts in a persistent scene-state registry [2605.19587]. RoomPilot parses text or CAD into IDSL, then refines the state with a dual-channel energy,
$$
E(\mathcal{S},t)=\alpha_{struct}(t)\mathcal{E}_{struct}(\mathcal{S})+\alpha_{sem}(t)\mathcal{E}_{sem}(\mathcal{S}),
$$
combined with progressive rule activation and operator scheduling, before instantiating walls, openings, retrieved assets, or category-specific procedural generators [2512.11234]. This suggests a continuum from scene code as a conditioning interface to scene code as an executable, optimizer-facing specification.

## 4. Editing, interaction, and state persistence

Code-as-Room is closely tied to editability because room attributes are exposed as addressable fields. In Ctrl-Room, object-level edits correspond directly to field updates in $o_i=[c_i,l_i,s_i,r_i]$: move updates $l_i$, resize updates $s_i$, rotate updates $r_i$, and replace updates $c_i$ with optional changes to size and orientation [2310.03602]. The modified code is reprojected to a semantic panorama and propagated through a mask-guided editing module. The paper defines source, target, and inpainting masks, then combines inpainting with a DIFT-based feature consistency loss,
$$
L_{obj}=-\cos\big(sg(F_t^{ori}\odot m_{src}),F_t^{edit}\odot m_{tar}\big),
$$
to preserve object identity during moved or resized edits without edit-specific retraining [2310.03602].

Programmable-Room exposes editing at the module level. Because the LLM-generated program binds intermediate variables, an edit such as changing room width or replacing a furniture item triggers only the affected modules: geometry edits recompute shape, layout, depth, and empty room; texture edits re-run panorama synthesis; furniture edits rewrite the CSS-like layout and reload the relevant assets [2506.17707]. SceneCode offers analogous locality through executable object programs and registry versioning: changing drawer count, hinge side, or joint limits modifies only the corresponding program and metadata, after which only that object’s artifacts are recompiled [2605.19587]. RoomPilot similarly permits edits by changing IDSL fields, rule weights, counts, styles, or specific placements, followed by partial re-optimization [2512.11234].

In ESCape the ClassRoom, editability and interaction are centered on room logic rather than geometry. Rooms, puzzles, doors, triggers, narrative panels, and timers are authored in HTML as A-Frame entities and Web Components, then orchestrated by JavaScript state machines and DOM events such as `game-state-event` and `game-state-updated` [2408.08273]. Unlocking a door corresponds to a state transition together with GLTF visibility changes and dynamic navmesh updates. Because the global state machine is stored on the `a-scene`, state persists across rooms and any entity can react to updates [2408.08273].

Symbolic interaction appears again in The Room and Code Park. The Room supports hybrid intelligence because humans and machines can read and write the same RDF-like symbolic structures in episodic and semantic memory stores, and multi-agent collaboration combines the most recent episodic answer or strongest semantic answer across agents [2204.01611]. Code Park makes code itself explorable as room interiors: users move between a bird’s-eye overview and first-person room mode, click syntax-highlighted wall content, invoke go-to definition through animated transitions, and rely on room placement and class-name roofs for wayfinding [1708.02174]. In all of these systems, the room is not merely displayed; it is a mutable state surface.

## 5. Evaluation regimes and reported empirical properties

The evaluation literature is heterogeneous because different Code-as-Room systems target different outputs: panoramas, meshes, executable scripts, physically interactive assets, WebXR experiences, or code-understanding workflows. Ctrl-Room reports panorama FID, CLIP Score, Inception Score, and time per $512\times 1024$ panorama, as well as mesh-based render metrics and a user study on Perceptual Quality and 3D Structure Completeness [2310.03602]. On Structured3D, Ctrl-Room achieves panorama FID 21.02, CLIP Score 21.58, Inception Score 3.429, and time 61.1 s, compared with Text2Light at FID 56.22 and 81.56 s and MVDiffusion at FID 34.76 and 208.5 s; in the mesh user study it reports PQ 3.89 and 3DS 3.746, both the best among the compared methods [2310.03602].

ControlRoom3D emphasizes global plausibility, structure completeness, and perceptual quality. Its averaged metrics include CLIP Score 28.4, Inception Score 5.75, 3D structure completeness 4.19, layout plausibility 4.54, and perceptual quality 4.07, outperforming the MVDiffusion-like and Text2Room baselines in layout plausibility and user-study metrics [2312.05208]. Its ablations attribute improvements to the proxy room, geometry alignment, panorama generation, and mesh completion, with the full system reaching PA 4.76, 3DS 4.22, and PQ 4.23 in user study scores [2312.05208].

Programmable-Room evaluates both image generation and mesh quality. For panorama generation, PR with BiLSTM reports FID 65.68 and KID 0.02354, improving over PR without BiLSTM at FID 84.89 and KID 0.03811, and over PanFusion at FID 72.66 and KID 0.03363 [2506.17707]. For 3D mesh generation, PR reports PQ 3.57, 3DS 3.82, and 154.61 s, compared with Text2Room at PQ 2.68, 3DS 2.39, and 5179.08 s, Holodeck at PQ 2.52 and 180.00 s, and SceneScape at PQ 2.18 and 9300.00 s [2506.17707].

The 2026 top-down-image Code-as-Room benchmark focuses on visual understanding, spatial reasoning, code generation, and scene quality. Gemini3-Flash with CaR reaches Object Recall 58.9%, Functional Accuracy 88.42%, SelfOverlap 2.57%, Layout IoU 72.0%, Rotation Accuracy 93.5%, Support Accuracy 93.5%, Agent Completion 100%, and Blender Execution Rate 100%; Gemini3.1-pro with CaR reaches Layout IoU 73.2%, Spatial Relation 79.8%, Agent Completion 100%, and Execution Rate 95.5% [2605.18451]. The memory ablation reports that removing cross-stage memory reduces Object Recall from 55.5% to 48.2%, Layout IoU from 73.2% to 58.0%, and Rotation Accuracy from 93.6% to 88.4% [2605.18451].

SceneCode and RoomPilot evaluate controllability and physical usability more directly. SceneCode reports CNT 79.4%, ATR 74.0%, NAV 100.0%, COL 11.3%, and OOB 0.4% at scene level, while object-level metrics show PBR 0.6066, MAT 1.5738, NME 0.0000, faces 6013.64, vertices 4945.07, and UV islands 22.03, all reported against an image-to-3D baseline [2605.19587]. RoomPilot reports Layout Fidelity 0.58, CSRcount 0.89, CSRrel 0.90 for the full model, zero out-of-boundary count, zero collisions across categories, and user-study scores of Visual Quality 4.10, Layout Reasonableness 4.31, and Physical Plausibility 4.42 [2512.11234].

The non-3D variants use different evidence. ESCape the ClassRoom does not report frame-rate or load-time metrics and does not present a new user study, but documents WebXR delivery tested on Meta Quest 2 and 3 and emphasizes installation-free deployment [2408.08273]. The Room compares cumulative reward and success rate for handcrafted memory policies, and reports that two agents with 16 memories each outperform one agent with 32 memories, with no statistical tests reported for the multi-agent figure [2204.01611]. Code Park provides two user studies: in the first, it reports that participants rated it easier to get familiar with than Visual Studio and significantly better for becoming familiar with the codebase structure, while Visual Studio remained faster for several targeted tasks; in forced-choice results, fun was 0 versus 28 in favor of Code Park, and preference for learning a codebase was 7 versus 21 in favor of Code Park [1708.02174].

## 6. Limitations, misconceptions, and research directions

A common misconception is that Code-as-Room implies hard-coded constraints and guaranteed physical correctness. Several systems explicitly reject that interpretation. Ctrl-Room states that explicit analytic constraints such as collision and no-overlap are not hard-coded, and that consistency is learned from Structured3D priors and further supported by loop-consistent panoramic sampling [2310.03602]. ControlRoom3D likewise notes that its published method achieves plausibility through proxy-conditioned diffusion and depth alignment rather than an explicit layout loss [2312.05208]. Programmable-Room states that furniture placement relies on LLM heuristics and that no explicit collision detection, snapping, or physics support is described [2506.17707]. By contrast, RoomPilot makes structural and semantic constraints explicit through energy terms and typed relations, and SceneCode compiles articulated assets into SDF with collision proxies and inertial properties [2512.11234] [2605.19587]. The literature therefore spans implicit-prior systems and explicit-rule systems.

A second misconception is that code-based rooms are necessarily photoreal or necessarily efficient. SceneCode reports an average wall-clock time of 7:25:36 per evaluation run and average token cost of $21.73, with roughly 62% devoted to object construction and 38% to agent planning [2605.19587]. The 2026 image-conditioned Code-as-Room framework is currently optimized for top-down images and notes difficulty with arbitrary-view inputs, highly detailed small objects, and layout drift under excessive feedback iterations [2605.18451]. SceneCode notes that primitive-based programs can look less photoreal than retrieved assets [2605.19587]. ESCape the ClassRoom identifies accessibility as a current issue needing work and does not yet implement multiplayer or teacher observers [2408.08273]. The Room notes brittleness to non-commonsense placements and future needs around trust and conflict resolution in human collaboration [2204.01611]. Code Park notes limited language support, modest codebase scales, and the latency cost of 3D traversal [1708.02174].

The reported future directions are correspondingly diverse. Ctrl-Room’s scene code is presented as amenable to future rule-based validators and a scene DSL with operations such as `add(o)`, `remove(i)`, `move(i,\Delta l)`, and `replace(i,c')` [2310.03602]. ESCape the ClassRoom proposes multiplayer, adaptive puzzles, formative assessment, and continued accessibility work [2408.08273]. Programmable-Room points toward multi-room hierarchical programs, stronger geometric constraints, explicit opening geometry, collision-aware furniture optimization, seam-aware texturing, and richer export and editing APIs [2506.17707]. SceneCode identifies neural texturing, parallel object generation, richer friction and damping models, and expansion beyond indoor environments [2605.19587]. RoomPilot proposes richer behavior schemas, learned constraint solvers, real-time IDSL editing, and physics-aware multi-agent interaction [2512.11234]. A plausible implication is that the paradigm is converging toward hybrid systems in which declarative scene code, executable generation, and explicit optimization coexist rather than compete.

In that sense, Code-as-Room is less a single method than a representational commitment: room structure and room behavior are surfaced as code-level objects, relations, and transformations. Whether the concrete substrate is a tensor, a set of boxes, a JSON DSL, an HTML scene graph, a symbolic POMDP state, or a collection of Blender programs, the research program consistently treats explicit representation as the basis for controllability, editability, reproducibility, and downstream execution [2310.03602] [2312.05208] [2506.17707] [2605.18451] [2605.19587] [2512.11234].

Source: https://www.emergentmind.com/topics/code-as-room