Large-Scale Architectural Models

• LAMs are models that operate on building or urban-scale data to produce structured, semantic, and geometry-aware representations across various layouts and spatial configurations.
• They integrate multimodal architectures with explicit geometric primitives and textual interfaces to process and interpret complex architectural scenes.
• LAMs enable practical applications such as indoor reconstruction, urban scene generation, CAD analysis, and visual question answering through extensive pretraining on large, diverse datasets.

Large-scale Architectural Models (LAMs), in the built-environment sense, denote models that operate on building-scale or urban-scale data, produce structured, semantic, geometry-aware representations, and generalize across layouts, objects, and spatial configurations. Current exemplars span room-scale indoor modeling from point clouds, steerable urban scene generation from compositional layout priors, large-scale CAD understanding, quantitative architectural style analysis, multimodal architectural visual question answering, and daylight-driven generative design (Mao et al., 9 Jun 2025, Lu et al., 2024, Luo et al., 28 Mar 2025, Zhong et al., 9 Jun 2025, Wang et al., 25 Sep 2025, Li et al., 2024). The acronym is also overloaded in adjacent literatures, where “LAM” can refer to Large Artificial Intelligence Models, Large Action Models, or Large Atomic Models; in architectural research, the intended meaning is therefore determined by modality, representation, and task formulation (Wang et al., 1 May 2025, Georgievski et al., 24 Jul 2025, Zhang et al., 2 Jun 2025).

1. Scope, definition, and conceptual boundaries

In the architectural literature represented here, LAMs are not a single standardized model family but a convergent design space. One explicit formulation defines them as models that operate on building-scale or multi-room architectural data, produce structured, semantic, geometry-aware representations, and generalize across many layouts and object configurations (Mao et al., 9 Jun 2025). A complementary urban-scale formulation describes them as models that capture the geometry, semantics, and organization of extensive built environments—entire city districts, road networks, and blocks—in a way that is scalable and controllable (Lu et al., 2024).

This yields two dominant scales of operation. The first is room-scale or indoor architectural understanding, where a model reconstructs walls, doors, windows, and furnishing layouts from scans. The second is city-block or corridor-scale urban modeling, where a model uses semantic primitives such as buildings, roads, sidewalks, vehicles, and vegetation to generate or edit large 3D scenes (Mao et al., 9 Jun 2025, Lu et al., 2024). A plausible implication is that present-day LAMs are best understood as a family of geometry-aware multimodal systems rather than a single canonical architecture.

The term also intersects with broader “foundation model” discourse. Several papers explicitly frame their systems as foundation-style or foundation-level models for architectural, urban, or CAD tasks, relying on large pretraining corpora, standard multimodal backbones, and transfer to downstream benchmarks (Mao et al., 9 Jun 2025, Luo et al., 28 Mar 2025, Wang et al., 25 Sep 2025). At the same time, the architectural sense of LAM should be distinguished from unrelated uses of the acronym in wireless communications, agentic service composition, and atomistic modeling (Wang et al., 1 May 2025, Georgievski et al., 24 Jul 2025, Zhang et al., 2 Jun 2025).

2. Representations, primitives, and multimodal architectures

A recurring pattern across current exemplars is the use of explicit architectural primitives as the target representation. In room-scale indoor modeling, the input is a point cloud

$P \in \mathbb{R}^{N \times 6},$

with points $\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$, and the output is a structured scene description containing architectural layout elements and object instances. SpatialLM represents layout elements as planar segments and objects as oriented 3D bounding boxes,

$$o_k = \big(x_k, y_k, z_k, h_k, w_k, l_k, \theta_k, c_k \big),$$

with center, size, yaw, and semantic category (Mao et al., 9 Jun 2025). The same work expresses the full scene as Python code, making the representation text-native and human-editable.

Urban-scale generation adopts a different but related abstraction. Urban Architect uses a compositional 3D layout prior made of semantic primitives with simple geometric structure: buildings and vehicles as cuboids, vegetation as ellipsoids, and roads or sidewalks as planes. For a camera pose $\mathbf{T}$, the layout is projected to semantic and depth maps, and the diffusion target distribution becomes conditioned on both text and layout,

$$p_0(\mathbf{x}_0 \mid y, \mathcal{F}(L(\mathbf{T}))).$$

This makes layout the structural prior and text the appearance controller (Lu et al., 2024).

In CAD understanding, the representation is vector-native rather than volumetric. ArchCAD-400K uses line-grained primitive annotations, where each primitive $k$ has a semantic category $l_k$ and an instance identifier $z_k$,

$(l_k, z_k).$

DPSS then fuses primitive features and raster features with a geometry-guided gate,

$$w = \sigma \big( W_1 \tanh( W_2 X_p + W_3 V_p ) \big), \qquad U_p = \mathrm{concat}(X_p, w V_p),$$

so that vector geometry anchors the multimodal representation (Luo et al., 28 Mar 2025).

Vision–language architectural analysis follows the same explicit alignment principle. ArchiLense maps architectural images and style descriptions into a joint embedding space and measures their match with cosine similarity,

$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$0

then ranks descriptions by their discriminative power across architect groups (Zhong et al., 9 Jun 2025). This suggests that current LAMs are less defined by one universal backbone than by a common insistence on explicit geometric, semantic, and linguistic interfaces.

3. Room-scale indoor LAMs

SpatialLM is the clearest room-scale instantiation of a LAM for indoor scenes. It processes 3D indoor point clouds and outputs structured architectural descriptions containing walls, doors, windows, and oriented 3D object boxes with semantic categories (Mao et al., 9 Jun 2025). Its target task is “structured indoor modeling”: given a raw RGBD-derived point cloud, reconstruct a compact, structured representation of the environment.

The model follows a standard multimodal pattern: a 3D encoder, a 2-layer MLP projector, and an open-source LLM backbone. The best-performing encoder is Sonata, a variant of PTv3, while the language backbone is Qwen2.5-0.5B. The model converts dense 3D input into a compact set of visual tokens,

$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$1

and then autoregressively generates Python scene scripts (Mao et al., 9 Jun 2025). An important engineering result is that single-stage end-to-end fine-tuning of encoder, projector, and LLM outperforms staged freezing schedules.

Its training corpus is unusually large by architectural standards: 12,328 indoor scenes, 54,778 rooms, 403,291 walls, 123,301 doors, 48,887 windows, total floor area of about 863,986 m², and 412,932 object instances spanning 59 categories (Mao et al., 9 Jun 2025). The data are synthetic but human-authored, derived from a large repository of real interior designs and rendered into dense RGBD point clouds with precise 3D annotations. This scale is central to the paper’s argument that LLM-based architectural models fail when trained only on small real datasets and become viable when pretrained on a large, realistic architectural corpus.

Evaluation is reported as F1-score at IoU thresholds 0.25 and 0.5 because autoregressive models do not produce confidence scores. On Structured3D layout estimation, SpatialLM pretrained on the SpatialLM dataset and fine-tuned on Structured3D reaches 86.5 at IoU$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$[email protected] and 84.6 at IoU$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$[email protected], exceeding RoomFormer and SceneScript. On ScanNet 3D object detection, the same pretrain–fine-tune strategy reaches 65.6 at IoU$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$[email protected] and 52.6 at IoU$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$[email protected], essentially matching V-DETR at the looser threshold and remaining competitive at the stricter one (Mao et al., 9 Jun 2025). Zero-shot experiments on 107 virtual indoor tour reconstructions further show consistent layouts and major-object detection without task-specific adaptation.

Architecturally, SpatialLM is notable because it does not emit a proprietary CAD or BIM file; it emits Python scripts describing room-scale architecture and furnishing layout. This directly supports floorplan extraction, scene editing, navigation maps, and BIM pre-processing as downstream uses. It does not model multi-floor building structure or room adjacency graphs explicitly, but its per-room outputs are already architectural primitives in a strong sense (Mao et al., 9 Jun 2025).

4. Urban-scale generative LAMs

Urban Architect provides the most explicit city-scale generative realization of a LAM. Its premise is that text alone is too ambiguous for urban-scale 3D generation, because urban scenes contain many interacting components, intricate arrangement relationships, and unbounded spatial extent (Lu et al., 2024). The proposed remedy is a compositional 3D layout prior that introduces semantic primitives and arrangement structure into the generation loop.

The core methodological contribution is Layout-Guided Variational Score Distillation. A differentiable renderer produces semantic and depth maps from the 3D layout for arbitrary camera poses; these are injected into a ControlNet, and both the diffusion model and LoRA network are conditioned on the resulting features. The gradient becomes

$\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$6

This tightens the target distribution by coupling text with semantic and geometric layout constraints (Lu et al., 2024).

To make generation scalable, the paper introduces a Scalable Hash Grid that decomposes the scene into stuff grids for static background and object grids for movable instances. Layout-constrained sampling ensures that rays allocate capacity to architecturally relevant space rather than empty regions. The resulting system reaches scenes covering about $\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$7 and driving distances over 1000 m, with 32 GB GPU memory for the largest experiments (Lu et al., 2024).

Quantitatively, on KITTI-360 the method reports FID 59.8 and KID 0.059 on 5000 rendered images, outperforming CC3D and other baselines. Large-scale results show geometry and appearance consistency over long trajectories, and editing demonstrations show instance-level changes such as adding or removing buildings and moving vehicles, as well as style-level changes such as “Night time,” “Foggy,” or “Vangogh paint” while preserving layout (Lu et al., 2024). The framework is therefore not only generative but explicitly steerable at the level of urban massing and scene organization.

From the perspective of LAMs, Urban Architect formalizes a key distinction between structural control and appearance control. Layout acts as the architectural prior, while text governs stylistic and atmospheric variation. This suggests a general recipe for urban-scale LAMs: compositional semantic geometry, differentiable rendering into 2D control signals, scalable neural scene representation, and explicit editing interfaces (Lu et al., 2024).

5. CAD, style, and multimodal architectural understanding

A complementary strand of LAM research addresses architectural drawings, style, and image-based interpretation rather than point clouds or generative fields. ArchCAD-400K is the largest CAD-centered example in the data provided. It contains 413,062 chunks extracted from 5,538 highly standardized drawings, with line-grained panoptic annotations over 27 classes and a total annotated area of $\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$8 m² (Luo et al., 28 Mar 2025). Chunks correspond to roughly $\mathbf{p}_i=(x_i,y_i,z_i,r_i,g_i,b_i)$9 physical patches, allowing large full-floor drawings to be tiled into manageable training units without discarding vector semantics. DPSS, the proposed baseline, uses HRNetW48 for raster context, PointTransformerV2 for primitive geometry, and a transformer decoder for panoptic symbol spotting.

This CAD line of work matters for LAMs because it provides large-scale supervision for primitive-level architectural semantics: walls, doors, columns, axis lines, stairs, furniture, and other symbolic entities. The paper argues that such data can serve as core pretraining material for CAD encoders in larger architectural systems, and that panoptic symbol spotting is a base capability for downstream tasks such as code checking, quantity takeoffs, BIM enrichment, and construction planning (Luo et al., 28 Mar 2025).

Image-based style understanding is represented by ArchiLense. Its dataset, ArchDiffBench, contains 1,765 high-quality architectural images from different regions and historical periods, organized into 10 architect-based groups and 81 paired subsets for comparative style analysis (Zhong et al., 9 Jun 2025). The framework combines GPT-4V, BLIP-2, CLIP ViT-G/14, LLaVA-1.5, and Vicuna-1.5 in a two-stage pipeline that first extracts style descriptions and then evaluates them quantitatively. Reported results include a 92.4% consistency rate with expert annotations and 84.5% classification accuracy, indicating that VLLM-based pipelines can move architectural style analysis beyond purely subjective reading (Zhong et al., 9 Jun 2025).

Multimodal architectural VQA is pushed further by ArchGPT. Its data-construction pipeline yields Arch-300K, approximately 315,000 image–question–answer triplets derived from 8,643 architectural scenes using Wikimedia Commons, 3D-reconstruction-based image filtering, semantic segmentation, and LLM-guided text verification (Wang et al., 25 Sep 2025). The model fine-tunes ShareGPT4V-7B with LoRA and multi-annotation fusion, then supports detailed descriptions and aspect-guided conversations about style, elements, materials, symbolism, and context. On its domain test set, ArchGPT reaches JudgeLM 7.713 and architecture-focused JudgeLM 7.107, surpassing ShareGPT4V-7B, LLaVA-1.5-7B, and InternVL3-8B, while largely preserving general multimodal performance (Wang et al., 25 Sep 2025).

Taken together, these systems extend the idea of a LAM beyond 3D reconstruction. They show that large-scale architectural modeling also includes vector-symbolic understanding, quantitative style discrimination, and image-grounded question answering, all of which are necessary if architectural models are to function as research, documentation, and design infrastructures rather than geometry-only predictors.

6. Applications, limitations, and open directions

The application envelope described across these papers is broad. SpatialLM explicitly targets augmented reality, embodied robotics, autonomous navigation and mapping, digital twins, and architectural design tools (Mao et al., 9 Jun 2025). Urban Architect emphasizes driving simulators, VR walk-throughs, and urban design exploration through instance-level and style-level editing (Lu et al., 2024). ArchCAD-400K connects CAD understanding to automated code checking, cost estimation, BIM enrichment, and construction sequencing (Luo et al., 28 Mar 2025). ArchGPT places architectural VQA in VR, MR, and AR systems for guided exploration and heritage interpretation (Wang et al., 25 Sep 2025). A design-oriented variant appears in daylight-driven generative architecture, where GPT-4 prompt generation, Stable Diffusion v1.5, ControlNet, and a daylighting LoRA model are assembled into a pipeline from parametric massing to facade/window design and final rendering (Li et al., 2024).

The limitations are correspondingly diverse and specific. SpatialLM is indoor-only, room-scale, and closed-vocabulary; it does not model multi-floor structure, structural systems, or MEP semantics, and it still benefits from target-domain fine-tuning (Mao et al., 9 Jun 2025). Urban Architect offers strong structural control but not pixel-level scene control, and its primitives remain coarse relative to full architectural detail or engineering feasibility (Lu et al., 2024). ArchCAD-400K is 2D, standardized, and regionally biased toward drawings from Chinese and Asian design institutes (Luo et al., 28 Mar 2025). ArchiLense is visually oriented, relatively small in scale, and biased toward modern Western architectural styles (Zhong et al., 9 Jun 2025). ArchGPT is limited by Wikimedia-style scene bias, dependence on what the teacher LLM “knows,” and the absence of explicit 3D or BIM input (Wang et al., 25 Sep 2025). The daylight-driven diffusion pipeline relies on a small daylight dataset, manual filtering of stochastic outputs, and only one performance objective—daylight—rather than multi-objective architectural analysis (Li et al., 2024).

Open directions are unusually consistent across papers. SpatialLM points toward further scaling of data and model size, open-vocabulary 3D detection, 3D VQA, multi-floor and multi-building modeling, BIM semantics, and outdoor–indoor integration (Mao et al., 9 Jun 2025). Urban Architect motivates richer layout semantics, integration with simulation, physical constraints, and co-design interfaces for planners and architects (Lu et al., 2024). CAD-centered work points toward linking 2D symbol understanding with 3D BIM, site plans, temporal project phases, and multimodal corpora that include text and sensor data (Luo et al., 28 Mar 2025). Style- and VQA-oriented work points toward larger, more geographically balanced datasets and spatially aware architectures that move beyond facade imagery into plans, sections, and reconstructed 3D environments (Zhong et al., 9 Jun 2025, Wang et al., 25 Sep 2025). The daylight-driven design pipeline explicitly calls for additional physical factors and broader typological validation (Li et al., 2024).

A plausible synthesis is that present LAMs are converging on a layered architectural stack: explicit geometric or symbolic primitives, large multimodal backbones, structured textual interfaces, and domain-specific corpora large enough to support transfer. What remains incomplete is the full integration of room-scale geometry, urban layout, CAD/BIM semantics, style knowledge, physical simulation, and interactive reasoning into one coherent architectural foundation model.