Semantic Walls: Multimodal and Lexical Insights

• Semantic Walls are multifaceted research constructs that reinterpret wall concepts using contextual semantics to resolve ambiguities in language, vision, structure, and urban analysis.
• They integrate methods from lexical selection, multimodal facade segmentation, and indoor wireframe detection to enhance accuracy in categorization and reconstruction.
• Applications range from precise translation distinctions and improved imaging under distortions to mapping non-physical urban barriers linking cultural, economic, and social segregation.

Semantic walls designate several distinct but technically related research objects in which the notion of a wall is interpreted through semantics rather than through geometry alone. In recent work, this includes fine-grained lexical distinctions such as English wall versus Spanish pared and muro (Chaudhary et al., 2021), multimodal segmentation of building facade walls and windows from images and natural-language instructions (Li et al., 9 Jun 2025), semantic wireframe detection in indoor scenes where wall boundaries are labeled edges in a graph (Gillsjö et al., 2022), and “invisible walls” in cities, defined as non-physical barriers rooted in culture, economics, social norms, and perceived inclusivity (Fan et al., 17 Feb 2025). Taken together, these lines of work show that wall semantics can refer to lexical selection, visual parsing, structural scene representation, or socially experienced segregation.

1. Principal research meanings

The phrase spans multiple research settings rather than a single formalism. In each case, semantics supplies information that pure surface form, texture, or geometry does not fully determine.

Research setting	Wall concept	Semantic signal
Lexical selection	English wall	contextual cues governing pared vs muro
Facade segmentation	building facade walls and windows	text descriptions fused with image features
Room wireframe detection	labeled line segments	edge labels such as wall, floor, ceiling, door, window
Urban segregation	invisible walls	review language about inclusivity, price, culture, and atmosphere

In the lexical-selection setting, the problem is that languages carve up semantic space differently: English uses a single noun, while Spanish splits the concept into pared and muro, and Greek likewise subdivides wall into different lexical items (Chaudhary et al., 2021). In architectural computer vision, the problem is to identify wall and window regions or wall edges under severe perspective distortion, stylization, low texture, or occlusion, and semantics is injected through textual prompts or semantic labels (Li et al., 9 Jun 2025, Gillsjö et al., 2022). In urban computing, semantic walls are explicitly defined as non-physical barriers inferred from language about places rather than from literal infrastructure (Fan et al., 17 Feb 2025).

A plausible unifying interpretation is that semantic walls arise when the category wall is underdetermined by raw observation and must be resolved by higher-level context: linguistic context in translation, cross-modal context in segmentation, scene semantics in wireframe recovery, or social meaning in urban experience.

2. Lexical selection: when “wall” is pared and when it is muro

Chaudhary et al. formulate the lexical problem from a parallel corpus

$D=\{(x^{(m)},y^{(m)})\}_{m=1}^{|D|},$

with word alignments

$A_m=\langle x_i,y_j\rangle,$

lemmatization, and POS tagging (Chaudhary et al., 2021). For each English focus word type $v_x$ with POS tag $t_x$, the target translations $\text{trans}(v_x,t_x)$ are collected, and ambiguity is filtered by the entropy of the translation-choice distribution:

$$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$

Only pairs with entropy above a threshold, such as $0.69$ for binary splits, are retained. The pipeline also filters out types aligned to fewer than two distinct translations appearing $\ge 50$ times and removes cases where target choices correspond to different WordNet senses.

Lexical selection is then cast as linear classification. For each focus word, the model builds contextual feature vectors $\phi(\cdot)\in\mathbb{R}^d$ using lemma features, word-sense tags via WSD, bigrams of lemmas in a fixed window, and dependency-head and dependent features. One-vs-rest LinearSVMs are trained for each target choice $v_{y_k}$, and prediction is

$A_m=\langle x_i,y_j\rangle,$0

The same feature weights are used to extract concise rules. Raw features are mapped into templates such as “Words,” “Short phrases,” and “Concepts,” yielding both human-readable and machine-readable JSON/YAML outputs.

For wall.N, the extracted rules make the distinction operational. Muro is associated with short phrases such as “climb wall,” “city wall,” and “brick wall,” words such as break and climb, and concepts such as city.n.01. Pared is associated with “face wall,” “hang wall,” and “picture wall,” words such as hang, paint, and room, and concepts such as gallery.n.01 (Chaudhary et al., 2021). This supports the claim that the distinction is conditioned by indoor/outdoor context signals, material, verbs, and semantic fields rather than by an arbitrary synonym split.

Automated evaluation places the interpretable linear model close to a stronger contextual baseline. The most-frequent-translation baseline achieves approximately $A_m=\langle x_i,y_j\rangle,$1 accuracy; Decision Tree reaches $A_m=\langle x_i,y_j\rangle,$2 for Spanish and $A_m=\langle x_i,y_j\rangle,$3 for Greek; LinearSVM reaches $A_m=\langle x_i,y_j\rangle,$4 for Spanish and $A_m=\langle x_i,y_j\rangle,$5 for Greek; and a BERT-based skyline reaches $A_m=\langle x_i,y_j\rangle,$6 for Spanish and $A_m=\langle x_i,y_j\rangle,$7 for Greek (Chaudhary et al., 2021). In a human learner study, participants with rules reached $A_m=\langle x_i,y_j\rangle,$8 accuracy in approximately $A_m=\langle x_i,y_j\rangle,$9 examples, whereas without rules they needed approximately $v_x$0 examples; significance was reported at $v_x$1 up to $v_x$2 examples for Spanish and across all examples for Greek. A common misconception is that the pared/muro choice is merely a dictionary matter; the extracted rules show that it is a contextual lexical-selection problem.

3. Multimodal wall semantics in facade segmentation

In facade analysis, SAAF—“Segment Any Architectural Facades”—treats wall segmentation as a multimodal semantic guidance problem (Li et al., 9 Jun 2025). The architecture has four blocks: an input module with a facade image $v_x$3 and a natural-language instruction $v_x$4; a frozen vision backbone $v_x$5, specified as ViT-H from SAM, producing dense features $v_x$6; a multimodal LLMF with a special “<SEG>” query token based on LLaVA-7B or LLaVA-13B with LoRA adapters; and a trainable multimodal decoder $v_x$7 that fuses image features and the segmentation embedding to output a full-resolution mask $v_x$8.

The core semantic mechanism is the embedding associated with the special token:

$v_x$9

followed by projection

$t_x$0

The decoder fuses $t_x$1 and $t_x$2 either through cross-modal attention or through concatenation followed by convolutions. The generic fusion operator is written as

$t_x$3

Optional cross-attention uses

$t_x$4

The stated rationale is that text prompts inject high-level architectural priors such as “wall,” “opening,” and “fenestration,” while the “<SEG>” token focuses the model on mask generation rather than pure classification.

Training is end-to-end with a joint text-plus-mask objective

$t_x$5

where $t_x$6 is autoregressive cross-entropy for text and

$t_x$7

The Dice term is

$t_x$8

The reported hyperparameters are $t_x$9, $\text{trans}(v_x,t_x)$0, $\text{trans}(v_x,t_x)$1, and $\text{trans}(v_x,t_x)$2. Training uses a facade dataset of $\text{trans}(v_x,t_x)$3 high-resolution images spanning photos, renderings, CAD drawings, sketches, pen drawings, watercolor, historical documents, and related styles, with a $\text{trans}(v_x,t_x)$4 train/val/test split and de-duplication. Implementation details include dual NVIDIA A6000 GPUs with $\text{trans}(v_x,t_x)$5 GB each, Adam with learning rate $\text{trans}(v_x,t_x)$6, total batch size $\text{trans}(v_x,t_x)$7, and $\text{trans}(v_x,t_x)$8 training steps; all $\text{trans}(v_x,t_x)$9 parameters are frozen, while LoRA adapters, the projection layer $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$0, $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$1, and the LLM’s embed_tokens and Im_head are trainable (Li et al., 9 Jun 2025).

Evaluation uses mean Intersection over Union:

$$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$2

with $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$3 the number of classes, namely walls and windows. On wall segmentation, SAAF reports $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$4 mIoU on pen-drawing, $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$5 on real perspective, $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$6 on renderings, and $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$7 on historical documents, compared with lower FCN, U-Net, and HRNet baselines in each case. The paper states that SAAF consistently outperforms all baselines by $$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$8–$$H(v_x,t_x) = -\sum_{v_y\in\mathrm{trans}(v_x,t_x)} p(v_y\mid v_x,t_x)\,\ln p(v_y\mid v_x,t_x).$$9 points in mIoU across diverse styles, transfers with mIoU $0.69$0 to highly stylized domains when trained on real photos, and maintains $0.69$1 on heterogeneous inputs while baselines drop below $0.69$2 (Li et al., 9 Jun 2025). The qualitative analysis emphasizes sharp, well-aligned wall boundaries under severe perspective distortion and accurate delineation on watercolor and line drawings where single-modal models fail.

4. Semantic wall structure in indoor wireframes

Semantic Room Wireframe Detection extends wall semantics from pixel masks to a labeled planar graph (Gillsjö et al., 2022). The output is

$0.69$3

where $0.69$4 is a set of $0.69$5D junctions and $0.69$6 is a set of undirected edges, each edge carrying a semantic label

$0.69$7

Junctions are typed as either proper, meaning true three-plane intersections that are reported even if occluded by furniture, or false, meaning artifacts due to limited field of view or occlusion. The formulation therefore treats wall boundaries as structural entities with explicit topological and semantic roles.

SRW-Net consists of an hourglass backbone taking $0.69$8 to a feature tensor $0.69$9, a junction proposal head, an AFM-based line-attraction proposal head, proposal matching with threshold $\ge 50$0 px, a semantic-classification head based on LoI-Pool that samples $\ge 50$1 points along each matched line segment, and a separate 4-layer GCN refinement on both the junction graph and the line graph (Gillsjö et al., 2022). The learned mapping is summarized as

$\ge 50$2

This pipeline converts local visual evidence into a connected semantic wireframe suitable for downstream layout recovery.

Training uses a multi-task objective

$\ge 50$3

where $\ge 50$4 combines pixel-wise cross-entropy over $\ge 50$5 with weighted ratio $\ge 50$6 and an $\ge 50$7 offset loss, $\ge 50$8 follows HAWP’s AFM-based regression and classification, and $\ge 50$9 is cross-entropy over $\phi(\cdot)\in\mathbb{R}^d$0 with hard-mining (Gillsjö et al., 2022). The dataset is produced from Structured3D by planarity cleanup, door-open detection based on semantic-mask consistency, and $\phi(\cdot)\in\mathbb{R}^d$1D annotation via back-projection, visibility clipping, and merged semantic labeling.

The key metrics are structural average precision for lines, $\phi(\cdot)\in\mathbb{R}^d$2, and junction AP, $\phi(\cdot)\in\mathbb{R}^d$3. On the test set, SRW-Net reports $\phi(\cdot)\in\mathbb{R}^d$4 for wall, $\phi(\cdot)\in\mathbb{R}^d$5 for window, $\phi(\cdot)\in\mathbb{R}^d$6 for door, $\phi(\cdot)\in\mathbb{R}^d$7 for ceiling, and $\phi(\cdot)\in\mathbb{R}^d$8 for floor, with $\phi(\cdot)\in\mathbb{R}^d$9; for junctions it reports $v_{y_k}$0 for false and $v_{y_k}$1 for proper, with $v_{y_k}$2 (Gillsjö et al., 2022). Compared with non-semantic baselines trained on the same data, SRW-Net achieves $v_{y_k}$3, versus $v_{y_k}$4 for HAWP and $v_{y_k}$5 for L-CNN. The paper argues that the model handles complex non-Manhattan scenes better than previous room layout estimation algorithms and can be used for full $v_{y_k}$6D layout fitting without cuboid assumptions.

A misconception addressed by this line of work is that wall understanding from a single image reduces to room-layout estimation with Manhattan priors. The semantic-wireframe formulation instead represents wall edges, door edges, and window edges jointly and explicitly supports later plane fitting and reconstruction.

5. Invisible walls in cities

In urban computing, invisible walls—also called semantic walls in the exposition—are defined as non-physical barriers rooted in culture, economics, social norms, and perceived inclusivity that systematically limit inter-group interactions even when physical proximity would permit them (Fan et al., 17 Feb 2025). The central empirical claim is that these barriers manifest in the language of user-generated reviews. Mentions of atmosphere, pricing, cultural themes, or inclusivity cues signal whether a Point of Interest is more or less welcoming to particular racial or ethnic groups.

The Reflective LLM Coder operationalizes this idea through two agentic subroutines. The Reflective Attributor first analyzes POI name, reviews, and images to list cultural, economic, or social factors that might attract or repel distinct groups, then revises those factors using real-world feedback from actual visitor proportions $v_{y_k}$7 and local residential proportions $v_{y_k}$8. The Code Summarizer applies a four-step Chain-of-Thought: extraction of atomic codes, semantic combination of redundant codes, quality-based removal of overly specific or non-generalizable codes, and explanation refinement into one-sentence dimension descriptions. The result is a nine-dimension codebook, described as nine orthogonal “bricks” (Fan et al., 17 Feb 2025).

The nine dimensions are Cultural Resonance and Appeal; Price Sensitivity and Economic Accessibility; Service Quality and Customer Experience; Atmosphere and Social Environment; Accessibility and Convenience; Visual and Aesthetic Appeal; Cultural and Social Inclusivity; Product Variety and Quality; and Community Engagement and Local Involvement. For each POI $v_{y_k}$9, demographic group $A_m=\langle x_i,y_j\rangle,$00, and dimension $A_m=\langle x_i,y_j\rangle,$01, the LLM generates a rating

$A_m=\langle x_i,y_j\rangle,$02

where $A_m=\langle x_i,y_j\rangle,$03 denotes strong repulsion, $A_m=\langle x_i,y_j\rangle,$04 neutrality, and $A_m=\langle x_i,y_j\rangle,$05 strong attraction. The $A_m=\langle x_i,y_j\rangle,$06 ratings are vectorized through a small MLP:

$A_m=\langle x_i,y_j\rangle,$07

These ratings form one channel of the RE’EM framework, which combines Reasoning, Embedding, and Population features. The embedding channel uses a fine-tuned GTE-base model over review subsets:

$A_m=\langle x_i,y_j\rangle,$08

The population channel uses the local racial composition vector $A_m=\langle x_i,y_j\rangle,$09 from census block groups within $A_m=\langle x_i,y_j\rangle,$10 km of each POI:

$A_m=\langle x_i,y_j\rangle,$11

A neighbor-aware multi-view predictor constructs three graphs, one per channel, and applies channel-specific GATs before fusion:

$A_m=\langle x_i,y_j\rangle,$12

optimized with MSE

$A_m=\langle x_i,y_j\rangle,$13

Experiments use Yelp Open Dataset reviews, images, and attributes; SafeGraph Patterns for ground-truth $A_m=\langle x_i,y_j\rangle,$14; and ACS demographics, across Philadelphia with $A_m=\langle x_i,y_j\rangle,$15 POIs, Tucson with $A_m=\langle x_i,y_j\rangle,$16, Tampa with $A_m=\langle x_i,y_j\rangle,$17, and New Orleans with $A_m=\langle x_i,y_j\rangle,$18 (Fan et al., 17 Feb 2025). On Philadelphia, RE’EM reports MSE $A_m=\langle x_i,y_j\rangle,$19, RMSE $A_m=\langle x_i,y_j\rangle,$20, MAE $A_m=\langle x_i,y_j\rangle,$21, and $A_m=\langle x_i,y_j\rangle,$22, compared with Population-only at MSE $A_m=\langle x_i,y_j\rangle,$23 and $A_m=\langle x_i,y_j\rangle,$24. The reported gains are a $A_m=\langle x_i,y_j\rangle,$25 elevation in $A_m=\langle x_i,y_j\rangle,$26 and a $A_m=\langle x_i,y_j\rangle,$27 reduction in MSE. Cross-city generalization using the Philadelphia-derived codebook yields $A_m=\langle x_i,y_j\rangle,$28 increases from $A_m=\langle x_i,y_j\rangle,$29 to $A_m=\langle x_i,y_j\rangle,$30 in Tucson, from $A_m=\langle x_i,y_j\rangle,$31 to $A_m=\langle x_i,y_j\rangle,$32 in Tampa, and from $A_m=\langle x_i,y_j\rangle,$33 to $A_m=\langle x_i,y_j\rangle,$34 in New Orleans, with paired $A_m=\langle x_i,y_j\rangle,$35-tests indicating significant MSE reductions at $A_m=\langle x_i,y_j\rangle,$36 in all cases (Fan et al., 17 Feb 2025). A user study with $A_m=\langle x_i,y_j\rangle,$37 domain-expert researchers reports approximately chance-level performance in baseline settings, rising to $A_m=\langle x_i,y_j\rangle,$38–$A_m=\langle x_i,y_j\rangle,$39 accuracy when codebook-guided summaries are shown, and $A_m=\langle x_i,y_j\rangle,$40–$A_m=\langle x_i,y_j\rangle,$41 preference for codebook-guided summaries.

The main misconception here is literalism: invisible walls are not physical walls. They are inferred social barriers whose evidence is linguistic, affective, and demographic.

6. Shared methodology, limitations, and open directions

Across these domains, semantics acts as a disambiguating prior. In the lexical case, entropy filtering identifies words whose translation choice genuinely depends on context, and interpretable LinearSVM weights are turned into rules (Chaudhary et al., 2021). In facade segmentation, textual instructions and the “<SEG>” query token provide high-level architectural priors that help on low-contrast or texture-less regions (Li et al., 9 Jun 2025). In room wireframes, semantic labels constrain line meaning and support full $A_m=\langle x_i,y_j\rangle,$42D layout fitting (Gillsjö et al., 2022). In urban segregation prediction, reflective coding converts reviews into a compact, interpretable codebook that can guide rating and multi-view prediction (Fan et al., 17 Feb 2025). This suggests a recurring pattern: semantic walls are useful precisely where local evidence is ambiguous.

The limitations are domain-specific and substantial. For SAAF, dataset coverage is still sparse for many regional or historical styles; the LLM-plus-vision fusion remains black-box; and current fusion uses only image-text rather than geometry such as point clouds or depth (Li et al., 9 Jun 2025). For SRW-Net, floor-edge recall remains relatively low at $A_m=\langle x_i,y_j\rangle,$43, the GCN refinement is trained separately rather than fully end-to-end, the model is trained purely on synthetic Structured3D, and domain adaptation to real scenes is still needed (Gillsjö et al., 2022). In the lexical-selection pipeline, the work states that the method scales with minimal human intervention and suggests applicability to less-resourced languages with similar semantic subdivisions, but the distinction remains conditioned by corpus-derived features and the set of retained ambiguous words (Chaudhary et al., 2021). In the urban setting, the codebook is generalizable across three cities, yet the framework remains anchored to reviews, images, names, and observed visitation patterns rather than to direct causal identification of segregation mechanisms (Fan et al., 17 Feb 2025).

The open directions are correspondingly varied. SAAF explicitly proposes self-supervised pretraining on unlabeled facades, integration of LiDAR or point-cloud data for true $A_m=\langle x_i,y_j\rangle,$44D facade reconstruction, richer component taxonomies such as doors, balconies, and cornices, panoptic facade parsing, and interactive refinement through follow-up text or GUI clicks (Li et al., 9 Jun 2025). SRW-Net identifies end-to-end graph-aware backbones, extension of the semantic set, and incorporation of multi-view and temporal consistency through SLAM integration as open directions (Gillsjö et al., 2022). The lexical work points toward broader deployment of automatically extracted rules for language learning and less-resourced languages (Chaudhary et al., 2021). The urban work indicates that codebook-guided summaries can produce cognitive gains for human perception of social inclusiveness and that the nine semantic bricks capture broadly applicable segregation mechanisms across cities (Fan et al., 17 Feb 2025).

Taken together, these results establish semantic walls as a cross-domain research theme in which the meaning of a wall—lexical, visual, structural, or social—is made explicit, modeled, and evaluated through semantic representations rather than through geometry or frequency alone.