Learning-Based Hierarchical Scene Graph Matching for Robot Localization Leveraging Prior Maps

Abstract: Accurate localization is a fundamental requirement for autonomous robots operating in indoor environments. Scene graphs encode the spatial structure of an environment as a hierarchy of semantic entities and their relationships, and can be constructed both online from robot sensor data and offline from architectural priors such as Building Information Models (BIM). Matching these two complementary representations enables drift correction in SLAM by grounding robot observations against a known structural prior. However, establishing reliable node-to-node correspondences between them remains an open challenge: existing combinatorial methods are prohibitively expensive at scale, and prior learned approaches address only flat graph matching, ignoring the multi-level semantic structure present in both representations. Here we present a learned, end-to-end differentiable pipeline that augments both graphs with semantically motivated edge types encoding intra- and inter- level relationships, explicitly exploiting this hierarchy to enable simultaneous matching from high-level room concepts down to low-level wall surfaces. Trained exclusively on floor plans, the proposed method outperforms the combinatorial baseline in F1 on real LiDAR environments while running an order of magnitude faster, demonstrating viable zero-shot generalization for BIM-assisted robot localization.

• The paper presents a novel end-to-end differentiable pipeline that leverages GNN-based hierarchical scene graph matching to align noisy sensor data with BIM priors.
• It employs a shared MLP encoder, context-aware GATv2 layers, and a Sinkhorn-based assignment to achieve robust, zero-shot localization with significant speed improvements on synthetic data.
• The approach excels in recall and efficiency on real-world LiDAR data, suggesting its applicability for SLAM backends and semantic mapping in complex indoor environments.

Learning-Based Hierarchical Scene Graph Matching for Robot Localization Leveraging Prior Maps

Introduction and Motivation

Autonomous mobile robots require accurate localization capabilities to support robust operation in indoor environments. Building Information Models (BIMs) provide detailed, structured architectural priors that encode both geometric and semantic aspects of real-world buildings. Exploiting these priors to refine robot localization is an established strategy, but the challenge lies in matching online-constructed scene graphs—generated from partial, noisy sensor data—with their BIM-derived architectural counterparts. The node-matching problem is nontrivial due to the heterogeneous, hierarchical structure of both representations, encompassing both high-level concepts (rooms) and low-level geometric entities (wall surfaces), and the incomplete exploration characteristic of real robotic deployments.

This paper introduces an end-to-end, differentiable pipeline that explicitly models the heterogeneous, two-level hierarchy of both scene and architectural graphs. The proposed approach leverages graph neural encoders, context-aware attention mechanisms, and a differentiable bipartite assignment based on Sinkhorn normalization, providing computation-efficient, scalable, and semantically informed scene graph matching. Targeted applications include online drift correction for SLAM, semantic map alignment, and BIM-assisted robot localization.

Hierarchical Scene Graph Representations and Feature Encoding

Both the architectural (A-graph) and situational (S-graph) representations are structured as two-level hierarchies. Nodes are of two types: rooms and wall surfaces (WS). Nodes are described by a seven-dimensional feature vector that encodes node type, 2D centroid coordinates, surface normal (for WS), and segment length (for WS), with feature homogenization achieved through a shared two-layer MLP. This MLP-based encoder transforms heterogeneous input features into a unified embedding space, allowing for cross-type comparison and message passing.

The paper augments the classical single-edge-type graph structure to incorporate:

• Room-to-WS Edges: Capture containment relations,
• Room-to-Room Edges: Encode spatial adjacency via shared walls,
• WS-to-WS Edges: Represent angular sequences of wall segments within the same room,

enabling contextual information propagation across both hierarchical levels during graph message passing. This edge augmentation is critical for leveraging the multi-level semantics of complex indoor environments.

Figure 1: The proposed pipeline uses a shared MLP for node feature homogenization, a shared GATv2 encoder for structure-aware node embeddings, and a Sinkhorn-normalized affinity for robust correspondences, supporting partial observations via dummy-padding.

Structure-Aware Embedding via Graph Attention

The node representations are further enriched by a shared GATv2 encoder. Leveraging attention-based message passing, GATv2 computes context-sensitive node embeddings that adaptively weight contributions from heterogeneous neighbors (e.g., a wall surface attending differently to its parent room and adjoining segments). This architecture is explicitly designed to address the fundamental heterogeneity of architectural scene graphs. The encoder applies multi-head attention and dropout for regularization and is parameterized to produce 32-dimensional embeddings through two GATv2 layers.

Differentiable Cross-Graph Assignment

Node matching across the A-graph and S-graph is posed as a rectangular bipartite assignment problem (given the partial nature of robot exploration). Similarity is computed as a dot-product affinity matrix between node embeddings, with instance normalization ensuring conditioning across variable graph sizes.

To handle the partial observation regime ($N_2 \leq N_1$), dummy columns are appended to square the affinity matrix prior to application of the Sinkhorn algorithm. This continuous relaxation yields a doubly stochastic soft assignment, accommodating nodes in the A-graph that have no observed counterparts in the S-graph. Matching is finalized using the Hungarian algorithm, extracting an optimal hard one-to-one correspondence.

Experimental Evaluation

The pipeline is systematically evaluated on both synthetic (MSD dataset) and real-world (LiDAR-acquired) environments. The synthetic evaluation employs perturbed graph pairs to simulate realistic noise and partiality, testing resilience and generalization to degradations encountered in practice.

Figure 2: Evaluation graph examples—synthetic floor plan (left) and a real-world indoor LiDAR environment with its corresponding A-graph (right).

Quantitative Results:

• On synthetic data, the method delivers an F1 score of 85%, competitive with the combinatorial iS-Graph baseline (95%), but with deterministic completion (100% of samples) and an 82-fold reduction in computation time (0.093 s vs. 7.66 s/sample).
• On real data, the proposed method achieves higher F1 (84% vs. 67%), due mainly to a substantial improvement in recall (88% vs. 53%), and a 9-fold lower inference time (0.021 s/sample). Notably, the model was trained exclusively on synthetic data, demonstrating robust zero-shot transfer.

The recall-focused improvement highlights the GNN-based matcher’s flexibility under semantically ambiguous or noisy conditions, outperforming strict combinatorial approaches that tend toward higher precision but at the expense of recall.

Implications, Limitations, and Future Directions

The presented pipeline provides a framework for real-time, semantic-aware robot localization by exploiting architectural priors via hierarchically structured scene graph matching. In practical terms, the significant reduction in matching time, coupled with competitive or superior assignment quality—especially under out-of-domain generalization—makes this approach suitable for deployment in SLAM back-ends, as well as semantic mapping and autonomous exploration tasks.

On the theoretical front, the method demonstrates that graph neural architectures, when designed to exploit hierarchical heterogeneity and equipped with permutation-invariant differentiable assignment, can scale semantic matching to realistic, large-scale indoor environments.

One limitation identified by the authors is the method’s current lack of explicit structural symmetry handling, which can result in ambiguous correspondences in highly symmetric environments. Future work may embed symmetry detection and awareness into the model architecture, enhancing robustness to repeated room configurations and further improving assignment fidelity. Improvements in feature representation, context propagation, and matching precision/recall are also anticipated.

Conclusion

This paper advances the state of hierarchical, heterogeneous graph matching for indoor robot localization through a learned, GNN-based pipeline that is both computation-efficient and semantically informed. Its capacity for accurate, zero-shot scene graph matching against BIM priors opens the path for broader adoption in real-world, resource-constrained robotics platforms, and motivates continued research on symmetry resolution and hierarchical generalization in scene graph representations.