TOL: Textual Localization with OpenStreetMap

Abstract: Natural language provides an intuitive way to express spatial intent in geospatial applications. While existing localization methods often rely on dense point cloud maps or high-resolution imagery, OpenStreetMap (OSM) offers a compact and freely available map representation that encodes rich semantic and structural information, making it well suited for large-scale localization. However, text-to-OSM (T2O) localization remains largely unexplored. In this paper, we formulate the T2O global localization task, which aims to estimate accurate 2 degree-of-freedom (DoF) positions in urban environments from textual scene descriptions without relying on geometric observations or GNSS-based initial location. To support the proposed task, we introduce TOL, a large-scale benchmark spanning multiple continents and diverse urban environments. TOL contains approximately 121K textual queries paired with OSM map tiles and covers about 316 km of road trajectories across Boston, Karlsruhe, and Singapore. We further propose TOLoc, a coarse-to-fine localization framework that explicitly models the semantics of surrounding objects and their directional information. In the coarse stage, direction-aware features are extracted from both textual descriptions and OSM tiles to construct global descriptors, which are used to retrieve candidate locations for the query. In the fine stage, the query text and top-1 retrieved tile are jointly processed, where a dedicated alignment module fuses textual descriptor and local map features to regress the 2-DoF pose. Experimental results demonstrate that TOLoc achieves strong localization performance, outperforming the best existing method by 6.53%, 9.93%, and 8.31% at 5m, 10m, and 25m thresholds, respectively, and shows strong generalization to unseen environments. Dataset, code and models will be publicly available at: https://github.com/WHU-USI3DV/TOL.

• The paper introduces a two-stage TOLoc framework combining contrastive place recognition and fine pose regression from textual queries.
• It leverages automated OSM tile generation and a 121K-text query benchmark to ensure scalability across diverse urban settings.
• Quantitative results show significant meter-level accuracy improvements over state-of-the-art methods with robust cross-city generalization.

Textual Localization with OpenStreetMap: TOL and the TOLoc Framework

Introduction

The paper "TOL: Textual Localization with OpenStreetMap" (2604.01644) confronts the problem of estimating accurate 2-degree-of-freedom (DoF) positions in urban environments from natural language descriptions, leveraging only OpenStreetMap (OSM) data. This setting, text-to-OSM (T2O) localization, requires alignment between semantic cues in free-form text and topological and semantic map representations, but without reliance on images, point clouds, or GNSS priors. The approach emphasizes scalability and lightweight deployment by capitalizing on the ubiquity and richness of OSM. To enable systematic study, the authors introduce TOL, a multi-city benchmark with 121K text queries linked to OSM tiles, and propose TOLoc, a two-stage, direction-aware localization pipeline combining contrastive place recognition and fine pose regression.

Figure 1: (a) T2O localization retrieves OSM tiles based on language and regresses positions; (b) comparison with point cloud and prior OSM retrieval methods, showing the proposed approach enables finer meter-level localization.

TOL Benchmark: Design and Data Generation

The TOL benchmark consists of urban scenes drawn from Singapore, Boston, and Karlsruhe, aggregating approximately 316 km of trajectories. Map tiles are constructed from OSM records centered on vehicle positions from nuScenes and KITTI-360 datasets. Tiles are rasterized into three channels (node, way, area), encoding a broad taxonomy of urban objects selected for structural and semantic diversity.

Figure 2: Spatial coverage of TOL—Boston, Singapore, and Karlsruhe sequences for broad geographic diversity.

Figure 3: Visual example of a rasterized OSM tile incorporating node, way, and area channelization.

Textual queries are generated using an automated map parsing pipeline: for each candidate position, the pipeline samples objects visible along cardinal directions (considering building-based occlusion) and describes their semantics and relative positions via templated sentences. This process is entirely independent of manual labeling or LLM dependency, supporting large-scale scalable data generation.

The TOLoc Framework: Coarse-to-Fine Localization

TOLoc implements T2O localization as a two-stage problem: first, coarse-grained retrieval (place recognition, PR) narrows the search to top-$K$ most semantically likely map tiles; second, a fine-grained pose estimation (PE) module predicts the positional offset within the highest-ranked tile.

Figure 4: TOLoc pipeline. The query text and OSM database are encoded. The PR stage uses direction-aware contrastive descriptors. The PE module employs cross-modal attention to localize within the retrieved tile using the TOA module.

In the PR stage, text and map tiles are encoded separately using visual-textual backbones (CLIP or SigLIP). The text encoder splits the multi-sentence description into directional components, each independently embedded and concatenated to yield a direction-aware descriptor. Map features are similarly regionally pooled according to relative direction. A trainable MLP projects these descriptors into a relational space. Retrieval is then via cosine similarity.

For pose refinement, the top candidate tile's patch-level feature map is fused with the text descriptor using the Text-to-OSM Alignment (TOA) module, which stacks self-attention over patch features and cross-attention between the fused map and text features. The output is regressed to a 2-DoF metric offset within the retrieved tile.

Experimental Results

Quantitative Performance

TOLoc consistently outperforms both state-of-the-art T2O place recognition methods (GOTPR and CVG-Text/CT2Loc) and strong ablation baselines in both recall and strict-meter threshold settings. On TOLoc-N, TOLoc (SigLIP-384 backbone) achieves 28.10% success within 25m, a 9.87% absolute improvement over CT2Loc. The gain at 10m and 5m thresholds remains substantial, indicating not just coarse but metric-level localization improvements. These results are robust across unseen test cities (TOLoc-K360), underscoring generalization.

Figure 5: Recall@25m for increasing top-$K$ retrieval; TOLoc dominates all baselines across all $K$ and both test and validation splits.

Figure 6: Qualitative examples—text queries (left) and corresponding ground-truth/predicted positions, demonstrating accurate tile retrieval and fine-grained pose estimation; errors are reported for retrieval and localization steps.

Ablations and Analysis

• Textual offset: Increased offset between text query and tile center (“text offset”) reduces recall, highlighting the importance of query-map spatial consistency.
• Fusion order: Permuting input sentence order for text fusion yields negligible performance differences, supporting order-invariant fusion robustness.
• Runtime: TOLoc achieves >14 FPS for the encoding step, with inference dominated by descriptor encoding rather than the PE module. The model is thus suitable for real-time applications.

Failure Modes

Figure 7: Place recognition failure—ambiguous or structurally redundant queries yield multiple plausible tiles, or rare semantic layouts create hard negatives.

Figure 8: Pose estimation failure—ambiguous or sparse local scene structure leads to uncertainty even in the correct retrieved tile; large parks and absence of distinctive objects cause spatial under-specification.

Implications and Future Directions

This study formalizes T2O global localization and offers a scalable, open benchmark and a competitive baseline model. The direction-aware cross-modal fusion strategy is empirically validated to surpass graph-matching and text aggregation methods, both in scene coverage and positional accuracy, while remaining computationally tractable. The theoretical contribution is significant: it shows that robust urban localization is possible solely from language and OSM data, contrasting previous reliance on dense geometry or high-resolution imagery.

Practically, the TOLoc pipeline provides a template for retrieval-plus-regression models in text-driven map interaction—relevant to robotics, emergency response, and HMI contexts where imagery or geometry may be unavailable. The strong zero-shot generalization across cities and continents suggests the method leverages structural regularities from OSM and compositional language effectively.

Looking forward, limitations remain for scenes with ambiguous or underspecified descriptions, and directions for future work include conversational/sequential query modeling, uncertainty-aware or active query strategies, and extension to multimodal setups (e.g., integrating weak image cues or user-provided pseudo-geometric hints).

Conclusion

"TOL: Textual Localization with OpenStreetMap" (2604.01644) establishes both a large-scale, automated benchmark and a direction-aware neural architecture for language-driven urban localization. TOLoc demonstrates strong metric accuracy and cross-city generalization, outperforming contemporaneous approaches that focus on coarse retrieval or graph-level scene matching. The foundational result is that accurate position estimation in city-scale OSM is achievable from compact natural language queries, thereby opening new research and application domains for rapid, accessible, and language-based geospatial understanding.