Vector Map as Language: Toward Unified Remote Sensing Vector Mapping

Abstract: Remote sensing vector mapping aims to generate structured maps of geospatial entities, such as buildings, roads, and water bodies, from remote sensing imagery. In practice, vector maps usually contain multiple category layers and heterogeneous entity structures, requiring a unified model for diverse mapping needs. However, existing methods typically represent vector objects as polygons or graphs, making them suitable only for specific categories: polygons poorly capture topological relations, while graphs often blur instance boundaries. We observe that language, as a natural medium for human communication, offers a flexible and expressive representation that can accommodate heterogeneous map elements, including geometry, semantics, and topolog. Motivated by this insight, we propose Vector Map as Language (VecLang), a unified paradigm that reformulates multiclass vector mapping as structured text generation. VecLang encodes the common elements of different geospatial entities into a GeoJSON-like vector language, enabling cross-category modeling within a shared textual format. To generate this language reliably, we design a progressive vision-language mapping framework that first localizes vectorization units and then generates structured map elements. We further introduce Hierarchical Vector Language Optimization, which uses reinforcement learning to improve syntax validity, content fidelity, and map executability. We also build VecMap-Bench with 54K images and 800K instances, supporting training and evaluation across standard and generalization settings. Extensive experiments demonstrate that VecLang handles both single-class and multiclass vector mapping while achieving strong cross-dataset and open-vocabulary generalization. The model and dataset are publicly available at https://github.com/yyyyll0ss/VecLang.

• The paper introduces VecLang, a language-based paradigm that unifies polygonal and network representations for remote sensing vector mapping.
• It employs a Progressive Vectorization Framework and hierarchical reinforcement learning to enhance mapping accuracy and computational efficiency.
• Empirical results on VecMap-Bench demonstrate improved mAP, IoU, and topological fidelity across multiclass and open-vocabulary mapping tasks.

Vector Map as Language: A Unified Paradigm for Remote Sensing Vector Mapping

Motivation and Challenges in Unified Vector Mapping

Remote sensing vector mapping (RSVM) seeks to extract explicit, structured maps from remotely sensed imagery, encompassing geospatial entities such as buildings, roads, and water bodies. Traditional approaches encode these entities either as polygons or graphs, each with inherent limitations. Polygon-based methods are effective for closed objects but unsuitable for capturing network topologies, while graph-based techniques naturally handle connectivity in roads and networks but lack precise instance boundaries for closed shapes. Consequently, these representational constraints preclude a unified, category-agnostic approach to vector mapping.

The authors observe that language, particularly in structured formats, inherently provides an extensible, hierarchical medium capable of representing heterogeneous information — semantics, geometry, topology — within a shared, parseable framework. This insight drives the introduction of VecLang, a language-based paradigm that formulates multiclass vector map generation as structured text generation, aiming to unify the modeling, generation, and optimization of diverse geospatial entities.

Figure 1: Comparison between existing methods and VecLang, highlighting VecLang's ability to unify closed (polygonal) and network-like (graph) geospatial structures within a language-based representation.

Hierarchical Vector Language: The VecLang Representation

VecLang introduces Structured Vector Language (SVL), a GeoJSON-inspired, GIS-compatible language that encodes geospatial entities as textual sequences. Entities are described as objects consisting of an identifier, semantic class label, geometry (which can be, e.g., polygons, polylines, or multi-lines), and optional topological fields (such as containment or intersection relations). This uniform representation supports both closed (e.g., building) and network-like (e.g., road) objects.

Figure 2: Motivation of VecLang — diverse map entities (buildings, roads, water bodies) share core descriptive elements (semantics, geometry, topology), which can be systematically organized via a hierarchical vector language.

Map annotations — which can be highly heterogeneous in source data — are serialized into SVL sequences that follow a shared grammar. Bidirectional conversion functions enable robust translation between map and language representations: a map-to-language function encodes vector data into SVL, while a language-to-map parser reconstructs executable geometry from SVL text.

Figure 3: Map-Language reversible conversion pipeline enabling top-down map-to-language and bottom-up language-to-map transitions for both polygonal and network-structured entities.

Progressive Vectorization Framework

Dense remote sensing scenes can encompass hundreds of objects, making end-to-end full-image SVL generation computationally demanding and unstable. The Progressive Vectorization Framework (PVF) proposed in VecLang addresses this by first localizing candidate vectorization units (e.g., instance bounding boxes for buildings, or local patches for road segments), then generating map elements as structured SVL text, conditioned on both the localized imagery and category-specific prompts.

This pipeline decomposes the generation problem, reducing output sequence lengths, enhancing robustness, and optimizing resource consumption.

Figure 4: The VecLang pipeline: (a) progressive vectorization (localization then generation), (b) hierarchical vector language optimization, (c) reversible map-language conversion.

Figure 5: PVF mitigates the drawbacks of direct, full-text generation—sequence length, accuracy, and memory—by localizing and segmenting the generation steps.

Hierarchical Vector Language Optimization

Given that outputting well-formed, executable SVL is more demanding than simple text matching, VecLang employs Hierarchical Vector Language Optimization (HVLO) based on reinforcement learning. Built atop the Group Relative Policy Optimization (GRPO) algorithm, SVL generation is rewarded at three hierarchical levels:

• Syntax Validity: Ensuring the output is parseable as JSON, adheres to the expected schema, and provides usable semantic and geometric fields.
• Content Consistency: Matching the structure and semantics (class, geometry type, structural plausibility) to ground truth across classes.
• Execution Fidelity: Direct spatial and topological evaluation through instance- and class-aware IoU, Hausdorff alignment, and topology (connectivity) scores (notably crucial for road graphs).

This RL paradigm bridges the gap between text sequence similarity and downstream map utility, ensuring that small textual or coordinate deviations do not undermine the coherence or executability of the resulting vector maps.

Figure 6: Pure textual accuracy is insufficient—for SVL, small coordinate errors can have large geometric/topological consequences, motivating hierarchical rewards.

Unified Benchmarking: VecMap-Bench

VecMap-Bench, constructed from over 54,000 images and 800,000 instances, provides a comprehensive evaluation platform for: single-class mapping, multiclass mapping, cross-dataset generalization (transfer learning), and open-vocabulary (zero-shot) generalization. All annotations are normalized and converted to SVL, facilitating consistent training and evaluation.

Figure 7: VecMap-Bench spans single-class, multiclass, cross-dataset, and open-vocabulary task regimes, supporting thorough generalization analysis.

Empirical Results and Key Findings

Single-Class Mapping: VecLang demonstrates competitive or superior accuracy across building, water body, and road extraction — achieving 88.96 mAP and 92.01 C-IoU for building polygons, and 75.96 recall for road graphs. Unlike prior work, these results are achieved via a single, category-agnostic model.

Figure 8: Visualizations for building, road, and water-body mapping; VecLang captures both instance geometry and network topology across categories.

Multiclass Mapping: SVL allows VecLang to jointly model closed and open graph structures, yielding higher unified F1 and IoU scores across categories in the multiclass regime.

Figure 9: VecLang consistently outputs accurate, interpretable multiclass vector maps, in contrast to the fragmented or structurally compromised outputs from polygon- or graph-only baselines.

Generalization:

• Cross-Dataset: VecLang maintains substantially higher F1 and IoU under domain shift compared to task-specific models.
• Open-Vocabulary: On previously unseen categories and natural scenes, post-trained VecLang significantly boosts open-set vectorization scores over base VLMs, supporting direct textual prompting.

  Figure 10: Post-training with SVL enables the base VLM to generalize robustly to new categories and domains in structured vector mapping tasks.

Scene Scalability: Progressive localization and generation enable VecLang to efficiently produce high-resolution, full-scene vectorizations without degradation in coherence or map utility.

Figure 11: Output slice from large-scale, high-res scenes; VecLang preserves both fine-grained object boundaries and network continuity at scale.

Ablations:

• Representation: SVL markedly outperforms freeform text and naive coordinate sequences in geometric fidelity and topological accuracy.
• Framework: PVF reduces GPU memory and inference time, while increasing parsing and mapping accuracy.
• Reward Structure: All three reward components are necessary for best performance, synergistically optimizing for well-formed, content-correct, and executable maps.

  Figure 12: Parsing accuracy and reward curves for alternative representations and RL reward combinations, highlighting the necessity of structured SVL and composite rewards.

Discussion and Theoretical Implications

VecLang's structured language approach harmonizes vision-language modeling and geospatial representation, circumventing previous expressivity limitations by integrating semantic, geometric, and topological reasoning within a hierarchical, parseable medium. This paradigm allows for unified learning and inference over spatially heterogeneous scene elements, facilitating direct generalization to novel categories or domains given only textual prompts.

Practically, this unlocks new avenues for zero-shot/cross-task geospatial analysis, scalable mapping, automated cartography, and potentially multi-temporal map updating or multimodal (e.g., text-based querying over maps) reasoning. Theoretically, VecLang's language-centric architecture suggests that generalist models can be incentivized to learn executable, structured spatial reasoning by nesting modality-specific inductive biases within their output representations, with reinforcement learning acting as an enforcement mechanism bridging the gap between sequence-level and task-specific rewards.

Conclusion

VecLang establishes a unified, language-based paradigm for remote sensing vector mapping, leveraging a structured, GeoJSON-like representation and progressive vision-language generation optimized via hierarchical RL. Empirical results on VecMap-Bench confirm substantial gains in accuracy, robustness, and generalization, addressing longstanding limitations in category- and representation-specific approaches. The adoption of language as the primary medium for geospatial map representation and reasoning provides a promising blueprint for future research in scalable, open-world geospatial AI.

Future research directions include extending to temporal domains, integrating richer spatial logics, and scaling towards holistic multimodal Earth observation systems.