GeoAI Agency Primitives

Abstract: We present ongoing research on agency primitives for GeoAI assistants -- core capabilities that connect Foundation models to the artifact-centric, human-in-the-loop workflows where GIS practitioners actually work. Despite advances in satellite image captioning, visual question answering, and promptable segmentation, these capabilities have not translated into productivity gains for practitioners who spend most of their time producing vector layers, raster maps, and cartographic products. The gap is not model capability alone but the absence of an agency layer that supports iterative collaboration. We propose a vocabulary of $9$ primitives for such a layer -- including navigation, perception, geo-referenced memory, and dual modeling -- along with a benchmark that measures human productivity. Our goal is a vocabulary that makes agentic assistance in GIS implementable, testable, and comparable.

• The paper introduces nine agency primitives that formalize interfaces for iterative, human-in-the-loop geospatial artifact creation.
• It details a modular framework integrating sensing, execution, enrichment, and inference primitives to improve GIS workflow control.
• The study benchmarks human-AI collaboration using use cases like image summarization, crop mapping, and flood damage assessment.

Agency Primitives for GeoAI: A Modular Foundation for Human-in-the-Loop GIS Assistants

Motivation and Problem Formulation

Geographic Information Systems (GIS) workflows are artifact-centric, demanding iterative user control over high-dimensional, spatio-temporal data. Despite the maturity of image captioning, VQA, and promptable segmentation in satellite imagery applications, these advancements have not bridged the productivity gap for GIS practitioners, who require the rapid, controlled generation of vector layers, raster products, and derived cartographic artifacts. The current integration of LLMs and VLMs with GIS is limited to knowledge extraction or isolated tool-calling, overlooking the necessity for human-in-the-loop, artifact-native interaction and iterative agentic workflows.

The paper "GeoAI Agency Primitives" (2604.01869) formulates this missing layer as a set of nine agency primitives, which formalize foundational interfaces and operations essential for GeoAI agents that truly augment human decision-making in spatial environments. The aim is not accuracy maximization of singular models, but scaffolding compositional, testable, and comparable agentic systems—providing a vocabulary and instrumentation for agent design and experimental benchmarking.

The GeoAI Agency Primitive Vocabulary

The proposed primitives serve as orthogonal capabilities rather than models, enabling granular grounding and control. They are grouped functionally into sensing, execution, enrichment, and inference primitives.

After parsing a user query and decomposing it into sub-tasks and dependencies, agents leverage these primitives recursively throughout the multi-step interaction workflow:

Figure 1: High-level workflow for the GeoAI agent, translating natural queries into decomposed tasks, and enabling iterative review before committing changes.

Core Sensing and Spatial-Temporal Abstractions

Four primitives—Navigation, Perception, GeoMemory, and Embeddings—constitute the agent’s mechanism for spatial exploration, semantic interpretation, context accumulation, and similarity reasoning:

• Navigation constructs context bundles across ROI, scale, stack, and sampling, subsetting the vast Earth surface into tractable queryable units. Different strategies (diversity, uncertainty, coverage, temporal contrast) are supported for exploration and quality control.
• Perception abstracts model selection and inference dispatch, interfacing with task-appropriate detectors, segmenters, or VQA modules. Critically, perceptions yield labeled suggestions with interpretable notes, surfacing model uncertainty and failure states for targeted user intervention.
• GeoMemory operationalizes spatially-indexed, session-persistent annotation storage, enabling efficient querying, curation, and retrieval over geometrically structured facts (geometry, timestamp, query, outputs, notes).
• Embeddings provide semantic vector representations suitable for exploratory search, diversity sampling, and lightweight modeling (e.g., PRESTO, SatCLIP, Prithvi). Embeddings support "find more like this" and act as the backbone for local propagation and feature extraction.

Figure 2: The four core primitives for spatial reasoning—context navigation, model-agnostic perception, structured geospatial memory, and foundation-model-driven embeddings.

Execution, Enrichment, and Inference Primitives

The remaining primitives allow expressiveness, scalable inference, attribution, and robust agent control:

• Compute Graphs formally represent operator graphs over spatial layers, composed either from user commands or LLM-generated plans, supporting traceable, reviewable derivation pipelines.
• Budgets enforce interactivity and computational tractability by constraining graph complexity, VLM inference, and tree depth, yielding partial and early-stoppable results for transparent user mediation.
• Propagation generalizes active search by ranking candidate regions similar to user-provided seeds, enabling local batch expansion of labels through guided exploration in embedding space.
• Attribution enriches geometries with heterogeneous evidence: textual facts, image previews, temporal plots, and on-the-fly statistics, facilitating user decision-making and reducing labeling ambiguity.
• Dual Modeling formalizes the minimization of human and VLM expense by iteratively alternating high-precision, low-throughput expert annotation and high-coverage, low-cost lightweight model inference. This enables uncertainty-driven correction and rapid convergence with human-in-the-loop guarantee.

Figure 3: Execution (graphs, budgets), enrichment (attribution), and scalable dual modeling for composable, extensible agentic reasoning and efficient user interaction.

Benchmarking Human-AI Collaboration

The study proposes a benchmark for agentic GeoAI, emphasizing human productivity, iteration dynamics, and agent effectiveness through quantifiable metrics: time-to-quality-threshold, progress AUC, rework rate, and suggestion bias. Sessions are drawn from a combinatorial task-region-time space, with task-dependent quality evaluated periodically. Four agent capability levels—ranging from manual baselines to full agentic stack—provide a framework for ablation and controlled evaluation of each primitive’s utility.

This focus shifts the evaluation of GeoAI from model-centric accuracy to operational metrics reflecting artifact development and realistic user workflows, directly addressing practitioner requirements for time-efficient, verifiable artifact creation.

User Stories and Illustrative Scenarios

The abstraction power of the proposed primitives is demonstrated in several canonical RS tasks.

Image Summarization leverages multi-scale navigation, embedding-based diversity sampling, VLM captioning, and spatial memory to synthesize scene-level overviews, seeding subsequent detailed workflows.

Crop Mapping with Sparse Labels exploits embedding-guided propagation, NDVI-based attribute attachment, local dual modeling, and uncertainty-driven review for rapid expansion and validation of land cover labels with minimal expert input.

Flood Damage Assessment operationalizes propagation around high-priority anchors, multi-modal evidence fusion (SAR changes, Sentinel-2 previews), dual modeling, and per-object aggregation, enabling fast, semantically rich, and geometrically structured situational awareness in disaster settings.

These scenarios highlight the agent stack’s modularity and the instrumental role of human oversight and artifact curation throughout the automated reasoning process.

Practical and Theoretical Implications

The modularization of agent capabilities into formal agency primitives delineates a blueprint for GeoAI assistant construction, addressing the human-in-the-loop gap left by most prior RS-focused LLM integrations (e.g., tool-calling, code-generation, hybrid systems [see comparative analyses in (Chen et al., 2024, Zhang et al., 2023, Du et al., 2023)]).

Practically, this architecture produces observable wins in transparent, auditable, and efficient GIS artifact production, paving the way for incremental adoption in legacy GIS environments. The primitives are amenable to interface standardization, compositional benchmarking, and systematic ablation—critical for robust and community-validated pipeline development.

Theoretically, this approach formalizes the boundary between foundation models and operational agentic intelligence, necessitating further research into advanced active navigation, semantic memory architectures, and cross-task generalization with safety and reproducibility guarantees. The agentic stack provides a testbed for human preference learning, failure explanation, and agent introspection at the artifact level—questions that remain largely open in GeoAI.

Conclusion

The "GeoAI Agency Primitives" framework (2604.01869) specifies a minimal, compositional vocabulary for building agentic assistants that can reason, perceive, and act within geospatial artifact workflows. By shifting the focus from monolithic accuracy improvements to structured, collaborative artifact creation and curation, this work lays the foundation for the next generation of GeoAI systems that are not only powerful but also inherently testable, extensible, and compatible with the iterative needs of GIS practitioners. The open benchmarking protocol invites future research into operationalizing, generalizing, and validating agentic workflows across the breadth of RS applications.