GeoLLM-Engine: Geospatial AI Environment

• GeoLLM-Engine is an integrated environment combining LLMs and multi-agent systems for complex geospatial and remote sensing workflows.
• It offers modular front-end geospatial integration and back-end agent orchestration to facilitate task decomposition and deterministic workflow control.
• The system demonstrates high accuracy and scalability with 175+ geospatial tools and rigorous benchmarking across diverse real-world tasks.

GeoLLM-Engine is an environment enabling LLMs and multi-agent systems to perform sophisticated geospatial analysis and remote sensing workflows through natural language commands. It provides a modular framework integrating geospatial APIs, map/UI primitives, external knowledge bases, orchestration protocols, and benchmarking infrastructure for both single-agent and multi-agent use cases. The engine supports task decomposition, agent collaboration, deterministic workflow control, error handling, and performance evaluation at scale. Recent research has demonstrated the engine’s utility in supporting realistic, multi-tool Earth Observation (EO) workflows, advancing the automation and correctness of geospatial copilots, and benchmarking LLM agents against complex data science and geospatial challenges (Singh et al., 2024, Lee et al., 27 Jan 2025, Luo et al., 10 Sep 2025, Chen et al., 2024).

1. System Architecture and Layered Design

GeoLLM-Engine is organized as a two-layered architecture comprising:

• Front-end Geospatial Integration: Exposes over 175 geospatial tools via function-callable Python APIs, including data loaders (e.g., rasterio, geopandas), map/UI primitives (Mapbox, React widgets), vision models (SwinModel, object detectors), and retrieval utilities (LangChain, FAISS-based RAG). All views and tool calls are synchronized between UI and agent state via WebSocket mirroring.
• Back-end Agent Orchestration: Implements deterministic workflow control and benchmark execution. Multi-agent protocols—such as AutoGen’s synchronous message passing—are layered on top, coordinating sub-agent scheduling and workflow state mutation via shared Redis pub/sub channels.

The agentic orchestration is modular: a root “Orchestrator” agent parses the user’s prompt, consults workflow memory and intent-based tool recommendations, and emits a program-like schedule of specialized sub-agents (e.g., Database, DataOps, Agriculture, Forestry, Urban). Each agent exposes a FastAPI endpoint and executes assigned workflow steps through function-calling APIs, returning structured JSON messages for aggregation.

Containerization (Docker, Kubernetes) ensures horizontal scaling, with each agent deployed as a microservice and GPU resource allocation governed by weighted-fair scheduling policies.

2. Task Parsing, Agent Collaboration, and Workflow Execution

The internal workflow follows a cyclical plan–execute–observe–replan loop:

• Task Parsing: The central Orchestrator or Planner agent ingests the user prompt and associated file metadata, decomposing it into ordered subtasks. Outputs include function identifiers with parameters (function calling mode) or textual code descriptions (code generation mode).
• Agent Execution:
  • Function-Calling Mode: Worker agents issue JSON “function_call” messages for specific geospatial APIs and process return values (GeoJSON fragments, images, status).
  • Code-Generation Mode: Worker agents generate and execute Python scripts in secure sandboxes; errors (tracebacks) trigger self-repair loops.
• Result Integration: Final artifacts (GeoJSON, image references) from Worker agents are aggregated, and the global environment state is updated before dispatching new subtasks or concluding with output to the user.

Message-passing strictly conforms to a triple schema: (sender, recipient, content), with synchronization and completion determined by state machine predicates and a shared chat history.

3. Geospatial Tooling and API Wrappers

GeoLLM-Engine integrates an extensive catalog of geospatial functions and APIs, standardized for agent invocation:

• API Stack:
  • GeoPandas (>=0.13.0), Rasterio (>=1.3.8), GDAL (>=3.6.0)
  • Shapely, Fiona, pyproj for I/O, projection, and spatial ops
  • Mapbox and React for dynamic map manipulation
• Agent-Specific Functions:
  • Database agent: load, filter, tile
  • DataOps agent: reprojection, resampling, clustering
  • Domain-specific agents (e.g., Agriculture, Urban): expose dedicated ops (up to 521 APIs system-wide)
• Container and Deployment:
  • Dockerfile images (Ubuntu 22.04 + CUDA 11.8) with necessary library pins
  • Kubernetes deployment with ConfigMaps/Secrets for secure resource provisioning (Earthdata, Sentinel-2/Modis endpoints, internal GIS DBs)

4. Multi-Agent Environment, Synchronization Protocols, and Formalism

The multi-agent paradigm is key for scaling geospatial workflows beyond monolithic LLM agents:

• Orchestrator Mapping: For user task prompt $T$ and initial memory $M_0$, deterministic scheduling is formalized by

$O : (T, M_0) \to S = [(a_1, p_1), … ,(a_n, p_n)]$

Each agent $a_i$ executes sub-prompt $p_i$ and returns message $m_i$, evolving the workflow state:

$M_{i+1} = \text{update}(M_i, m_i)$

Task completion is signaled when $\text{Done}(S, M_k)$ holds.

• Resource Scheduling: Policy assigns GPU/CPU slots per agent, minimizing the maximum weighted sum:

$$w(a) = \alpha \cdot \text{cpu\_req}(a) + \beta \cdot \text{gpu\_req}(a)$$

• Message Schema:

| Field | Description | Example Values | |-----------|---------------------------------|----------------------| | sender | Role issuing message | "Planner", "Worker" | | task_id | Unique task thread identifier | "T12345" | | type | Message type | "function_call", "code", "result" | | payload | Message content / file paths | JSON object, string |

5. Code Generation, Static Analysis, and RAG Pipelines

GeoLLM-Engine supports both stable function-calling (deterministic API invocation) and flexible code generation (dynamic Python code synthesis):

• Function Calling: JSON schema is defined for each API; calls are parsed and validated for correct arguments.
• Code Generation Mode:
  • LLM emits Python code blocks for each subtask.
  • Execution is sandboxed; tracebacks are captured and routed to the agent for self-repair (up to five error-repair cycles).
  • Static code analysis (Python AST, Jedi) is used to diagnose undefined variables, missing imports, or invalid calls.
  • If libraries or function usage are ambiguous, Retrieval-Augmented Generation (RAG) fetches relevant code examples from an embedded knowledge base using FAISS/BBAI embeddings.

This dual strategy enables robust, flexible automation of geospatial workflows, accommodating both novice and advanced user requests.

6. Benchmarking, Performance Measurement, and Scaling

GeoLLM-Engine incorporates benchmarking frameworks on realistic, large-scale datasets:

• Benchmarking: Tasks (‘GeoCode’, GeoLLM-Engine-10k) include single-turn and multi-turn workflows utilizing 28+ geospatial libraries (Earth Engine, geemap, xarray, scikit-eo).
• Metrics:
  • Agentic correctness:

  $$Crct = \frac{\sum_{i=1}^N \mathbf{1}\{\hat{f}_i = f^*_i\}}{\sum_{i=1}^N 1}$$ - Mean-square percentage error:

  $$\epsilon = \frac{1}{N} \sum_{i=1}^N \left( \frac{\hat d_i - d^*_i}{d^*_i} \right)^2$$

• Performance Statistics:

  • Multi-agent GeoLLM-Squad achieves a 17% improvement in correctness over monolithic baselines and maintains sub-5% RS-metric error over 2k workflows (Lee et al., 27 Jan 2025).
  • Function-calling based GeoJSON agents achieve 85.71% accuracy (92.5% on basic tasks); code generation mode reaches 97.14% (100% intermediate) (Luo et al., 10 Sep 2025).
  • Benchmarks span over ½ million complex multi-tool tasks, distributed across 100 GPT-4-Turbo nodes for parallel evaluation (Singh et al., 2024).

Scalability is realized by Kubernetes autoscaling and parallel agent deployment, maintaining sub-second orchestration latency.

7. Contemporary Implications and Future Directions

GeoLLM-Engine’s modularity and agentic orchestration have demonstrated robust scaling, functional correctness, and flexibility in real-world, long-horizon geospatial analysis. Benchmark outcomes show significant improvements in both function call accuracy and data output fidelity compared to single-agent and general-purpose baselines. The systematic inclusion of tool-centric APIs, error-repair via static analysis, retrieval-augmentation, and deterministic scheduling positions GeoLLM-Engine as a standard for evaluating LLMs and multi-agent systems in geospatial domains.

Recent findings underscore the need for further research in multi-step, multi-agent workflows, robust error handling, and flexible function invocation, particularly for advanced spatial tasks (e.g., clustering, spatial joins, multimodal integration) and scaling beyond simple captioning or template-based protocols (Chen et al., 2024, Lee et al., 27 Jan 2025, Singh et al., 2024).

GeoLLM-Engine thus constitutes a comprehensive environment for building, benchmarking, and deploying geospatial agents in both academic and applied contexts, serving as a reference architecture for future GeoAI developments.