GeoMMAgent: Multi-Agent Geoscience Orchestration

• GeoMMAgent is a multi-agent orchestration framework that integrates retrieval, remote sensing perception, and reasoning for expert-level geoscience intelligence.
• It decomposes complex image–query problems into subgoals and assigns them to specialized agents, reflecting a plan–execute–evaluate paradigm.
• Performance benchmarks show GeoMMAgent outperforms standalone models by up to 21.7 points, underscoring its effective multi-agent coordination.

GeoMMAgent is a multi-agent orchestration framework for geoscience and remote sensing (RS) introduced in “GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing” (Xiao et al., 10 Apr 2026). It is designed to address the proposition that no single current multimodal LLM can reliably achieve expert-level multimodal intelligence in geoscience and RS, a setting characterized by wide-ranging disciplinary knowledge, heterogeneous sensor modalities, and a fragmented spectrum of tasks. The framework follows a plan–execute–evaluate paradigm and strategically integrates retrieval, perception, and reasoning through domain-specific RS models and tools, with self-evaluation used for verification and possible re-execution (Xiao et al., 10 Apr 2026).

1. Problem setting and motivation

GeoMMAgent is grounded in a diagnosis of geoscience and RS as an unusually demanding multimodal domain. The paper identifies four recurring obstacles. The first is the breadth of disciplinary knowledge: GeoMMBench spans RS, photogrammetry, GIS, and GNSS, while also relying on mathematics, physics, geography, cartography, and spectroscopy. The second is sensor heterogeneity: optical RGB, multispectral and hyperspectral imagery, SAR, LiDAR, DEM, thermal imagery, maps, plots, tables, and diagrams all appear as relevant inputs. The third is task fragmentation, ranging from theoretical knowledge and preprocessing to perception, spatial reasoning, temporal analysis, and high-level applications. The fourth is domain-specific reasoning difficulty, including compass reading, scale bars, coordinate grids, spectral-index interpretation, electromagnetic spectrum diagrams, and temporal interpretation across images and maps (Xiao et al., 10 Apr 2026).

The paper’s central motivation is that current MLLMs remain weak precisely where geoscience requires reliable performance. Reported weakness patterns include visual-linguistic misalignment, misinterpretation of geospatial relationships, sensor modality confusion, deficiencies in specialized disciplines such as GIS and photogrammetry, and image-perception failures in domain-specific tasks (Xiao et al., 10 Apr 2026). GeoMMAgent responds to these deficits by decomposing a question into subgoals and routing them across specialized capabilities for knowledge retrieval, RS-specific perception, multimodal reasoning, and self-evaluation.

A plausible implication is that GeoMMAgent should be understood less as a single foundation model than as a structured systems response to multimodal specialization: the framework externalizes knowledge, visual grounding, and verification instead of requiring a monolithic model to solve all three internally.

2. GeoMMBench as the empirical basis

GeoMMAgent is inseparable from GeoMMBench, the benchmark introduced in the same paper. GeoMMBench is an expert-crafted, image-based multiple-choice benchmark for multimodal intelligence in geoscience and remote sensing. It contains 1,053 image-based multiple-choice questions with a single correct answer, divided into a validation set of 37 and a test set of 1,016. Questions were created by domain experts, including PhD researchers and doctoral students, and were reviewed to remove text-only inference bias, ambiguous samples, and incorrect samples (Xiao et al., 10 Apr 2026).

The benchmark covers four disciplines—RS, photogrammetry, GIS, and GNSS—six sensor modalities—optical, MSI/HSI, SAR, DEM, LiDAR, and thermal—and a task spectrum including principles, perception, spatial relation analysis, quality tasks, time-series analysis, and applications. Evaluation is zero-shot, uses default multiple-choice QA prompts, parses outputs by automated rule-based regex extraction, treats invalid outputs as incorrect, and reports micro-averaged accuracy. The paper evaluates 36 open-source and proprietary models under this protocol (Xiao et al., 10 Apr 2026).

GeoMMBench functions as both benchmark and failure taxonomy. Human expert validation accuracy is reported as 86.5%. The best standalone closed-source model, Gemini-1.5 Pro, reaches 70.7% on the test set, and the best open-source model, Qwen3-VL-30B, reaches 66.7% (Xiao et al., 10 Apr 2026). These results are used as direct motivation for GeoMMAgent: retrieval compensates for missing factual knowledge, RS-specific perception tools compensate for weak grounding, and a dedicated reasoning agent compensates for failures in multimodal synthesis.

3. Multi-agent architecture

GeoMMAgent is described as a multi-agent and tool-augmented orchestration framework. Given an image–query pair, it dynamically decomposes complex problems into structured subgoals, assigns them to specialized agents, and integrates their outputs into a coherent final answer. The workflow is organized into three stages: Planning, Multi-Agent Execution, and Self-Evaluation (Xiao et al., 10 Apr 2026).

The coordinator, or Coordinate Agent, is the orchestration hub. It interprets the task, creates an execution plan, assigns subtasks, selects tools and agents, schedules dependencies, and aggregates outputs. The supplementary prompt characterizes it as an “Intelligent Orchestration Expert in the field of Remote Sensing” that creates execution plans through the coordination of multiple specialized agents and toolkits (Xiao et al., 10 Apr 2026).

The Knowledge Agent performs external knowledge retrieval and summarization. It is responsible for querying knowledge bases and web resources, retrieving, filtering, and summarizing factual information about remote sensing, geophysics, and geographic entities. The Perception Agent extracts visual evidence from multi-sensor imagery through scene classification, object detection, and semantic segmentation, and provides calibrated predictions for downstream reasoning. The Reasoning Agent integrates visual features, retrieved knowledge, and task context to conduct step-by-step analysis and produce logically consistent answers. The Self-Evaluation Agent reviews logic, consistency, and completeness, and can trigger selective re-execution or refinement when initial outputs are weak or inconsistent (Xiao et al., 10 Apr 2026).

The paper does not define a persistent memory module as a separate architectural component. The nearest equivalent is an execution log or reasoning trace passed into self-evaluation and possible re-execution. The framework is therefore best characterized as a prompt-driven, plan-based controller over specialized agents and external tools rather than as a learned end-to-end routing policy.

4. Toolkits, models, and execution path

GeoMMAgent’s tool library is divided into a general toolkit, a knowledge toolkit, a perception toolkit, and a reasoning toolkit. The system is described as fully training free and extensible, with plug-and-play tool addition and no model fine tuning or architectural modification required to incorporate new tools (Xiao et al., 10 Apr 2026).

The general toolkit includes format conversion, patch tiling and merging, filtering, cropping, scaling, super resolution, area counting, and box counting. These utilities support compatibility across RS data sources, large-image processing, image enhancement, ROI extraction, resolution adaptation, quantitative region measurement, and instance counting. They also implicitly address the mismatch between geospatial image scale and the input constraints of conventional MLLMs (Xiao et al., 10 Apr 2026).

The knowledge toolkit includes the Google API, Wikimedia API, and GME. Google is used for open-domain and up-to-date web information; Wikimedia provides encyclopedic knowledge about technical terminology, landforms, and geospatial entities; GME is used for multimodal retrieval in a unified embedding space, reducing hallucination and improving image-grounded reasoning (Xiao et al., 10 Apr 2026).

The perception toolkit uses RS-specific supervised models rather than relying only on general MLLM visual encoders. Scene classification is performed by a YOLO11-based classifier with a CSPNet backbone trained on Million-AID, returning top-5 predictions with confidence scores across 51 scene categories. Detection is handled by a pre-trained YOLO11 detector with CSPNet backbone on DOTA-v2, producing oriented bounding boxes, categories, confidence scores, object counts, spatial distributions, and detection reports. Segmentation is handled by DeepLabv3+ with an Xception backbone trained on LoveDA, producing semantic masks with per-pixel class labels (Xiao et al., 10 Apr 2026).

The reasoning toolkit uses Qwen-VL-Max as the main reasoning model. It integrates perception outputs, retrieved knowledge, the original image, and question context, and performs multi-step inference, semantic matching, consistency checking, option filtering, and final answer generation. The toolkit list also includes a Spatial-Temporal Analysis tool and a Multiple Choice Matching tool for temporal characterization and final option alignment (Xiao et al., 10 Apr 2026).

Tools are integrated under Model Context Protocol (MCP) for modularity. The end-to-end execution path is straightforward: the coordinator analyzes the image and question, decomposes the problem, dispatches retrieval and perception subtasks, passes structured evidence to the reasoning agent, aggregates the outputs, and then submits the result to self-evaluation. If self-evaluation identifies insufficient evidence or inconsistency, the system can revise the search or tool strategy and re-run selected steps (Xiao et al., 10 Apr 2026).

5. Performance, ablation, and exemplars

GeoMMAgent is reported at 86.5 on the validation set and 88.4 on the test set, the best result in the paper (Xiao et al., 10 Apr 2026). The best standalone closed-source model, Gemini-1.5 Pro, reaches 70.7, and the best open-source model, Qwen3-VL-30B, reaches 66.7. The paper states that GeoMMAgent exceeds the best closed-source model by 17.7 points and the best open-source model by 21.7 points (Xiao et al., 10 Apr 2026).

Performance is also broken down across disciplines, modalities, and task types. Reported discipline accuracies are 87.6 for RS, 89.8 for photogrammetry, 93.2 for GIS, and 97.6 for GNSS. Reported sensor-modality accuracies are 78.2 for optical, 94.0 for DEM, 91.2 for SAR, 89.2 for HSI, and 74.2 for LiDAR. Reported task-spectrum results are 89.3 for principles, 92.2 for perception, 82.9 for spatial, 97.3 for quality, 76.9 for time series, and 97.8 for applications (Xiao et al., 10 Apr 2026). This suggests that the largest benefits arise in high-level applications, quality-oriented tasks, and non-RGB modalities where generic MLLMs are comparatively weaker.

The component-wise ablation study is explicit. Removing knowledge yields 83.8 on validation and 87.4 on test. Removing perception yields 83.8 and 80.3. Removing reasoning yields 59.5 and 67.3. Removing self-evaluation yields 81.1 and 80.1. The full model yields 86.5 and 88.4 (Xiao et al., 10 Apr 2026). The paper identifies reasoning as the most critical component: removing it causes the largest degradation, while perception and self-evaluation also have substantial effects. The knowledge module gives a smaller but consistent gain.

Two qualitative examples illustrate the mechanism. In a spectral-band identification problem, an initial retrieval strategy was too generic; self-evaluation identified low confidence and insufficient evidence, after which the system refined its retrieval query around microwave penetration of vegetation and subsurface features and corrected the answer. In an aircraft-counting problem, the detection toolkit found 12 aircraft, matched the count to option C, and self-evaluation confirmed the result (Xiao et al., 10 Apr 2026). These examples make the framework’s intended division of labor concrete: perception anchors the answer visually, retrieval supplies technical facts, reasoning links them, and self-evaluation audits the chain.

6. Position in the literature, limitations, and scope

GeoMMAgent occupies a distinct position within recent geospatial and spatial-reasoning agent research. “GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis” focuses on evaluating GIS agents in a real execution sandbox with 117 atomic GIS tools and 53 tasks across six GIS domains (Yu et al., 15 Apr 2026). “GeoJSON Agents: A Multi-Agent LLM Architecture for Geospatial Analysis—Function Calling vs Code Generation” studies a planner–worker architecture for GeoJSON-centered vector GIS automation and reports 85.71% accuracy for Function Calling and 97.14% for Code Generation on a 70-task benchmark (Luo et al., 10 Sep 2025). “Geometrically-Constrained Agent for Spatial Reasoning” targets the semantic-to-geometric gap in spatial reasoning by formalizing a task constraint before tool use, rather than focusing on geoscience and RS multimodal interpretation (Chen et al., 27 Nov 2025). Relative to these systems, GeoMMAgent is defined by benchmark-driven geoscience QA over image–query pairs and by its explicit integration of retrieval, RS-specific perception, reasoning, and self-evaluation (Xiao et al., 10 Apr 2026).

The paper states several limitations. GeoMMBench cannot represent the full breadth and depth of geoscience. GeoMMAgent’s tools are mostly targeted to benchmark-covered tasks rather than the full wider geoscience task range. The toolkit is therefore not exhaustive. No explicit latency, token-cost, or deployment-efficiency analysis is reported; no end-to-end learned planner is described; and no extensive trust-calibration or human-in-the-loop protocol is presented (Xiao et al., 10 Apr 2026).

Within those limits, GeoMMAgent’s main significance is methodological. It reframes expert-level geoscience intelligence as a coordination problem over knowledge, perception, and reasoning rather than as a property of a single multimodal model. In the paper’s formulation, agentic tool use is not ancillary infrastructure but the mechanism by which expert-level performance becomes attainable in geoscience and remote sensing (Xiao et al., 10 Apr 2026).