AgriWorld:A World Tools Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agents

Abstract: Foundation models for agriculture are increasingly trained on massive spatiotemporal data (e.g., multi-spectral remote sensing, soil grids, and field-level management logs) and achieve strong performance on forecasting and monitoring. However, these models lack language-based reasoning and interactive capabilities, limiting their usefulness in real-world agronomic workflows. Meanwhile, LLMs excel at interpreting and generating text, but cannot directly reason over high-dimensional, heterogeneous agricultural datasets. We bridge this gap with an agentic framework for agricultural science. It provides a Python execution environment, AgriWorld, exposing unified tools for geospatial queries over field parcels, remote-sensing time-series analytics, crop growth simulation, and task-specific predictors (e.g., yield, stress, and disease risk). On top of this environment, we design a multi-turn LLM agent, Agro-Reflective, that iteratively writes code, observes execution results, and refines its analysis via an execute-observe-refine loop. We introduce AgroBench, with scalable data generation for diverse agricultural QA spanning lookups, forecasting, anomaly detection, and counterfactual "what-if" analysis. Experiments outperform text-only and direct tool-use baselines, validating execution-driven reflection for reliable agricultural reasoning.

• The paper introduces AgriWorld, a World–Tools–Protocol framework that combines typed agricultural data, executable tools, reflective agents, and checkers for verifiable reasoning.
• The full Agro-Reflective agent achieves 57.6% benchmark accuracy, while Qwen3-32B reaches 73.84% choice accuracy versus 36.75% for GPT-4o, recovering 92% of initial code-generation bugs.
• Tool grounding improves robustness and complex reasoning: reflective agents reduce forecasting NRMSE to 0.18, raise counterfactual success to 71.4%, and limit spatial and temporal OOD degradation to 12% and 6%.

"AgriWorld: A World–Tools–Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agents" (2602.15325) addresses a concrete mismatch in scientific LLM applications: agronomic questions require computation over heterogeneous spatiotemporal data (parcel geometries, Sentinel-2-style raster time series, soil grids, weather streams, management logs), which neither text-only LLMs nor narrow agricultural foundation models can handle end-to-end. The authors, from Sun Yat-sen University, propose a three-part abstraction — an executable environment (AgriWorld), an execution-driven agent (Agro-Reflective), and a verifiable evaluation protocol with executable checkers (instantiated as AgroBench). The central claim is that grounding and reflection, not model scale, determine reliability on these tasks; the experiments support this with large margins over GPT-4o and other frontier baselines.

A task instance is formalized as $$\mathcal{I}=\langle q,\ \mathcal{B},\ \mathcal{S}_{out},\ \mathcal{V},\ B\rangle$$: a natural-language query, bindings to concrete data handles (parcel IDs, time intervals, sensor products), a rigid output schema, an executable checker $\mathcal{V}$, and a step budget $B$. The checker returns a binary validity indicator plus a structured diagnostic report of constraint violations, so evaluation goes beyond final-answer matching to failure-mode attribution (spatial misalignment, temporal window errors, unit mismatches, ungrounded claims).

AgriWorld's state space comprises five typed entities: vector parcels with CRS metadata, 4D remote-sensing raster time series with validity masks, static grid fields (soil, DEM), weather streams, and management event logs. Tool executions are modeled as state transitions producing both a return value and an inspectable artifact carrying units, CRS, resolution, and a provenance hash over inputs and tool version. Two mechanisms address domain-specific correctness hazards:

• Canonical alignment operator $\Pi$: composition of spatial reprojection to a task CRS (e.g., EPSG:3857) and temporal resampling to a canonical frequency under fixed interpolation policies.
• Robust zonal statistics with validity-ratio gating: aggregation raises a LowCoverageError when the cloud-masked valid-pixel ratio falls below a threshold, preventing conclusions from corrupted imagery.

The protocol enforces four hierarchical constraints: schema validity, numeric tolerance against reference simulator outputs, counterfactual consistency (the proposed intervention must satisfy $M(s,a_{intervene}) - M(s,a_{baseline}) \le -\Delta$ under the simulator), and physical sanity (dimensional homogeneity and coverage).

The agent operates an execute–observe–refine loop: it emits code conditioned on accumulated memory, executes it, and on failure invokes a diagnostic reflection function that inspects artifact metadata, categorizes errors into typed classes (e.g., SpatialMisalignment, UnitError), and synthesizes targeted patches. The loop terminates early once the checker validates the output.

Ablation results quantify each component's contribution:

Variant	Interaction	Tool scope	Alignment	Avg. turns	Acc (%)
Text-only (Kimi/GPT)	Chat	None	–	1.0	21.9
AgriWorld-Direct (one-shot)	One-shot	All	✓	1.0	48.2
w/o Remote Sensing	Reflective	w/o RS	✓	2.4	41.5
w/o Alignment	Reflective	All	✗	3.1	51.3
w/o Reflection	One-shot	All	✓	1.0	50.6
Full Agro-Reflective	Reflective	All	✓	3.5	57.6

Notably, removing alignment increases accuracy relative to one-shot but degrades versus full reflection while inflating turns — consistent with the paper's observation that without automatic CRS transformation, agents join WGS84 coordinates against UTM-projected rasters and obtain empty intersections. The authors report that 35% of initial code generations contained bugs, of which the reflection loop recovered 92%.

The benchmark uses LoRA-finetuned Qwen3 models (8B/32B) with three interaction paradigms, compared against open-source and proprietary baselines including DeepSeek-V3, Yi-1.5-34B, LLaMA 3, GPT-4o, and Gemini-2.0. The strongest result is Qwen3-32B-LoRA-Reflective achieving 73.84% choice accuracy versus 36.75% for GPT-4o — a roughly twofold margin over a far larger proprietary model, attributed primarily to API-schema hallucination by general-purpose models (e.g., inventing a nonexistent get_soil_moisture() instead of grid.sample()). Text-only baselines cluster at 21–40%, supporting the claim that dynamic field-level data cannot be memorized parametrically.

Fine-grained task breakdown shows gains concentrated where multi-step reasoning is required:

Model	Lookup Acc.	Forecasting NRMSE ↓	Anomaly IoU ↑	Counterfactual Success ↑
Text-only	41.2%	0.89	0.12	14.5%
One-shot	82.5%	0.34	0.51	43.8%
Reflective	86.7%	0.18	0.68	71.4%

Forecasting error drops by 47% relative to one-shot because reflection catches resampling/alignment mistakes (e.g., mixing monthly precipitation aggregates with daily temperature). Counterfactual success rises from 43.8% to 71.4%, with text-only below 15%; here the simulator serves as an experimental sandbox for causal verification. Lookups show only marginal improvement, as expected for single-call tasks.

Efficiency analysis shows logarithmic saturation: accuracy climbs from 48.2% to 68.5% within four turns (fixing syntax and parameter errors) and gains only +5.3% from turn 4 to 20. Average convergence requires 3.5 turns despite a budget of 20, indicating effective early stopping via the verifier.

Out-of-distribution robustness is the most consequential finding. On held-out Brazilian fields (spatial OOD), text-only accuracy falls 55% — the paper attributes this to hemisphere-inappropriate crop-calendar priors — while the reflective agent degrades only 12%. On the El Niño year 2023 (temporal OOD), text models relying on historical averages drop 46%, whereas tool-grounded querying limits degradation to 6%. The implication is that grounding confers generalization precisely because behavior is driven by retrieved data rather than fitted regional priors.

Several caveats deserve emphasis. The headline comparison against GPT-4o and Gemini-2.0 is confounded: those baselines were not finetuned on the benchmark's training split, whereas the proposed agents were LoRA-tuned on it, so part of the gap may reflect task-specific adaptation rather than the framework alone. Counterfactual consistency is verified against the environment's own simulator $M$ — circularity is inherent, since "correct" causal answers are defined by the same simulation backend the agent queries; real-world agronomic validity of those simulators is assumed, not established. The evaluation suite is built by converting an existing QA benchmark into executable form, and its coverage of agricultural reasoning breadth is bounded by that source. The OOD analysis covers two scenarios (Brazil, 2023); whether robustness holds under more systematic distribution shift remains untested. Finally, the reported aggregate "QA Total" scores mix different denominators across baseline groups (150 tasks for some baselines vs. 541 for others), complicating direct comparison.

The paper demonstrates that for spatiotemporal scientific domains, executable grounding plus structured self-correction outperforms scale: a LoRA-tuned 32B agent with reflective execution roughly doubles GPT-4o's accuracy and exhibits markedly better OOD robustness. The World–Tools–Protocol decomposition — typed state, artifact-producing tools with provenance, and executable checkers — provides a reusable template for verifiable agentic evaluation. The main unresolved issues are the training asymmetry in baseline comparisons, reliance on internal simulators as ground truth for counterfactuals, and limited OOD coverage.