---
title: 'AgriBench-13K: Agricultural Language Benchmark'
url: https://www.emergentmind.com/topics/agribench-13k
type: topic
---

# AgriBench-13K: Agricultural Language Benchmark

Searching arXiv for the cited benchmark and related source papers.
arXiv search query: AgriGPT-VL agricultural vision-language understanding suite 2510.04002
AgriBench-13K is a text-only benchmark for agricultural language understanding that, in the AgriGPT-VL study, serves as the text-only counterpart to the vision-language benchmark AgriBench-VL-4K. Within that work, it has a dual function: a held-out split is used to calibrate and early-stop Stage-1 textual grounding, and the benchmark as a whole is used as the principal text-only evaluation suite for testing whether a progressive multimodal curriculum preserves pure language capability in agriculture. The AgriGPT-VL paper reports that the full curriculum causes no noticeable degradation on text-only language tasks and instead improves domain terminology and answer formatting [2510.04002].

## 1. Definition, provenance, and scope

AgriBench-13K originates from the earlier AgriGPT ecosystem and is treated in the AgriGPT-VL paper as an established benchmark rather than a newly constructed dataset [2510.04002]. Its size is given as 13K items, as implied by the name and cited from the AgriGPT work. In the AgriGPT-VL paper, it is consistently characterized as a text-only benchmark used to measure agricultural language abilities without images.

The paper does not enumerate a task taxonomy or domain inventory for AgriBench-13K. It references the benchmark as inherited from AgriGPT and evaluates it with generation-style metrics—BLEU, METEOR, and ROUGE variants—which indicates open-ended QA-style evaluation. Detailed domain coverage, such as crop health, pests and diseases, or agronomy, is not described there. Likewise, the paper does not state the languages used, nor does it revisit source collection or annotation procedures beyond noting that these details belong to the earlier AgriGPT source.

This limited exposition is itself significant. AgriBench-13K is central to the claims about text-only competence in AgriGPT-VL, yet many of its corpus-construction properties remain external to the 2025 paper. A plausible implication is that AgriBench-13K should be understood less as an independently re-specified benchmark in AgriGPT-VL and more as a fixed reference point inherited from prior work.

## 2. Roles in the AgriGPT-VL training curriculum

In AgriGPT-VL, AgriBench-13K plays two explicit roles. First, a held-out split is used for early stopping and calibration before multimodal alignment begins. Second, the full benchmark functions as the main text-only evaluation suite for verifying that the multimodal curriculum preserves and even strengthens agricultural language capability [2510.04002].

The relevant training pipeline begins with Stage-1 textual grounding. AgriGPT-VL starts from Qwen2.5-VL, undergoes continual pretraining on about 200K agricultural documents, corresponding to approximately 2.2B tokens, and is then supervisedly instruction-tuned on Agri-342K. The held-out AgriBench-13K split is used during this stage for early stopping and calibration, thereby anchoring agricultural language proficiency before any vision-language alignment.

Stage-2 multimodal alignment is described as a sequence of increasingly permissive updates. Shallow alignment is performed on 1M image-caption pairs while both the vision encoder and the LLM are frozen and only connector or adapter layers are trained. The stated purpose of this design is to minimize catastrophic forgetting in the LLM and preserve text-only competence established in Stage-1. Deep alignment then uses 2M image-QA samples and gradually unfreezes components via LoRA, transitioning toward full multimodal reasoning. Finally, GRPO optimization uses a 15K reward dataset and 50K expert-polished supervision, with rewards emphasizing image-text consistency, internal logic, and verifiable terminology.

These design choices are directly tied to the reported behavior on AgriBench-13K. The authors connect shallow alignment to preservation of language capability, gradual unfreezing to mitigation of forgetting, and GRPO reward design to reinforcement of precise terminology and logical structure. Although the ablations reported in the paper focus on AgriBench-VL-4K rather than AgriBench-13K, the narrative is that Shallow Alignment, Deep Alignment, and GRPO each contribute additive gains in multimodal reasoning without eroding text-only performance.

## 3. Evaluation protocol and measurement

The AgriGPT-VL paper evaluates AgriBench-13K using automatic generation metrics rather than judge-based preference scoring [2510.04002]. The reported metrics are BLEU, METEOR, Rouge-1-f, Rouge-2-f, and Rouge-L-f. The paper notes that the definitions and computation formulas appear in Appendix A.5, but those formulas are not included in the visible main text.

LLM-as-a-judge is not used for AgriBench-13K. In the AgriGPT-VL framework, LLM-as-a-judge is reserved for pairwise preference evaluation on the vision-language benchmark AgriBench-VL-4K. For the text-only benchmark, evaluation remains fully metric-based.

The paper also leaves several protocol details unspecified. Exact train, validation, and test ratios are not given. Input-output formats and prompt instructions are not detailed. Reference construction, annotator information, and inter-annotator agreement are likewise not described. As a result, the AgriBench-13K evaluation in AgriGPT-VL is transparent at the level of reported metrics and system comparisons, but not at the level of benchmark reconstruction.

## 4. Comparative performance on text-only agricultural tasks

AgriGPT-VL is compared against InternVL-3-8B, InternVL-3-14B, LLaVA-1.5-7B, MiniCPM-V-2.6, Yi-VL-6B, Yi-VL-34B, Qwen-VL-7B, Qwen2.5-VL-72B-Instruct, Qwen-QVQ, DeepSeek-VL-1.2, Gemini-2.5-Flash, and Gemini-2.5-Pro. Although many of these are multimodal models, the comparison here is strictly on text-only tasks [2510.04002].

The reported scores are as follows:

- **InternVL-3-8B**: BLEU 5.52; METEOR 23.07; ROUGE-1-f 24.14; ROUGE-2-f 5.69; ROUGE-L-f 23.08
- **InternVL-3-14B**: BLEU 8.53; METEOR 27.56; ROUGE-1-f 26.75; ROUGE-2-f 6.46; ROUGE-L-f 25.56
- **LLaVA-1.5-7B**: BLEU 1.44; METEOR 13.62; ROUGE-1-f 21.67; ROUGE-2-f 4.88; ROUGE-L-f 20.60
- **MiniCPM-V-2.6**: BLEU 1.15; METEOR 12.50; ROUGE-1-f 21.41; ROUGE-2-f 5.25; ROUGE-L-f 20.16
- **Yi-VL-6B**: BLEU 1.03; METEOR 12.38; ROUGE-1-f 20.93; ROUGE-2-f 4.36; ROUGE-L-f 20.09
- **Yi-VL-34B**: BLEU 1.69; METEOR 14.27; ROUGE-1-f 21.82; ROUGE-2-f 4.74; ROUGE-L-f 20.27
- **Qwen-VL-7B**: BLEU 7.70; METEOR 30.17; ROUGE-1-f 24.16; ROUGE-2-f 4.97; ROUGE-L-f 22.86
- **Qwen2.5-VL-72B-Instruct**: BLEU 6.52; METEOR 30.27; ROUGE-1-f 25.84; ROUGE-2-f 5.95; ROUGE-L-f 24.43
- **Qwen-QVQ**: BLEU 2.48; METEOR 17.31; ROUGE-1-f 17.54; ROUGE-2-f 3.63; ROUGE-L-f 16.75
- **DeepSeek-VL-1.2**: BLEU 6.37; METEOR 29.67; ROUGE-1-f 22.10; ROUGE-2-f 4.93; ROUGE-L-f 20.93
- **Gemini-2.5-Flash**: BLEU 6.12; METEOR 27.49; ROUGE-1-f 24.85; ROUGE-2-f 5.59; ROUGE-L-f 23.73
- **Gemini-2.5-Pro**: BLEU 4.34; METEOR 23.69; ROUGE-1-f 24.16; ROUGE-2-f 4.45; ROUGE-L-f 22.30
- **AgriGPT-VL**: BLEU 10.84; METEOR 32.53; ROUGE-1-f 27.73; ROUGE-2-f 6.36; ROUGE-L-f 26.36

The summary reported in the paper is precise: AgriGPT-VL achieves the highest BLEU, METEOR, ROUGE-1-f, and ROUGE-L-f among all compared models, and the second-best ROUGE-2-f at 6.36, behind InternVL-3-14B at 6.46. The authors interpret this pattern as evidence that the curriculum improves standardized agricultural terminology and canonical answer style while preserving text-only capability. The paper further notes that generalization tests on unrelated text benchmarks—MMLU, ARC, and OpenBookQA—suggest minimal forgetting, consistent with the strategy of training text first and vision later.

## 5. Relation to AgriBench-VL-4K and distinction from the 2024 AgriBench benchmark

AgriBench-13K is paired in the AgriGPT-VL paper with AgriBench-VL-4K. The former measures language ability without images; the latter measures multimodal performance with open-ended and image-grounded questions. Together, they are used to test whether adding visual alignment compromises text-only competence [2510.04002].

This pairing should not be conflated with the earlier benchmark named AgriBench. The 2024 paper "AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models" introduces a hierarchical MM-LLM benchmark backed by MM-LUCAS, with 1,784 rural or agricultural landscape RGB images, 1,784 semantic segmentation masks, 1,784 depth maps, and associated multimodal annotations. That paper explicitly states that the term “AgriBench-13K” does not appear there and that no 13K-sample variant is defined or released in that work [2412.00465].

The distinction matters conceptually. In the 2024 benchmark, “AgriBench” denotes a multimodal, hierarchical framework spanning levels from basic recognition to human-aligned suggestion, with qualitative evaluation and no standardized splits yet. In the 2025 AgriGPT-VL paper, “AgriBench-13K” denotes a text-only benchmark inherited from the AgriGPT ecosystem, used for calibration and evaluation of agricultural language understanding. A common misconception is therefore to treat AgriBench-13K as a subset or scale-up of MM-LUCAS; the available evidence does not support that identification.

At the same time, the two lines of work are adjacent in purpose. Both are concerned with domain-grounded agricultural evaluation, and both address the tension between general-purpose multimodal systems and agriculture-specific competence. A plausible implication is that the naming overlap reflects convergence on agriculture benchmarking rather than a shared underlying dataset.

## 6. Availability, limitations, and unresolved aspects

The AgriGPT-VL paper states that datasets, models, and evaluation tools will be open-sourced to support reproducible research and deployment in low-resource agricultural settings [2510.04002]. However, the main text does not provide specific download links or license terms for AgriBench-13K.

Several limitations follow directly from what is not specified in the paper. It does not discuss AgriBench-13K’s domain coverage, biases, or limitations in detail. It does not specify languages, prompt templates, exact split ratios, or construction procedures. It does not report ablations directly on AgriBench-13K, even though its central claim is that multimodal curriculum stages do not harm text-only performance. It also does not provide explicit LaTeX formulas for textual grounding losses, alignment objectives, or GRPO objectives in the visible main text, instead deferring those details to appendices.

These omissions constrain what can be concluded about benchmark design itself. What is well supported is narrower but still important: within AgriGPT-VL, AgriBench-13K functions as the definitive text-only yardstick and calibration set for agricultural language understanding, and the reported metrics support the claim that progressive multimodal alignment can preserve or improve text-only performance when anchored by early textual grounding, conservative shallow alignment, gradual LoRA-based unfreezing, and reward signals that emphasize terminology and logical consistency [2510.04002].

Source: https://www.emergentmind.com/topics/agribench-13k