AgriGPT-VL Suite: Agricultural VLM Ecosystem
- The paper presents AgriGPT-VL Suite, a unified agricultural vision-language ecosystem combining data generation, model specialization, and benchmark design.
- Agri-3M-VL is the largest agricultural image-text corpus, synthesized via multi-agent refinement to capture robust crop, pest, and disease features.
- Empirical evaluations demonstrate that AgriGPT-VL outperforms baselines on metrics like Acc, BLEU, and METEOR while preserving text-only capabilities.
AgriGPT-VL Suite is a unified agricultural vision-language ecosystem consisting of three tightly coupled components: Agri-3M-VL, a large-scale agricultural vision-language corpus; AgriGPT-VL, an agriculture-specialized vision-LLM; and AgriBench-VL-4K, a dedicated benchmark for agricultural multimodal evaluation, with AgriBench-13K retained for text-only assessment. It was introduced to address a specific domain-gap claim: contemporary multimodal LLMs perform well on internet-scale general data but are “ill-equipped” for agriculture, where competent reasoning depends on crop science, pest diagnosis, weed recognition, fruit maturity, symptom interpretation, and management reasoning that are largely absent from standard pretraining corpora (Yang et al., 5 Oct 2025).
1. Conceptual scope and historical placement
The suite is defined less as a single model than as an integrated stack for agricultural multimodal research. Its novelty lies in combining data generation, model specialization, and benchmark design inside one framework. The paper repeatedly argues that prior agricultural multimodal work was fragmented, small-scale, or weakly controlled, whereas AgriGPT-VL Suite couples supervision, curriculum, and evaluation in a single ecosystem.
This framing places the suite in a broader sequence of agricultural AI systems. AgroGPT showed that agricultural multimodal dialogue could be bootstrapped from vision-only datasets by constructing the 70k-sample AgroInstruct corpus and expert-tuning LLaVA- and Mipha-based backbones, but its emphasis was efficient conversational specialization rather than a unified corpus-model-benchmark stack (Awais et al., 2024). ShizishanGPT instead exemplified a tool-augmented agricultural QA architecture built around retrieval, search, a knowledge graph, and external predictive tools; it is suite-like in orchestration, but it does not establish a clear vision-language pipeline in the same sense (Yang et al., 2024).
A concise view of the AgriGPT-VL Suite is given below.
| Component | Primary role | Reported contents |
|---|---|---|
| Agri-3M-VL | Training corpus | 1M image-caption, 2M image-grounded VQA, 50K expert VQA, 15K GRPO |
| AgriGPT-VL | Agriculture-specialized VLM | Qwen2.5-VL-based model trained with text grounding, multimodal alignment, and GRPO |
| AgriBench-VL-4K | Multimodal evaluation | 2,018 open-ended QA and 1,858 image-grounded single-choice questions |
| AgriBench-13K | Text-only evaluation | Retained for language-only agricultural assessment |
2. Agri-3M-VL as the data foundation
Agri-3M-VL is presented as, to the authors’ knowledge, the largest agricultural vision-language corpus to date. It is built from raw agricultural image datasets spanning pests and diseases, insects, crops, weeds, and fruits, and the paper reports the source composition precisely (Yang et al., 5 Oct 2025).
| Source dataset | Images | Classes / categories |
|---|---|---|
| PlantVillage | 54,305 | 38 |
| Species196 | 6,878 | 166 |
| Insect Foundation | 317,128 | 38,867 |
| SelectDataset | 558,930 | 2,958 |
| Fruits-360 | 97,255 | 206 |
| Fresh-Rotten Fruit | 30,357 | 18 |
| Total | 1,064,853 | 42,253 |
The corpus includes an insect total of 324,006 images and 39,033 classes, and a fruit total of 157,969 images and 224 classes. The authors state that this nearly covers the “full agricultural visual landscape,” but that is a qualitative claim rather than a formally validated completeness guarantee.
Transformation from raw repositories into Agri-3M-VL is handled by a transferable, reusable multi-agent Data Generator with four stages. In caption generation, raw images are converted into structured descriptions emphasizing observable agronomic attributes such as crop growth stage, leaf color, lesion spread, pest morphology, fruit maturity, and external texture, yielding about 1M image-caption pairs. In instruction synthesis, strong external VLMs—explicitly Qwen2.5-VL 72B and GPT-4o—generate recognition questions, factual visual questions, reasoning-oriented questions, and interactive multimodal dialogues, producing approximately 2M VQA samples in open-ended and single-choice formats. The paper’s examples include questions such as “What species of weed is shown?” and “What is the likely impact of this weed on crop yield?”
The third stage, multi-agent refinement, uses roughly two VQA pairs per image and a loop among three agents: Feedback, Evaluation, and Rethinking. Feedback proposes revisions, Evaluation scores samples on dimensions such as factual consistency and image grounding, and Rethinking rewrites them with self-consistency checks. Samples iterate until preset thresholds are met, producing roughly 2M high-quality VQA pairs. Separately, GPT-4o verifies and polishes an additional 50K subset for supervised fine-tuning, and the authors construct a 15K GRPO dataset. The fourth stage, instruction filtering, removes irrelevant, generic, or hallucinated instructions. The paper contrasts a rejected question—“How many legs does a seven-spotted ladybug have?”—with a retained grounded prompt such as “What kind of insect is in the picture?”
The final corpus composition is therefore: 1M image-caption pairs, 2M image-grounded VQA pairs, 50K expert-level VQA instances, and a 15K GRPO reinforcement learning dataset. The paper further specifies that the 15K GRPO set contains 10K VQA and 5K single-choice questions. The modalities are strictly image plus text; there is no mention of time series, tabular data, or geospatial signals in the suite’s core training corpus. Annotation is synthetic-curated rather than purely human-labeled: large models generate captions and instructions, agents iteratively refine them, filtering removes weak grounding, and the expert subset is manually verified.
3. AgriGPT-VL model and curricular training
AgriGPT-VL itself is an agriculture-specialized VLM built by starting from Qwen2.5-VL. The visible paper text is deliberately more detailed about the training curriculum than about architectural internals. It explicitly refers to a vision encoder, LLM component, connector, and adapter layers, later combined with gradual unfreezing through LoRA. What can therefore be stated firmly is that AgriGPT-VL uses a pretrained Qwen2.5-VL vision-language backbone with connector/adapter modules mediating multimodal alignment; the exact vision encoder variant, projector design, tokenization strategy, image resolution, and fusion equations are not specified in the provided text (Yang et al., 5 Oct 2025).
The training curriculum is the paper’s central technical mechanism. Stage 1 is text-only domain grounding. Starting from Qwen2.5-VL, the authors perform continual pretraining on approximately 200K documents comprising about 2.2B tokens to inject agricultural terminology and background knowledge. This is followed by supervised instruction tuning on Agri-342K, inherited from earlier AgriGPT work. A held-out split of AgriBench-13K is used for early stopping and calibration. This stage is designed to establish agricultural language competence before any multimodal adaptation.
Stage 2 is curricular multimodal alignment, divided into three sub-stages. In Stage 2a, shallow alignment, the model is trained on the 1M image-caption pairs with both the vision encoder and the LLM fully frozen; only the connector and adapter layers are trained. The paper characterizes this as “captioning, vision frozen,” with the stated purpose of establishing a stable semantic bridge without disrupting pretrained text abilities. In Stage 2b, deep alignment, training moves to the 2M image-QA samples, which cover recognition, attributes, diagnosis, and basic multi-hop reasoning. At this point the model uses LoRA and gradually unfreezes the vision encoder and the LLM, moving from coarse image-text correspondence to full multimodal reasoning. The paper notes “two questions per image for cross-validation,” consistent with a broader emphasis on cross-consistency. In Stage 2c, GRPO optimization, the model undergoes reward-guided fine-tuning on the 15K GRPO dataset. The visible text does not define the full expansion of GRPO, but it does state that reward signals target image-text consistency, internal logic, and verifiable terminology.
A notable feature of the curriculum is its explicit attempt to preserve text-only capability while improving multimodal reasoning. The proposed mechanism is ordering: agricultural text grounding first, conservative multimodal bridge learning second, broader multimodal adaptation third, and reinforcement-style refinement last. The paper argues that this “text-first then easy-to-hard multimodal” sequence mitigates catastrophic forgetting. The visible text does not provide explicit loss functions, optimization equations for the supervised stages, or detailed hyperparameters such as learning rates, batch sizes, epochs, or compute budget.
4. AgriBench-VL-4K and the evaluation protocol
AgriBench-VL-4K is the suite’s dedicated benchmark for agricultural vision-language understanding. It contains two parts: 2,018 open-ended question-answer pairs and 1,858 image-grounded single-choice questions. Earlier in the paper, the single-choice split is described as having “two per image for cross-consistency,” indicating that each image is paired with two related discriminative questions. Open-ended questions are synthesized from structured captions of held-out images and cover recognition, symptom/mechanism analysis, management recommendations, and simple multi-step reasoning. The single-choice split is constructed with distractors mined from confusable taxa and co-occurring conditions, and stems and option templates are explicitly designed not to overlap with training prompts, to reduce prompt leakage (Yang et al., 5 Oct 2025).
Quality control is emphasized. The benchmark is strictly de-duplicated against all training splits, then human-reviewed. De-duplication is carried out at both image and text level: perceptual hashing or visual-feature similarity is used for images, while lexical or embedding similarity is used for question-answer strings. Items then undergo a two-pass human review by “agriculture-literate annotators,” checking factual correctness, image-grounded evidence, and ambiguity, with disagreements resolved by adjudication. The paper does not provide annotator counts, agreement statistics, or similarity thresholds.
Evaluation is multi-metric. For single-choice questions, the benchmark reports Acc and Acc. Acc is ordinary per-question single-choice accuracy. Acc is the stricter image-level cross-consistency metric: for an image paired with two single-choice questions, both must be answered correctly for the image to count as correct. For open-ended generation, the benchmark reports BLEU, METEOR, Rouge-1-f, Rouge-2-f, and Rouge-L-f. The paper interprets these as capturing terminology conformity, semantic coverage, and structural completeness.
In addition to automatic metrics, the suite uses an LLM-as-a-judge protocol based on JudgeLM. It performs blind pairwise comparisons between AgriGPT-VL and competing systems, swaps left/right order to reduce order bias, and averages the two outcomes. The visible text states that it reports “three preference metrics: WR (ties excluded),” but the excerpt is truncated before listing the other names or formulas. AgriBench-13K functions as the text-only complement to AgriBench-VL-4K, enabling explicit measurement of whether multimodal specialization degrades agricultural language ability.
5. Empirical performance, ablations, and generalization
The main reported results are strong on both the text-only and multimodal benchmarks. On AgriBench-13K, AgriGPT-VL achieves BLEU 10.84, Meteor 32.53, Rouge-1-f 27.73, Rouge-2-f 6.36, and Rouge-L-f 26.36, outperforming all compared general VLMs on the reported metrics. The second-best systems vary by metric: for example, InternVL-3-14B is second on BLEU, Rouge-1-f, and Rouge-L-f, while Qwen2.5-VL-72B-Instruct is second on Meteor (Yang et al., 5 Oct 2025).
On AgriBench-VL-4K, AgriGPT-VL is best on every reported metric. It reaches Acc 85.84% and Acc 74.17%, compared with InternVL-3-14B at 83.05% / 69.21% and Qwen2.5-VL-72B-Instruct at 81.70% / 67.49%. In open-ended generation it records BLEU 26.27, Meteor 47.55, Rouge-1-f 46.52, Rouge-2-f 20.09, and Rouge-L-f 43.81. The strongest baseline on these open-ended metrics is mostly Qwen2.5-VL-72B-Instruct, but with substantially lower scores: BLEU 15.41, Meteor 41.38, Rouge-1-f 39.14, Rouge-2-f 13.25, and Rouge-L-f 36.63. The LLM-as-a-judge analysis is described qualitatively rather than numerically in the visible text: AgriGPT-VL achieves consistently higher pairwise win rates against most strong baselines and remains competitive even against top large models, but exact percentages are not reproduced.
The ablation study clarifies the effect of the curriculum. Starting from Base (Qwen2.5-VL-7B), the model scores 77.20% Acc, 60.32% Acc, BLEU 13.42, Meteor 38.24, Rouge-1-f 35.52, Rouge-2-f 10.78, and Rouge-L-f 32.73. Adding shallow alignment raises performance to 78.23% / 62.47%, with BLEU 15.54 and Rouge-L-f 37.95; Meteor decreases slightly to 36.47, but descriptive metrics improve. Adding deep alignment yields 81.18% / 66.67%, BLEU 21.68, Meteor 44.38, Rouge-1-f 43.04, Rouge-2-f 15.62, and Rouge-L-f 40.37. Adding GRPO produces the full model at 85.84% / 74.17%, BLEU 26.27, Meteor 47.55, Rouge-1-f 46.52, Rouge-2-f 20.09, and Rouge-L-f 43.81. The paper’s interpretation is correspondingly staged: shallow alignment improves lexical and descriptive consistency, deep alignment is the main driver of cross-modal understanding and reasoning, and GRPO most strongly boosts factual faithfulness and robustness, particularly on Acc.
The suite also includes a generalization check outside agriculture. Comparing base Qwen2.5-VL against AgriGPT-VL on six public benchmarks shows mostly preserved broad capability: MMLU 0.6783 vs 0.6741, ARC 0.9043 to 0.8462, OpenBookQA 0.8501 to 0.8412, MMBench 0.8398 to 0.8312, SEED-Bench 0.7565 to 0.7574, and MMMU 0.4329 to 0.4599. The paper interprets this as evidence that the text-first curriculum mitigates forgetting while allowing some transfer of visual reasoning gains beyond agriculture.
6. Relation to adjacent agricultural multimodal systems, limitations, and significance
Within agricultural multimodal research, AgriGPT-VL Suite occupies the role of an integrated image-text foundation stack. Other systems address adjacent but different axes. AgroBench is an expert-annotated evaluation benchmark spanning seven agricultural topics and emphasizing fine-grained disease, pest, weed, management, machinery, and traditional-practice QA; it is complementary because it probes expert correctness rather than training-time supervision (Shinoda et al., 28 Jul 2025). AgriVLN, together with later extensions T-araVLN and SUM-AgriVLN, opens a distinct embodied line centered on agricultural vision-and-language navigation, instruction refinement, and spatial memory rather than static image-grounded QA (Zhao et al., 10 Aug 2025, Zhao et al., 8 Sep 2025, Zhao et al., 16 Oct 2025). AgriGPT-Omni extends the AgriGPT family from image-text to multilingual speech-vision-text interaction, reusing the same ecosystemal logic of data, curriculum, and benchmark while adding agricultural speech as a first-class modality (Yang et al., 11 Dec 2025). AgroVG specializes in agricultural visual grounding as generalized set prediction, and AgroNVILA explicitly targets Ground/UAV/Satellite scale confusion through multi-view training and perception-reasoning decoupling (Li et al., 21 May 2026, Zhang et al., 15 Mar 2026). AgriWorld moves in a different direction again, emphasizing tool execution, geospatial querying, simulation, and verifiable code-based agricultural reasoning rather than a pure VLM stack (Zhang et al., 17 Feb 2026).
The suite’s limitations are also explicit or inferable from what is and is not reported. Much of Agri-3M-VL is synthesized using GPT-4o and Qwen2.5-VL 72B, so corpus quality depends partly on upstream model behavior. The authors attempt to control this with multi-agent refinement, filtering, and manual verification, but residual noise is not quantified (Yang et al., 5 Oct 2025). The visual corpus is assembled from existing datasets concentrated in particular categories, so source biases in image style, geography, or crop coverage remain plausible. The paper reports no formal inter-annotator agreement for the training corpus, no exact thresholds for the refinement agents, and no detailed taxonomy of all question categories beyond examples and broad types. Benchmark leakage is addressed through de-duplication and human review, but no formal memorization audit beyond that process is reported. GRPO is described only partially in the visible text: rewards are specified at a high level, but exact policy objective details, sampling strategy, KL treatment, and optimization equations are deferred to appendices not included in the supplied content. The suite is also restricted to image plus text; time series, tabular agronomic records, geospatial rasters, and speech are outside its core scope.
Its significance therefore lies in demonstrating that agricultural specialization can be organized as a full multimodal research stack rather than as an isolated fine-tuned model. The suite combines million-scale synthetic-curated supervision, a staged curriculum that tries to preserve domain language competence, and a benchmark that measures not only answer accuracy but grounded consistency and judge-preference quality. In that sense, AgriGPT-VL Suite marks a transition from narrow agricultural assistants toward reusable, domain-specific multimodal infrastructure for research, reproducibility, and low-resource deployment.