T-araVLN: Agricultural VLN with LLM Translation

• T-araVLN is an agricultural Vision-and-Language Navigation approach that employs an LLM-based Instruction Translator to clean and refine noisy spoken instructions.
• The method decouples the translation of natural language from navigation, removing irrelevant content and clarifying high-level goals versus low-level commands.
• Quantitatively, T-araVLN improves Success Rate from 0.47 to 0.63 and reduces Navigation Error from 2.91 m to 2.28 m on the A2A benchmark.

Searching arXiv for the primary paper and a few related works named in the provided data. T-araVLN, short for Translator for Agricultural Robotic Agents on Vision-and-Language Navigation, is a method for agricultural Vision-and-Language Navigation (VLN) that augments the AgriVLN navigation architecture with an LLM-based Instruction Translator. It is designed for field robots that must move from a start pose to a target pose in agricultural environments by following natural-language instructions that are spoken-style, noisy, and often imperfect. Within the A2A benchmark introduced by AgriVLN, T-araVLN rewrites an original instruction into a refined and precise form before navigation, and on the full A2A benchmark it improves Success Rate from 0.47 to 0.63 while reducing Navigation Error from 2.91 m to 2.28 m, yielding state-of-the-art performance in the agricultural domain (Zhao et al., 8 Sep 2025).

1. Problem setting and domain characteristics

T-araVLN is formulated for navigation episodes whose input consists of a natural-language instruction

$W=\langle w_1,w_2,\dots,w_L\rangle$

and, at each time step $t$, a front-facing RGB image $I_t$. The output is a low-level action selected from

$\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$

with the goal of moving a field robot from a start pose to a target pose in an agricultural environment by grounding language in visual observations from fields, orchards, and greenhouses (Zhao et al., 8 Sep 2025).

The motivating difficulty is that agricultural VLN combines linguistic irregularity with visual ambiguity. The instructions are described as spoken-style utterances from agricultural workers and may contain fillers, redundant phrases, grammatical mistakes, ambiguous references, and mixtures of high-level goals with low-level motion hints. The environments themselves are visually repetitive, with rows of crops, similar plants, and long corridors between ridges, and they also exhibit outdoor variability through lighting changes, foliage occlusion, and seasonal appearance changes. A literal reading of such instructions can therefore become misaligned with what the robot can actually perceive and execute (Zhao et al., 8 Sep 2025).

The method is positioned as a response to limitations in the original AgriVLN/A2A setting. AgriVLN and the A2A benchmark are described as the first to bring VLN into agriculture, but the base model is reported to handle simple instructions more effectively than complex, noisy ones. In particular, it must jointly learn navigation and language cleanup inside a single model, even though the A2A instructions are deliberately noisy and error-prone to mimic real workers. This motivates a decoupled design in which instruction quality is improved first, before the navigation policy processes the text (Zhao et al., 8 Sep 2025).

2. Overall framework and formal pipeline

Conceptually, T-araVLN is AgriVLN navigation policy + an LLM-based Instruction Translator. For each episode, the system first takes the original instruction $W$, then passes it to an Instruction Translator $\mathcal{T}$ together with a prompt $P_{\mathcal{T}}$, and finally feeds the translated instruction $W'$ into the AgriVLN base model $\mathcal{D}$ along with the RGB image sequence. The translator outputs both a reasoning trace $R$ and a refined instruction $t$0: $t$1 The navigation model then predicts an action sequence from the translated instruction and observations: $t$2 At each step, $t$3 fuses visual and language features to choose $t$4 (Zhao et al., 8 Sep 2025).

Episode termination follows three conditions. The episode ends when $t$5, or when the predicted action subsequence $t$6 deviates from the ground-truth label sequence $t$7 by a defined threshold $t$8, or when $t$9 reaches the maximum allowed number of steps. The deviation rule is inherited from AgriVLN (Zhao et al., 8 Sep 2025).

The paper emphasizes that the novelty lies not in a new policy architecture but in a pre-policy translation step that transforms the effective instruction from $I_t$0 to $I_t$1. The underlying policy can be viewed conceptually as

$I_t$2

where $I_t$3 encodes $I_t$4 and history. This arrangement keeps the multimodal navigation core intact while changing the linguistic input distribution presented to it. This suggests that the method targets instruction quality as the primary bottleneck rather than low-level action modeling (Zhao et al., 8 Sep 2025).

3. Instruction Translator module

The Instruction Translator $I_t$5 is described as a large-language-model-based natural language processing module used as-is via API rather than trained on A2A. Its role is to reformulate instructions from “noisy, mistaken, spoken-style, full of irrelevant details” into language that is “formal, concise, precise, easy-to-follow for a robot.” The output $I_t$6 is intended not merely as a paraphrase but as a cleaned instruction that removes inessential content, corrects errors, separates high-level from low-level information, and regularizes directional and motion expressions (Zhao et al., 8 Sep 2025).

The prompt template is given as

$I_t$7

where $I_t$8 describes the rewriting task, $I_t$9 specifies the expected input-output format, and $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$0 are five translation principles, each accompanied by a one-shot example $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$1. The five principles are Inessential Removal, Errors Revision, High-Low Separation, Representational Rotation, and Representational Movement (Zhao et al., 8 Sep 2025).

These principles define the translator’s behavior. Inessential Removal deletes filler words, chit-chat, redundant phrases, and non-instructional content, preserving only information relevant to navigation decisions. Errors Revision corrects obvious linguistic errors, including wrong crop names, mis-stated directions, and inconsistent references when possible from context, while normalizing inconsistent terminology. High-Low Separation disentangles high-level goals from low-level motion hints and rephrases them into clearer step-by-step navigational descriptions. Representational Rotation makes orientation instructions explicit and unambiguous, and Representational Movement normalizes vague movement expressions into clearer descriptions such as moving forward along a row until a junction (Zhao et al., 8 Sep 2025).

The translator’s architecture is intentionally external to the learned navigation system. The paper lists GPT‑4.1, Claude‑3.7‑sonnet, and DeepSeek‑r1 as the LLMs used via API. There is no supervised loss $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$2 over A2A; the translation process is treated as the black-box mapping

$\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$3

All learning remains inside the AgriVLN navigation model. The reasoning trace $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$4 is generated by the LLM and used to guide translation, but it is not used by the navigation model itself (Zhao et al., 8 Sep 2025).

4. Vision–language integration and training methodology

T-araVLN reuses the AgriVLN base model for multimodal processing. On the language side, the system supplies $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$5 instead of $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$6, and AgriVLN encodes this text into embeddings and subtask representations. On the visual side, each front-facing RGB image $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$7 is processed by a visual backbone to produce visual features. AgriVLN then maintains a Subtask List and a decision mechanism that fuses the current subtask representation, current visual features, and possibly historical state in order to choose the next action $\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$8. The multimodal decision can be summarized conceptually as

$\{FORWARD,\ LEFT\ ROTATE,\ RIGHT\ ROTATE,\ STOP\},$9

T-araVLN does not alter these fusion networks; it modifies the language input so that the existing network can interpret it more reliably (Zhao et al., 8 Sep 2025).

The dataset is the A2A benchmark from AgriVLN. It contains agricultural environments including fields, greenhouses, and orchards; deliberately noisy spoken-style instructions with errors and informal phrasing; expert trajectories associated with each instruction; front-facing RGB images rendered along paths; and ground-truth low-level action sequences $W$0 (Zhao et al., 8 Sep 2025).

Training of the navigation model follows the same training scheme as AgriVLN. Using instruction–path pairs $W$1, the base model is trained with supervised learning to minimize action prediction error: $W$2 There is no additional translation loss and no reinforcement-learning component introduced in the paper: $W$3 The translator is applied at both training time and test time to produce $W$4 for all instructions (Zhao et al., 8 Sep 2025).

The paper does not describe explicit curriculum learning, synthetic augmentation, or instruction simplification beyond translation itself. It characterizes the key augmentation effect as the translation process, which effectively produces a cleaner instruction dataset on the fly. A plausible implication is that T-araVLN changes the supervision signal received by the navigation model without changing the action labels or visual observations (Zhao et al., 8 Sep 2025).

5. Evaluation protocol and quantitative performance

Evaluation is conducted on A2A using three partitions: a two-subtask portion, a $W$5-subtask portion, and full A2A. The baselines listed are Random, Fixed, SIA-VLN, DILLM-VLN, AgriVLN, and Human. The T-araVLN variants are defined by translator choice: $W$6-araVLN$W$7 with Claude‑3.7‑sonnet, $W$8-araVLN$W$9 with DeepSeek‑r1, and $\mathcal{T}$0-araVLN$\mathcal{T}$1 with GPT‑4.1 (Zhao et al., 8 Sep 2025).

The primary metrics are Success Rate (SR) and Navigation Error (NE). With $\mathcal{T}$2 episodes, ground-truth goal position $\mathcal{T}$3, predicted final position $\mathcal{T}$4, a distance metric $\mathcal{T}$5, and a success threshold $\mathcal{T}$6, SR is

$\mathcal{T}$7

and NE is

$\mathcal{T}$8

For translation analysis, the paper additionally uses BERTScore$\mathcal{T}$9 between the original instruction $P_{\mathcal{T}}$0 and translated instruction $P_{\mathcal{T}}$1, where a higher value indicates a more conservative translation and a lower value indicates a more aggressive one (Zhao et al., 8 Sep 2025).

On the whole A2A benchmark, the reported results are: SIA-VLN with SR $P_{\mathcal{T}}$2, NE $P_{\mathcal{T}}$3; DILLM-VLN with SR $P_{\mathcal{T}}$4, NE $P_{\mathcal{T}}$5; AgriVLN with SR $P_{\mathcal{T}}$6, NE $P_{\mathcal{T}}$7; and T-araVLN$P_{\mathcal{T}}$8 with SR $P_{\mathcal{T}}$9, NE $W'$0, and BERTScore$W'$1 $W'$2. Relative to AgriVLN, this corresponds to an SR gain of $W'$3 and an NE reduction of $W'$4 m (Zhao et al., 8 Sep 2025).

On the two-subtask portion, AgriVLN attains SR $W'$5 and NE $W'$6, whereas T-araVLN$W'$7 reaches SR $W'$8 and NE $W'$9. The Human upper bound is SR $\mathcal{D}$0 and NE $\mathcal{D}$1. The paper states that T-araVLN approaches human performance on simpler two-subtask instructions. On the $\mathcal{D}$2-subtask portion, gains are smaller but still positive, moving from SR $\mathcal{D}$3 to $\mathcal{D}$4 and from NE $\mathcal{D}$5 to $\mathcal{D}$6, indicating continued benefit under greater compositional complexity (Zhao et al., 8 Sep 2025).

6. Ablations, qualitative analysis, and research positioning

The ablation over LLM choice compares translator conservativeness and navigation performance. T-araVLN$\mathcal{D}$7 with Claude‑3.7‑sonnet obtains SR $\mathcal{D}$8, NE $\mathcal{D}$9, and BERTScore $R$0, which the paper describes as the most conservative translation. T-araVLN$R$1 with DeepSeek‑r1 obtains SR $R$2, NE $R$3, and BERTScore $R$4, described as the most aggressive translation. T-araVLN$R$5 with GPT‑4.1 yields the best result, SR $R$6, NE $R$7, and BERTScore $R$8. The paper interprets this as a balance: overly conservative translation leaves too much noise, whereas overly aggressive translation may remove essential semantics (Zhao et al., 8 Sep 2025).

The ablation over translation principles starts from the baseline without a translator, SR $R$9 and NE $t$00. Using only one principle at a time with GPT‑4.1 gives: Inessential Removal, SR $t$01, NE $t$02, BERTScore $t$03; Errors Revision, SR $t$04, NE $t$05, BERTScore $t$06; High-Low Separation, SR $t$07, NE $t$08, BERTScore $t$09; Representational Rotation, SR $t$10, NE $t$11, BERTScore $t$12; and Representational Movement, SR $t$13, NE $t$14, BERTScore $t$15. Combining all five principles produces the full T-araVLN result, SR $t$16, NE $t$17, and BERTScore $t$18. The paper explicitly notes that every principle individually improves over the baseline and that Representational Movement is especially beneficial (Zhao et al., 8 Sep 2025).

The qualitative example described in the paper contrasts an original instruction containing several noises and mistakes with a translated instruction that is substantially shorter and more explicit. The reported pattern is that irrelevant temporal references and ambiguous comparisons are removed, the target object is specified more clearly, and the action structure becomes easier to follow. The accompanying visualization shows the baseline AgriVLN model veering off under ambiguous language, whereas T-araVLN follows a path more closely aligned with ground truth and stops near the correct position. This suggests that translation is particularly helpful when redundant context or slight linguistic errors would otherwise alter target or directional grounding (Zhao et al., 8 Sep 2025).

Within the broader VLN landscape, the paper situates T-araVLN against conventional benchmarks such as R2R, R4R, TOUCHDOWN, and AerialVLN, and against translator-oriented methods such as VLN-Trans and MTST. The stated distinction is that earlier training-based translators are tied to particular error distributions, whereas A2A contains richer and more diverse errors. T-araVLN therefore uses a general-purpose LLM guided by explicit principles and prompt engineering rather than training a translator on A2A. In this positioning, AgriVLN is the first agricultural VLN agent that directly consumes noisy instructions, and T-araVLN is the layer on top of it that improves instruction quality without redesigning the navigation policy (Zhao et al., 8 Sep 2025).

The strengths stated in the paper are significant performance gains, robust handling of messy real-world language, and modularity. The listed limitations are dependence on external LLM APIs, lack of end-to-end coupling between translator and navigator, evaluation only on A2A agricultural environments, and more modest gains on very complex instructions. The future directions implied by the work include broader agricultural scenarios, real robot deployment, tighter integration between translator and navigator, and richer grounding with maps, 3D structure, depth, or semantic maps. Taken together, T-araVLN is presented as a modular VLN system for agricultural robots in which instruction translation serves as the decisive interface between informal human language and robot-executable navigation (Zhao et al., 8 Sep 2025).