GuirlVG: Efficient RL for GUI Visual Grounding

• GuirlVG is a reinforcement learning framework that employs rule-based reinforcement fine-tuning to localize GUI elements based on natural language instructions.
• The framework introduces an Adversarial KL Factor to stabilize reward optimization, significantly enhancing spatial accuracy and data efficiency.
• Systematic ablation studies reveal that GuirlVG outperforms traditional supervised methods with only a fraction of the training samples, ideal for rapid UI adaptation.

GuirlVG is a reinforcement learning (RL) framework for data-efficient graphical user interface visual grounding (GUI-VG), which is the task of localizing screen coordinates or bounding boxes of actionable elements in a screenshot in response to a natural-language instruction (e.g., “click the ‘Save’ icon”). GuirlVG addresses the inefficiency and scalability limitations of supervised fine-tuning (SFT) approaches on large multimodal LLMs (MLLMs), providing empirical advances in reinforcement-based GUI-VG through a formal decomposition of rule-based reinforcement fine-tuning (RFT), a novel stabilization method, and systematic ablations demonstrating significant gains in data efficiency and model performance (Kang et al., 6 Aug 2025).

1. Problem Definition and Motivation

GUI-VG requires mapping free-form user intents to actionable targets on complex screen images. While traditional approaches employ SFT using large datasets of screenshot–instruction–bounding box triples, this methodology incurs substantial annotation overhead (up to tens of millions of samples) and repeat retraining for every new MLLM backbone. As modern MLLMs increasingly cover GUI-relevant domains via pretraining, the value of exhaustive post-hoc SFT is increasingly questioned. Rule-based reinforcement fine-tuning (RFT) offers several intrinsic advantages:

• Data efficiency: RFT attains strong generalization with only thousands of examples.
• Exploration-driven reasoning: The model receives feedback on semantically correct outputs, even if they diverge from reference tokens, facilitating hypothesis-driven learning.
• No external critic: Rewards depend on deterministic rule checks (such as format correctness and target accuracy), not on learned reward models or human preferences.

Despite these strengths, naive RFT underperforms large-scale SFT, necessitating methodological analysis and stabilization (Kang et al., 6 Aug 2025).

2. Formalization of Rule-Based Reinforcement Fine-Tuning

GuirlVG builds on the Group Relative Policy Optimization (GRPO) paradigm. The current policy $\pi_\theta$ (an MLLM with LoRA adapters) and a frozen reference policy $\pi_\mathrm{ref}$ are defined over multimodal input states $s$ (screenshot and instruction). For a given $s$, $N$ candidate outputs $\{o_i\}$ are sampled. Each $o_i$ is assigned a rule-based reward $r_i = r_i^f + r_i^a$ where:

• $r_i^f \in [0,1]$: format reward, measuring adherence to required output tags (e.g., presence of > … and <answer>…</answer> tokens).
• $r_i^a \in [0,1]$: accuracy reward, determined by spatial correctness (e.g., point-in-bounding-box or intersection-over-union (IoU) tests).

Group-level statistics are computed:

• Mean $\pi_\mathrm{ref}$0 and standard deviation $\pi_\mathrm{ref}$1 across $\pi_\mathrm{ref}$2,
• Standardized advantage for each $\pi_\mathrm{ref}$3: $\pi_\mathrm{ref}$4.

The GRPO objective for input $\pi_\mathrm{ref}$5 is:

$\pi_\mathrm{ref}$6

where $\pi_\mathrm{ref}$7 is the Kullback-Leibler divergence and $\pi_\mathrm{ref}$8 is a regularization coefficient. The loss to be minimized is:

$\pi_\mathrm{ref}$9

(Kang et al., 6 Aug 2025).

3. Adversarial KL Factor and Stabilization

A central innovation in GuirlVG is the introduction of the Adversarial KL Factor to stabilize RFT and prevent reward over-optimization (reward hacking). The base KL regularization coefficient $s$0 is dynamically scaled by a per-sample factor $s$1, with $s$2 the maximum achievable reward (here, $s$3). The modified per-sample objective is:

$s$4

and the overall group loss is:

$s$5

The gradient is given by:

$s$6

In this formulation, high-reward samples incur a larger penalty for divergence from the reference, anchoring the policy and suppressing reward exploitation. This mechanism is critical for stable and reliable RFT in the GUI-VG domain (Kang et al., 6 Aug 2025).

4. Implementation, Training Configurations, and Ablative Analysis

Model backbone: Qwen2.5-VL with vision encoder frozen and LoRA adapters (rank=64, $s$7) modulating visual-language connections.

Key training parameters:

• Group size $s$8, batch size $s$9
• Learning rate $s$0 (LoRA), $s$1 (Full-FT)
• Number of training steps: $s$2 ($s$3K samples for final run)
• Best $s$4 found to be $s$5 with Adversarial KL Factor

Reward and output format:

• Point prediction with in-bounding-box binary reward was most effective.
• Soft Reward Function for format (+0.7% improvement over strict JSON): partial credit for presence of tags and coordinate fields.

Major ablations:

• Strict vs. Soft Reward: 79.2% $s$6 79.9%
• Reward Type ([email protected] $s$7 In-bbox): 79.9% $s$8 83.4%
• KL tuning and Adversarial KL: 83.4% ($s$9) $N$0 87.4% (+Adversarial KL, $N$1)
• Full-FT vs. LoRA-FT: 87.5% vs. 87.4% (with LoRA 25$N$2 faster)
• Group/Batch size: (6,4) optimal; (8,4) reduced by ~3.5%
• Adding prompt resolution at test time marginally improved accuracy (+0.6%) (Kang et al., 6 Aug 2025).

5. Experimental Results and Comparative Performance

GuirlVG demonstrates substantial data efficiency and accuracy improvements over SFT-based models such as OS-Atlas (7B). With only 5.2K training samples (versus 13.6M for SFT), results include:

ScreenSpot Benchmark:

Method	#Train	Avg. Acc. (%)	Δ over SFT
OS-Atlas (7B)	13.6M	81.0	—
GuirlVG (2K)	2K	88.0	+7.0
GuirlVG (5.2K)	5.2K	88.7	+7.7

ScreenSpotV2:

Method	#Train	Avg. Acc. (%)	Δ over SFT
OS-Atlas (7B)	13.6M	84.1	—
GuirlVG (2K)	2K	90.9	+6.8
GuirlVG (5.2K)	5.2K	91.9	+7.8

ScreenSpot-Pro (Full-desktop):

Method	#Train	Avg. Acc. (%)	Δ over SFT
OS-Atlas (7B)	13.6M	18.9	—
GuirlVG (2K)	2K	31.6	+12.7
GuirlVG (5.2K)	5.2K	36.1	+17.2

All improvements are statistically significant ($N$3 with bootstrap resampling). This suggests rule-based RFT, when optimally configured, can substantially exceed large-scale SFT with less than 0.05% of the data (Kang et al., 6 Aug 2025).

6. Contextualization and Significance in GUI Automation

GuirlVG advances sample-efficient grounding in GUI automation. RFT methods, when properly stabilized and ablated, can replace SFT for GUI-VG tasks under constraints of annotation scarcity, compute, or rapid domain adaptation. The system’s reliance on simple, verifiable reward signals makes it robust to distributional shifts and ongoing MLLM updates.

A plausible implication is that, as vision-LLMs increasingly cover UI-domain distributions during pretraining, the role of carefully engineered, small-sample RL pipelines such as GuirlVG will become more prominent for specialized GUI autonomy. This shift is especially pertinent as high-quality, large-scale UI datasets remain expensive to curate (Kang et al., 6 Aug 2025).

GuirlVG exemplifies a broader trend toward data-efficient RL-based adaptation for real-world MLLM applications, suggesting future research may focus further on scalable reward engineering and transferability across rapidly-evolving UI environments.