Valley-to-Peak GUI Grounding

• Valley-to-Peak (V2P) is a GUI grounding framework that reshapes patch-attention by creating low-attention valleys for background regions and high-attention peaks at target centers.
• It integrates a gradient-weighted suppression loss with a Fitts’ Law-inspired, size-adaptive Gaussian peak model to focus click prediction on actionable UI elements.
• Empirical evaluations demonstrate significant accuracy improvements on benchmarks, particularly for small, dense, and overlapping UI targets.

Valley-to-Peak (V2P) is a training framework for GUI grounding that calibrates the attention distribution of a vision-LLM over a screenshot so that irrelevant regions become low-attention “valleys” and the actionable point of the target UI element becomes a high-attention “peak.” Implemented on top of the coordinate-free GUI-Actor formulation, V2P does not alter inference—click prediction still selects the highest-attention patch—but changes supervision by combining explicit background suppression with a Fitts’ Law-inspired, size-adaptive Gaussian prior over the target element (Chen et al., 11 Jan 2026).

1. Conceptual definition and problem setting

V2P is motivated by two failure modes in attention-based GUI grounding. The first is Background Distraction: prior attention losses reward attention on the target but do not penalize attention on background patches, so softmax normalization permits non-target regions to absorb probability mass and can shift the attention peak away from the true target. The second is Center–Edge Confusion: supervision based on uniform bounding boxes treats all pixels inside the box equally, so the model does not distinguish the center of a widget from its edges, which leads to diffuse or edge-biased attention and imprecise clicks, especially for small or overlapping elements (Chen et al., 11 Jan 2026).

The task setting is standard GUI grounding. A model receives a screenshot $I$ from a desktop, mobile, or web interface and a natural-language instruction $q$, such as “Click the Send button,” and must output a click location. In V2P, this location is not generated as explicit coordinate tokens. Instead, grounding is represented as a normalized attention map over image patches, and the training objective reshapes that map to follow a coarse-to-fine “valley-to-peak” pattern: suppressing non-target regions first, then sharpening the center of the target (Chen et al., 11 Jan 2026).

An earlier preprint reported the same core formulation and described V2P as moving “from background suppression to center peaking for robust GUI grounding task,” with earlier benchmark results of $92.3\%$ on ScreenSpot-v2 and $50.5\%$ on ScreenSpot-Pro (Chen et al., 19 Aug 2025). The later version reports improved results and a more detailed analysis of failure attribution and generalization (Chen et al., 11 Jan 2026).

2. Placement within coordinate-free GUI grounding

V2P is implemented on top of GUI-Actor. GUI-Actor introduces a special output token $\mathtt{<ACTOR>}$; at the top transformer layer, the hidden state of this token is denoted $h_{\mathtt{<ACTOR>}}$, while the vision encoder produces patch features $\{v_i\}_{i=1}^M$ over $M = H \times W$ patches. A self-attention layer refines visual features,

$$\tilde{v}_1, \dots, \tilde{v}_M = \mathrm{SelfAttn}(v_1, \dots, v_M),$$

after which two MLPs project text and vision into a shared space,

$$z = \mathrm{MLP}_T(h_{\mathtt{<ACTOR>}}), \qquad z_i = \mathrm{MLP}_V(\tilde{v}_i).$$

Attention logits and the patch-level attention distribution are then

$q$0

where $q$1 is the embedding dimension (Chen et al., 11 Jan 2026).

Within this formulation, V2P is strictly a training-time modification. The inference mechanism remains unchanged: the patch with highest attention is still interpreted as the click position. This is a common point of confusion. V2P is not a new decoding rule, and it is not a coordinate-regression alternative inserted at inference. Its contribution lies in replacing uniform target supervision with two attention-shaping losses: a Suppression Attention Loss for background patches and a Fitts-Gaussian Action Attention Loss for target patches (Chen et al., 11 Jan 2026).

3. The “valley” component: background suppression

The valley component begins by defining which patches are background. If patch $q$2 corresponds to region $q$3 in the screenshot and the ground-truth element has bounding box $q$4, then the background set is

$q$5

These are the patches that should receive low attention (Chen et al., 11 Jan 2026).

A simple background penalty is the total attention mass on $q$6,

$q$7

The paper then develops a gradient-weighted variant. Let $q$8 denote attention on the true target patch $q$9, with logits $92.3\%$0. For a non-target patch $92.3\%$1,

$92.3\%$2

This shows that increasing a non-target logit decreases target attention, and that $92.3\%$3 measures how strongly patch $92.3\%$4 interferes with the target. The resulting Suppression Attention Loss is defined as

$92.3\%$5

The interpretation given in the paper is that background patches that most strongly compete with the target should be penalized more heavily. This suggests that suppression is not merely a sparsity prior; it is an interference-aware redistribution of attention mass away from harmful background regions (Chen et al., 11 Jan 2026).

Functionally, this component “carves attention valleys” outside the target box. In cluttered GUIs, where many non-target elements are visually salient or semantically similar, that explicit negative supervision is intended to reduce drift and eliminate spurious high-attention regions before finer localization is attempted.

4. The “peak” component: Fitts-Gaussian center peaking

The peak component addresses the mismatch between uniform box labels and actual click behavior. V2P adopts Fitts’ Law as a behavioral prior,

$92.3\%$6

with the interpretation that larger targets tolerate more pointing error and smaller targets require more precise clicks. Rather than treating the entire box as equally positive, V2P models click likelihood as a 2D Gaussian centered on the element, with variance proportional to target size (Chen et al., 11 Jan 2026).

For a ground-truth box $92.3\%$7, the centroid is

$92.3\%$8

and width and height are

$92.3\%$9

The standard deviations are set proportionally to size,

$50.5\%$0

and the Gaussian density is

$50.5\%$1

with $50.5\%$2 (Chen et al., 11 Jan 2026).

Because the model operates over patches rather than pixels, the continuous Gaussian is integrated over each patch region $50.5\%$3,

$50.5\%$4

Using the axis-aligned decomposition into normal CDFs, these masses are normalized into a target distribution

$50.5\%$5

and the Action Attention Loss is the KL divergence

$50.5\%$6

This yields center-biased supervision that decays smoothly toward the edges and adapts automatically to element size: larger elements produce broader peaks, while smaller elements induce sharper peaks (Chen et al., 11 Jan 2026).

A frequent misconception is that this is simply Gaussian label smoothing. In V2P, the Gaussian is tied to an HCI prior about interaction tolerance, is integrated at patch level, and is paired with explicit negative supervision outside the target. The intended effect is not only denser supervision, but disambiguation between center and edge patches in dense professional interfaces where edge-biased attention can lead to systematic misclicks.

5. Optimization procedure and implementation details

The full training objective combines language modeling with the two attention-shaping terms,

$50.5\%$7

Here, $50.5\%$8 is the standard next-token prediction loss, while $50.5\%$9 and $\mathtt{<ACTOR>}$0 balance background suppression and peak shaping. The paper characterizes optimization as a coarse-to-fine “Valley-to-Peak” paradigm: the valley phase pulls attention mass out of distractor regions, and the peak phase sharpens the distribution within the target region around the Gaussian center (Chen et al., 11 Jan 2026).

At each training step, the model computes attention logits $\mathtt{<ACTOR>}$1 and attention weights $\mathtt{<ACTOR>}$2, identifies the background set $\mathtt{<ACTOR>}$3, constructs the Gaussian target distribution $\mathtt{<ACTOR>}$4 from the bounding box and Gaussian factor, computes the KL loss, computes the gradient-weighted suppression loss, and backpropagates the combined objective. V2P uses Qwen2.5-VL-Instruct 7B and 3B backbones, adds one extra self-attention layer over patches to enhance spatial coherence before patch attention is computed, and follows GUI-Actor’s training recipe with approximately $\mathtt{<ACTOR>}$5M elements across approximately $\mathtt{<ACTOR>}$6M screenshots from diverse GUI datasets (Chen et al., 11 Jan 2026).

The Gaussian factor is a sensitive hyperparameter. The reported sensitivity study finds best ScreenSpot-Pro performance near $\mathtt{<ACTOR>}$7, while larger $\mathtt{<ACTOR>}$8 values broaden the peak and degrade performance by introducing label noise through background inclusion. The paper also analyzes element size relative to patch area $\mathtt{<ACTOR>}$9, defining Small as $h_{\mathtt{<ACTOR>}}$0, Medium as $h_{\mathtt{<ACTOR>}}$1, and Large as $h_{\mathtt{<ACTOR>}}$2 for stratified evaluation (Chen et al., 11 Jan 2026).

6. Empirical behavior, generalization, and limitations

With a Qwen2.5-VL-7B backbone, V2P-7B achieves $h_{\mathtt{<ACTOR>}}$3 accuracy on ScreenSpot-v2 and $h_{\mathtt{<ACTOR>}}$4 on ScreenSpot-Pro. On ScreenSpot-v2, GUI-Actor-7B reaches $h_{\mathtt{<ACTOR>}}$5, so the gain is small; the paper attributes this to ScreenSpot-v2 being dominated by large elements for which coarse localization is often sufficient. On ScreenSpot-Pro, GUI-Actor-7B reaches $h_{\mathtt{<ACTOR>}}$6, so V2P-7B improves by $h_{\mathtt{<ACTOR>}}$7 absolute points, or $h_{\mathtt{<ACTOR>}}$8 relative improvement, and exceeds strong baselines including UI-TARS-72B at $h_{\mathtt{<ACTOR>}}$9 and JEDI-7B at $\{v_i\}_{i=1}^M$0 (Chen et al., 11 Jan 2026).

Ablation results localize the gain. On ScreenSpot-Pro, full V2P-7B reaches $\{v_i\}_{i=1}^M$1; removing Fitts-Gaussian Peak Modeling reduces performance to $\{v_i\}_{i=1}^M$2, and removing both Fitts-Gaussian Peak Modeling and Suppression Attention reduces it to $\{v_i\}_{i=1}^M$3. Error attribution on 182 cases that V2P-7B corrects relative to GUI-Actor-7B assigns $\{v_i\}_{i=1}^M$4 of improvements to correcting Background Distraction, $\{v_i\}_{i=1}^M$5 to fixing Center-Edge Confusion, and the remaining $\{v_i\}_{i=1}^M$6 to other attention improvements. This directly links the dual-loss design to the intended failure modes (Chen et al., 11 Jan 2026).

The strongest gains appear on small elements. For Small targets on ScreenSpot-Pro, GUI-Actor-7B improves from $\{v_i\}_{i=1}^M$7 to $\{v_i\}_{i=1}^M$8; on ScreenSpot-v2, small-element performance rises from $\{v_i\}_{i=1}^M$9 to $M = H \times W$0. For Medium elements on ScreenSpot-Pro, performance improves from $M = H \times W$1 to $M = H \times W$2, and for Large elements from $M = H \times W$3 to $M = H \times W$4. The paper therefore frames V2P primarily as a precision-enhancing method for dense, high-resolution, professional GUIs rather than as a generic large-element localization improvement (Chen et al., 11 Jan 2026).

The reported generalization evidence spans six benchmarks—ScreenSpot-v2, ScreenSpot-Pro, OSWorld-G, UI-Vision, UI-I2E, and MMBench-GUI L2—where V2P-7B beats or matches strong baselines on most datasets. It matches JEDI-7B on OSWorld-G despite much less PC data, outperforms specialized GUI models on UI-Vision, UI-I2E, and MMBench-GUI L2, and exhibits training curves that avoid the overfitting observed in some baselines such as Aguvis-7B on out-of-distribution ScreenSpot-Pro (Chen et al., 11 Jan 2026).

The limitations are correspondingly specific. Attention calibration alone does not resolve semantically similar targets when multiple UI elements share similar icons or labels; stronger context reasoning is then required. Unconventional or cluttered designs can still disperse attention, reflecting uncertainty. The added self-attention layer and auxiliary attention losses introduce modest computational cost relative to simpler coordinate regression, particularly at high resolutions. The paper also implies that extensions to multi-click actions, drag-and-drop, gestures, or reinforcement-learning-style reward shaping remain open directions rather than established capabilities (Chen et al., 11 Jan 2026).

7. Terminological scope and other uses of “V2P”

The acronym V2P is overloaded across arXiv, and the GUI-grounding meaning should be distinguished from several unrelated usages.

Domain	Meaning of V2P
GUI grounding	Valley-to-Peak attention calibration via background suppression and center peaking (Chen et al., 11 Jan 2026)
DSERN CMOS sensors	Valley-Peak Modulation, a normalized valley-to-peak contrast in photon-counting histograms (Hendrickson et al., 1 Mar 2026)
Microbeam radiation therapy	Valley-to-peak ratio $M = H \times W$5 derived from macro valley and macro peak dose fields (Mentzel et al., 2022)
Video-language evaluation	Video Visual Prompt in V2P-Bench (Zhao et al., 22 Mar 2025)
NDR electronics	Typically discussed through peak-to-valley current ratio, with valley-to-peak as its reciprocal (Nguyen et al., 2011)

In the GUI-grounding literature, however, Valley-to-Peak has a precise and narrower meaning: a training objective that reshapes a patch-attention distribution so that irrelevant regions are explicitly suppressed and the center of the actionable element is explicitly emphasized. That definition is specific to the attention-based grounding regime introduced by GUI-Actor and should not be conflated with histogram modulation statistics, dose ratios, or visual-prompt benchmarks (Chen et al., 11 Jan 2026).