PosPad: Visual Token Compression for Efficient UI Grounding

• PosPad is a position-preserving compression method that converts dropped contiguous visual tokens into a single learnable marker (<pos>) to maintain spatial order.
• It preserves the spatial layout by inserting the marker at the end of each dropped token span, avoiding harmful positional jumps in M-RoPE embeddings.
• Empirical results show PosPad maintains grounding precision close to dense baselines while reducing sequence length and computational load.

Searching arXiv for the cited papers to ground the article. PosPad is a position-preserving visual token compression strategy introduced in the UI-grounding framework FocusUI. It is designed for the setting in which a high-resolution screenshot is patchified into a long visual-token sequence and then aggressively reduced before multimodal decoding. The central problem is that direct visual token pruning shortens the sequence but creates positional jumps in the $(h,w)$ dimensions of M-RoPE, which is especially harmful for fine-grained UI grounding. PosPad addresses this by compressing each contiguous sequence of dropped visual tokens into a single learnable marker, written as <pos>, placed at the sequence’s last index so that the reduced sequence preserves positional continuity while still achieving substantial length reduction (Ouyang et al., 7 Jan 2026).

1. Problem formulation and motivation

FocusUI studies efficient UI grounding: given a high-resolution screenshot $I$ and a natural-language instruction $q$, the model must identify the target UI region while using fewer visual tokens than a dense VLM baseline. This setting is unusually stringent because screenshots are tokenized into thousands of visual tokens; for 2K screenshots the paper notes roughly 4700 visual tokens, and across its benchmark statistics visual tokens account for at least 84.3%–98.1% of the entire input sequence, depending on model and benchmark (Ouyang et al., 7 Jan 2026).

The resulting bottlenecks are twofold. First, long visual sequences create substantial computation and memory overhead. Second, attention is diluted because many patches correspond to redundant or homogeneous regions. A natural response is to prune visual tokens, but UI grounding differs from coarse visual understanding tasks because it is a fine-grained localization task: small spatial offsets can cause the model to click the wrong button, icon, or text field. In Qwen-style VLMs, visual patches are processed with multimodal rotary position embeddings (M-RoPE) over temporal, height, and width dimensions. When tokens are simply removed, the sequence collapses across omitted regions, and neighboring kept patches become adjacent in the sequence even when they were far apart in the original screenshot. The paper attributes large localization offsets to exactly this mechanism, describing direct pruning as creating positional jumps in the $(h,w)$ dimensions of the M-RoPE sequence (Ouyang et al., 7 Jan 2026).

PosPad is the mechanism introduced to preserve enough spatial structure after pruning that precise grounding remains possible. It does not determine which tokens are relevant; rather, it preserves the ordered spatial progression of screenshot patches once many of them have been removed. This design targets the UI-specific failure mode in which sequence shortening destroys the layout scaffold needed for precise localization.

2. Formal definition of the PosPad transformation

Let the original visual token sequence be $x_{1:M}$, where $M$ is the number of screenshot patch tokens. FocusUI first computes per-patch saliency scores and retains the top-$K$ patches under a retention ratio $r \in (0,1]$, with

$K = \lfloor rM \rfloor.$

If $\mathcal{K}$ denotes the kept index set and $I$0 the dropped index set, the selection rule is defined from the per-patch saliency scores $I$1 by thresholding at the $I$2-th largest score (Ouyang et al., 7 Jan 2026).

PosPad then partitions the dropped set $I$3 into maximal contiguous subsequences

$I$4

where each $I$5 is consecutive in flattened 1D patch order. For each dropped span $I$6, PosPad retains only its last index

$I$7

and forms the set of sequence-end indices

$I$8

The preserved index set is then

$I$9

The transformed sequence is

$q$0

Each contiguous dropped span is therefore compressed into a single special learnable token, written as <pos>, and that marker is placed at the last index of the dropped span. All other indices in that span are removed, while kept visual tokens remain unchanged. If a total of $q$1 tokens are dropped and these drops form $q$2 contiguous spans, the final visual length is

$q$3

The number of <pos> markers is exactly $q$4 (Ouyang et al., 7 Jan 2026).

The design choice of using the last index is the central positional-preservation mechanism. The marker inherits that index’s $q$5 positional information, which preserves continuity more faithfully than removing the entire span or inserting a marker earlier in the span. The paper explicitly states that PosPad alters only sequence sparsity and not token indices or rotary bases, so it is compatible with common M-RoPE implementations and requires no modifications to the downstream LM architecture (Ouyang et al., 7 Jan 2026).

3. Integration within the FocusUI pipeline

PosPad sits between token selection and multimodal decoding. FocusUI has three major components: Instruction-to-Patch supervision, a Query-Guided Saliency Scorer, and PosPad. The scorer, not PosPad, decides what to keep. It operates over visual patch embeddings $q$6 and instruction token embeddings $q$7, computing

$q$8

These scores drive top-$q$9 selection under the specified retention ratio (Ouyang et al., 7 Jan 2026).

The supervision for the scorer combines instruction-conditioned localization and a rule-based UI prior. The fused supervision is

$(h,w)$0

Here the UI-graph term down-weights large homogeneous regions, while the instruction-conditioned term identifies patches relevant to the task. PosPad is applied after visual encoding and after saliency scoring, but before the visual token sequence is consumed by the downstream LM decoder. In this sense, the scorer answers which patches to keep, whereas PosPad determines how to represent what was dropped without collapsing the spatial layout (Ouyang et al., 7 Jan 2026).

FocusUI’s downstream grounding component follows the coordinate-free grounding scheme from GUI-Actor. The LM outputs an action token representation $(h,w)$1, and an action head attends over selected patch features. PosPad is not a grounding head by itself; its contribution is to ensure that the reduced visual sequence still retains enough layout structure for the grounding head to align the instruction-conditioned action representation with candidate UI regions.

This separation of roles is conceptually important. PosPad is neither a saliency estimator nor a positional-embedding rewrite. It is a sequence-structure transformation over already selected visual tokens.

4. Positional continuity, ablations, and empirical behavior

The clearest empirical finding is that continuity preservation is crucial. FocusUI compares four sequence constructions on ScreenSpot-Pro at $(h,w)$2 retention: the original visual sequence, direct dropping, full padding, and PosPad. The ablation isolates the difference between reducing sequence length and preserving positional continuity (Ouyang et al., 7 Jan 2026).

Strategy	ScreenSpot-Pro accuracy	Effect
Baseline $(h,w)$3	40.9	No pruning
CLIP score + direct drop	28.5	Continuity broken
CLIP score + full padding	38.7	Continuity preserved, no length reduction
CLIP score + PosPad	38.2	Continuity preserved, compressed
Ins2Patch score + direct drop	29.2	Continuity broken
Ins2Patch score + full padding	42.1	Continuity preserved, no length reduction
Ins2Patch score + PosPad	42.3	Continuity preserved, compressed

These results show that most of the performance collapse under pruning comes from breaking positional continuity, not merely from discarding content. Direct drop is described as catastrophic, whereas continuity-preserving schemes recover most of the lost accuracy. PosPad is slightly better than full padding in the Ins2Patch setting while still reducing sequence length.

The paper also studies marker placement. The appendix compares three variants: sequence-first, sequence-middle, and sequence-end. Sequence-end is best at all tested retention ratios. On ScreenSpot-Pro, at $(h,w)$4, the scores are 39.8 for first, 38.7 for middle, and 40.4 for end; at $(h,w)$5, they are 36.8, 33.5, and 37.7, respectively. This supports the stated interpretation that end placement is most compatible with raster-scan ordering and M-RoPE, whereas earlier placement pulls the omitted region toward earlier positions in the flattened sequence (Ouyang et al., 7 Jan 2026).

At system level, PosPad supports aggressive pruning with modest degradation. For FocusUI-7B on ScreenSpot-Pro, accuracy changes from 48.3 at $(h,w)$6 to 46.5 at $(h,w)$7 and 45.1 at $(h,w)$8. In the same comparison, reducing to $(h,w)$9 retention yields 1.44× faster inference and about 17% lower peak GPU memory. The efficiency table notes that the reported number of visual tokens does not include <pos> tokens, so the actual multimodal sequence is slightly longer than the raw kept-token count, but the end-to-end reduction remains substantial (Ouyang et al., 7 Jan 2026).

Against generic VLM pruning baselines, the gap is larger. At $x_{1:M}$0 retention on ScreenSpot-Pro, generic pruning methods such as Fast-V, HiPrune, and Vision-Zip suffer severe drops, while FocusUI with PosPad remains much closer to its dense baseline. The broader implication is that PosPad’s benefit is strongest in precise localization-heavy UI grounding, especially under aggressive token retention ratios and on high-resolution screenshots (Ouyang et al., 7 Jan 2026).

5. Implementation characteristics and operational constraints

PosPad introduces a learnable special token <pos>. The paper specifies that it is inserted only at sequence-end indices of dropped spans. It does not provide an explicit initialization formula for the <pos> embedding, a PosPad-specific attention mask, or any modified rotary-position formula beyond inheriting the original index (Ouyang et al., 7 Jan 2026).

Several implementation details delimit the method’s scope. Dropped spans are defined with respect to the 1D flattened sequence order, and the appendix ties the sequence-end design to the raster-scan ordering used by the vision encoder and M-RoPE. If pruning produces several disjoint contiguous regions, each region receives its own <pos> token. PosPad therefore preserves continuity at the level of contiguous omissions rather than at the level of every dropped patch.

FocusUI is trained end-to-end with PosPad present in the loop. The retention ratio $x_{1:M}$1 is sampled uniformly from $x_{1:M}$2 during training, which exposes the model to varying sparsity patterns. For reproduction, the paper reports patch size $x_{1:M}$3 for Qwen2.5-VL and $x_{1:M}$4 for Qwen3-VL, with $x_{1:M}$5 and $x_{1:M}$6 in supervision construction. The framework is also described as natively compatible with FlashAttention-2, precisely because PosPad changes only the input sequence structure and does not depend on intermediate attention or activation information (Ouyang et al., 7 Jan 2026).

The paper does not present PosPad as a purely inference-only plug-in and does not test a “PosPad only at inference without retraining” setting. It also does not compare PosPad against interpolation, learned pooling over dropped spans, or explicit index-remapping schemes. Those alternatives therefore remain outside the reported evidence base.

A further operational limitation is that PosPad preserves continuity only coarsely. A whole dropped run is summarized by one token. Very aggressive pruning still hurts: at $x_{1:M}$7 and especially $x_{1:M}$8 retention, accuracy declines noticeably. PosPad thus preserves the layout scaffold more effectively than direct dropping, but it does not reconstruct the omitted fine-grained content (Ouyang et al., 7 Jan 2026).

6. Conceptual scope, misconceptions, and related methods

PosPad is best understood as a sequence-level compression mechanism for preserving positional continuity after visual token pruning. Several nearby interpretations are inaccurate. It is not the token-selection policy itself; selection is handled by the saliency scorer. It is not ordinary padding in the NLP sense; the inserted <pos> markers summarize omitted contiguous regions and remain tied to inherited spatial indices. It is also not a redesign of positional encoding, since FocusUI explicitly leaves token indices and rotary bases unchanged (Ouyang et al., 7 Jan 2026).

A frequent source of confusion is nomenclature. The paper “Pose-Aware Diffusion for 3D Generation” uses the exact term PAD, not PosPad, for a 3D generation framework that synthesizes geometry directly in observation space using a monocular-depth-derived partial point cloud as a geometric anchor (Zhou et al., 1 May 2026). That method is pose-aware 3D generation rather than position-preserving token compression, and the overlap is only lexical.

A more substantive conceptual parallel appears in “Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level LLMs”, which does not mention PosPad explicitly but is described as a close conceptual match if PosPad is taken to mean adding extra intra-patch positions or latent updates to reduce an information bottleneck inside coarse patches. In that setting, Scratchpad Patching inserts transient internal states inside patches to refresh patch-level context without increasing the persistent patch sequence length or the inference-time KV-cache footprint. The paper emphasizes, however, that these added positions are semantic transient compute states, not dummy padding tokens; it is therefore more accurately a patch-local latent update mechanism than a positional padding mechanism (Zheng et al., 10 May 2026).

Within the UI-grounding context, the more precise characterization is narrower and more concrete. PosPad is a method for compressing dropped contiguous visual-token spans into single learned positional markers placed at sequence ends, thereby preserving coarse spatial continuity in a reduced multimodal sequence. Its significance lies in showing that efficient screenshot grounding depends not only on selecting the right tokens, but also on representing where discarded regions used to be.