UI-Ins-32B: Advanced GUI Grounding Model

• UI-Ins-32B is a 32-billion parameter, multimodal model designed for GUI grounding that leverages diverse natural language instructions as dynamic reasoning pathways.
• It employs a two-stage training pipeline with multi-perspective supervised fine-tuning and reinforcement learning to optimize reasoning pathway selection.
• The model achieves state-of-the-art performance on benchmarks and underpins real-world mobile GUI agent applications through effective device–cloud collaboration.

UI-Ins-32B is a 32-billion parameter, instruction-as-reasoning, multimodal model designed for GUI grounding and agentic control of graphical user interfaces. It serves as both a benchmarked research baseline and a deployable foundation agent, particularly for mobile and real-world GUI navigation. The architecture, training strategies, and benchmark outcomes for UI-Ins-32B are detailed in multiple research papers, with core technical contributions centered on leveraging diverse natural language instructions as reasoning pathways, multi-stage supervised and reinforcement learning protocols, and large-scale grounding/navigation datasets.

1. Concept and Motivations

UI-Ins-32B is introduced as the largest and highest-capacity member of the UI-Ins family (“UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning” (Chen et al., 23 Oct 2025)). Its design principle is to solve GUI grounding: given a UI screenshot and a natural language instruction (e.g., “tap the top-right gear icon”), the model identifies and outputs the coordinates or bounding box of the actionable UI element corresponding to the instruction. Unlike prior approaches treating instructions as static intent surrogates, UI-Ins-32B treats instructions as dynamic analytical pathways that encode different reasoning perspectives—e.g., by location, appearance, function, or intent. This paradigm is termed “Instruction-as-Reasoning,” motivated by the finding that instruction diversity unlocks significant performance gains. The model addresses observed flaws in existing datasets (23.3% flaw rate in instruction annotations) and demonstrates that inference-time exploitation of instruction diversity can boost grounding performance by up to 76% (Chen et al., 23 Oct 2025).

2. Model Architecture and Training Pipeline

UI-Ins-32B is built atop a Qwen2.5-VL backbone, scaled to 32B parameters. The end-to-end training pipeline has two main stages:

1. Supervised Fine-Tuning (SFT) with Multi-Perspective Reasoning: The training data is cleaned with OmniParser V2 to refine UI element ground truth and filter poor instructions. Diverse instructions are synthesized per sample using GPT-4.1, generating four perspectives: appearance, functionality, spatial, and goal/intent. Each training sequence concatenates a perspective-specific reasoning string and the grounded coordinate, effectively conditioning grounding on reasoning diversity.
2. Reinforcement Learning (RL) for Pathway Selection and Synthesis: The RL stage uses Grouped Regularized Policy Optimization (GRPO) to select, compose, or synthesize optimal reasoning pathways at inference. The reward is binary (1 if the predicted point lies in the correct bounding box), and Z-score-normalized advantage estimation stabilizes optimization. The RL prompt is open-ended, encouraging the model to develop emergent reasoning strategies not limited to the fixed set of instruction perspectives. Batch size, learning rate, and other hyperparameters are set per scale; 33k instruction-augmented instances are further expanded to 100k via multi-perspective generation for RL.

This two-stage protocol, along with expert-guided instruction synthesis and data cleaning, is shown to prevent policy collapse typically observed in SFT+RL pipelines trained only on coordinate outputs, preserving diverse reasoning and effective RL exploration (Chen et al., 23 Oct 2025).

3. Model Capabilities and Benchmark Results

UI-Ins-32B achieves new state-of-the-art across several GUI grounding and navigation benchmarks. Key results include (Chen et al., 23 Oct 2025, Zhou et al., 26 Dec 2025):

Benchmark	UI-Ins-7B	UI-Ins-32B	Previous Bests
UI-I2E-Bench	81.1	87.3	GTA1-32B: 83.5
ScreenSpot-Pro	52.2	57.0	OpenCUA-32B: 55.3
MMBench-GUI L2	83.1	84.9	GTA1-32B: 83.4
ScreenSpot-V2	94.0	94.9	UI-Venus-7B: 94.1
Showdown	73.1	73.8	GTA1-32B: 71.1
AndroidWorld*	74.1†	73.3‡	Gemini 2.5: 69.7, UI-TARS-2: 73.3

* 74.1% is reported with UI-Ins-7B as executor under GPT-5 planner (Chen et al., 23 Oct 2025); 73.3% reported in the end-to-end MAI-UI-32B agent (Zhou et al., 26 Dec 2025). † As UI executor; ‡ As integrated foundation agent.

The most significant gains appear on complex, semantically ambiguous, and instruction-rich subsets, validating the emergent reasoning approach. UI-Ins-32B’s ability to compose or invent reasoning strategies improves generalization and robustness—e.g., performance on the “Advanced” subset of MMBench-GUI L2 notably exceeds prior models (Chen et al., 23 Oct 2025).

4. System Integration and Foundation Agent Role

In “MAI-UI Technical Report: Real-World Centric Foundation GUI Agents” (Zhou et al., 26 Dec 2025), UI-Ins-32B is instantiated as MAI-UI-32B, a foundation-scale GUI agent built on Qwen3-VL spanning grounding, navigation, and device-cloud collaborative deployment. The model participates as both cloud-side executor and grounding component in a native mobile agentic architecture with core features:

• Self-evolving data pipeline: Co-evolution of model and navigation task corpus via iterative SFT and model rollouts with rejection sampling. Partial trajectories and failure-handling are retained.
• Augmented action space: Supports actions such as click, drag, system_button, ask_user, and mcp_call (external tool invocation via Model Context Protocol).
• Online RL in dynamic GUIs: Trained in live, stateful environments with strict on-policy asynchronous rollouts across 512+ environments.
• Device–cloud collaboration: Local agents (e.g., 2B models) execute tasks; cloud-side MAI-UI-32B resumes when local trajectories diverge or require escalated capability. This arrangement yields strong quantitative improvements (33.4% in on-device performance, 42.7% reduction in cloud calls, 40% of tasks kept on-device for privacy).

MAI-UI-32B surpasses comparable models (e.g., Gemini-3-Pro, UI-Tars-2, Seed1.8, GTA1-32B) on major grounding and navigation datasets, especially when zoom-in inference and error-recovery logic are enabled (Zhou et al., 26 Dec 2025).

5. Underlying Principles: Instruction-as-Reasoning and Robust Grounding

The “Instruction-as-Reasoning” paradigm is foundational for UI-Ins-32B. Empirically, instruction rewriting across diverse perspectives addresses the incomplete and noisy intent signal typical in real-world deployment. The RL component encourages emergent behaviors:

• Strategic perspective selection: The model adapts to select the most effective reasoning pathway per query.
• Compositional reasoning: Multiple perspectives can be concatenated into a single chain-of-thought trace.
• Novel synthesis: The model can invent previously unseen reasoning strategies, e.g., by grouping elements or inferring UI state.

This method is shown to enhance grounding robustness, mitigate policy collapse, and improve agentic performance in interactive scenarios (Chen et al., 23 Oct 2025). The approach is further validated in ablation, where disabling instruction-as-reasoning leads to SFT+RL degradation rather than improvement.

6. Limitations and Open Challenges

Documented limitations for UI-Ins-32B are as follows:

• Grounding precision remains challenging: Despite gains, fine-grained, element-level localization and dense annotation of small or nested UI components are still error-prone, particularly under real deployment conditions or in benchmarks like MUIAnno (Parvez et al., 17 May 2026).
• Static screenshot bias: Many evaluations remain restricted to static screens or limited app interaction, requiring further generalization to highly dynamic, user-driven sequences.
• Cloud scale versus privacy/cost: The strongest models (UI-Ins-32B, MAI-UI-32B) rely on cloud execution for peak performance, raising privacy and latency concerns that are only partly mitigated by device–cloud orchestration.
• Benchmark dependence: While UI-Ins-32B leads on tested benchmarks, broader adoption and cross-domain generalization require additional study, especially as new classes of UI elements and task specifications arise.
• Reproducibility: Reliance on proprietary model APIs for benchmarks like MUIAnno can introduce experiment variability over time.

7. Relationship to Related Work and Datasets

UI-Ins-32B is positioned relative to several research lines:

• Fine-grained UI extraction: MUIAnno (Parvez et al., 17 May 2026) provides an expert-annotated, compositional benchmark for element-level detection. UI-Ins-32B’s architecture aligns with element-level semantic grounding requirements but existing models, including UI-Ins-32B, still struggle with nested and small components in dense screens.
• Attribute inference: Approaches such as “Learning to Infer User Interface Attributes from Images” (Schlattner et al., 2019) show that attribute-space reasoning, rather than pixel matching, is essential for UI implementation recovery. This principle is conceptually related to the instruction-as-reasoning design of UI-Ins-32B.
• Multimodal icon and element annotation: Studies such as “Multimodal Icon Annotation For Mobile Applications” (Zang et al., 2021) demonstrate the effectiveness of combining pixel and view-hierarchy information for robust detection, especially for icons—a core subset for UI-Ins-32B tasks.
• Navigation and agentic control: Foundation agent frameworks such as MAI-UI-32B (Zhou et al., 26 Dec 2025) build on grounding capabilities and expand them into real-world, multi-environment, and tool-augmented agentic pipelines.

A plausible implication is that UI-Ins-32B, though highly performant on current grounding and agentic benchmarks, represents an intermediate step toward fully robust, compositional, and context-aware GUI agents needed for truly general multimodal interfaces.

---

References

• UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning (Chen et al., 23 Oct 2025)
• MAI-UI Technical Report: Real-World Centric Foundation GUI Agents (Zhou et al., 26 Dec 2025)
• MUIAnno: An Expert-Annotated Dataset and Evaluation Benchmark for Mobile UI Understanding (Parvez et al., 17 May 2026)
• Learning to Infer User Interface Attributes from Images (Schlattner et al., 2019)
• Multimodal Icon Annotation For Mobile Applications (Zang et al., 2021)