---
title: 'Mobile-Agent-v3: Multi-Agent GUI Automation'
url: https://www.emergentmind.com/topics/mobile-agent-v3
type: topic
---

# Mobile-Agent-v3: Multi-Agent GUI Automation

Mobile-Agent-v3 is a general-purpose, open-source multi-agent GUI automation framework that builds on and extends the capabilities of GUI-Owl, achieving state-of-the-art results on a spectrum of benchmark environments spanning mobile (AndroidWorld) and desktop (OSWorld) platforms. Its architecture fuses large-scale, cross-platform environment infrastructure, modular multi-agent designs, advanced UI-grounding and planning, and a scalable reinforcement learning mechanism optimized for trajectory-level policy refinement. Mobile-Agent-v3 not only addresses the challenges of reliable, long-horizon GUI automation but also establishes itself as a foundation for further research and deployment in GUI, digital assistant, and multimodal automation scenarios [2508.15144].

## 1. Performance and Benchmarking

Mobile-Agent-v3 attains leading performance metrics across benchmark suites for GUI automation, most notably:

| Platform                | Mobile-Agent-v3 Score | GUI-Owl Baseline | OpenCUA Baseline |
|-------------------------|----------------------|------------------|------------------|
| AndroidWorld            | 73.3                 | 66.4             | <66.4            |
| OSWorld-Verified        | 37.7                 | 29.4             | Lower/NA         |

- AndroidWorld—a mobile interaction benchmark—witnessed a boost from 66.4 (GUI-Owl) to 73.3 (Mobile-Agent-v3), underlining robust mobile-specific improvements.
- OSWorld, targeting desktop environments, saw an increase from 29.4 to 37.7, surpassing prior open-source models such as OpenCUA.
- These gains reflect not only improved single-action decision-making but heightened reliability in executing long-horizon and multi-step tasks.

The system incorporates step-level critics and trajectory-level multimodal critics, ensuring that each trajectory (sequence of screen states and actions) is rigorously validated, and the feedback loop drives both data cleaning and model improvement.

## 2. Large-Scale, Cross-Platform Environment Infrastructure

Mobile-Agent-v3 is underpinned by a cloud-based virtual infrastructure that spans Android, Ubuntu, macOS, and Windows.

- **Self-Evolving GUI Trajectory Production Framework**: Synthetic and real interaction trajectories are generated using automated, high-quality query generation emulating realistic user instructions. Each action step is automatically validated—incorrect or suboptimal trajectory segments are flagged, pruned, and recycled to refine the data.
- The iterative data production forms a self-improving loop: GUI-Owl is used to bootstrap new data, which is then further validated and used for model retraining.
- This continuous, closed-loop pipeline allows for scalable buildout of diverse, high-fidelity training corpora for both mobile and desktop workflows, dramatically reducing dependence on manual annotation.

## 3. Modular Agent Capabilities and Multi-Agent System Design

Mobile-Agent-v3 systematically decomposes automation challenges into specialized, collaborative agent modules, serving both as unified standalone agents and as interoperable components in larger agentic systems:

- **UI Grounding**: Robust localization of screen elements using segmentation and detection techniques.
- **Task Planning and Action Semantics**: Extraction and generalization of procedural knowledge—breaking down high-level tasks into actionable subgoals, semantically mapping both primitive actions (click, drag, type) and their consequences (as seen in before/after state pairs).
- **Diverse Reasoning Patterns**: The system leverages offline hint-guided rejection sampling, collaborative multi-agent distillation, and multimodal critics. This diversity allows the agent to reason adaptively, not only following fixed scripts but flexibly adjusting in response to real-time feedback.
- **Multi-Agent Roles**: Distinct roles are implemented, including Manager (task decomposition), Worker (action execution), Reflector (critic/judgment), and Notetaker (context preservation).

This modular architecture supports composability—agents can interact, share context, and collaborate on complex, heterogeneous instruction sequences involving multiple GUIs or devices.

## 4. Reinforcement Learning and Trajectory-Aware Policy Optimization

Mobile-Agent-v3 introduces a scalable, asynchronous reinforcement learning pipeline specifically adapted for GUI environments:

- **TRPO (Trajectory-aware Relative Policy Optimization)** is used to optimize agent policies at the trajectory level, rather than individual steps:
    $$
    \hat{A}_\tau = \frac{R(\tau) - \bar{R}}{\sigma_R + \varepsilon}
    $$
    Here, $R(\tau)$ denotes the holistic reward for trajectory $\tau$, while $\bar{R}$ and $\sigma_R$ are its running mean and standard deviation.
- Rewards are assigned globally per trajectory and then uniformly distributed across its constituent actions, a method designed to address the credit assignment problem endemic to long-horizon GUI tasks with sparse rewards.
- The policy objective is given by the TRPO loss:
    $$
    L_{TRPO} = -\frac{1}{N} \sum_{i=1}^{G} \sum_{s=1}^{S^i} \sum_{t=1}^{|o_{i,s}|} \min \left[ r_t(\theta) \hat{A}_{\tau_i}, \; \text{clip}\left(r_t(\theta), 1\!-\!\varepsilon, 1\!+\!\varepsilon\right) \hat{A}_{\tau_i} \right]
    $$
    where $r_t(\theta)$ is the ratio of policy probabilities, effectively regularizing updates to ensure stable learning.

This approach stabilizes training and improves convergence, outperforming standard RL methods on long-horizon GUI tasks. For example, with TRPO, OSWorld scores reach 34.9 (surpassing prior baselines).

## 5. Data Pipelines and Self-Improving Loops

The training regime continuously fuses human-like query generation, extensive environment rollouts, and automated judgment:

- **Query Generation**: High-quality, task-diverse DAGs and GUI metadata are used to synthesize queries mimicking authentic human commands.
- **Automated Critic Modules**: Both step-level and trajectory-level automated critics (leveraging multimodal models) assess and filter the generated trajectories for both correctness and diversity.
- **Feedback and Self-Evolution**: Cleaned and validated data is reintroduced into the model training loop. This approach produces distributions of exemplars covering grounding, reasoning, and procedural semantics—foundational for robust generalization.

## 6. Open Source Release and Community Impact

The Mobile-Agent-v3 system, including GUI-Owl models and trajectory data pipelines, is available as open source at https://github.com/X-PLUG/MobileAgent.

- This release provides the research and practitioner community with a state-of-the-art multi-agent GUI automation system, facilitating both reproducibility and extensibility.
- The framework is designed for application in automated digital assistants, wide-scale UI testing, workflow automation, and as a baseline for further multi-agent and reinforcement learning research in GUI contexts.

## 7. Significance and Future Directions

Mobile-Agent-v3 establishes new standards for open-source, multi-agent GUI automation with substantive empirical advances on challenging real-world benchmarks. It enables robust, generalizable, and efficient long-horizon interaction—integrating scalable cloud infrastructure, modular agent design, diverse data pipelines, and trajectory-aware reinforcement learning. The open-source contribution is poised to accelerate both academic inquiry and practical deployments, fostering further work in autonomous GUI interaction, digital task assistance, and the evolution of agent-based systems on heterogeneous device ecologies [2508.15144].

Source: https://www.emergentmind.com/topics/mobile-agent-v3