EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

Abstract: The development of native computer-use agents (CUA) represents a significant leap in multimodal AI. However, their potential is currently bottlenecked by the constraints of static data scaling. Existing paradigms relying primarily on passive imitation of static datasets struggle to capture the intricate causal dynamics inherent in long-horizon computer tasks. In this work, we introduce EvoCUA, a native computer use agentic model. Unlike static imitation, EvoCUA integrates data generation and policy optimization into a self-sustaining evolutionary cycle. To mitigate data scarcity, we develop a verifiable synthesis engine that autonomously generates diverse tasks coupled with executable validators. To enable large-scale experience acquisition, we design a scalable infrastructure orchestrating tens of thousands of asynchronous sandbox rollouts. Building on these massive trajectories, we propose an iterative evolving learning strategy to efficiently internalize this experience. This mechanism dynamically regulates policy updates by identifying capability boundaries -- reinforcing successful routines while transforming failure trajectories into rich supervision through error analysis and self-correction. Empirical evaluations on the OSWorld benchmark demonstrate that EvoCUA achieves a success rate of 56.7%, establishing a new open-source state-of-the-art. Notably, EvoCUA significantly outperforms the previous best open-source model, OpenCUA-72B (45.0%), and surpasses leading closed-weights models such as UI-TARS-2 (53.1%). Crucially, our results underscore the generalizability of this approach: the evolving paradigm driven by learning from experience yields consistent performance gains across foundation models of varying scales, establishing a robust and scalable path for advancing native agent capabilities.

• The paper presents a novel framework (EvoCUA) that evolves computer use agents by integrating verifiable synthetic experience generation, high-throughput interactive simulation, and iterative policy optimization.
• The paper details a robust simulation infrastructure that orchestrates tens of thousands of virtual sandboxes, ensuring precise execution and scalable experience collection.
• The paper demonstrates superior performance on the OSWorld benchmark, with EvoCUA-32B achieving a 56.7% success rate under stricter step constraints compared to existing models.

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

Motivation and Paradigm Shift

The EvoCUA framework introduces a substantive transition in GUI-based agentic AI, moving from static behavior cloning on passively collected datasets to a dynamic, self-sustaining cycle of experience acquisition, interaction, and policy refinement. Existing computer use agents are constrained by data bottlenecks and suboptimal generalization due to their reliance on static imitation; this impedes progress on long-horizon tasks and fails to capture causal feedback. EvoCUA addresses these challenges by integrating verifiable synthetic experience generation, high-throughput interactive infrastructure, and an iterative learning paradigm—collectively enabling a native agent that continuously evolves its capabilities via active interaction.

System Architecture

EvoCUA operationalizes its paradigm shift through three tightly integrated pillars:

• Verifiable Synthesis Engine autonomously creates diverse computer use tasks and their corresponding executable validators, leveraging domain taxonomies and hybrid resource injection to ensure high variability and realism. This "generation-as-validation" strategy eliminates ambiguous supervision and grounds rewards in environment-executable correctness.
• Scalable Interaction Infrastructure decentralizes rollout orchestration over tens of thousands of QEMU-KVM virtualized sandboxes, with robust abstractions for environment instantiation and precise control over simulation determinism, rendering consistency, and resource elasticity. Gateway and scheduling microservices eliminate I/O bottlenecks, matching burst training needs and ensuring strict session isolation.
• Iterative Experience-Driven Optimization operates a continuous feedback loop, using policy-guided curriculum synthesis, rejection sampling, and direct preference optimization to consolidate successful routines and mine rich corrective supervision from failure. The empirical success rate is estimated through massive-scale Monte Carlo sampling over the evolving task distribution, continuously recalibrated by the synthesis engine in response to policy performance.

  Figure 1: Conceptual depiction of EvoCUA's evolving experience learning cycle, unifying verifiable data synthesis, scalable infrastructure, and iterative optimization.

Verifiable Synthetic Data Generation

EvoCUA's data pipeline is engineered to break the limitations of manual curation and annotation-heavy methods. Task scenarios are hierarchically decomposed by domain and capability; environmental realism is introduced via hybrid resource strategies that blend parametric document synthesis with nonparametric injection of real-world artifacts. Agentic dual-stream synthesis produces (instruction, validator) pairs using closed-loop execution for guaranteed executability and consistency. Critical quality assurance steps enforce cross-modal decontamination, reference agent cross-validation, and manual inspection to maintain ground-truth integrity and avoid benchmark leakage.

Figure 2: Three-stage architecture of verifiable synthesis: task space construction, dual-stream agentic generation, and rigorous filtering yield high-consistency, executable supervision.

Scalable Infrastructure and Experience Collection

The platform's virtualization stack enables hundreds of thousands of daily sandboxed sessions with kernel-level isolation and optimized I/O. Tools define immutable simulation environments, and dynamic clusters handle rapid environment scaling on demand. High-throughput asynchronous routing and distributed scheduling maintain resource elasticity, enabling on-policy RL with minimal latency between experience generation and policy update. Kernel and userspace patches guarantee HID mapping consistency, rendering fidelity, and robust runtime stability.

Figure 3: Scalable infrastructure orchestrating interaction requests through asynchronous gateway, distributed scheduler, and massive parallel sandbox clusters.

Evolving Learning Paradigm

EvoCUA's learning regimen proceeds through:

• Cold-Start: Behavioral priors established by high-quality synthesized trajectories, unified action schema, and hindsight reasoning generate explicit cognitive chains for interpretable decision-making.
• Rejection Sampling Fine-Tuning (RFT): Dynamic compute budgeting allocates rollout resources adaptively, prioritizing boundary queries. Step-level denoising and judge-aided curation refine the experience pool for maximum SNR.
• Step-Level Direct Preference Optimization (DPO): RL procedures discover critical forking points in failed trajectories, constructing preference pairs for both immediate action correction and reflection-induced recovery. DPO is used to optimize marginal preference between chosen and rejected agentic responses, yielding robust expansion of the capability boundary.

  Figure 4: Dual-paradigm DPO at critical forking: action correction optimizes preference for correct execution; reflection prioritizes remedial reasoning over blind continuation.

Empirical Evaluation

On the OSWorld benchmark, EvoCUA-32B achieves a verified success rate of 56.7%, surpassing OpenCUA-72B (45.0%) and the closed-source UI-TARS-2 (53.1%). Notably, EvoCUA's efficiency permits higher precision under stricter step constraints (50 steps vs. 100), and scaling analysis demonstrates consistent gains across varying backbone sizes (e.g., EvoCUA-8B reaches 46.1%). Ablation confirms monotonic additive improvements from unified action space, cold start, RFT, and DPO iterations; the paradigm generalizes to larger models with robustness against distributional shifts. Experience scaling studies highlight that the signal-to-noise ratio and active curation of boundary cases are critical to sustained improvement.

Figure 5: OSWorld-Verified benchmark highlights EvoCUA-32B's state-of-the-art open-weights performance.

Trajectory Visualization and Reasoning Alignment

Inspection tools validate alignment between user instructions, reasoning traces, and atomic execution. Visualization of long-horizon spreadsheet manipulation tasks shows explicit goal clarification, precise stateful interactions (e.g., Shift-select), and visual evidence-based termination. These modalities guarantee the agent's decision-making remains logically and perceptually grounded.

Figure 6: Initial goal clarification aligning agent reasoning with instruction, validating synthetic data's grounding.

Figure 7: Stateful interaction: agent correctly executes Shift-click sequence for complex selection.

RL Recipes and Online Interaction

A key limitation of trajectory-level RL is training-inference discrepancy due to context compression. EvoCUA proposes Step-Level Policy Optimization (STEPO) to allocate advantage values uniformly and encourage optimal trajectory length, addressing discrepancies and driving both efficiency and robustness. Empirical studies show STEPO significantly outperforms trajectory-level GRPO, although computational costs remain high; future directions will seek scalable alternatives.

Figure 8: Comparative STEPO performance: significant reward improvements over GRPO across repeated trials.

Implications and Prospects

EvoCUA establishes a scalable, generalizable methodology for evolving computer use agents, demonstrating practical superiority for open-weights models on public benchmarks and theoretical contributions in experience-driven policy optimization. Its synthesis-guided curriculum, high-throughput simulation infrastructure, and iterative supervision enable robust agentic capabilities that generalize beyond static imitation. Remaining discrepancies with closed-source and human baselines highlight limitations in offline experience scaling and environmental stochasticity; future research will extend online RL, environment diversity, and advanced credit assignment schemes.

Conclusion

EvoCUA represents a rigorous advance in native computer use agent training by synergizing verifiable synthetic experience generation, elastic simulation infrastructure, and evolving experience-driven learning. Its empirical performance confirms the practical viability and generalizability of the evolving paradigm. Future developments will explore online RL, increasing environmental variability, and scalable step-level optimization strategies to close the gap to human-level performance and fully autonomous open computer-use agents.