ClawGUI-Agent: RL-Powered GUI Deployment

• ClawGUI-Agent is a deployment component that enables RL-trained agents to control mobile GUIs across major operating systems.
• It integrates with ClawGUI-RL’s multi-component training loop, employing hierarchical advantage estimation and dense step-level supervision.
• The system supports hybrid CLI-GUI workflows with persistent personalized memory, validated by state-of-the-art benchmarks on real devices.

ClawGUI-Agent is a component of the ClawGUI framework designed to facilitate the deployment and end-user interaction of reinforcement learning (RL) trained agents in graphical user interface (GUI) environments. Functioning as part of an integrated stack for GUI agent research and application, ClawGUI-Agent enables models trained through ClawGUI-RL to control mobile devices across major operating systems, bridging the gap between research prototypes and production-scale usability. The deployment harness supports both automated GUI actions and hybrid CLI-GUI workflows, with persistent, personalized memory for end-user sessions (Tang et al., 13 Apr 2026).

1. System Overview

ClawGUI-Agent operates as the deployment and user-facing layer within the ClawGUI framework, responsible for executing RL-trained GUI agents on Android, HarmonyOS, and iOS devices. ClawGUI-Agent enables these agents to interact with a wide range of mobile applications through direct manipulation of the device's visual interface. Interaction support is broad, encompassing taps, swipes, keystrokes, and text entry, and ClawGUI-Agent exposes access to these capabilities through integration with 12+ chat platforms, facilitating a conversational interface for diverse deployment scenarios. The harness also introduces hybrid CLI-GUI control schemes and maintains persistent user-specific memory per session.

2. Integration with ClawGUI-RL Infrastructure

ClawGUI-Agent draws on models trained via the ClawGUI-RL infrastructure, which utilizes a multi-component training loop centered on parallel virtual and real-device environments, GiGPO for hierarchical advantage estimation, and the Process Reward Model (PRM) for dense step-level supervision. After the training cycle, ClawGUI-Agent takes over by deploying the trained agent onto mobile operating systems, where it can execute tasks through either autonomous RL policy or via chat-based CLI-GUI mediation. This architecture closes the loop from epoch-based RL experimentation to interactive user-facing deployment, with seamless transition across research, evaluation (ClawGUI-Eval), and deployment contexts (Tang et al., 13 Apr 2026).

3. Cross-Platform Deployment and Control Paradigms

Supporting deployment on Android, HarmonyOS, and iOS, ClawGUI-Agent abstracts lower-level device interactions via ADB, cloud-phone provider SDKs, and platform-specific APIs. The system provides a uniform API for issuing UI actions—taps, swipes, and typing—to both emulators and physical devices. A salient feature is hybrid CLI-GUI operation: agents can receive textual instructions through chat platforms, map them to visual sequences, and execute the corresponding UI actions, or alternatively, autonomously infer and perform multi-step mobile UI policies. Persistent personalized memory allows session state to be maintained across user interactions, facilitating context-aware behaviors.

4. Empirical Performance and Benchmarking

Deployed agents trained end-to-end within the ClawGUI pipeline, most notably the ClawGUI-2B model, demonstrate state-of-the-art results on standardized benchmarks. On the MobileWorld GUI-Only suite (117 tasks), ClawGUI-2B achieves a 17.1% success rate, outperforming the same-scale MAI-UI-2B baseline by 6.0 percentage points. This empirical result reinforces the impact of the full-stack infrastructure, particularly the synergy between the training, evaluation, and deployment components encapsulated in ClawGUI-Agent (Tang et al., 13 Apr 2026).

Model	Success Rate (%)
MAI-UI-2B	11.1
Qwen3-VL-32B	11.9
UI-Venus-72B	16.4
ClawGUI-2B (Ours)	17.1

ClawGUI-Agent ensures that deployed models are not constrained to simulated environments but are validated on-device, and that evaluation is run through standardized protocols identical to ClawGUI-Eval. A reported 95.8% reproduction against official baselines across 6 benchmarks and 11+ models further attests to the standardization and reliability of the deployment route.

5. Example Usage and API Illustration

ClawGUI exposes a Python API that abstracts environment management, policy loading, reward modeling, and RL training, culminating in models exportable to the deployment harness. Below is an example illustrating the interface for configuring and executing RL agents prior to deployment with ClawGUI-Agent.

from clawgui.rl import EnvironmentManager, PRMRewardModel, RLTrainer env_mgr = EnvironmentManager( virtual_count=64, real_devices=['device_id1','device_id2'], # optional spare_pool_size=10, health_check_interval=10, ) policy = load_llm_policy('maiu-2b', device='cuda') prm = PRMRewardModel( judge_model='qwen3.5-72b', threshold=0.5, ) trainer = RLTrainer( env_manager=env_mgr, policy=policy, reward_model=prm, algorithm='GiGPO', rollout_group_size=8, max_steps=50, lr=1e-6, batch_size=8, epochs=3, gamma=0.99, temperature=0.7, ) trainer.train() trainer.save('clawgui-2b-rl.pt')

Through this API, researchers can complete the RL training pipeline and subsequently load the trained agent into ClawGUI-Agent for deployment across supported platforms.

6. Limitations and Future Directions

ClawGUI-Agent inherits several practical limitations from the broader framework. Real-device tasks necessitate manual authoring and verification using MLLM-as-judge protocols, introducing annotation overhead. PRM-based reward shaping depends on frozen judge models, which are susceptible to reward misclassification. The orchestration of numerous cloud devices and GPUs presents scaling challenges, particularly for smaller laboratories (Tang et al., 13 Apr 2026).

Envisioned directions include adoption of world models to facilitate model-based planning through predictive UI dynamics, privacy-preserving on-device RL pipelines leveraging real user signals, a unified action harness handling CLI, GUI, and API routes with meta-RL for control policy selection, and automation for scale-out task creation via mock app code synthesis.

7. Significance in the GUI Agent Ecosystem

ClawGUI-Agent, as the deployment and user interaction module of ClawGUI, establishes a reproducible, scalable solution for bringing RL-trained GUI agents to operational contexts on real devices. Its cross-platform, chat-integrated deployment capabilities and persistent personalized memory mark a maturation from research sandboxing to production-level utility. The system’s standardized protocols, hybrid control paradigms, and empirical competitiveness position it as a reference implementation for RL-based GUI automation research and future application development (Tang et al., 13 Apr 2026).