LiteCUA: Lightweight Computer-Use Agent

• LiteCUA is a lightweight computer-use agent that abstracts GUIs and event-level noise into semantic states using an MCP server.
• It employs a modular perceiver-reason-act architecture integrated with AIOS 1.0, standardizing actions into a compact, atomic command set.
• Benchmarking on OSWorld tasks shows LiteCUA’s superior performance (14.66% success) compared to traditional LLM-based agents.

LiteCUA is a lightweight computer-use agent designed to enable robust interaction with real-world computer environments by leveraging a semantic abstraction of the system states and actions. Developed atop the AIOS 1.0 platform, LiteCUA exemplifies the principle that decoupling interface complexity from agent reasoning can unlock significant capability gains even for relatively simple agent architectures. This paradigm is realized by representing the computer as a Model Context Protocol (MCP) server, providing an interpretable semantic context to LLMs and standardizing agent actions into a compact, atomic space.

1. Foundational Principles and Overview

LiteCUA reframes the agent–environment interface by transforming the native pixel, GUI, and event-level complexity of conventional computer environments into structured semantic representations that are directly consumable by LLM-based agents. This is achieved through the AIOS 1.0 operating platform, which deploys an MCP server to expose the environment as a set of semantically annotated states and actions.

The agent operates through a modular orchestrator-worker architecture, divided functionally into perception (semantic state construction), reasoning (deliberation over tasks and next actions), and acting (atomic GUI command execution). This cycle is tightly integrated with the AIOS 1.0 layered system, which abstracts core applications and manages sandboxed execution.

2. Architectural Components and Semantics

AIOS 1.0 delineates two principal layers:

• Application Layer: Abstracts interactions for key system components such as Terminal, Code Editor, Browser, and Document tools through agent-centric SDKs.
• Kernel Layer: Manages LLM cores, context/memory flows, and a tool manager that incorporates both a virtual machine controller and an MCP server for environment abstraction.

Model Context Protocol (MCP) Server

The MCP server operationalizes the semantic state abstraction. The environmental state at any time is represented as

$s = \{E_1, E_2, ..., E_n\}$

where each interface element $E_i$ is defined structurally as

$$E_i = \{\text{type}, \text{label}, \text{position}, \text{usage}, \text{state}, ...\}$$

For example:

{"type": "button", "label": "Open File", "position": [x, y], "usage": "Open a file dialog"}

Actions are constrained to a discrete, finite set:

$$\mathcal{A} = \{\text{CLICK}(x, y), \text{SCROLL}(direction), \text{TYPE}(\text{string}), \text{DRAG}(x_1, y_1, x_2, y_2), ...\}$$

All actions are executed within a sandbox, enforced by the VM Controller, preserving system integrity and agent reproducibility.

3. Interface Complexity and Reasoning Efficiency

Legacy agent frameworks entangled reasoning with the presentation complexity of arbitrary GUIs, resulting in brittle performance and overwhelming planning requirements. LiteCUA's approach via MCP and AIOS 1.0 achieves strict separation:

$$\text{Decision Complexity} = f(\text{Semantic State, Task}) \ll g(\text{Raw Interface, Task})$$

This means that even basic agents become substantially more capable, as the act of mapping between tasks and agent-understandable states is both compact and losslessly informative. The agent logic is simplified to deliberate on tasks using context-native abstractions, with the execution layer demoted to precise, atomic commands.

4. Performance Benchmarking: OSWorld Evaluation

The effectiveness of LiteCUA was assessed on the OSWorld benchmark, encompassing 369 diverse desktop and web computer-use tasks. LiteCUA achieved a success rate of 14.66%, outperforming vanilla LLM baselines such as GPT-4o-mini (6.21%), GPT-4o (11.21%), and Gemini-1.5-pro (5.1%), as well as specialized agents Friday (11.11%), Open-Interpreter (8.94%), and AgentStore (13.55%). Performance breakdown illustrates domain-specific strengths and limitations:

• OS-related tasks: 54.2% success, ~18 operational steps.
• VSCode tasks: 34.8% success, 25.6 steps.
• Productivity applications (Libreoffice Calc/Thunderbird): 0% success, with agents consistently hitting operation step limits (50 steps), signifying environment complexity bottlenecks.
• Multi-app and Chrome/VLC: Intermediate results, e.g., 10.9% success and 38.4 steps average.

Method	OSWorld Success Rate (%)
GPT-4o-mini	6.21
GPT-4o	11.21
Gemini-1.5-pro	5.1
Friday	11.11
Open-Interpreter	8.94
AgentStore	13.55
LiteCUA	14.66

5. Comparative Innovations and Contextualization

LiteCUA's key innovations include:

• Explicit context adaptation: Rather than forcing LLM agents to parse GUIs designed for humans, the environment is restructured to provide maximally informative, agent-native semantics.
• Tool abstraction: The MCP server acts as the sole interface for tool communication, ensuring agents operate on a standardized set of environment and action schemas.
• Perceive-reason-act modularity: The agent architecture is minimal yet demonstrates that environment reframing is a critical driver of agentability.

Relative to prior agents, LiteCUA's results establish that the complexity-laden pipelines and dense model architectures are less influential than the degree of semantic contextualization of the system environment. By abstracting away pixel and event-level noise and presenting the operational context in a compact, atomic format, even simple agents surpass previous benchmarks.

6. Software Availability and Ecosystem Integration

LiteCUA is open source (github.com/agiresearch/LiteCUA) and integrated into the AIOS main branch (github.com/agiresearch/AIOS), facilitating continued development and research. The modular implementation enables direct experimental manipulation of environment abstraction, memory, kernel facilities, and agent workflow for both benchmarking and production deployment.

7. Implications and Prospects

The LiteCUA paradigm—using MCP-based environment contextualization—suggests that the advancement of computer-use agent intelligence will increasingly depend on the design of native, semantically rich digital environments. This reframing places the burden of interface adaptation on environment architecture instead of agent complexity.

Potential future directions include enhanced multimodal perception, deeper temporal and causal state tracking, and extension of the atomic action space to encompass probabilistic or dynamic action generators. The stable foundation provided by AIOS and MCP is agnostic to agent model scale and design, positioning LiteCUA as a base for scalable, generalizable computer-use agents.

A plausible implication is that as agent research incorporates MCP-style contextualization, subsequent gains in task generalization and long-horizon reasoning can be realized with much simpler agent algorithms—moving research focus towards agent cognition and away from interface engineering.