CUAWright: Unified Interface for Digital Agents

Updated 7 October 2026

CUAWright is a minimal computer-use harness that treats the computer as a programmable environment where the filesystem serves as both a persistent workspace and a dynamic tool substrate, and Bash commands as the agent's primary action interface.
CUAWright replaces domain-specific harnesses with a programmable terminal-centred environment to automate browsers, execute scripts, construct reusable tools, and request visual observations on demand which improves task performance and lowers estimated cost.
It successfully handles long-horizon tasks, reports cost and success improvement over baseline models, and demonstrates potential in CAD tasks despite focus on prompt injection security and future security and agent error diagnosis features.

CUAWright is a minimal computer-use harness that exposes Bash commands as the agent’s sole primary action interface, while using the filesystem as both persistent workspace and dynamically extensible tool substrate. Introduced in “CUAWright: A Minimal Unified Interface for Digital Agents” (Lu et al., 2 Oct 2026), it replaces domain-specific browser, desktop, and application harnesses with a programmable terminal-centered environment. The system can inspect and modify files, execute scripts, automate browsers, invoke GUI controls when necessary, construct reusable tools, and request visual observations on demand. Its reported evaluations cover web, desktop, long-horizon, and CAD tasks, with results indicating improved task performance and lower estimated cost relative to several GUI-native or hybrid baselines.

1. Conceptual foundations and design rationale

Computer-use agents traditionally operate through domain-specific harnesses. Browser agents may receive screenshots, DOM trees, or accessibility trees and return coordinate or element actions; desktop agents generally produce mouse and keyboard events; CAD systems may expose application-specific APIs or registered tools. CUAWright instead treats the computer as a programmable environment whose underlying state can often be accessed more directly than through pixels.

The paper identifies several limitations of GUI-native interaction. Screenshots expose only a partial projection of system state; primitive actions force the model to reproduce procedures manually; repeated screenshots increase context and inference cost; GUI trajectories leave behind relatively little reusable structure; and browser, desktop, and CAD settings commonly require different action schemas. Hybrid CLI and multi-agent systems reduce some of these limitations but remain GUI-centric and statically designed because tools are selected and engineered before execution.

CUAWright adopts a minimal unified interface. Bash is not merely one tool in a larger catalog: it is the mechanism through which the agent invokes other tools, writes programs, launches applications, controls browsers, captures screenshots, and submits tasks. The central design claim is that a capable coding model can synthesize task-specific procedures more flexibly through a terminal than through a fixed collection of human-engineered GUI actions.

The harness is approximately 3K lines of code. Its conceptual components include a model-to-terminal execution loop, command execution and output collection, optional screenshot capture and image loading, persistent workspace access, environment-specific setup and submission mechanisms, timeouts, output truncation, context compaction, and optional auxiliary agents exposed as command-line tools.

A simplified state formulation models the environment as $s_t=(x_t,f_t)$ , where $x_t$ is application and system state and $f_t$ is persistent filesystem state. Given task $T$ , observation $o_t$ , and history $h_t$ , the model emits a terminal action $a_t=\pi(T,o_t,h_t)$ , and the environment transitions after executing that command. The action interface remains terminal-based even when the command invokes a browser automation library, a GUI utility, a CAD package, or another application.

2. Terminal-centered interaction and visual observation

Bash provides access to a broad set of operations:

inspecting, searching, parsing, transforming, and summarizing files;
launching and controlling applications;
invoking Python and other scripts;
automating browsers through Playwright or the Chrome DevTools Protocol;
creating and modifying artifacts;
capturing and loading screenshots;
using xdotool or pyautogui for keyboard and mouse actions;
executing CadQuery or Build123d programs;
submitting completed tasks.

GUI interaction is therefore retained as a fallback rather than used as the default action language. Browser tasks can use Playwright or CDP to query structured browser state and perform operations in batches. Desktop tasks can manipulate files or application state directly and invoke GUI automation when structured access is insufficient. CAD tasks can generate parametric programs, render them, inspect the resulting images, and revise the programs.

CUAWright uses “vision on demand” rather than attaching screenshots at every interaction step. The agent first invokes a screenshot utility with a target save path and then invokes an image-loading utility for that path. The normal observation is textual terminal output, while visual information is requested when needed for tasks such as reading a visual dashboard, checking a slide layout, or validating a CAD render. This design is intended to reduce image-processing cost and context growth during long trajectories.

The terminal-centered interface is particularly relevant to tasks in which the underlying state is more structured than the visual interface suggests. Browser pages may expose DOM, accessibility, network, and debugging state; desktop applications may store state in files, document packages, databases, or predictable application interfaces; and CAD systems support parametric construction and rendering. A terminal interface can combine these forms of access within a single action space.

The design does not eliminate visual reasoning. Instead, it connects visual interpretation to programmatic execution:

$\text{visual interpretation} \longrightarrow \text{parametric program} \longrightarrow \text{rendered result} \longrightarrow \text{visual verification}.$

This combination is important for CAD and visually constrained desktop tasks, where purely programmatic access is insufficient but repeated low-level GUI interaction is inefficient.

3. Filesystem workspace and dynamic tool construction

CUAWright uses the filesystem as persistent workspace, externalized memory, and an evolvable tool substrate. If $w_t\subseteq f_t$ denotes the workspace at time $t$ , interaction produces a persistent sequence $x_t$ 0. The agent can create notes, logs, extracted values, task requirements, progress records, failed approaches, generated scripts, and intermediate artifacts.

Filesystem persistence allows information to survive context compaction. Facts need not remain in the active conversational transcript: they can be written to files and later retrieved, searched, reorganized, corrected, or deleted. This gives the workspace a mutable structure distinct from an immutable interaction history.

The filesystem also expands the effective program set available to the agent. The paper represents this as:

$x_t$ 1

where $x_t$ 2 contains environment-provided programs and $x_t$ 3 contains executable artifacts created in the workspace. A typical workflow is to identify a repetitive operation, write a script, run and inspect it, revise or debug it, invoke it with new arguments, and reuse it later.

In 89 of 105 OSWorld 2.0 tasks, GPT-5.6 Sol created at least one saved code or executable-like file, corresponding to 84.8% of tasks. It created 776 executable files, of which 192, or 24.7%, were reused in at least two distinct execution-command events. The artifacts included inspection and extraction utilities, interface automation, artifact construction or editing, verification and testing, reusable CDP click helpers, mail lookup scripts, bank-search tools, and CAD-generation programs.

One reported example created a utility that combined 13 medical-document images into a labeled contact sheet, allowing the model to inspect the evidence in one multimodal operation rather than opening each image separately. The filesystem therefore functions not only as storage but as a mechanism for constructing task-specific interfaces.

This property distinguishes CUAWright from a fixed tool library. A static harness determines capabilities before task execution; CUAWright permits the model to specialize its own tools during execution. The approach shifts part of the engineering burden from harness designers to the model’s coding, testing, and verification behavior.

4. Execution settings, resource controls, and safety boundaries

For Online-Mind2Web and Odysseys, the workspace runs on the host and commands are executed through the system shell. Interactive pagers are disabled, observations are truncated at 24,000 characters, transcripts are compacted every 20 steps, per-command timeouts are applied, and each run is capped at 100 model steps.

For OSWorld 2.0, tasks run in the container-hosted Ubuntu virtual machine supplied by OSWorld 2.0. Hardware acceleration is used where available; commands are capped at 300 seconds; command output is capped at 16,000 characters; long output preserves its head and tail around an explicit truncation notice; and the total run is capped at 300 model steps across all roles.

The reported controlled OSWorld executions include the following single-agent results:

Model and harness	Partial reward	Strict success	Cost per task
CUAWright, GPT-5.5 xhigh	63.2%	25.7%	$10.10
CUAWright, GPT-5.6 Sol	67.9%	27.6%	$4.40
GPT-5.5 official baseline	47.5%	13.9%	$16.10
GPT-5.6 Sol official baseline	62.7%	27.3%	$13.90

The experimental environments provide practical containment through OSWorld virtual-machine and container infrastructure, execution timeouts, output truncation, designated workspaces, and benchmark-specific isolation. These mechanisms are operational controls rather than a general-purpose security architecture.

The paper explicitly leaves robust transactional execution, permission control, validation, sandboxing, rollback, and recovery as future work. Direct access to files, application state, configuration, and other persistent resources increases the potential impact of incorrect commands. The principal trade-offs are therefore:

Simplicity versus constrained affordances: Bash is minimal and universal but less semantically constrained than typed APIs.
Programmability versus safety: direct access enables flexible automation but increases the blast radius of mistakes.
Text efficiency versus visual awareness: optional screenshots reduce overhead but require the agent to recognize when visual evidence is necessary.
Single-agent autonomy versus orchestration: a single agent avoids coordination overhead, while specialist agents may identify errors the main agent misses.
Flexible tool creation versus implementation reliability: generated tools can be reusable but may encode bugs or incorrect assumptions.

These safety limitations are significant in light of OS-level CUA threat research. AdvCUA evaluates terminal-oriented agents in multi-host enterprise sandboxes and distinguishes willingness to perform malicious actions from verifiable environmental success using deterministic checks (Luo et al., 8 Oct 2025). Its threat model is not a direct evaluation of CUAWright, but the Bash-centered interface makes command privileges, filesystem scope, network access, credential handling, and multi-host reachability relevant evaluation dimensions.

5. Empirical evaluation across digital tasks

CUAWright is evaluated on Online-Mind2Web, Odysseys, OSWorld 2.0, WeaveBench, BenchCAD, and CadGenBench.

Online-Mind2Web

Online-Mind2Web contains 300 tasks over 136 live websites. With GPT-5.4, CUAWright achieves 88.1% overall success compared with 83.4% for a GUI baseline; CUAWright with GPT-5.5 reaches 89.4%. The improvement over the GUI GPT-5.4 baseline is 4.7 percentage points.

Harness and model	Easy	Medium	Hard	Overall
GUI baseline GPT-5.4	89.9	81.0	76.6	83.4%
CUAWright GPT-5.4	90.3	86.5	88.8	88.1%
CUAWright GPT-5.5	92.9	87.0	90.5	89.4%

The larger gains on medium and hard tasks are consistent with the use of programmatic browser state, reusable scripts, and reduced dependence on coordinate prediction.

Odysseys

Odysseys contains 200 realistic long-horizon web tasks composed of multiple atomic operations across websites. With GPT-5.4, CUAWright reaches 77.5% success compared with 33.5% for the GUI baseline. CUAWright GPT-5.5 reaches 88.0%.

Harness and model	Easy	Medium	Hard	Overall
GUI GPT-5.4	80.0	58.7	3.7	33.5%
CUAWright GPT-5.4	95.6	78.3	69.7	77.5%
CUAWright GPT-5.5	93.3	89.1	85.3	88.0%

The 44.0-percentage-point overall gain over the GUI baseline is particularly pronounced on hard tasks. Persistent files, reusable scripts, and programmatic browser state are plausible contributors to this result.

OSWorld 2.0

CUAWright runs on 105 of 108 public OSWorld 2.0 tasks. Task 026 and task 041 require an unavailable GitLab server, while task 082 requires nested Docker that is not reliably supported by the setup. The maximum trajectory length is 300 steps.

For GPT-5.5, CUAWright improves partial reward from the official baseline’s 47.5% to 63.2%, a 15.7-percentage-point increase and approximately 33.2% relative improvement. Estimated cost decreases from $x_t$ 410.10 per task, approximately 37.5% as reported in the abstract. For GPT-5.6 Sol, partial reward increases from 62.7% to 67.9%, while cost decreases from $x_t$ 54.40.

Partial reward and strict success measure different properties. If $x_t$ 6 is the graded score for task $x_t$ 7, mean partial reward is:

$x_t$ 8

Strict success measures the fraction of tasks satisfying all evaluator criteria. The larger and more consistent partial-reward gains indicate that CUAWright often produces substantially more complete or precise results even when it does not achieve all-or-nothing completion.

Filesystem ablation

Disabling programmatic filesystem writes and changes substantially degrades performance for GPT-5.6 Sol:

full CUAWright: 67.9% partial reward, 27.6% strict success, and $4.40 per task;
no filesystem: 55.5% partial reward, 15.5% strict success, and $4.90 per task.

The supplied discussion contains inconsistencies between some printed difference values and the displayed table, but the substantive direction is clear: restricting filesystem access causes a large degradation. This supports the interpretation that filesystem access provides memory, intermediate state, reusable programs, and task-specific interface construction rather than functioning merely as incidental storage.

Single-agent and multi-agent execution

On OSWorld, the single-agent configuration performs at least as well as the reported multi-agent configuration:

GPT-5.5: 63.2% single-agent versus 60.4% multi-agent;
GPT-5.6 Sol: 67.9% single-agent versus 67.3% multi-agent.

Multi-agent execution increases median task turns from 101 to 151 for GPT-5.6 Sol and from 73 to 137 for GPT-5.5. The reported explanation is that planners sometimes produce inaccurate success criteria, while frontier models can already perform self-verification. Multi-agent review can improve some artifacts but can also consume the step budget through repeated or inconclusive verification.

WeaveBench

WeaveBench contains 114 long-horizon tasks across eight work domains. CUAWright with GPT-5.5 reaches a 53.5% PassRate and an overall score of 66.4, compared with 35.1% and 49.9 for Codex CLI with GPT-5.5.

PassRate is the fraction of tasks with score at least 0.80. CUAWright passes 61 of 114 tasks. The result indicates that a terminal-native interface can compete with harnesses that combine Bash with predefined benchmark-specific GUI tools.

CAD benchmarks

On BenchCAD, CUAWright with GPT-5.5 achieves a mean voxel intersection-over-union of 79.0%, compared with 55.8% for the official GPT-5.5 configuration with a tool. The benchmark asks agents to reconstruct a 3D solid from four orthographic views by generating a CadQuery program. CUAWright scores 78.6% on easy, 81.4% on medium, and 76.9% on hard samples.

The reported workflow includes writing CadQuery, rendering the geometry, comparing the render with reference views, testing geometric hypotheses, and revising the program. One heat-sink example used 249 model turns and 67 image inspections and received voxel IoU 1.0000.

On CadGenBench, which contains 81 public fixtures, CUAWright outperforms a Codex plus Build123d-MCP baseline despite receiving no CAD-specific registered tool interface. With GPT-5.5, CUAWright reaches 49.4% overall compared with 45.7%; with GPT-5.6, it reaches 55.7% compared with 53.2%.

6. Failure modes, security implications, and future directions

CUAWright performs best when tasks expose structured state or can be scripted. Mixed tasks requiring direct visual manipulation remain difficult, including games requiring key presses, drag-and-drop puzzles, focus-sensitive operations, and visually precise GUI interactions. The paper also reports persistent model-level errors: GPT-5.5 most often misread the task, while GPT-5.6 Sol more often produced apparently correct but insufficiently precise outputs or placed correct content in the wrong location.

The system’s security implications are inseparable from its design. Bash as a universal action interface enables commands that can inspect, alter, delete, compile, execute, transmit, and reconfigure system state. A minimal action space does not by itself impose least privilege, provenance, rollback, or authorization boundaries.

Indirect prompt injection is a relevant threat when untrusted web or document content can influence terminal commands. SIR evaluates adaptive web-to-OS attacks in which malicious content is composed from reusable principles, refined through feedback from failed victim trajectories, and scored by deterministic filesystem, service, permission, and artifact checks (Xiong et al., 31 Aug 2026). Its reported attack success rises from 4% to 24% on Claude Opus 4.8 and from 0% to 28% on Gemini 3.5 Flash when adaptive feedback is introduced. These results do not evaluate CUAWright directly, but they indicate why fixed injection suites and refusal-only metrics are insufficient for terminal-capable agents.

Branch steering presents a related issue. An attacker may not need to inject a new command if untrusted data can select a hazardous branch, destination, or parameter already authorized by a plan. “Securing Computer-Use Agents Against Branch Steering Attacks” distinguishes control-flow integrity from data-flow integrity and introduces COBRA, which uses trusted branching plans, ahead-of-time capability constraints, deterministic branch resolution, HTTP and MCP proxies, and authenticated tool descriptions (Zingrillo et al., 2 Oct 2026). On STEER-Bench, COBRA reports 0% attack success under its threat model and 97% benign utility. The result suggests that preventing unauthorized actions is insufficient if untrusted content can determine the parameters or branches of authorized actions.

Failure diagnosis and repair are another open direction. CUADebug studies multimodal root-cause analysis over failed CUA trajectories, using paired before-and-after screenshots, action traces, structured error taxonomies, and debugger-guided re-execution (Zhang et al., 31 Jul 2026). Its 204 human-annotated failures include task reasoning and control, perception, grounding and interaction, external/system, and infeasible-task cases. On the principal split, joint subtype-and-step diagnosis improves from 11.2% to 19.6%, while structured repair improves single re-execution task completion from 13.89% to 29.90% and continual re-execution from 12.20% to 25.86%. For CUAWright, a corresponding recovery layer could inspect command output, filesystem changes, screenshots, process state, and intended effects before deciding whether to retry, revise, roll back, or escalate.

Future work indicated by the combined research context includes:

Permission and capability control: constrain filesystem, process, network, and service access rather than relying solely on prompts.
Transactional execution and rollback: support atomic changes, checkpoints, recovery, and restoration after failed commands.
Deterministic postcondition verification: distinguish issued commands from actual environmental effects.
Provenance-aware input handling: separate user authorization from untrusted web, document, tool, or terminal content.
Branch and parameter constraints: prevent untrusted observations from selecting security-sensitive destinations or arguments.
Adaptive red-team evaluation: test against attackers that learn from refusals and failed trajectories.
Failure-aware recovery: preserve command traces, visual evidence, filesystem artifacts, and causal hypotheses for repair.
Broader platform evaluation: extend beyond the reported Ubuntu-oriented environments and benchmark configurations.
Matched comparison protocols: resolve infrastructure exclusions, differing baseline conditions, and inconsistencies in printed evaluation values.

CUAWright’s broader significance lies in its treatment of digital environments as programmable rather than primarily visual. Its evidence indicates that a small, compositional, terminal-focused interface can improve web automation, desktop interaction, long-horizon execution, and CAD reconstruction while reducing estimated cost. The same design also transfers substantial responsibility to the model and execution environment. The central unresolved problem is therefore not whether a terminal can unify computer-use tasks, but how to preserve its programmability while adding enforceable permissions, provenance, verification, recovery, and security guarantees.