Cross-Platform Generalization of Computer Use Agents

Establish agents for autonomous computer use that generalize across web, desktop, and mobile environments without relying on environment-specific interfaces, thereby enabling robust cross-platform deployment.

Background

A central motivation of the work is the difficulty of building agents that operate reliably across heterogeneous GUI environments. Prior approaches often depend on environment-specific interfaces such as DOM parsers for web, accessibility trees for mobile, or specialized APIs for desktop applications, which impedes generalization and cross-platform deployment.

Surfer 2 is proposed as a unified architecture designed to work purely from visual observations across web, desktop, and mobile. The paper positions this design as a step toward addressing the broader open challenge of cross-platform generalization without environment-specific dependencies.

References

Building agents that generalize across web, desktop, and mobile environments remains an open challenge, as prior systems rely on environment-specific interfaces that limit cross-platform deployment.

— Surfer 2: The Next Generation of Cross-Platform Computer Use Agents  (2510.19949 - Andreux et al., 22 Oct 2025) in Abstract, Page 1

These mechanisms suggest possible implementations of Affora's semantic substrate beyond the DOM; transfer to them has not been evaluated here. The same principles can therefore be asked of desktop software: whether visible controls correspond to machine-operable elements, whether state and available actions are recoverable, and whether task state survives transitions between views or windows. Testing Affora across native and Electron applications would determine how far these rules generalize beyond the web.

— Affora: A Design System for Agent-Friendly Interfaces  (2609.19125 - Gao, 16 Sep 2026) in Section 7.4, “Beyond the DOM: desktop interfaces and software for human–agent collaboration”

Finally, application adaptation assumes software that is open-source or scriptable enough to expose a command-line surface, and our environments target desktop Linux; the coding-agent-driven construction is not tied to these choices in principle, but broadening to closed-source or non-desktop platforms remains open.

— CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents  (2609.05374 - Shi et al., 4 Sep 2026) in Section 6, “Limitations”