Cross-Platform Generalization of Computer Use Agents

Establish agents for autonomous computer use that generalize across web, desktop, and mobile environments without relying on environment-specific interfaces, thereby enabling robust cross-platform deployment.

Background

A central motivation of the work is the difficulty of building agents that operate reliably across heterogeneous GUI environments. Prior approaches often depend on environment-specific interfaces such as DOM parsers for web, accessibility trees for mobile, or specialized APIs for desktop applications, which impedes generalization and cross-platform deployment.

Surfer 2 is proposed as a unified architecture designed to work purely from visual observations across web, desktop, and mobile. The paper positions this design as a step toward addressing the broader open challenge of cross-platform generalization without environment-specific dependencies.