MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents

Abstract: Current benchmarks for computer-use agents evaluate models in impersonal environments. This leaves a gap between evaluation and deployment where personal assistants are expected to work across a user's whole digital life, including their context, historical data, and logged-in accounts. This gap is widest on web tasks, where live web evaluations cannot exercise sites that require logging in or personal information, the kind of site a real personal assistant has to drive. We introduce MyPCBench, which tests computer-use agents as personal assistants on a Linux desktop populated with 17 simulated real-world web applications and a full desktop stack, all seeded for one canonical persona, Michael Scott from The Office. We define 184 tasks in this environment, each inspired by a real request drawn from the OpenClaw community, and benchmark six closed and open-weight models with a uniform computer+bash tool surface. We find that the best model, Claude Opus 4.6, fully solves 55.4\% of the tasks, the only model above 50\%. Model failures cluster on tasks that span many applications and on long trajectories, where personalization stresses an assistant the most. We release the environment, task set, and agent harness at https://mypcbench.com.

• The paper introduces a persona-grounded benchmark that simulates realistic, multi-app digital environments for personal computer-use agents.
• It employs 184 diverse tasks targeting multi-step workflows, rigorous cross-app coordination, and long-horizon planning with precise rubric evaluation.
• Experimental results reveal that while some models perform well in single-app scenarios, they struggle with compositional tasks and cross-app interactions.

MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents

Motivation and Problem Formulation

The evaluation of LLM-based personal computer-use agents frequently occurs in impersonal, tabula-rasa environments—empty desktops, generic or ephemeral user data, and hand-structured app states. Such setups are disconnected from the deployment scenarios where agents must act over a user’s full, cross-app, cross-history digital context, leveraging persistent logins, realistic personal information, and the inherent complexity of “lived-in” desktop environments. MyPCBench addresses this central limitation by introducing a benchmark environment with rich, persona-grounded digital history and inter-app consistency.

Figure 1: The MyPCBench environment—a reproducible Linux desktop populated with 17 pre-logged-in web apps and extensive personal data for one canonical persona.

Environment and Persona Synthesis

MyPCBench instantiates a reproducible, deterministic Ubuntu desktop environment seeded for the canonical persona “Michael Scott.” The environment hosts 17 locally-served, Next.js-based web applications mirroring real-world services across finance, travel, food delivery, productivity, chat, and more. The desktop is pre-authenticated for each app and populated with correlated synthetic user data: thousands of emails, financial transactions, events, ride orders, food deliveries, and browsing logs. Cross-app consistency ensures, for example, that a trip event will generate entries in calendar, banking, rideshare, mail, and messaging ecosystems simultaneously, introducing data redundancy, contradictions, and realistic dependencies that stress a genuine personal-assistant agent.

Persona generation is anchored in a single JSON document hierarchically describing identity, history, routines, financial state, and social graph. The environment build process deterministically seeds all databases, cookies, file system, browser state, and session contexts to guarantee experimental repeatability while enabling complex, multi-faceted task scenarios.

Figure 2: The 17 pre-authenticated applications span six major usage domains and support cross-app tasks that mimic real user behavior and preference dependencies.

Task Suite Design

The benchmark defines 184 tasks, each directly inspired by authentic user requests sourced from the OpenClaw community and rewritten to reference only entities and contexts present in Michael Scott’s digital life. Task types intentionally target the limitations of stateless or minimally-seeded evaluation:

• Bounded action: Tightly-scoped, single-app writes
• Multi-step orchestration: Cross-app workflows (e.g., scheduling, notifications)
• Cross-source reconciliation: Consistency checks, counterfactual reasoning
• Aggregation/reporting: Data rollups requiring multi-app queries and result synthesis
• Personal lookup: Retrieval of specific personal records
• Pattern inference: Inference over long-span behaviors (e.g., “usual tip amount”)

Task composition explicitly stresses the need for multi-app coordination and long-horizon planning, with 68% requiring interaction with more than one application. Each task is accompanied by a structured rubric specifying both mandatory side-effects and observable state changes, supporting precise per-item evaluation rather than anecdotal textual inspection.

Figure 4: Distribution of task complexity—most require actions across domains, favoring the emergence of compositional and personalized agent competencies.

Agent Harness and Evaluation Protocol

Agents interface with MyPCBench via an OSWorld-compatible harness. At every time-step, the agent observes a desktop screenshot and action history and emits a command (click, type, key, drag, bash, etc.) mapped onto the native API of each model’s agent protocol. The evaluation policy ensures that each run is independent, resetting the environment to a base snapshot before every trajectory to preclude state leakage.

For metric reporting, a “strict” perfect rate scores a task only if all rubric items are successfully completed; a “rubric score” assigns partial credit proportional to satisfied sub-tasks. An LLM-based judge (Gemini 3.1) evaluates performance, mirroring the MT-Bench and Odysseys protocols and supporting transparent, verifiable scoring.

Experiments and Comparative Analysis

MyPCBench benchmarks six agents from three families—Claude Opus 4.6 / Sonnet 4.6, GPT-5.5 / GPT-5.4 mini, and Qwen 3.5 (35B-A3B and 9B)—using a harmonized computer+bash action schema. Key findings:

• Claude Opus 4.6 achieves 55.4% strict perfect rate, the only model above 50% on this metric.
• GPT-5.5 and GPT-5.4 mini fall to 29.3% and 19.0% respectively, while Qwen models underperform, especially in long-horizon or multi-app tasks.
• For tasks spanning 7+ apps, no open-weight model solves a single instance, and only Claude Opus and GPT-5.5 register any strict successes.
• Task category breakdown reveals that pattern inference and aggregation/reporting—unique to realistic, cross-app personal workflows—are particularly challenging, with only the top Claude model exceeding 80% perfect for pattern inference but all other families below 50%.

Figure 5: Left—Per-task-type strict perfect rate across models; Right—Performance as a function of the number of applications touched by a task. Multi-app, compositional tasks sharply reduce SOTA agent competence.

Analysis of trajectory efficiency (rubric points per agent step) and failure signatures demonstrates that Claude agents employ bash judiciously, sometimes circumventing the GUI surface via direct state queries (“console-script shortcuts”), whereas the GPT and Qwen families are either overly bash-dependent (often to their detriment) or struggle to coherently integrate multiple tool affordances, with Qwen 9B collapsing almost entirely under dual-tool complexity.

Figure 3: Visualization of a Claude Opus 4.6 trajectory, exemplifying a long-horizon, cross-application plan that satisfies all rubric criteria across 99 steps and 10 applications.

Failure analysis classifies errors as premature terminations, app skips, surface-error abandonments, partial artifacts, and hallucinated persona data. The GPT family exhibits a high rate of premature “DONE” predictions, while Qwen models compound hallucination and failure-to-recover errors, especially under bash-enhanced action surfaces. Claude family models, while dominant, are penalized for over-reliance on non-UI actions on tasks requiring explicit UI-visible side-effects rather than only latent database changes.

Figure 7: Family- and model-level breakdown of failed rubric cases, highlighting systematic trends and recurring pitfalls per agent architecture.

Implications and Potential Future Directions

The experimental results conclusively demonstrate that evaluation on static, minimally-personalized or generic desktop benchmarks fundamentally misrepresents the progress and readiness of personal computer-use agents. MyPCBench's strict rubric-driven, persona-grounded, and compositional task design exposes significant remaining gaps even in SOTA agents—especially in multi-app coordination and long-horizon personal data reasoning. Notably, a majority of agents fail to generalize when required to synthesize and effect visible, user-facing changes across contextually entangled applications, with strong overfitting to bash-only or single-surface workflows.

The rigorous environment and agent harness, coupled with fully-open assets and deterministic persona seeding, make MyPCBench a robust foundation for training and validating future personal agents capable of handling complex, high-fidelity real-world settings. The pipeline reveals the need for:

• Enhanced multi-tool policies and affordance arbitration to enforce correct UI-vs-shell action allocation.
• Improved long-horizon planning and execution recovery, especially under compound dependencies or conflicting historical records.
• Systematic evaluation of robustness to hallucinated or incomplete persona data, as well as expansion beyond a single persona for generalized deployment.

Conclusion

MyPCBench establishes a new methodological baseline for evaluating personally intelligent computer-use agents, enforcing competence in realistic, cross-app, contextually coherent settings. The results unambiguously indicate that current frontier agents do not yet match human-level expectations even after adaptation to the personal-assistant use case, particularly on long-horizon, compositional, and side-effect-requiring workflows. Open issues center on tool integration, efficient planning, and robust, user-first persona reasoning. The release of MyPCBench will facilitate reproducible and discriminative empirical scrutiny and drive the community toward architectures that genuinely “use a computer like its owner” rather than excelling only in contrived, stateless testbeds.

[See (2606.16748) for full details and code access.]