MyPCBench: Personal Agent Benchmark

• MyPCBench is a benchmark evaluating personal computer-use agents by simulating realistic user environments with persistent persona data.
• It tests agents across 184 tasks that span multi-application workflows and visible side effects verified through deterministic snapshot resets.
• The benchmark highlights challenges in long-horizon planning and cross-application consistency, revealing notable performance differences among models.

Searching arXiv for the cited benchmark papers to ground the article in current paper metadata and identifiers. MyPCBench is a benchmark for personally intelligent computer-use agents that evaluates models as personal assistants on a Linux desktop populated with authenticated, simulated consumer applications and persistent personal context (Jang et al., 15 Jun 2026). It is designed to address a gap between common agent evaluations in impersonal environments and the deployment setting in which assistants are expected to operate across a user’s digital life, including historical data, logged-in accounts, and cross-application state (Jang et al., 15 Jun 2026). The benchmark introduces a reproducible Ubuntu virtual machine seeded for one canonical persona, Michael Scott from The Office, defines 184 tasks derived from anonymized OpenClaw requests, and grades agent behavior through rubric-based evaluation of visible user-side effects under deterministic snapshot reset (Jang et al., 15 Jun 2026). In this setting, the reported best model, Claude Opus 4.6, fully solves 55.4\% of tasks, while failures concentrate on long trajectories and tasks spanning many applications (Jang et al., 15 Jun 2026).

1. Concept and motivation

MyPCBench is motivated by the claim that most prior evaluations of computer-use agents are impersonal: they typically run on stock desktops, use minimal seeding, and provide little or no personal history (Jang et al., 15 Jun 2026). The paper identifies web tasks as a particularly acute case, because live-web evaluations generally avoid sites that require authentication or personal data, even though those are central to the workflows expected of a real personal assistant (Jang et al., 15 Jun 2026). The benchmark therefore focuses on a setting in which agents must act over persistent identity, long personal histories, and authenticated application surfaces rather than isolated GUI flows (Jang et al., 15 Jun 2026).

The benchmark’s novelty is described as threefold: personalization through one coherent persona, logged-in multi-application coverage across 17 realistic consumer applications, and reproducibility through a deterministic Ubuntu VM image with snapshot reset and consistent cross-application state (Jang et al., 15 Jun 2026). The persona-centered design extends beyond profile metadata: seeded records are linked across email, calendar, travel, banking, messaging, shopping, and browser history, so that an event in one application corresponds deterministically to traces in others (Jang et al., 15 Jun 2026).

This emphasis places MyPCBench within the broader research program on benchmark realism. The benchmark contrasts itself with web and desktop evaluations such as WebArena, VisualWebArena, Mind2Web, WebVoyager, OSWorld, Windows Agent Arena, MacOSWorld, WorkArena, and AndroidWorld, which are characterized as using synthetic or lite personal states, avoiding authenticated sites, or evaluating stock-state desktops (Jang et al., 15 Jun 2026). It also differs from personalization benchmarks such as LaMP, LongMemEval, PersonalWAB, and Persona2Web by embedding identity into the entire desktop environment rather than handing the model an explicit profile or memory store (Jang et al., 15 Jun 2026).

2. Environment architecture and persona seeding

The released environment is a QEMU/KVM Ubuntu 24.04 virtual machine running GNOME Shell, shipped either as a Docker image or a standalone qcow2 disk (Jang et al., 15 Jun 2026). Firefox is preloaded with a profile, the full LibreOffice suite is installed, and two web applications, HooliWork and HooliChat, are also exposed as native desktop applications (Jang et al., 15 Jun 2026). Observation is screenshot-based: agents see the desktop as screenshots and interact through the OSWorld-compatible HTTP Control API, specifically GET /screenshot and POST /execute on port 5000 (Jang et al., 15 Jun 2026).

The benchmark includes 17 simulated web applications, implemented as Next.js clones of real consumer products, served locally, backed by SQLite, and pre-logged-in (Jang et al., 15 Jun 2026). Together they span six SimilarWeb top-level domains—Computers/Tech, Finance, Travel, Food, Ecommerce, and Gambling—and expose 226 database tables with approximately 42,000 rows of user-facing state (Jang et al., 15 Jun 2026). Product images are real photos, with image–title alignment checked by a vision–LLM (Jang et al., 15 Jun 2026).

The applications and their seeded scopes are as follows:

Application	Real-world analogue	Seeded scope
HooliMail	Gmail	2,398 seeded messages
HooliCalendar	Google Calendar	679 personal/work events
HooliWork	Slack	Branch channels, DMs, read state
SprintBoard	Jira	Sprints and tickets
HooliChat	WhatsApp	One-to-one and group threads
LockedIn	LinkedIn	18 jobs across 9 companies
HangryDash	DoorDash	28 restaurants, 165 menu items
TableFind	OpenTable	4,128-slot inventory
Kwik-E-Mart	Instacart	7 stores, 109 curated SKUs
HooliShop	Amazon	90-SKU catalog
Dinoco Airlines	Delta	14 booked flights across 5 AVP hubs
Cheskepdia	Airbnb	Trips, wishlists, search
eTaxi	Uber	Trip history, saved places
SpeedTax	TurboTax	Prior-year returns, W-2s, 1099s
Gringotts	Chase Bank	1,812 transaction log
BatBucks	Robinhood	Holdings, watchlist
OddsMarket	Polymarket	Open positions, watchlist

The canonical persona is Michael Scott, chosen as a low-sensitivity fictional identity to avoid real personally identifiable information while preserving realistic cross-application coupling (Jang et al., 15 Jun 2026). The seed includes 1,812 bank transactions, 2,398 emails, 679 calendar events, 2,526 chat and work messages, 126 rideshare requests, 402 food-delivery orders, 155 retail orders, 29 grocery orders, 32 restaurant reservations, 35 bookmarks, and 10,746 browsing-history visits (Jang et al., 15 Jun 2026). Persona specification is encoded as JSON with top-level fields for identity, contacts, financials, investments, prediction markets, routines, trips, work, tax information, planted contradictions and dependencies, browsing patterns, shopping, application overrides, and cross-application events (Jang et al., 15 Jun 2026).

Cross-application consistency is deterministic. A single event, such as a trip or dinner, generates correlated records across relevant services (Jang et al., 15 Jun 2026). The paper gives examples in which a Philadelphia trip produces a Cheskepdia booking, Gringotts charges, a HooliCalendar block, Dinoco boarding passes, relevant browsing history, emails, and HooliChat messages; similarly, a dinner-plan chain for Cooper’s Seafood House generates a TableFind reservation, a Gringotts charge, a calendar event, browser history, and chat messages (Jang et al., 15 Jun 2026). Runtime effects are also coupled, as when a HangryDash order triggers a Gringotts charge and a HooliMail confirmation (Jang et al., 15 Jun 2026).

Determinism is enforced through seeders keyed on persona and seeder, and a base snapshot is captured after first boot and restored between tasks (Jang et al., 15 Jun 2026). The default VM allocation is 4 vCPUs and 8 GB RAM, with a boot-to-ready time of approximately 90 seconds (Jang et al., 15 Jun 2026). By default, all applications are hosted locally inside the guest and there is no live-web traffic (Jang et al., 15 Jun 2026).

3. Task suite and behavioral taxonomy

The benchmark defines 184 tasks sourced from 2,749 anonymized OpenClaw community requests, after filtering near-duplicates, infeasible-in-VM requests such as “call my mom,” and requests requiring applications outside the hosted set (Jang et al., 15 Jun 2026). Named entities are rewritten to match Michael Scott’s seeded data, and each task is packaged with a rubric in the Odysseys format (Jang et al., 15 Jun 2026). Every task was audited by at least two authors end-to-end in the live VM (Jang et al., 15 Jun 2026).

The benchmark organizes tasks into six behavioral types:

Behavioral type	Count	Share
Bounded action	64	35\%
Multi-step orchestration	48	26\%
Cross-source reconciliation	25	14\%
Aggregation and reporting	23	12\%
Personal lookup	13	7\%
Pattern inference	11	6\%

Representative examples illustrate the intended workload structure. A bounded-action task may ask the agent to “Zelle Pam a hundred bucks” after checking HooliChat contacts and adding a memo; an orchestration task may involve reviving a fan club by consulting group chat, recruiting contacts through email, and booking a watch party on the calendar; a cross-source reconciliation task may ask whether two trips four weeks apart are affordable given the credit-card balance; an aggregation task may require computing monthly Zelle sends by recipient for the last two complete months and producing a ranked LibreOffice Calc spreadsheet; and a pattern-inference task may require inferring typical delivery tipping behavior from historical orders (Jang et al., 15 Jun 2026).

The suite is explicitly multi-application and history-dependent. Sixty-eight percent of tasks touch multiple applications, ranging from 1 to 19 apps per task, and 40\% span at least two SimilarWeb top-level domains (Jang et al., 15 Jun 2026). Rubrics contain between 3 and 13 items per task, with mean 6.5, totaling 1,191 items (Jang et al., 15 Jun 2026). Crucially, rubric items grade visible state changes and artifacts—such as files saved, cards moved, and reservations made—rather than only the final textual answer (Jang et al., 15 Jun 2026).

This rubric design has methodological consequences. It rewards partial task completion while still distinguishing workflows that require persistent, user-visible side effects from workflows that only require extracting information (Jang et al., 15 Jun 2026). A plausible implication is that MyPCBench evaluates both epistemic competence, in the sense of correctly reconciling personal data, and operational competence, in the sense of producing externally verifiable desktop outcomes.

4. Agent interface and evaluation protocol

MyPCBench models the environment as a partially observable Markov decision process, following the POMDP framing cited in the paper (Jang et al., 15 Jun 2026). At each step, the agent receives a $1280 \times 800$ screenshot and the action history, then returns an action executed through the OSWorld HTTP Control API (Jang et al., 15 Jun 2026). At most 20 recent screenshots are retained in context; older screenshots are replaced by a text placeholder (Jang et al., 15 Jun 2026).

The action surface is based on the OSWorld pyautogui vocabulary: click, type, key, scroll, drag, wait, screenshot, done, and fail (Jang et al., 15 Jun 2026). All agents in cua+bash mode also receive a shell tool, bash (Jang et al., 15 Jun 2026). Anthropic’s str_replace_based_edit_tool is available only to Claude because other providers do not document an equivalent (Jang et al., 15 Jun 2026). For OpenAI and Qwen models, the harness appends a short dual-tool usage hint directing the shell toward read-only parsing and computation and reserving the GUI for visible side effects; Claude receives no such hint (Jang et al., 15 Jun 2026).

The harness restores a fresh base snapshot before every task and runs a ReAct-style loop until DONE, FAIL, or exhaustion of the 100-turn budget (Jang et al., 15 Jun 2026). VNC and noVNC are available only for human observation (Jang et al., 15 Jun 2026). Logging records each action, screenshot, wall time, and final answer, and replay is deterministic because of snapshot reset (Jang et al., 15 Jun 2026).

Evaluation uses an LLM-as-a-judge protocol. Each task contains rubric items $\{r_1,\ldots,r_N\}$ with normalized per-task weights, and the judge, gemini-3.1-flash-lite-preview, runs once per rubric item over the full trajectory (Jang et al., 15 Jun 2026). It receives the task instruction, the rubric item, the complete action history, and every screenshot in chronological order, and returns either “success” or “failure” for that item (Jang et al., 15 Jun 2026). The paper defines the per-item success indicator $s_{i,r} \in \{0,1\}$, per-task score

$s_i = \Sigma_r w_{i,r} s_{i,r},$

model-level rubric score

$\overline{s}=\frac{1}{T}\sum_i s_i,$

perfect rate

$\frac{1}{T}\sum_i \mathbb{1}[s_i = 1],$

and Trajectory Efficiency

$$\text{Traj.\ Eff.} \;=\; \frac{1}{T}\sum_{i=1}^{T} \frac{s_i}{n_i},$$

reported as percent per step (Jang et al., 15 Jun 2026).

The reported protocol uses single canonical runs per task rather than best-of-$N$ evaluation, and confidence intervals are not reported because of the cost of end-to-end VM rollout (Jang et al., 15 Jun 2026). This means reported values are directly tied to one execution trace per task per model.

5. Baselines, scaling behavior, and empirical results

MyPCBench reports results for six models, each driven by its provider’s native computer-use agent with the shared computer-plus-bash surface: Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.5, GPT-5.4 mini, Qwen 3.5 35B-A3B, and Qwen 3.5 9B (Jang et al., 15 Jun 2026). All runs use the same persona context block, the same 100-turn budget, and the same Gemini judge (Jang et al., 15 Jun 2026).

The headline results are as follows:

Model	Perfect rate	Rubric score
Claude Opus 4.6	55.4\%	81.8\%
Claude Sonnet 4.6	39.1\%	65.4\%
GPT-5.5	29.3\%	54.1\%
GPT-5.4 mini	19.0\%	48.8\%
Qwen 3.5 35B-A3B	7.6\%	42.5\%
Qwen 3.5 9B	2.7\%	7.0\%

Claude Opus 4.6 is the only model above 50\% perfect and fully solves 55.4\% of tasks (Jang et al., 15 Jun 2026). Average steps and Trajectory Efficiency further differentiate models: Opus records 46.5 average steps and 3.61 Trajectory Efficiency; Sonnet 45.8 and 3.03; GPT-5.5 45.8 and 1.45; GPT-5.4 mini 43.7 and 1.65; Qwen 35B 66.0 and 1.41; Qwen 9B 69.2 and 0.65 (Jang et al., 15 Jun 2026).

Per-task-type performance shows that bounded action, pattern inference, and personal lookup are materially easier than aggregation/reporting and multi-step orchestration (Jang et al., 15 Jun 2026). For pattern inference, Opus reaches 94.7\% rubric and perfects 82\% of those tasks, whereas GPT-5.5 reaches 59.1\% rubric and 45\% perfect (Jang et al., 15 Jun 2026). For aggregation/reporting and orchestration, GPT-5.5 stays below 35\% rubric on aggregation and 32.5\% on orchestration (Jang et al., 15 Jun 2026). Qwen 9B perfects zero tasks across personal lookup, aggregation, pattern inference, and cross-source reconciliation (Jang et al., 15 Jun 2026).

Performance degrades sharply with application count. For Opus, single-app tasks achieve 66\% perfect and 87.4\% rubric, while tasks involving seven or more apps drop to 36\% perfect and 67.9\% rubric (Jang et al., 15 Jun 2026). Sonnet drops from 46\% to 14\% perfect across the same range (Jang et al., 15 Jun 2026). GPT-5.4 mini, Qwen 35B, and Qwen 9B all reach 0\% perfect at 7+ apps, while GPT-5.5 records 4.5\% perfect in that slice (Jang et al., 15 Jun 2026).

Step-budget scaling provides another lens on horizon limitations. Opus continues improving at the 100-step cap, GPT-family models flatten around step 60, and Qwen saturates around step 25 (Jang et al., 15 Jun 2026). Some cua+bash trajectories exceed 100 steps despite the 100-turn cap because a single turn can emit several pyautogui actions plus a shell call (Jang et al., 15 Jun 2026).

Tool-surface ablation indicates that adding bash and a dual-tool hint to computer-only agents produces modest changes for GPT-5.5 and GPT-5.4 mini, a mixed effect for Qwen 35B, and a large regression for Qwen 9B (Jang et al., 15 Jun 2026). The paper states that equalizing the tool surface does not overturn the model ordering (Jang et al., 15 Jun 2026).

6. Failure modes, interpretation, and methodological significance

The paper groups failed-rubric judge explanations into five modes: premature termination (DONE), skipped required app, surface error treated as terminal, partial artifact, and hallucinated persona data (Jang et al., 15 Jun 2026). Aggregated counts across models are 354, 323, 129, 47, and 31 respectively (Jang et al., 15 Jun 2026). These categories emphasize that failure is often procedural rather than purely perceptual or semantic.

Family-specific patterns are also reported. GPT-5.4 mini and GPT-5.5 overuse premature DONE, with 130 and 105 such failures respectively, compared with 28 for Opus and 38 for Sonnet (Jang et al., 15 Jun 2026). Qwen 35B leads in hallucinated persona data and surface-error abandonment, while Qwen 9B is reported to collapse under the dual-tool schema, zero-scoring 164 of 184 tasks and emitting malformed tool calls that splice bash and computer schemas (Jang et al., 15 Jun 2026). Claude models exhibit a different pathology: console-script shortcuts, including JavaScript console access and curl to REST endpoints via bash, can retrieve application state without producing the user-visible side effects required by certain rubrics (Jang et al., 15 Jun 2026).

One consequence is that perfect rate falls faster than rubric score. Skipped-app and premature-termination failures can invalidate an entire task even if many rubric items have already passed (Jang et al., 15 Jun 2026). This design makes the benchmark sensitive to end-to-end completion discipline rather than merely local competence. The benchmark therefore measures not only whether an agent can infer the correct state of the world, but whether it can carry a multi-application workflow through to the externally visible result.

The benchmark’s emphasis on visible artifacts and deterministic reset aligns it with desktop-evaluation infrastructure descended from OSWorld, while extending that setup into a personalized regime (Jang et al., 15 Jun 2026). This suggests a methodological shift from evaluating generic GUI navigation to evaluating identity-grounded task execution under persistent, coherent personal history.

7. Relation to other benchmarking traditions and stated limitations

Despite sharing a name fragment with earlier PC benchmarking work, MyPCBench is unrelated in subject matter to the 2017 IAC study on benchmarking institutional computing resources with Polyhedron Fortran workloads (Caon et al., 2017). That work concerns HTC scheduling, compiler comparisons, and normalized execution-time measurement on heterogeneous desktops and racks (Caon et al., 2017). It does, however, illustrate a distinct benchmark tradition centered on reproducibility, normalization, and operational deployment constraints, which provides a useful contrast with MyPCBench’s focus on computer-use agents rather than CPU performance (Caon et al., 2017). A plausible implication is that both efforts treat benchmarking as a systems problem, but at different abstraction layers: one at machine-performance level, the other at agent-environment interaction level.

Another useful contrast is with nanoBench, a low-overhead x86 microbenchmarking tool designed for isolated instruction sequences and cache characterization (Abel et al., 2019). nanoBench measures latency, throughput, port usage, and cache behavior with hardware counters and kernel-level control (Abel et al., 2019). MyPCBench operates at the opposite end of the granularity spectrum, evaluating long-horizon, screenshot-conditioned, multi-application workflows rather than microarchitectural primitives (Abel et al., 2019). This suggests that “PC benchmarking” now spans at least three distinct layers present in the cited materials: hardware resource benchmarking (Caon et al., 2017), microarchitectural benchmarking (Abel et al., 2019), and agentic desktop-task benchmarking (Jang et al., 15 Jun 2026).

The paper states several limitations. The benchmark uses one canonical persona and one Linux/GNOME/Firefox stack, favoring depth over diversity (Jang et al., 15 Jun 2026). Grading relies on a single Gemini judge, so absolute counts are described as structural rather than precise prevalence estimates (Jang et al., 15 Jun 2026). The environment is simulated rather than live-web, which avoids privacy and variability concerns but places behaviors tied to genuinely sensitive data out of scope by design (Jang et al., 15 Jun 2026). Dual-use concerns are explicitly acknowledged: the skills evaluated generalize to agents that could operate real logged-in accounts, though the released benchmark mitigates this through synthetic persona data, local hosting, and a recommendation to benchmark offline against the released image only (Jang et al., 15 Jun 2026).

Future directions identified in the paper include extending to multiple personas, locales, and life stages; dynamic updates to personal contexts and cross-application events; multi-judge ensembles and additional automatic checks; more evaluation metrics such as recovery rate after errors and explicit multi-app reconciliation scores; improved agent memory and planning; stronger hybrid GUI/shell policies; and expansion to Windows, macOS, alternative browsers, mobile surfaces, and a broader application suite (Jang et al., 15 Jun 2026).

MyPCBench’s main significance lies in converting personalization from auxiliary metadata into the core substrate of evaluation. By seeding one coherent identity across authenticated clones of banking, travel, delivery, messaging, productivity, and browser environments, then scoring visible side effects under deterministic reset, it defines a benchmark regime in which cross-application consistency, long-horizon planning, and completion discipline become first-class empirical variables (Jang et al., 15 Jun 2026).