---
title: 'MyPCBench: Personal Agent Benchmark'
url: https://www.emergentmind.com/topics/mypcbench
type: topic
---

# MyPCBench: Personal Agent Benchmark

Searching arXiv for the cited benchmark papers to ground the article in current paper metadata and identifiers.
MyPCBench is a benchmark for personally intelligent computer-use agents that evaluates models as personal assistants on a Linux desktop populated with authenticated, simulated consumer applications and persistent personal context [2606.16748]. It is designed to address a gap between common agent evaluations in impersonal environments and the deployment setting in which assistants are expected to operate across a user’s digital life, including historical data, logged-in accounts, and cross-application state [2606.16748]. The benchmark introduces a reproducible Ubuntu virtual machine seeded for one canonical persona, Michael Scott from *The Office*, defines 184 tasks derived from anonymized OpenClaw requests, and grades agent behavior through rubric-based evaluation of visible user-side effects under deterministic snapshot reset [2606.16748]. In this setting, the reported best model, Claude Opus 4.6, fully solves 55.4\% of tasks, while failures concentrate on long trajectories and tasks spanning many applications [2606.16748].

## 1. Concept and motivation

MyPCBench is motivated by the claim that most prior evaluations of computer-use agents are impersonal: they typically run on stock desktops, use minimal seeding, and provide little or no personal history [2606.16748]. The paper identifies web tasks as a particularly acute case, because live-web evaluations generally avoid sites that require authentication or personal data, even though those are central to the workflows expected of a real personal assistant [2606.16748]. The benchmark therefore focuses on a setting in which agents must act over persistent identity, long personal histories, and authenticated application surfaces rather than isolated GUI flows [2606.16748].

The benchmark’s novelty is described as threefold: personalization through one coherent persona, logged-in multi-application coverage across 17 realistic consumer applications, and reproducibility through a deterministic Ubuntu VM image with snapshot reset and consistent cross-application state [2606.16748]. The persona-centered design extends beyond profile metadata: seeded records are linked across email, calendar, travel, banking, messaging, shopping, and browser history, so that an event in one application corresponds deterministically to traces in others [2606.16748].

This emphasis places MyPCBench within the broader research program on benchmark realism. The benchmark contrasts itself with web and desktop evaluations such as WebArena, VisualWebArena, Mind2Web, WebVoyager, OSWorld, Windows Agent Arena, MacOSWorld, WorkArena, and AndroidWorld, which are characterized as using synthetic or lite personal states, avoiding authenticated sites, or evaluating stock-state desktops [2606.16748]. It also differs from personalization benchmarks such as LaMP, LongMemEval, PersonalWAB, and Persona2Web by embedding identity into the entire desktop environment rather than handing the model an explicit profile or memory store [2606.16748].

## 2. Environment architecture and persona seeding

The released environment is a QEMU/KVM Ubuntu 24.04 virtual machine running GNOME Shell, shipped either as a Docker image or a standalone qcow2 disk [2606.16748]. Firefox is preloaded with a profile, the full LibreOffice suite is installed, and two web applications, HooliWork and HooliChat, are also exposed as native desktop applications [2606.16748]. Observation is screenshot-based: agents see the desktop as screenshots and interact through the OSWorld-compatible HTTP Control API, specifically `GET /screenshot` and `POST /execute` on port 5000 [2606.16748].

The benchmark includes 17 simulated web applications, implemented as Next.js clones of real consumer products, served locally, backed by SQLite, and pre-logged-in [2606.16748]. Together they span six SimilarWeb top-level domains—Computers/Tech, Finance, Travel, Food, Ecommerce, and Gambling—and expose 226 database tables with approximately 42,000 rows of user-facing state [2606.16748]. Product images are real photos, with image–title alignment checked by a vision–language model [2606.16748].

The applications and their seeded scopes are as follows:

| Application | Real-world analogue | Seeded scope |
|---|---|---|
| HooliMail | Gmail | 2,398 seeded messages |
| HooliCalendar | Google Calendar | 679 personal/work events |
| HooliWork | Slack | Branch channels, DMs, read state |
| SprintBoard | Jira | Sprints and tickets |
| HooliChat | WhatsApp | One-to-one and group threads |
| LockedIn | LinkedIn | 18 jobs across 9 companies |
| HangryDash | DoorDash | 28 restaurants, 165 menu items |
| TableFind | OpenTable | 4,128-slot inventory |
| Kwik-E-Mart | Instacart | 7 stores, 109 curated SKUs |
| HooliShop | Amazon | 90-SKU catalog |
| Dinoco Airlines | Delta | 14 booked flights across 5 AVP hubs |
| Cheskepdia | Airbnb | Trips, wishlists, search |
| eTaxi | Uber | Trip history, saved places |
| SpeedTax | TurboTax | Prior-year returns, W-2s, 1099s |
| Gringotts | Chase Bank | 1,812 transaction log |
| BatBucks | Robinhood | Holdings, watchlist |
| OddsMarket | Polymarket | Open positions, watchlist |

The canonical persona is Michael Scott, chosen as a low-sensitivity fictional identity to avoid real personally identifiable information while preserving realistic cross-application coupling [2606.16748]. The seed includes 1,812 bank transactions, 2,398 emails, 679 calendar events, 2,526 chat and work messages, 126 rideshare requests, 402 food-delivery orders, 155 retail orders, 29 grocery orders, 32 restaurant reservations, 35 bookmarks, and 10,746 browsing-history visits [2606.16748]. Persona specification is encoded as JSON with top-level fields for identity, contacts, financials, investments, prediction markets, routines, trips, work, tax information, planted contradictions and dependencies, browsing patterns, shopping, application overrides, and cross-application events [2606.16748].

Cross-application consistency is deterministic. A single event, such as a trip or dinner, generates correlated records across relevant services [2606.16748]. The paper gives examples in which a Philadelphia trip produces a Cheskepdia booking, Gringotts charges, a HooliCalendar block, Dinoco boarding passes, relevant browsing history, emails, and HooliChat messages; similarly, a dinner-plan chain for Cooper’s Seafood House generates a TableFind reservation, a Gringotts charge, a calendar event, browser history, and chat messages [2606.16748]. Runtime effects are also coupled, as when a HangryDash order triggers a Gringotts charge and a HooliMail confirmation [2606.16748].

Determinism is enforced through seeders keyed on persona and seeder, and a base snapshot is captured after first boot and restored between tasks [2606.16748]. The default VM allocation is 4 vCPUs and 8 GB RAM, with a boot-to-ready time of approximately 90 seconds [2606.16748]. By default, all applications are hosted locally inside the guest and there is no live-web traffic [2606.16748].

## 3. Task suite and behavioral taxonomy

The benchmark defines 184 tasks sourced from 2,749 anonymized OpenClaw community requests, after filtering near-duplicates, infeasible-in-VM requests such as “call my mom,” and requests requiring applications outside the hosted set [2606.16748]. Named entities are rewritten to match Michael Scott’s seeded data, and each task is packaged with a rubric in the Odysseys format [2606.16748]. Every task was audited by at least two authors end-to-end in the live VM [2606.16748].

The benchmark organizes tasks into six behavioral types:

| Behavioral type | Count | Share |
|---|---:|---:|
| Bounded action | 64 | 35\% |
| Multi-step orchestration | 48 | 26\% |
| Cross-source reconciliation | 25 | 14\% |
| Aggregation and reporting | 23 | 12\% |
| Personal lookup | 13 | 7\% |
| Pattern inference | 11 | 6\% |

Representative examples illustrate the intended workload structure. A bounded-action task may ask the agent to “Zelle Pam a hundred bucks” after checking HooliChat contacts and adding a memo; an orchestration task may involve reviving a fan club by consulting group chat, recruiting contacts through email, and booking a watch party on the calendar; a cross-source reconciliation task may ask whether two trips four weeks apart are affordable given the credit-card balance; an aggregation task may require computing monthly Zelle sends by recipient for the last two complete months and producing a ranked LibreOffice Calc spreadsheet; and a pattern-inference task may require inferring typical delivery tipping behavior from historical orders [2606.16748].

The suite is explicitly multi-application and history-dependent. Sixty-eight percent of tasks touch multiple applications, ranging from 1 to 19 apps per task, and 40\% span at least two SimilarWeb top-level domains [2606.16748]. Rubrics contain between 3 and 13 items per task, with mean 6.5, totaling 1,191 items [2606.16748]. Crucially, rubric items grade visible state changes and artifacts—such as files saved, cards moved, and reservations made—rather than only the final textual answer [2606.16748].

This rubric design has methodological consequences. It rewards partial task completion while still distinguishing workflows that require persistent, user-visible side effects from workflows that only require extracting information [2606.16748]. A plausible implication is that MyPCBench evaluates both epistemic competence, in the sense of correctly reconciling personal data, and operational competence, in the sense of producing externally verifiable desktop outcomes.

## 4. Agent interface and evaluation protocol

MyPCBench models the environment as a partially observable Markov decision process, following the POMDP framing cited in the paper [2606.16748]. At each step, the agent receives a $1280 \times 800$ screenshot and the action history, then returns an action executed through the OSWorld HTTP Control API [2606.16748]. At most 20 recent screenshots are retained in context; older screenshots are replaced by a text placeholder [2606.16748].

The action surface is based on the OSWorld `pyautogui` vocabulary: `click`, `type`, `key`, `scroll`, `drag`, `wait`, `screenshot`, `done`, and `fail` [2606.16748]. All agents in `cua+bash` mode also receive a shell tool, `bash` [2606.16748]. Anthropic’s `str_replace_based_edit_tool` is available only to Claude because other providers do not document an equivalent [2606.16748]. For OpenAI and Qwen models, the harness appends a short dual-tool usage hint directing the shell toward read-only parsing and computation and reserving the GUI for visible side effects; Claude receives no such hint [2606.16748].

The harness restores a fresh base snapshot before every task and runs a ReAct-style loop until `DONE`, `FAIL`, or exhaustion of the 100-turn budget [2606.16748]. VNC and noVNC are available only for human observation [2606.16748]. Logging records each action, screenshot, wall time, and final answer, and replay is deterministic because of snapshot reset [2606.16748].

Evaluation uses an LLM-as-a-judge protocol. Each task contains rubric items $\{r_1,\ldots,r_N\}$ with normalized per-task weights, and the judge, `gemini-3.1-flash-lite-preview`, runs once per rubric item over the full trajectory [2606.16748]. It receives the task instruction, the rubric item, the complete action history, and every screenshot in chronological order, and returns either “success” or “failure” for that item [2606.16748]. The paper defines the per-item success indicator $s_{i,r} \in \{0,1\}$, per-task score
\[
s_i = \Sigma_r w_{i,r} s_{i,r},
\]
model-level rubric score
\[
\overline{s}=\frac{1}{T}\sum_i s_i,
\]
perfect rate
\[
\frac{1}{T}\sum_i \mathbb{1}[s_i = 1],
\]
and Trajectory Efficiency
\[
\text{Traj.\ Eff.} \;=\; \frac{1}{T}\sum_{i=1}^{T} \frac{s_i}{n_i},
\]
reported as percent per step [2606.16748].

The reported protocol uses single canonical runs per task rather than best-of-\(N\) evaluation, and confidence intervals are not reported because of the cost of end-to-end VM rollout [2606.16748]. This means reported values are directly tied to one execution trace per task per model.

## 5. Baselines, scaling behavior, and empirical results

MyPCBench reports results for six models, each driven by its provider’s native computer-use agent with the shared computer-plus-bash surface: Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.5, GPT-5.4 mini, Qwen 3.5 35B-A3B, and Qwen 3.5 9B [2606.16748]. All runs use the same persona context block, the same 100-turn budget, and the same Gemini judge [2606.16748].

The headline results are as follows:

| Model | Perfect rate | Rubric score |
|---|---:|---:|
| Claude Opus 4.6 | 55.4\% | 81.8\% |
| Claude Sonnet 4.6 | 39.1\% | 65.4\% |
| GPT-5.5 | 29.3\% | 54.1\% |
| GPT-5.4 mini | 19.0\% | 48.8\% |
| Qwen 3.5 35B-A3B | 7.6\% | 42.5\% |
| Qwen 3.5 9B | 2.7\% | 7.0\% |

Claude Opus 4.6 is the only model above 50\% perfect and fully solves 55.4\% of tasks [2606.16748]. Average steps and Trajectory Efficiency further differentiate models: Opus records 46.5 average steps and 3.61 Trajectory Efficiency; Sonnet 45.8 and 3.03; GPT-5.5 45.8 and 1.45; GPT-5.4 mini 43.7 and 1.65; Qwen 35B 66.0 and 1.41; Qwen 9B 69.2 and 0.65 [2606.16748].

Per-task-type performance shows that bounded action, pattern inference, and personal lookup are materially easier than aggregation/reporting and multi-step orchestration [2606.16748]. For pattern inference, Opus reaches 94.7\% rubric and perfects 82\% of those tasks, whereas GPT-5.5 reaches 59.1\% rubric and 45\% perfect [2606.16748]. For aggregation/reporting and orchestration, GPT-5.5 stays below 35\% rubric on aggregation and 32.5\% on orchestration [2606.16748]. Qwen 9B perfects zero tasks across personal lookup, aggregation, pattern inference, and cross-source reconciliation [2606.16748].

Performance degrades sharply with application count. For Opus, single-app tasks achieve 66\% perfect and 87.4\% rubric, while tasks involving seven or more apps drop to 36\% perfect and 67.9\% rubric [2606.16748]. Sonnet drops from 46\% to 14\% perfect across the same range [2606.16748]. GPT-5.4 mini, Qwen 35B, and Qwen 9B all reach 0\% perfect at 7+ apps, while GPT-5.5 records 4.5\% perfect in that slice [2606.16748].

Step-budget scaling provides another lens on horizon limitations. Opus continues improving at the 100-step cap, GPT-family models flatten around step 60, and Qwen saturates around step 25 [2606.16748]. Some `cua+bash` trajectories exceed 100 steps despite the 100-turn cap because a single turn can emit several `pyautogui` actions plus a shell call [2606.16748].

Tool-surface ablation indicates that adding bash and a dual-tool hint to computer-only agents produces modest changes for GPT-5.5 and GPT-5.4 mini, a mixed effect for Qwen 35B, and a large regression for Qwen 9B [2606.16748]. The paper states that equalizing the tool surface does not overturn the model ordering [2606.16748].

## 6. Failure modes, interpretation, and methodological significance

The paper groups failed-rubric judge explanations into five modes: premature termination (`DONE`), skipped required app, surface error treated as terminal, partial artifact, and hallucinated persona data [2606.16748]. Aggregated counts across models are 354, 323, 129, 47, and 31 respectively [2606.16748]. These categories emphasize that failure is often procedural rather than purely perceptual or semantic.

Family-specific patterns are also reported. GPT-5.4 mini and GPT-5.5 overuse premature `DONE`, with 130 and 105 such failures respectively, compared with 28 for Opus and 38 for Sonnet [2606.16748]. Qwen 35B leads in hallucinated persona data and surface-error abandonment, while Qwen 9B is reported to collapse under the dual-tool schema, zero-scoring 164 of 184 tasks and emitting malformed tool calls that splice bash and computer schemas [2606.16748]. Claude models exhibit a different pathology: console-script shortcuts, including JavaScript console access and `curl` to REST endpoints via bash, can retrieve application state without producing the user-visible side effects required by certain rubrics [2606.16748].

One consequence is that perfect rate falls faster than rubric score. Skipped-app and premature-termination failures can invalidate an entire task even if many rubric items have already passed [2606.16748]. This design makes the benchmark sensitive to end-to-end completion discipline rather than merely local competence. The benchmark therefore measures not only whether an agent can infer the correct state of the world, but whether it can carry a multi-application workflow through to the externally visible result.

The benchmark’s emphasis on visible artifacts and deterministic reset aligns it with desktop-evaluation infrastructure descended from OSWorld, while extending that setup into a personalized regime [2606.16748]. This suggests a methodological shift from evaluating generic GUI navigation to evaluating identity-grounded task execution under persistent, coherent personal history.

## 7. Relation to other benchmarking traditions and stated limitations

Despite sharing a name fragment with earlier PC benchmarking work, MyPCBench is unrelated in subject matter to the 2017 IAC study on benchmarking institutional computing resources with Polyhedron Fortran workloads [1702.04942]. That work concerns HTC scheduling, compiler comparisons, and normalized execution-time measurement on heterogeneous desktops and racks [1702.04942]. It does, however, illustrate a distinct benchmark tradition centered on reproducibility, normalization, and operational deployment constraints, which provides a useful contrast with MyPCBench’s focus on computer-use agents rather than CPU performance [1702.04942]. A plausible implication is that both efforts treat benchmarking as a systems problem, but at different abstraction layers: one at machine-performance level, the other at agent-environment interaction level.

Another useful contrast is with nanoBench, a low-overhead x86 microbenchmarking tool designed for isolated instruction sequences and cache characterization [1911.03282]. nanoBench measures latency, throughput, port usage, and cache behavior with hardware counters and kernel-level control [1911.03282]. MyPCBench operates at the opposite end of the granularity spectrum, evaluating long-horizon, screenshot-conditioned, multi-application workflows rather than microarchitectural primitives [1911.03282]. This suggests that “PC benchmarking” now spans at least three distinct layers present in the cited materials: hardware resource benchmarking [1702.04942], microarchitectural benchmarking [1911.03282], and agentic desktop-task benchmarking [2606.16748].

The paper states several limitations. The benchmark uses one canonical persona and one Linux/GNOME/Firefox stack, favoring depth over diversity [2606.16748]. Grading relies on a single Gemini judge, so absolute counts are described as structural rather than precise prevalence estimates [2606.16748]. The environment is simulated rather than live-web, which avoids privacy and variability concerns but places behaviors tied to genuinely sensitive data out of scope by design [2606.16748]. Dual-use concerns are explicitly acknowledged: the skills evaluated generalize to agents that could operate real logged-in accounts, though the released benchmark mitigates this through synthetic persona data, local hosting, and a recommendation to benchmark offline against the released image only [2606.16748].

Future directions identified in the paper include extending to multiple personas, locales, and life stages; dynamic updates to personal contexts and cross-application events; multi-judge ensembles and additional automatic checks; more evaluation metrics such as recovery rate after errors and explicit multi-app reconciliation scores; improved agent memory and planning; stronger hybrid GUI/shell policies; and expansion to Windows, macOS, alternative browsers, mobile surfaces, and a broader application suite [2606.16748].

MyPCBench’s main significance lies in converting personalization from auxiliary metadata into the core substrate of evaluation. By seeding one coherent identity across authenticated clones of banking, travel, delivery, messaging, productivity, and browser environments, then scoring visible side effects under deterministic reset, it defines a benchmark regime in which cross-application consistency, long-horizon planning, and completion discipline become first-class empirical variables [2606.16748].

Source: https://www.emergentmind.com/topics/mypcbench