PARE: Proactive Agent Research Environment

• PARE is a comprehensive simulation framework for proactive digital agents that leverages finite state machines to model stateful, multi-app environments.
• It employs LLM-based active user simulation to enable realistic multi-turn interactions and assess agent strategies in context-rich scenarios.
• PARE benchmarks performance using detailed metrics such as plan acceptance rate and goal achievement across diverse, real-world application domains.

The Proactive Agent Research Environment (PARE) comprises a set of simulation frameworks, benchmarks, and methodologies for developing, evaluating, and analyzing proactive agents—systems that anticipate and initiate actions in digital environments without explicit user prompts. PARE formalizes a unified, stateful, and context-rich evaluation architecture, spanning granular user modeling, scenario synthesis, precise state-action semantics, and rigorous protocol-driven agent assessment. The environment is foundational for research on anticipatory digital assistants that must move beyond rote interface navigation to infer latent goals, calibrate intervention, seek clarification, and act autonomously within authentic digital workflows (Nathani et al., 1 Apr 2026, Lu et al., 2024, Tang et al., 4 Feb 2026, Chen et al., 9 Apr 2026).

1. Formal Environment Structure and Application Semantics

PARE instantiates digital environments by modeling each application as a finite state machine (FSM) $M = (S, A, T, s_0)$, where $S$ is the set of states (representing screens or UI contexts), $A$ is the set of user actions (e.g., taps, submissions), $T$ is the deterministic transition function, and $s_0$ the initial state. The environment maintains a global state $S_{\mathrm{env}}$, capturing the stack of current apps, their FSM states, and a shared backend database (contacts, messages, calendar, etc.).

Concretely, the state space and action space are bifurcated:

• User simulator: Observes a restricted state $s \in S_U$ (union of app FSM + system) and acts via a subset $A_U(s) \subseteq A$ determined by the current UI screen, faithfully encoding stateful navigation constraints (e.g., "send" only available after required fields are filled). Observations are limited to screen-level UI elements and truncated notifications.
• Proactive assistant: Observes the full environment state (all backend, user actions, and untruncated notifications), and possesses a flat, global API: read-only queries (e.g., fetch events), natural-language proposals (propose_task), and a no-operation action (wait). The agent faces no per-screen action bottlenecks.

This duality enables ecologically valid evaluation: user policies reflect realistic sequential UI interaction, while agents confront unconstrained but semantically coherent digital environments (Nathani et al., 1 Apr 2026).

2. Active User Simulation and Policy Realization

PARE implements active user simulation via an LLM-based agent (typically ReAct-style), effectively casting the user as a Partially Observable Markov Decision Process (POMDP) agent. At each timestep, the user simulator receives its UI-observable state and notifications, reasons via an explicit "Thought," and issues a valid UI action or responds to agent proposals through accept/reject. The simulation loop proceeds as:

$S$6

This pattern enforces a stackelberg structure: user acts first, agent then observes and intervenes. Transitions are deterministic except in scenarios configured with nonzero tool failure probability $p_{\mathrm{fail}}$, introducing controlled stochasticity. The user simulator operates solely for environment fidelity; it is not reward-driven (Nathani et al., 1 Apr 2026).

3. Benchmark Design: Scenario Composition and Task Types

The PARE-Bench suite consists of 143 manually vetted scenarios encompassing four core domains: communication (email, messaging), productivity (notes, shopping), scheduling (calendar, reminders), and lifestyle (cab booking, apartment search). Each scenario specifies:

1. App initialization: Backend seeded with structured data (e.g., pre-populated contacts, saved apartments).
2. Event flow: Exogenous events (inbox arrivals, reminders), possibly adversarial distractors (Poisson noise).
3. Oracle validation: Strict final-state predicates, formalizing scenario goals ($G_U$).

Task objectives span context maintenance, latent goal inference, intervention timing, and multi-application orchestration. Examples include proactive budget adjustment in apartment search or orchestrating cross-app workflows in response to ambient context changes. The scenario distribution in PARE-Bench is balanced to stress-test both information gathering and action-planning across app types (Nathani et al., 1 Apr 2026).

4. Evaluation Metrics and Decision Protocols

Evaluation within PARE employs metrics capturing both proposal quality and task success:

• Plan acceptance rate: $S$0, with $S$1 the agent's proposals.
• Goal achievement rate: $S$2, with $S$3 the user's true objectives.

Additional quantitative metrics include:

• Success@k: Binary indicator if at least one of $S$4 runs achieves $S$5.
• ProposalRate: (# proposal turns)/(# total agent turns).
• ReadActions: Mean read-only API calls for information acquisition.

Agents operate under a decision chain: observe → propose (if needed) → execute upon acceptance, or continue monitoring. User acceptance, rejection, or inaction is explicitly modeled and influences agent restraint and reactivity. This structure directly supports experiments on optimal intervention timing, contextualization, and consent handling (Nathani et al., 1 Apr 2026, Chen et al., 9 Apr 2026).

5. Comparative Evaluation and Empirical Findings

Empirical studies in environments configured per PARE reveal clear performance stratifications:

• On explicit, general tasks, agents such as Claude Sonnet 4.6 surpass 90% success, with failures dominated by rare GUI errors.
• On tasks requiring preference inference or intervention calibration, performance drops below 50%, with the primary failure modes being improper clarification queries (93.8% of personalized failures) and incorrect intervention timing (80% of proactive failures). GUI execution errors account for only 4.2% (personalized) or 15% (proactive) of failures (Chen et al., 9 Apr 2026).

These findings underscore that agent competence depends fundamentally on the capacity to model user preferences, resolve ambiguity via multi-turn interaction, and dynamically adjudicate between acting, proposing, and abstaining within context. Merely increasing the number of clarification queries is ineffective without quality and compositional fidelity in reasoning (Chen et al., 9 Apr 2026).

6. Architectural and Methodological Insights

PARE's delineation of roles and observability channels between user and agent constitutes its core architectural advance. Key methodological elements include:

• Separation of stateful navigation (user) vs. backend orchestration (agent), enhancing both simulation fidelity and agent flexibility.
• Event-driven, scenario-based evaluation: External events interleave with user/agent actions to simulate real digital complexity.
• LLM-based user simulation: Realistic sequential interaction, including handling of ambiguous proposals, consent negotiation, and rejection restraint.
• Integration of both deterministic rule-based and semantic (LLM-Judge) scoring: Supports evaluation on both objective and context-sensitive axes (Chen et al., 9 Apr 2026).

A notable implication is that advances in proactive agent design necessitate parallel progress in environment modeling: only richly structured, stateful, and interaction-grounded environments such as PARE can truly evaluate the transition from reactive to proactive digital assistance (Nathani et al., 1 Apr 2026, Chen et al., 9 Apr 2026).

7. Broader Impact and Ongoing Directions

PARE, through its FSM-centric architecture, structured benchmarks, and hybrid evaluation metrics, establishes a comprehensive framework for research on anticipatory digital assistants. Its scenario complexity, precision in agent-user-environment interaction, and documented empirical bottlenecks drive ongoing research into: robust user modeling, ambiguity resolution, adaptive intervention, privacy-compliant scenario synthesis, and transfer across domains (e.g., smart home, healthcare).

Open-source releases and extensive documentation have fostered reproducibility and cross-benchmark comparability, anchoring PARE's role in the evolving landscape of task-oriented and proactive agent research (Nathani et al., 1 Apr 2026, Lu et al., 2024, Tang et al., 4 Feb 2026, Chen et al., 9 Apr 2026).