KwaiClawEnv: Multimodal Agent Environments

• KwaiClawEnv is a family of stateful, sandboxed service environments that benchmark multi-turn, multimodal agent tasks across diverse scenarios.
• It underpins ClawMark and related systems with persistent state, deterministic verification, and dynamic exogenous updates to assess agent adaptability.
• It also functions as a scalable framework for tool-use trajectory synthesis, enabling the generation of high-quality training and evaluation traces.

KwaiClawEnv is not a uniformly canonical identifier in the 2026 arXiv literature. The term is most concretely attached to the execution environment of ClawMark, where it denotes the stateful sandboxed service stack used to evaluate multi-turn, multi-day, multimodal coworker agents. In other sources, however, the same label is either absent from the paper text or used as an informal alias for adjacent constructs, including a service-synthesis framework for general tool-use training, a GUI-app environment instantiated inside ClawGUI, and broader evaluation or personal-assistant environments (Meng et al., 26 Apr 2026, Huang et al., 6 Jul 2026).

1. Terminological status and cross-paper mapping

The primary ambiguity around KwaiClawEnv is terminological rather than architectural. Some papers define a concrete environment under that name or treat it as a common external label, whereas others explicitly state that the exact term does not appear in the paper and only map it to the nearest internal construct.

Source	Precise construct in the paper	Relation to “KwaiClawEnv”
ClawMark (Meng et al., 26 Apr 2026)	Stateful sandboxed services	Commonly-used name for the environment shipped with the benchmark
KAT-Coder-V2.5 (Huang et al., 6 Jul 2026)	Service–Task–Eval environment synthesis framework	Directly used for large-scale tool-use trajectory generation
ClawGUI (Tang et al., 13 Apr 2026)	Per-app GUI environment under the Environment Manager	Exact term absent; closest mapping for a Kwai/Kuaishou app environment
Claw-Eval (Ye et al., 7 Apr 2026)	Isolation Environment plus Mock Services	Exact term absent; repository/local alias only
Claw-Anything (Lin et al., 25 May 2026)	Training/evaluation environments plus simulator pipeline	Exact term absent; closest broad-context mapping

A common misconception is that KwaiClawEnv names a single package-level artifact with a stable definition across papers. The cited material does not support that reading. Instead, the label spans at least three distinct environment classes: persistent office-service stacks, scalable tool-use synthesis sandboxes, and app- or benchmark-specific execution environments. This suggests that “KwaiClawEnv” functions more as a cross-project shorthand than as a strictly standardized technical term (Tang et al., 13 Apr 2026, Ye et al., 7 Apr 2026, Lin et al., 25 May 2026).

2. ClawMark’s stateful sandboxed service environment

In ClawMark, KwaiClawEnv denotes the benchmark’s execution backbone: a living-world environment built from five stateful sandboxed services—filesystem, email, calendar, knowledge base, and spreadsheet—designed for coworker agents that operate across multiple in-universe working days (Meng et al., 26 Apr 2026). Each task runs inside an isolated docker-compose stack. The filesystem is a Docker-mounted workspace with artifacts and per-turn injection directories; email is implemented with GreenMail SMTP/IMAP; calendar with a Radicale CalDAV server; the knowledge base exposes a Notion-compatible API against a per-task workspace; and the spreadsheet service exposes a Google-Sheets-compatible API against a per-task spreadsheet. State persists across turns within the same task stack, while stacks are torn down between tasks.

The environment is explicitly organized around persistence, exogenous change, and raw multimodal evidence. Files, emails, calendar entries, knowledge-base pages, and spreadsheet cells survive from one turn to the next. The benchmark also injects changes between turns that are not caused by the agent. These changes can be “loud,” meaning announced in the wake-up prompt, or “silent,” meaning unannounced and discoverable only through active re-synchronization. The corpus-wide mean share of silent versus loud updates is 46.7% versus 53.3%.

The modality model is similarly strict. Supported artifacts include text, images, scanned PDFs, audio, video, and spreadsheets, but the environment does not provide automatic OCR or transcription. Artifacts are stored on the filesystem or as service attachments and are delivered untranscribed and unparsed. Agents must therefore bring or invoke their own perception tools for audio, video, PDF, image, or spreadsheet processing.

This design differentiates KwaiClawEnv from static, single-episode, text-centric benchmarks. The environment mutates independently of agent actions, spans multiple days per task, and evaluates whether an agent can track and revise its beliefs when the world changes underneath it.

3. State evolution, task structure, and deterministic verification

ClawMark’s task corpus contains 100 tasks across 13 professional scenarios and 87 distinct in-task roles. Tasks span 2–6 turns with mean 3.6 turns, and each task contains 6–29 checkers with mean 15.4. Representative scenarios include executive assistant, HR, content operation, e-commerce, journalist, project management, real estate, research assistant, clinical assistant, insurance, legal assistant, investment analyst, and EDA (Meng et al., 26 Apr 2026).

Verification is rule-based and deterministic. The benchmark uses 1,537 deterministic Python checkers over post-execution service state, including 55 red-line constraints with fixed high weights. Checker categories include filesystem and artifact inspection, backend state queries over the knowledge base, spreadsheet, and calendar, email-state queries, and numeric-tolerance or semantic-equivalence checks. No LLM-as-judge is invoked during scoring.

The two reported aggregate metrics are weighted score and strict Task Success. Their definitions are:

$$\mathrm{score}(m, \tau) = \frac{\sum_{c \in \mathcal{C}(\tau)} w_c \cdot \mathbf{1}\big[\mathrm{pass}_c(m, \tau)\big]} {\sum_{c \in \mathcal{C}(\tau)} w_c} \in [0,1].$$

$$\mathrm{Succ}(m) = \frac{100}{|\mathcal{T}|} \sum_{\tau \in \mathcal{T}} \mathbf{1}\!\left[ \forall c \in \mathcal{C}(\tau),\, \mathrm{pass}_c(m,\tau) \right] \in [0,100].$$

Weighted score rewards partial progress across rubrics of different length, whereas Task Success requires complete end-to-end satisfaction of all rubric items. Turn-level checking is also explicit: checkers are assigned per turn, each turn is executed and checked against post-turn service state, and failures do not halt later turns.

Benchmark results show substantial headroom. The strongest weighted score is 75.8, achieved by Claude Sonnet 4.6, while the strongest strict Task Success is only 20.0%, achieved by Claude Opus 4.6. Across 10,759 pooled checker evaluations, 3,404 failures are recorded, corresponding to a 31.6% failure rate. The dominant failure modes are silent-change detection at 56.5% and backend writeback at 53.6%; other categories include cross-source consistency at 34.0%, deliverable at 31.4%, evidence extraction at 23.6%, compliance guardrails at 21.5%, and red-line violations at 7.1%.

Turn-level analysis further clarifies the environment’s difficulty. On the 73 tasks with exactly three turns, six of seven evaluated models drop after the first exogenous update on Day 2. This empirically localizes the core challenge to state refresh, invalidation of stale assumptions, and robust writeback under evolving service state.

4. KwaiClawEnv as a trajectory-synthesis framework in KAT-Coder-V2.5

In KAT-Coder-V2.5, KwaiClawEnv is not a benchmark-specific office stack but an environment synthesis framework for general agentic tool-use training (Huang et al., 6 Jul 2026). Its role is to provide high-quality, scalable, and reproducible interaction settings grounded in real business tasks and “Claw”-style workflows. The report organizes it into three layers: Service, Task, and Eval.

The Service layer begins from human-authored Skills, such as OpenClaw, and category-guided LLM-generated Services. Skills are parsed to extract API specifications, parameter schemas, and usage constraints, then converted into deployable services with OpenAPI specifications, container configurations, and fixture data. Automatic checks validate executability, interface consistency, endpoint reachability, inter-service dependency connectivity, and logical correctness. Composite capabilities are then formed by chaining or nesting multiple services.

The Task layer seeds tasks from real business tasks with clear objectives, tool-use exemplars, and machine-verifiable success criteria. It expands these seeds via parameter expansion, constraint augmentation, and tool-chain orchestration, controlled by difficulty ratio, tool-chain length bounds, and tool source composition. Rollouts run in sandboxed containers and record full interaction traces: model decisions, tool invocations, tool outputs, and state transitions.

The Eval layer converts raw traces into a unified training format, including SFT-ready samples, then applies two-layer filtering. Hard-rule filtering removes unsafe, invalid, or hallucinated trajectories through checks such as tool-blacklist detection, file-existence validation, mandatory tool-coverage verification, and state-consistency enforcement. The remaining trajectories are then scored by LLM-as-Judge along semantic correctness, execution efficiency, and interaction naturalness. Quality signals feed back into the Service and Task layers through a closed-loop refinement process that classifies samples as repairable failures, rejected defects, or production-ready outputs.

The reported scale is substantial. Parsing curated Skills and converting them into deployable services achieves a generation success rate exceeding 90%. The system generates millions of candidate tasks, retains over one hundred thousand high-quality instances after multi-stage verification, and produces trajectories averaging 15 tool calls, with the longest exceeding 100 steps. Within KAT-Coder-V2.5, these environments provide structured rewards and executable traces for asymmetric actor–critic PPO with hindsight-augmented value estimation and for Multi-Teacher On-Policy Distillation.

5. GUI, evaluation-suite, and always-on-assistant mappings

In ClawGUI, the exact term “KwaiClawEnv” does not appear, and there is no Kwai/Kuaishou-specific environment or benchmark named in the paper. The closest construct is a per-app GUI environment defined within ClawGUI-RL’s Environment Manager, which abstracts Docker-based Android virtual emulators via MobileWorld and physical devices behind a unified interface (Tang et al., 13 Apr 2026). A Kwai or Kuaishou app can be registered as one of the application tasks under that manager and used for training, evaluation, and deployment. Observations explicitly include screenshots and action history; the Process Reward Model consumes the previous screenshot, the current screenshot, and the full history of actions. Episode-level reward is binary, with success $=1$ and failure $=0$, while dense step-level reward is provided by the PRM. The paper prints the combined reward formula as “$R = R_{\text{outcome} + R_{\text{step}}$” and gives no further policy-loss equations. Evaluation on MobileWorld GUI-Only uses a maximum of 50 interaction steps. Within the unified pipeline, ClawGUI-2B achieves 17.1% Success Rate on MobileWorld GUI-Only, outperforming the same-scale MAI-UI-2B baseline by 6.0%, and the GiGPO versus GRPO ablation moves from 14.5% to 17.1%.

In Claw-Eval, the exact name is also absent. If a local repository uses “KwaiClawEnv,” it maps to the Isolation Environment plus attached Mock Services in Claw-Eval’s architecture (Ye et al., 7 Apr 2026). That environment uses an isolated Docker sandbox, system-layer tools, task-specific service-layer tools, and three evidence channels: execution traces, service-side audit logs, and environment snapshots. The suite contains 300 human-verified tasks, 2,159 rubric items, and evaluates Completion, Safety, and Robustness across three trials. A vanilla LLM judge misses 44% of safety violations and 13% of robustness failures relative to the hybrid evidence pipeline.

In Claw-Anything, “KwaiClawEnv” likewise does not appear in the paper text, but the closest mapping is the released Claw-Anything environment suite plus its automated simulator and data-generation pipeline (Lin et al., 25 May 2026). Here the environment is formalized as $E=(P,D,F,L)$, where $P$ is persona, $D$ is devices, $F$ is fixtures, and $L$ is long-horizon logs. It binds a Linux CLI Docker and an Android GUI Docker to the same backend state, supports 35 services in benchmark and training sets, and yields environments with average context of approximately 191.7k words. The benchmark contains 200 evaluation tasks and 2,000 training environments. Average service count is 10.1 per task, with a maximum of 18. GPT-5.5 reaches 34.5% Pass@1, and fine-tuning Qwen3.5-27B on 1,500 successful trajectories from the 2,000 training environments improves Pass@1 from 9.8% to 33.5%, a gain of 23.7 absolute points.

6. Research significance, misconceptions, and limits of the label

The strongest evidence-supported interpretation is that KwaiClawEnv names a family of executable, persistent, and verifiable agent environments rather than a single universally fixed artifact. In ClawMark it is a living-world service stack; in KAT-Coder-V2.5 it is a scalable synthesis framework for tool-use trajectories; in ClawGUI it is only a plausible internal convention for a Kwai/Kuaishou app environment; and in Claw-Eval and Claw-Anything it is best understood as an informal alias for broader environment substrates (Meng et al., 26 Apr 2026, Huang et al., 6 Jul 2026).

Another misconception is that the term necessarily implies GUI automation. The evidence is mixed. ClawGUI indeed centers on GUI control through taps, swipes, and keystrokes on Android, HarmonyOS, and iOS, but ClawMark’s environment is service-centric rather than GUI-centric, and KAT-Coder-V2.5’s KwaiClawEnv is defined around containerized services, tool calls, and trajectory synthesis rather than screen-based interaction.

The literature also imposes clear limits on what can be asserted. ClawGUI explicitly states that the exact term does not appear in the paper. Claw-Eval does not provide code-level step() or reset() interfaces. KAT-Coder-V2.5 does not specify exact message schemas, low-level connectors, or per-step timeout policies for KwaiClawEnv itself. Claw-Anything provides a formal state model and evaluation formulas but still describes several services as controllable mock environments rather than full production systems. These gaps matter because they delimit the portability of the name across implementations.

Taken together, the cited works indicate a broader shift in agent research away from static prompt-response tasks and toward environments with persistent state, exogenous dynamics, multimodal artifacts, explicit tool interfaces, and verifier-backed scoring. A plausible implication is that “KwaiClawEnv” has become useful precisely because it compresses this design pattern into a single informal label, even though the underlying systems remain heterogeneous and paper-specific.