Agent Execution Harness

Updated 4 July 2026

Agent Execution Harness is a system layer that transforms language model outputs into concrete, verifiable actions by orchestrating context, tools, and state updates.
It integrates execution, tool mediation, context management, scheduling, and observability to ensure controlled and safe agent operation.
The architecture employs layered responsibilities—ranging from execution to governance—that facilitate benchmarking, recovery, and optimization of agent performance.

An agent execution harness is the system layer or runtime wrapper that surrounds a LLM and turns model outputs into actions in an external workspace or operating environment. Recent work defines the harness as the layer that conditions model calls; organizes context, tools, state, permissions, constraints, tracing, budget control, recovery, and retries; and mediates long-running execution so that an agent can produce concrete, verifiable artifacts rather than only text. A compact characterization used across multiple papers is Agent = Model + Harness, emphasizing that reported capability is jointly determined by the base model and the execution substrate rather than by the model alone (Yao et al., 27 May 2026, Ning et al., 18 May 2026, Zhang et al., 7 May 2026).

1. Conceptual foundations

In the control-theoretic framing, the LLM is an open-loop stochastic policy, while the harness is the controller that closes the loop around environment feedback and state updates. The harness constructs the model-visible context $c(s_t)$ , invokes the policy $\pi_\theta(\cdot \mid c(s_t))$ , parses and routes actions through tools, validates observations, and updates state according to a controller rule $s_{t+1} = \mathcal{C}_H(s_t, a_t, o_t; H)$ . This formulation makes stability, context drift, and control lag harness properties rather than model-only properties (Zhang et al., 7 May 2026).

A complementary systems view defines the harness as the software layer that turns a stateless LLM into a reliable, long-running agent by surrounding it with executable tools, sandboxes, memory, validators, permission boundaries, telemetry, and closed-loop control. In the “code as agent harness” perspective, code is the operational substrate because it is executable, inspectable, and stateful; plans, tests, logs, repositories, and simulators become first-class execution artifacts rather than latent prompt content (Ning et al., 18 May 2026).

This notion generalizes across domains. In mobile automation, the harness is the runtime substrate that connects planning to concrete device actions and verifies side effects; in finance, it is the inline wrapper that intercepts every proposed tool call before it commits state; in OS-level systems, it is the agent-native substrate that exposes system and app capabilities through structured interfaces while enforcing information-flow control (Li et al., 12 Jun 2026, Jia et al., 26 May 2026, Zhao et al., 22 Jun 2026).

2. Architectural surfaces and runtime responsibilities

Recent work converges on a layered decomposition of harness responsibilities. One disclosure taxonomy organizes them as Execution, Tool, Context, Scheduling, Observability, Verification, Governance; another uses Execution, Tooling, Context, Lifecycle, Observability, Verification, Governance. The naming differs, but both taxonomies place the same engineering burden on runtime substrate, tool mediation, context construction, orchestration, tracing, validation, and policy enforcement (Zhang et al., 7 May 2026, Chen et al., 4 Jun 2026).

Surface	Runtime role
Execution	sandboxing, runtime substrate, side-effect boundaries
Tool / Tooling	tool registry, schemas, adapters, parameter validation
Context	prompt assembly, retrieval, memory, compression, cache policy
Scheduling / Lifecycle	loops, retries, timeouts, delegation, rollback, stopping rules
Observability	traces, checkpoints, latency/cost logging, state diffs
Verification	parsers, validators, tests, anomaly detection, final checks
Governance	permissions, allowlists/denylists, approvals, audit trails

Within this decomposition, the harness owns concrete artifacts: prompt templates, tool schemas and adapters, orchestration code, configuration files, telemetry hooks, verification scripts, checkpoint stores, and permission policies. Several systems also externalize durable state into files or workspaces rather than leaving it implicit in conversation history. Natural-Language Agent Harnesses, for example, separate runtime state and final artifacts into stable file-backed surfaces such as /sa-output/runtime and /sa-output/artifacts, while LemonHarness constrains all state-changing operations to a controlled workspace and records the resulting file and process effects as observations for later turns (Pan et al., 26 Mar 2026, Ren et al., 23 Jun 2026).

The architectural consequence is that harness quality is not reducible to prompt wording. It depends on how the runtime exposes tools, constrains side effects, reconstructs state, logs evidence, and decides when an execution is complete. This suggests that “prompt engineering” is only one narrow slice of harness engineering.

3. Control-flow models and execution semantics

One major axis of variation is who owns control flow. In conventional agent loops, the model decides what to do next from an ever-growing transcript. Agentic Programming argues that looping, branching, sequencing, and termination should instead be deterministic program logic, with the LLM invoked only as a typed subroutine for reasoning or generation. Under that design, execution history forms a call-tree-derived DAG, and prompt context for a live call scales with call depth rather than with flat transcript length:

$L(d) \le \sum_{i=0}^{d}(S_i + B_i),$

which yields $O(d)$ bounded live context under bounded frame and summary budgets (Qi et al., 14 Jun 2026).

A related scheduler-theoretic proposal is the Structured Graph Harness, which lifts control flow into an explicit static DAG with immutable plan versions, deterministic scheduling, and a strict three-level recovery escalation protocol. Node readiness is computed from explicit dependencies rather than inferred from model memory, and bounded retries and timeouts guarantee that every node reaches a terminal state in a finite acyclic graph (Wei, 13 Apr 2026).

Declarative harness languages occupy another point in the design space. AgentSPEX specifies typed steps, branching, loops, parallel execution, reusable submodules, explicit state management, checkpointing, verification, and logging in YAML, with the harness interpreting task, step, if, switch, while, for_each, parallel, gather, call, set_variable, increment, input, and return constructs directly (Wang et al., 14 Apr 2026). Natural-Language Agent Harnesses externalize high-level control logic into editable natural language interpreted by an Intelligent Harness Runtime, with explicit contracts, durable artifacts, child-agent semantics, and file-backed state (Pan et al., 26 Mar 2026).

A different approach makes the harness itself learnable. HarnessBridge parameterizes the harness as a bidirectional interface policy with an observation projection that distills raw history into a compact generator-visible state and an action projection that maps a proposed action into either an executable transition or a trajectory-grounded rejection. The generator policy remains frozen; only the lightweight harness controller is trained (Wang et al., 11 Jun 2026).

4. Measurement, benchmarking, and reporting

A central development in the literature is the move from model-centric evaluation to model–harness evaluation. Harness-Bench was introduced specifically to isolate configuration-level harness effects under fixed user-facing task prompts, fixtures, initial sandbox state, budget, timeout, and evaluator, while preserving each harness’s native prompting, tool interfaces, state-management policy, recovery behavior, and action formats. The benchmark contains 106 sandboxed offline tasks and reports 5,194 execution trajectories. Across the same task set and model-backend pool, it finds a 23.8-point aggregate score gap between the best configurable harness, NanoBot at 76.2, and the lowest, OpenClaw at 52.4, with substantial differences in completion, process quality, tokens, turns, and failure behavior (Yao et al., 27 May 2026).

The evaluation formalism in Harness-Bench makes explicit that outcome quality, security, and process quality are all harness-sensitive:

$\mathrm{TaskScore}_i = \mathrm{Security}_i \cdot \mathrm{Completion}_i \cdot \mathrm{Process}_i,$

$\mathrm{Process}_i = \frac{\mathrm{Robustness}_i + \mathrm{ToolUse}_i + \mathrm{Consistency}_i}{3}.$

This scoring scheme records not only final artifacts but also execution traces, usage statistics, validator outputs, and security-gate outcomes (Yao et al., 27 May 2026).

The position paper “Stop Comparing LLM Agents Without Disclosing the Harness” pushes the same argument further through variance decomposition. Let $B(M,H)$ denote benchmark score for model $M$ under harness $H$ . The paper defines harness-induced variance $\pi_\theta(\cdot \mid c(s_t))$ 0 and model-induced variance $\pi_\theta(\cdot \mid c(s_t))$ 1, and in a controlled factorial study on SWE-bench Verified “subset100” reports an average HV of 18.48 pp², an average MV of 2.37 pp², and HV/MV = 7.80×, with 6 out of 9 model-pair/harness-pair comparisons flipping ordering (Zhang et al., 7 May 2026).

Trajectory-level safety evaluation makes the same point from a different angle. HarnessAudit-Bench instantiates 210 tasks across eight real-world domains in both single-agent and multi-agent configurations, and reports that task completion is misaligned with safe execution, that violations accumulate with trajectory length, and that most violations concentrate in resource access and inter-agent information transfer (Liu et al., 14 May 2026).

For optimization workflows, VeRO treats harnessed evaluation itself as the object of study. It provides versioned agent snapshots, budget-controlled evaluation, structured execution traces, and a standardized observation interface so that coding-agent optimizers can be compared under identical edit–execute–evaluate budgets. Its budget is defined as the number of evaluator invocations, $\pi_\theta(\cdot \mid c(s_t))$ 2, and every evaluation is tied to a specific Git commit and stored trace (Ursekar et al., 25 Feb 2026).

Framework	Primary focus	Reported scale
Harness-Bench	configuration-level harness effects	106 tasks, 5,194 trajectories
HarnessAudit-Bench	trajectory safety in single- and multi-agent harnesses	210 tasks, eight domains
VeRO	agent optimization under versioned, budgeted evaluation	versioned snapshots and structured traces

5. Safety, execution alignment, and repair

Safety-oriented work treats the harness as the main enforcement point for permission boundaries, information flow, rollback, and pre-execution validation. Harness-Bench introduces the notion of execution alignment, defined as the degree to which the harness preserves correspondence among the agent’s reasoning, observed workspace state, tool actions and feedback, and evaluator-checked conditions of success. Among failed trajectories, it reports recurring failure symptoms with concrete rates: contract/format violations 36.4%, tool/recovery lapses 24.6%, evidence/grounding gaps 14.6%, artifact commitment misses 11.1%, and state/continuation failures 9.3% (Yao et al., 27 May 2026).

SafeHarness integrates security directly into the lifecycle through four layers: adversarial context filtering at input processing, tiered causal verification at decision making, privilege-separated tool control at action execution, and safe rollback with adaptive degradation at state update. Across benchmark datasets, harness configurations, and attack scenarios, it reports an average reduction of approximately 38% in UBR and 42% in ASR relative to the unprotected baseline while preserving core task utility (Lin et al., 15 Apr 2026).

FinHarness specializes the same inline principle for finance agents. It combines a Query Monitor, a Tool Monitor, and a Cascade module that routes each step to a lightweight or advanced-tier LLM judge, then reinjects fired risk factors into the agent input as ex-ante evidence. On FinVault, routed FinHarness reduces ASR from 38.3% to 15.0% while preserving benign approval from 41.1% to 39.3%, and uses 4.7× fewer advanced-judge calls than an always-advanced ablation (Jia et al., 26 May 2026).

When harness flaws are already present, HarnessFix provides a repair workflow. It compiles execution traces and harness code into a Harness-aware Trace Intermediate Representation, attributes failures to responsible trajectory steps and harness layers, consolidates recurring diagnoses into flaw records, and applies scoped repair operators under regression-aware validation. Across SWE-Bench Verified, Terminal-Bench 2.0 Verified, GAIA, and AppWorld, it reports held-out improvements of 15.2%–50.0% over initial harnesses (Chen et al., 4 Jun 2026).

6. Domain-specific, OS-level, and training-oriented harnesses

The literature also includes purpose-built harnesses that encode domain-specific execution constraints rather than generic tool loops.

System	Domain	Characteristic mechanism
CAX-Agent	MAPDL/APDL automation	orchestrator-owned recovery ladder
LemonHarness	long-horizon workspace agents	controlled workspace, rule knowledge, time-aware execution
PhoneHarness	phone-use agents	mixed GUI, CLI, and tool routing with side-effect verification
AOHP	Android OS-level agents	personalized service composition, efficient agent interfaces, secure information flow
Polar	agentic RL over existing harnesses	proxy-based token-faithful trajectory reconstruction

CAX-Agent is a lightweight agent harness for MAPDL automation organized as a three-layer stack—routing, local lightweight model, unified external LLM API—under an orchestrator that owns the ReAct loop, retry budgets, execution trace, and checkpoints. Its recovery ladder escalates from deterministic rule patching through model-driven regeneration to context enrichment and human intervention. On 50 structural APDL prompts and 450 total case-runs, the model_only strategy achieved completion rate 0.9267, task score 3.5867/4, total score 9.1600/10, and zero-intervention rate 0.8400 (Lin et al., 12 May 2026).

LemonHarness defines an explicit execution boundary by constraining state-changing operations to a controlled workspace, loading rule knowledge at task start, and exposing elapsed and remaining budget to the model through a phase schedule with P1 Explore, P2 Implement, P3 Validate, and P4 Reserve. On Terminal-Bench 2.0, LemonHarness_GPT-5.3-CodeX reached 84.49% accuracy over 445 trials, and pairing the same framework with GPT-5.5 raised average accuracy to 86.52% across five jobs (Ren et al., 23 Jun 2026).

PhoneHarness targets mobile workflows that require deciding among GUI actions, device-side CLI, and host-side tools. It uses deterministic-first routing, bounded GUI delegation, nested traces, and side-effect verifiers. On its annotated evaluation split, it reports a 75.0% pass rate, outperforming the strongest non-PhoneHarness setting by 12.9 percentage points (Li et al., 12 Jun 2026).

AOHP moves the harness into the operating system itself by treating agents as first-class OS actors on top of AOSP. Its core mechanisms are personalized service composition, efficient agent interfaces, and secure information flow. In preliminary experiments it reports 75.56% average completion versus 54.44% on stock Android, together with −51.55% token cost, −44.21% duration, and −44.64% tool calls on a subset of fully solved tasks (Zhao et al., 22 Jun 2026).

Polar addresses training rather than inference-time execution. It treats the agent harness as a black box, proxies LLM API calls at the provider boundary, records token IDs and log-probabilities, reconstructs token-faithful trajectories, and exposes rollout-as-a-service endpoints to independent trainers. Using simple GRPO, it improved Qwen3.5-4B by 22.6, 4.8, 0.6, and 6.2 points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code, and Pi harnesses, respectively (Xu et al., 22 May 2026).

7. Open problems and future directions

Several limitations recur across the literature. Harness-Bench improves reproducibility by using offline, sandboxed workflows, but this limits coverage of live services, changing external state, and long-term production memory; it also evaluates complete harness configurations descriptively rather than as causal decompositions of specific mechanisms (Yao et al., 27 May 2026). The variance-decomposition literature likewise notes that harness-induced variance depends on the sampling distribution over harnesses, that the community lacks a principled distance notion between harnesses, and that comparability is still anchored to harness-confounded leaderboards (Zhang et al., 7 May 2026).

The “code as agent harness” survey frames the broader research agenda as evaluation beyond final task success, verification under incomplete feedback, regression-free harness improvement, consistent shared state across multiple agents, human oversight for safety-critical actions, and multimodal extensions (Ning et al., 18 May 2026). These are not narrow implementation problems; they are questions about how a harness should represent evidence, preserve invariants, and coordinate state under partial observability and long horizons.

System papers add further unresolved issues. LemonHarness identifies long-chain tasks, concurrency, external network effects, and stronger sandboxing details as open problems; AOHP highlights capability discovery, structured-UI coverage for custom-rendered apps, resource scheduling for virtual displays and sandbox runtimes, and policy usability for approvals and audit review (Ren et al., 23 Jun 2026, Zhao et al., 22 Jun 2026).

A plausible implication is that future work will increasingly treat harnesses as learnable, inspectable, and evolvable objects rather than static glue code. That direction is explicit in automated harness-evolution work, where an inner loop evolves a worker harness $\pi_\theta(\cdot \mid c(s_t))$ 3 for a task and an outer meta-evolution loop optimizes the evolution protocol $\pi_\theta(\cdot \mid c(s_t))$ 4 itself, although empirical results for that framework have not yet been reported (Seong et al., 22 Apr 2026). Across the surveyed literature, the common endpoint is not a single preferred architecture, but a shift in unit of analysis: reliable agent capability is increasingly a property of the model–harness configuration, the execution policies encoded around it, and the evidence those policies preserve throughout the trajectory.