---
title: Agent Execution Harness
url: https://www.emergentmind.com/topics/agent-execution-harness
type: topic
---

# Agent Execution Harness

An agent execution harness is the system layer or runtime wrapper that surrounds a language model and turns model outputs into actions in an external workspace or operating environment. Recent work defines the harness as the layer that conditions model calls; organizes context, tools, state, permissions, constraints, tracing, budget control, recovery, and retries; and mediates long-running execution so that an agent can produce concrete, verifiable artifacts rather than only text. A compact characterization used across multiple papers is **Agent = Model + Harness**, emphasizing that reported capability is jointly determined by the base model and the execution substrate rather than by the model alone [2605.27922] [2605.18747] [2605.23950].

## 1. Conceptual foundations

In the control-theoretic framing, the language model is an open-loop stochastic policy, while the harness is the controller that closes the loop around environment feedback and state updates. The harness constructs the model-visible context $c(s_t)$, invokes the policy $\pi_\theta(\cdot \mid c(s_t))$, parses and routes actions through tools, validates observations, and updates state according to a controller rule $s_{t+1} = \mathcal{C}_H(s_t, a_t, o_t; H)$. This formulation makes stability, context drift, and control lag harness properties rather than model-only properties [2605.23950].

A complementary systems view defines the harness as the software layer that turns a stateless LLM into a reliable, long-running agent by surrounding it with executable tools, sandboxes, memory, validators, permission boundaries, telemetry, and closed-loop control. In the “code as agent harness” perspective, code is the operational substrate because it is executable, inspectable, and stateful; plans, tests, logs, repositories, and simulators become first-class execution artifacts rather than latent prompt content [2605.18747].

This notion generalizes across domains. In mobile automation, the harness is the runtime substrate that connects planning to concrete device actions and verifies side effects; in finance, it is the inline wrapper that intercepts every proposed tool call before it commits state; in OS-level systems, it is the agent-native substrate that exposes system and app capabilities through structured interfaces while enforcing information-flow control [2606.14832] [2605.27333] [2606.23449].

## 2. Architectural surfaces and runtime responsibilities

Recent work converges on a layered decomposition of harness responsibilities. One disclosure taxonomy organizes them as **Execution, Tool, Context, Scheduling, Observability, Verification, Governance**; another uses **Execution, Tooling, Context, Lifecycle, Observability, Verification, Governance**. The naming differs, but both taxonomies place the same engineering burden on runtime substrate, tool mediation, context construction, orchestration, tracing, validation, and policy enforcement [2605.23950] [2606.06324].

| Surface | Runtime role |
|---|---|
| Execution | sandboxing, runtime substrate, side-effect boundaries |
| Tool / Tooling | tool registry, schemas, adapters, parameter validation |
| Context | prompt assembly, retrieval, memory, compression, cache policy |
| Scheduling / Lifecycle | loops, retries, timeouts, delegation, rollback, stopping rules |
| Observability | traces, checkpoints, latency/cost logging, state diffs |
| Verification | parsers, validators, tests, anomaly detection, final checks |
| Governance | permissions, allowlists/denylists, approvals, audit trails |

Within this decomposition, the harness owns concrete artifacts: prompt templates, tool schemas and adapters, orchestration code, configuration files, telemetry hooks, verification scripts, checkpoint stores, and permission policies. Several systems also externalize durable state into files or workspaces rather than leaving it implicit in conversation history. Natural-Language Agent Harnesses, for example, separate runtime state and final artifacts into stable file-backed surfaces such as `/sa-output/runtime` and `/sa-output/artifacts`, while LemonHarness constrains all state-changing operations to a controlled workspace and records the resulting file and process effects as observations for later turns [2603.25723] [2606.24311].

The architectural consequence is that harness quality is not reducible to prompt wording. It depends on how the runtime exposes tools, constrains side effects, reconstructs state, logs evidence, and decides when an execution is complete. This suggests that “prompt engineering” is only one narrow slice of harness engineering.

## 3. Control-flow models and execution semantics

One major axis of variation is who owns control flow. In conventional agent loops, the model decides what to do next from an ever-growing transcript. Agentic Programming argues that looping, branching, sequencing, and termination should instead be deterministic program logic, with the LLM invoked only as a typed subroutine for reasoning or generation. Under that design, execution history forms a call-tree-derived DAG, and prompt context for a live call scales with call depth rather than with flat transcript length:  
$$
L(d) \le \sum_{i=0}^{d}(S_i + B_i),
$$
which yields $O(d)$ bounded live context under bounded frame and summary budgets [2606.15874].

A related scheduler-theoretic proposal is the Structured Graph Harness, which lifts control flow into an explicit static DAG with immutable plan versions, deterministic scheduling, and a strict three-level recovery escalation protocol. Node readiness is computed from explicit dependencies rather than inferred from model memory, and bounded retries and timeouts guarantee that every node reaches a terminal state in a finite acyclic graph [2604.11378].

Declarative harness languages occupy another point in the design space. AgentSPEX specifies typed steps, branching, loops, parallel execution, reusable submodules, explicit state management, checkpointing, verification, and logging in YAML, with the harness interpreting `task`, `step`, `if`, `switch`, `while`, `for_each`, `parallel`, `gather`, `call`, `set_variable`, `increment`, `input`, and `return` constructs directly [2604.13346]. Natural-Language Agent Harnesses externalize high-level control logic into editable natural language interpreted by an Intelligent Harness Runtime, with explicit contracts, durable artifacts, child-agent semantics, and file-backed state [2603.25723].

A different approach makes the harness itself learnable. HarnessBridge parameterizes the harness as a bidirectional interface policy with an observation projection that distills raw history into a compact generator-visible state and an action projection that maps a proposed action into either an executable transition or a trajectory-grounded rejection. The generator policy remains frozen; only the lightweight harness controller is trained [2606.12882].

## 4. Measurement, benchmarking, and reporting

A central development in the literature is the move from model-centric evaluation to model–harness evaluation. Harness-Bench was introduced specifically to isolate configuration-level harness effects under fixed user-facing task prompts, fixtures, initial sandbox state, budget, timeout, and evaluator, while preserving each harness’s native prompting, tool interfaces, state-management policy, recovery behavior, and action formats. The benchmark contains **106 sandboxed offline tasks** and reports **5,194 execution trajectories**. Across the same task set and model-backend pool, it finds a **23.8-point aggregate score gap** between the best configurable harness, NanoBot at **76.2**, and the lowest, OpenClaw at **52.4**, with substantial differences in completion, process quality, tokens, turns, and failure behavior [2605.27922].

The evaluation formalism in Harness-Bench makes explicit that outcome quality, security, and process quality are all harness-sensitive:
$$
\mathrm{TaskScore}_i = \mathrm{Security}_i \cdot \mathrm{Completion}_i \cdot \mathrm{Process}_i,
$$
$$
\mathrm{Process}_i = \frac{\mathrm{Robustness}_i + \mathrm{ToolUse}_i + \mathrm{Consistency}_i}{3}.
$$
This scoring scheme records not only final artifacts but also execution traces, usage statistics, validator outputs, and security-gate outcomes [2605.27922].

The position paper “Stop Comparing LLM Agents Without Disclosing the Harness” pushes the same argument further through variance decomposition. Let $B(M,H)$ denote benchmark score for model $M$ under harness $H$. The paper defines harness-induced variance $HV(M)$ and model-induced variance $MV(H)$, and in a controlled factorial study on SWE-bench Verified “subset100” reports an average **HV of 18.48 pp²**, an average **MV of 2.37 pp²**, and **HV/MV = 7.80×**, with **6 out of 9** model-pair/harness-pair comparisons flipping ordering [2605.23950].

Trajectory-level safety evaluation makes the same point from a different angle. HarnessAudit-Bench instantiates **210 tasks across eight real-world domains** in both single-agent and multi-agent configurations, and reports that task completion is misaligned with safe execution, that violations accumulate with trajectory length, and that most violations concentrate in resource access and inter-agent information transfer [2605.14271].

For optimization workflows, VeRO treats harnessed evaluation itself as the object of study. It provides versioned agent snapshots, budget-controlled evaluation, structured execution traces, and a standardized observation interface so that coding-agent optimizers can be compared under identical edit–execute–evaluate budgets. Its budget is defined as the number of evaluator invocations, $n_{\mathcal E} \le B$, and every evaluation is tied to a specific Git commit and stored trace [2602.22480].

| Framework | Primary focus | Reported scale |
|---|---|---|
| Harness-Bench | configuration-level harness effects | 106 tasks, 5,194 trajectories |
| HarnessAudit-Bench | trajectory safety in single- and multi-agent harnesses | 210 tasks, eight domains |
| VeRO | agent optimization under versioned, budgeted evaluation | versioned snapshots and structured traces |

## 5. Safety, execution alignment, and repair

Safety-oriented work treats the harness as the main enforcement point for permission boundaries, information flow, rollback, and pre-execution validation. Harness-Bench introduces the notion of **execution alignment**, defined as the degree to which the harness preserves correspondence among the agent’s reasoning, observed workspace state, tool actions and feedback, and evaluator-checked conditions of success. Among failed trajectories, it reports recurring failure symptoms with concrete rates: **contract/format violations 36.4%**, **tool/recovery lapses 24.6%**, **evidence/grounding gaps 14.6%**, **artifact commitment misses 11.1%**, and **state/continuation failures 9.3%** [2605.27922].

SafeHarness integrates security directly into the lifecycle through four layers: adversarial context filtering at input processing, tiered causal verification at decision making, privilege-separated tool control at action execution, and safe rollback with adaptive degradation at state update. Across benchmark datasets, harness configurations, and attack scenarios, it reports an average reduction of approximately **38% in UBR** and **42% in ASR** relative to the unprotected baseline while preserving core task utility [2604.13630].

FinHarness specializes the same inline principle for finance agents. It combines a Query Monitor, a Tool Monitor, and a Cascade module that routes each step to a lightweight or advanced-tier LLM judge, then reinjects fired risk factors into the agent input as ex-ante evidence. On FinVault, routed FinHarness reduces **ASR from 38.3% to 15.0%** while preserving benign approval from **41.1% to 39.3%**, and uses **4.7× fewer advanced-judge calls** than an always-advanced ablation [2605.27333].

When harness flaws are already present, HarnessFix provides a repair workflow. It compiles execution traces and harness code into a Harness-aware Trace Intermediate Representation, attributes failures to responsible trajectory steps and harness layers, consolidates recurring diagnoses into flaw records, and applies scoped repair operators under regression-aware validation. Across SWE-Bench Verified, Terminal-Bench 2.0 Verified, GAIA, and AppWorld, it reports held-out improvements of **15.2%–50.0%** over initial harnesses [2606.06324].

## 6. Domain-specific, OS-level, and training-oriented harnesses

The literature also includes purpose-built harnesses that encode domain-specific execution constraints rather than generic tool loops.

| System | Domain | Characteristic mechanism |
|---|---|---|
| CAX-Agent | MAPDL/APDL automation | orchestrator-owned recovery ladder |
| LemonHarness | long-horizon workspace agents | controlled workspace, rule knowledge, time-aware execution |
| PhoneHarness | phone-use agents | mixed GUI, CLI, and tool routing with side-effect verification |
| AOHP | Android OS-level agents | personalized service composition, efficient agent interfaces, secure information flow |
| Polar | agentic RL over existing harnesses | proxy-based token-faithful trajectory reconstruction |

CAX-Agent is a lightweight agent harness for MAPDL automation organized as a three-layer stack—routing, local lightweight model, unified external LLM API—under an orchestrator that owns the ReAct loop, retry budgets, execution trace, and checkpoints. Its recovery ladder escalates from deterministic rule patching through model-driven regeneration to context enrichment and human intervention. On **50** structural APDL prompts and **450** total case-runs, the **model_only** strategy achieved **completion rate 0.9267**, **task score 3.5867/4**, **total score 9.1600/10**, and **zero-intervention rate 0.8400** [2605.15218].

LemonHarness defines an explicit execution boundary by constraining state-changing operations to a controlled workspace, loading rule knowledge at task start, and exposing elapsed and remaining budget to the model through a phase schedule with **P1 Explore**, **P2 Implement**, **P3 Validate**, and **P4 Reserve**. On Terminal-Bench 2.0, LemonHarness_GPT-5.3-CodeX reached **84.49%** accuracy over **445** trials, and pairing the same framework with GPT-5.5 raised average accuracy to **86.52%** across five jobs [2606.24311].

PhoneHarness targets mobile workflows that require deciding among GUI actions, device-side CLI, and host-side tools. It uses deterministic-first routing, bounded GUI delegation, nested traces, and side-effect verifiers. On its annotated evaluation split, it reports a **75.0% pass rate**, outperforming the strongest non-PhoneHarness setting by **12.9 percentage points** [2606.14832].

AOHP moves the harness into the operating system itself by treating agents as first-class OS actors on top of AOSP. Its core mechanisms are personalized service composition, efficient agent interfaces, and secure information flow. In preliminary experiments it reports **75.56%** average completion versus **54.44%** on stock Android, together with **−51.55% token cost**, **−44.21% duration**, and **−44.64% tool calls** on a subset of fully solved tasks [2606.23449].

Polar addresses training rather than inference-time execution. It treats the agent harness as a black box, proxies LLM API calls at the provider boundary, records token IDs and log-probabilities, reconstructs token-faithful trajectories, and exposes rollout-as-a-service endpoints to independent trainers. Using simple GRPO, it improved Qwen3.5-4B by **22.6**, **4.8**, **0.6**, and **6.2** points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code, and Pi harnesses, respectively [2605.24220].

## 7. Open problems and future directions

Several limitations recur across the literature. Harness-Bench improves reproducibility by using offline, sandboxed workflows, but this limits coverage of live services, changing external state, and long-term production memory; it also evaluates complete harness configurations descriptively rather than as causal decompositions of specific mechanisms [2605.27922]. The variance-decomposition literature likewise notes that harness-induced variance depends on the sampling distribution over harnesses, that the community lacks a principled distance notion between harnesses, and that comparability is still anchored to harness-confounded leaderboards [2605.23950].

The “code as agent harness” survey frames the broader research agenda as evaluation beyond final task success, verification under incomplete feedback, regression-free harness improvement, consistent shared state across multiple agents, human oversight for safety-critical actions, and multimodal extensions [2605.18747]. These are not narrow implementation problems; they are questions about how a harness should represent evidence, preserve invariants, and coordinate state under partial observability and long horizons.

System papers add further unresolved issues. LemonHarness identifies long-chain tasks, concurrency, external network effects, and stronger sandboxing details as open problems; AOHP highlights capability discovery, structured-UI coverage for custom-rendered apps, resource scheduling for virtual displays and sandbox runtimes, and policy usability for approvals and audit review [2606.24311] [2606.23449].

A plausible implication is that future work will increasingly treat harnesses as learnable, inspectable, and evolvable objects rather than static glue code. That direction is explicit in automated harness-evolution work, where an inner loop evolves a worker harness $\mathcal H$ for a task and an outer meta-evolution loop optimizes the evolution protocol $\Lambda$ itself, although empirical results for that framework have not yet been reported [2604.21003]. Across the surveyed literature, the common endpoint is not a single preferred architecture, but a shift in unit of analysis: reliable agent capability is increasingly a property of the **model–harness configuration**, the execution policies encoded around it, and the evidence those policies preserve throughout the trajectory.

Source: https://www.emergentmind.com/topics/agent-execution-harness