Harness-Aware Post-Training

• Harness-aware post-training is a method that adapts the runtime infrastructure (harness) around language models, focusing on interface-level improvements rather than updating model parameters.
• It employs techniques like learnable controllers, stateful memory management, and trace-guided repair to optimize context construction, tool invocation, and performance validation.
• Empirical evaluations show that optimized harnesses can significantly boost long-horizon task reliability and transfer performance while reducing token usage and error rates.

Harness-aware post-training denotes a class of post-training methods in which the optimization target includes the runtime harness that mediates how a model observes, reasons, acts, and is evaluated, rather than treating that harness as a fixed engineering wrapper. In the recent agent literature, the harness includes context construction, tool exposure and invocation, orchestration, validation, observability, memory, and governance, and benchmark performance is increasingly described as a property of the composed system rather than of the backbone model alone (Zhang et al., 7 May 2026, Wang et al., 11 Jun 2026, Kim et al., 24 Jun 2026). In this sense, harness-aware post-training generalizes post-training from parameter updates to interface-level adaptation: the object being improved may be a bidirectional controller between agent and environment, a stateful execution scaffold, an editable set of prompts, skills, memories, and tools, or a typed collection of runtime processors that can be evolved from traces (Chen et al., 12 Jun 2026, Lin et al., 28 May 2026).

1. Conceptual scope and formal object of optimization

The modern agent papers define the harness as the infrastructure layer around the LLM. One formulation describes it as the code that formats history, decides what context the generator sees, validates actions, and mediates environment feedback; another defines it as the full infrastructure layer that governs context construction, tool interaction, orchestration, and verification; still another restricts it to the wrapper acting through the system prompt $p$ and tool environment $TE$ (Wang et al., 11 Jun 2026, Zhang et al., 7 May 2026, Kim et al., 24 Jun 2026). Across these formulations, the common point is that the harness determines the effective interface between policy and environment.

This change in perspective is visible in the formalizations used by the literature. HarnessBridge defines the harness as a policy

$$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$

where $\widetilde{H}_t$ is the generator-visible projected state and $a_t'$ is the environment-facing action; the generator itself stays fixed, and optimization is only over the harness (Wang et al., 11 Jun 2026). In harness self-evolution, the agent at evolution step $t$ is written as

$A_t = (f, H_t),$

with fixed backbone model $f$ and editable harness state $H_t$ (Lin et al., 28 May 2026). In the ALFWorld study of harness design, the step state is

$s_t = (p, q, \mathcal{T}_{t-1}),$

making harness variation explicit in both prompt design and the structure of accumulated tool interaction history (Kim et al., 24 Jun 2026).

This scope makes harness-aware post-training broader than prompt tuning but narrower than unrestricted system redesign. What changes is the mediating runtime structure; what often remains fixed is the base generator, the tool environment, or both. That distinction is explicit in HarnessBridge, which frames the method as a learnable interface layer rather than a new agent policy, and in the self-evolution literature, which studies persistent harness updates while keeping model weights unchanged (Wang et al., 11 Jun 2026, Lin et al., 28 May 2026).

2. Why the harness is treated as a binding constraint

A central theoretical claim is that, on long-horizon tasks with comparable frontier models, performance variance is often governed more by harness configuration than by model choice. The position paper "Stop Comparing LLM Agents Without Disclosing the Harness" formalizes this as the Binding Constraint Thesis using benchmark score $TE$0, model variance $TE$1, and harness variance $TE$2:

$TE$3

It argues that in the relevant regime, harness variance is often comparable to or larger than model variance, and supports this with a controlled $TE$4 factorial experiment on a 100-task SWE-bench Verified subset that reports average $TE$5, average $TE$6, ratio $TE$7, and 6 ranking reversals out of 9 model-pair/harness-pair comparisons (Zhang et al., 7 May 2026).

The same paper gives a control-theoretic framing in which the model is an open-loop stochastic policy,

$TE$8

while the harness is the controller that closes the loop by deciding what the model sees, how tool outputs are filtered, how failures are recovered, and when execution terminates (Zhang et al., 7 May 2026). Its structural reliability channels—stability, context drift, and control lag—are harness properties rather than model properties. This suggests that long-horizon reliability is not exhausted by model capability; it also depends on context retention, corrective feedback timing, and recovery logic.

Complementary papers restate the same intuition in algorithmic terms. HarnessBridge argues that manual harnesses become a bottleneck because long trajectories grow noisy and redundant on the environment-to-agent side while the agent-to-environment side accumulates repeated, invalid, or poorly grounded actions (Wang et al., 11 Jun 2026). Harness-1 makes the analogous claim for retrieval: append-only transcripts force the model to reconstruct candidate sets, evidence status, and budget state from text history, whereas this routine bookkeeping can be maintained more reliably by the environment-side harness (Jiang et al., 1 Jun 2026).

3. Main algorithmic forms

The current literature contains several distinct but related forms of harness-aware post-training.

System	Optimized harness object	Reported result
HarnessBridge	Bidirectional projection policy	33.7% on Terminal-Bench 2.0 with 1.23M tokens
Harness-1	Stateful search harness	0.730 average curated recall
HarnessLLM	Test harness code generation	Better bug finding and testing strategy diversity
ALFWorld harness study	Training under h-low / h-mid / h-high	Richer harnesses improve ID and OOD performance
HarnessX	Typed harness primitives evolved from traces	+14.5% average gain, up to +44.0%

One line of work learns the interface directly. HarnessBridge trains a lightweight Qwen3.5-0.8B controller on a harness supervision dataset of 5,405 examples after deduplication, filtering, and LLM-as-judge validation, using Adam with learning rate $TE$9, batch size 64, bf16 precision, and 3 epochs (Wang et al., 11 Jun 2026). Its observation projection compresses raw trajectory history into a projected state with an active-state index and per-history-unit decisions $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$0, while its action projection predicts $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$1 and, when rejecting, returns trajectory-grounded feedback $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$2 (Wang et al., 11 Jun 2026). On Terminal-Bench 2.0 with Qwen3.5-35B-A3B it reaches 33.7% success rate with 1.23M tokens, compared with Terminus 2’s 30.3% and 2.31M tokens; on SWE-bench Verified with the same generator it achieves 60.2% success with 1.13M tokens, versus Terminus 2’s 61.6% and 1.47M tokens (Wang et al., 11 Jun 2026).

A second line externalizes state into the harness and then post-trains the policy over that state. Harness-1 introduces a per-episode WorkingMemory with state

$$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$3

where the harness maintains the candidate pool, curated set, importance tags, full-text store, evidence graph, verification cache, search history, and budget marker (Jiang et al., 1 Jun 2026). The policy retains semantic decisions—what to search, what to verify, which documents to keep or discard, and when to stop—while the harness handles deduplication, compression, evidence rendering, capacity management, and budget-aware prompt degradation (Jiang et al., 1 Jun 2026). Across eight retrieval benchmarks, Harness-1 reports 0.730 average curated recall, outperforming the next strongest open search subagent by +11.4 points, with gains especially strong on held-out transfer benchmarks (Jiang et al., 1 Jun 2026).

A third line studies the harness as an explicit training condition. The ALFWorld work defines three harnesses—h-low, h-mid, and h-high—by varying tool descriptions and per-step auxiliary information such as valid tools and the currently carried object (Kim et al., 24 Jun 2026). Its core result is that richer harnesses improve zero-shot and post-trained performance, and that applying the harness only after training is much worse than training with it in place. For Qwen2.5-7B with GRPO, h-mid shows training-time 76.2 versus post-hoc 55.5, and h-high shows training-time 77.9 versus post-hoc 55.4 (Kim et al., 24 Jun 2026). This indicates that the harness is part of the learned deployment interface rather than a removable presentation layer.

A fourth line uses harness generation itself as the post-training target. HarnessLLM trains models to write executable testing harnesses—Python code with input generators and output validators—via supervised fine-tuning followed by RLVR with a customized reward (Liu et al., 2 Nov 2025). Instead of predicting input-output pairs, the model writes code that synthesizes inputs and validates outputs through hardcoded checks, reference computation, or invariant checking. On Qwen3-4B, the RL Harness model reports TBR 74.1 on MBPP+Fix (Hard), 69.9 on LiveCodeBench, and 74.4 on Codeforces, each above the corresponding RL Input/Output baseline (Liu et al., 2 Nov 2025). In this setting, “harness-aware” refers to executable testing logic rather than an agent-execution scaffold, but the underlying principle is parallel: post-training targets the mediating runtime structure.

4. Trace-guided diagnosis and repair

Harness-aware post-training is not limited to learning a controller from successful trajectories; it also includes diagnosing and repairing harness flaws from failed ones. HarnessFix proposes a four-agent pipeline—trace abstraction agent, diagnosis agent, repair agent, and validation agent—that compiles raw execution traces and harness code into a Harness-aware Trace Intermediate Representation (HTIR) (Chen et al., 4 Jun 2026). HTIR normalizes heterogeneous traces into TraceStep nodes with step attributes, temporal links, input provenance links, control-flow links, node-local diagnostic evidence, and a harness layer responsibility facet mapped to ETCLOVG layers: Execution Environment and Sandbox, Tool Interface, Context and Memory, Lifecycle and Orchestration, Observability, Verification and Evaluation, and Governance and Security (Chen et al., 4 Jun 2026).

The diagnosis stage localizes failure symptoms, backtracks evidence through temporal and provenance structure, adjudicates candidate steps, and maps responsible evidence to implicated harness layers (Chen et al., 4 Jun 2026). Diagnoses are then consolidated into flaw records and mapped to scoped repair operators such as loop guarding, summary/retrieval policy repair, tool-schema narrowing, effect-evidence completion guarding, or finalization-check strengthening (Chen et al., 4 Jun 2026). Patch acceptance is governed by explicit validation criteria:

$$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$4

Across SWE-Bench Verified, Terminal-Bench 2.0 Verified, GAIA, and AppWorld, HarnessFix improves held-out test performance over the initial harnesses by 15.2%--50.0% and exceeds the strongest human-designed harness on every benchmark (Chen et al., 4 Jun 2026).

This repair perspective changes the meaning of post-training. Instead of improving prompts or optimizing final outcome statistics alone, it treats failures as attributable to specific runtime steps and specific harness layers. A plausible implication is that harness-aware post-training becomes closer to program repair than to conventional policy optimization when the dominant errors are infrastructural rather than representational.

5. Self-evolution and harness–model co-evolution

Another branch of the literature treats the harness as an editable persistent artifact that can evolve from execution evidence. The self-evolution study formalizes the update process as

$$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$5

and distinguishes two capabilities: harness-updating, the capability to produce useful persistent harness updates, and harness-benefit, the capability of the task-solving agent to exploit updated harnesses (Lin et al., 28 May 2026). Its main findings are that harness-updating is flat in base capability and harness-benefit is non-monotonic in base capability. The gap between the best and worst evolver is at most 3.1 percentage points on any benchmark, while benefit peaks at mid-tier models rather than increasing monotonically with scale (Lin et al., 28 May 2026).

The same paper identifies two failure modes for weak-tier models: harness activation failure and harness adherence failure (Lin et al., 28 May 2026). On SkillsBench, reported skill-load rates are 0.957 for Opus 4.6, 0.959 for Sonnet 4.6, 0.961 for Qwen3-235B, but 0.446 for GPT-OSS-120B and 0.251 for Qwen3-32B; reported Harness-Following Rate values include 0.757 for Opus 4.6, 0.730 for Sonnet 4.6, 0.350 for Qwen3-235B, 0.442 for GPT-OSS-120B, 0.600 for Haiku 4.5, and 0.142 for Qwen3-32B (Lin et al., 28 May 2026). This suggests that useful harness updates are not sufficient: the agent must reliably invoke and follow harness artifacts over long horizons.

HarnessX extends this idea into a foundry for composable, adaptive, and evolvable harnesses. It defines a harness as $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$6, with harness configuration $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$7, and implements runtime behavior through typed processors attached to eight lifecycle hooks (Chen et al., 12 Jun 2026). Edits are performed through a substitution algebra that can insert, replace, or remove processors while preserving interface contracts, and adaptation is carried out by AEGIS, a four-stage pipeline of Digester, Planner, Evolver, and Critic (Chen et al., 12 Jun 2026). Across ALFWorld, GAIA, WebShop, $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$8-Bench, and SWE-bench Verified, HarnessX reports an average gain of +14.5% and up to +44.0%, with gains largest where baselines are lowest (Chen et al., 12 Jun 2026).

HarnessX also closes the loop between harness adaptation and model training by reusing the same trajectories as diagnostic evidence for harness evolution and as reward-bearing experience for cross-harness GRPO (Chen et al., 12 Jun 2026). The grouped objective compares trajectories for the same task across different harness versions, turning harness variation into a source of exploration rather than a nuisance variable. This suggests a stronger notion of harness-aware post-training in which the harness is not only optimized alongside the model but also used to structure the model’s training signal.

6. Evaluation standards, robustness, and relation to other post-training notions

The growing emphasis on harness effects has produced a corresponding evaluation methodology. The disclosure proposal recommends a Harness Card organized by the seven-layer ETCSOVG taxonomy—Execution, Tool, Context, Scheduling, Observability, Verification, Governance—and a variance decomposition protocol whose minimal valid design is a $$\pi_h : (s,q,H_t,a_t) \mapsto (\widetilde{H}_t,a_t'),$$9 model-by-harness grid (Zhang et al., 7 May 2026). It also recommends trajectory-level metrics such as Recovery Rate $\widetilde{H}_t$0, Context Retention, and Control Lag $\widetilde{H}_t$1 to explain why harnesses differ (Zhang et al., 7 May 2026). In this view, benchmark scores without harness disclosure are incomplete because they collapse model effects, harness effects, and model–harness interaction into a single number.

Robustness results reinforce the same point. In the ALFWorld study, under stronger tool environment shift v2.0, Qwen2.5-7B + GRPO + h-low drops to 2.7, which is 10.8 points below the untrained base model’s 13.5; under the same shift, h-mid reaches 63.6 and h-high 53.0 (Kim et al., 24 Jun 2026). In HarnessBridge, a controller instruction-tuned on SWE-bench trajectories transfers to Terminal-Bench 2.0 and to unseen generators, including GPT-5.4-Nano, GPT-5.4, DeepSeek-V4-Flash/Pro, and Claude-Opus-4.7; for GPT-5.4-Nano it raises success from 18.0% to 22.5% while cutting tokens from 9.80M to 0.91M, and for GPT-5.4 it keeps success at 53.9% while reducing token use by about 89% (Wang et al., 11 Jun 2026). In Harness-1, the gain over the closest open baseline is about +7.9 points on source-family benchmarks and +17.0 points on held-out transfer benchmarks, a 2.2× larger gain on the benchmarks furthest from training data (Jiang et al., 1 Jun 2026). These results support the claim that harness informativeness and structured state can act as transferable priors.

The term nonetheless sits beside older, different uses of “post-training.” Backend-aware post-training quantization treats hardware constraints as the target of post-training adaptation rather than the runtime agent interface, using a declarative hardware specification and search over effective bit-widths (Jiang et al., 2021). Analog-based in-memory computing post-training optimizes DAC range and conductance range after training to recover deployment accuracy on AIMC hardware (Lammie et al., 2024). Much earlier, “post-training” in deep learning referred to freezing the learned representation and re-optimizing only the last layer, with a kernel-theoretic interpretation (Moreau et al., 2016). These are post-training procedures, but not harness-aware in the agent-execution sense developed by recent LLM work.

Several limitations recur across the harness-aware literature. HarnessBridge notes that evaluation is confined to coding tasks and reports single-run results due to evaluation cost (Wang et al., 11 Jun 2026). HarnessFix depends on trace quality, correctness of attribution, expressiveness of repair operators, and representative held-out validation (Chen et al., 4 Jun 2026). The self-evolution study does not evaluate parametric fine-tuning, RL on model weights, or hybrid methods combining weights and harness updates, and it notes that broader model grids could clarify dependence on family, scale, and training recipe (Lin et al., 28 May 2026). The ALFWorld study shows that minimal harnesses are especially brittle when tool names, parameter keys, or tool grouping change (Kim et al., 24 Jun 2026). Taken together, these caveats delimit the present state of the field: the strongest empirical support is for long-horizon, tool-heavy environments where the interface layer can dominate failure modes, but the general claim that the harness is a first-class post-training target is now supported by multiple, methodologically distinct lines of evidence (Zhang et al., 7 May 2026, Chen et al., 12 Jun 2026).