---
title: Model-Harness Co-design
url: https://www.emergentmind.com/topics/model-harness-co-design
type: topic
---

# Model-Harness Co-design

Model-Harness Co-design is the systematic design and optimization of an intelligent system as a coupled pair comprising a foundation model or policy and a surrounding executable harness. The model supplies learned capabilities such as reasoning, coding, planning, and action selection; the harness determines what information the model receives, which actions and tools are available, how memory and state are maintained, how execution proceeds, how failures are diagnosed, and how success is verified. The central premise is that measured capability is a property of the model–harness–environment system rather than of model weights in isolation. Depending on the domain, the harness may be a prompt-and-tool wrapper, a stateful coding-agent runtime, an embodied-agent controller, a model–accelerator implementation stack, or an evolvable program that generates and evaluates further harnesses.

## 1. Conceptual foundations and scope

A model harness is the software and execution substrate surrounding a model. It mediates information flow from the task and environment to the model, converts model outputs into actions, returns observations, preserves state, and determines when an episode or task is complete. In LLM applications, the harness can control memory, retrieval, prompt construction, tool interaction, orchestration, verification, retries, termination, and final presentation. In coding agents, it additionally governs repository navigation, shell execution, middleware, sub-agents, skills, sandboxing, tracing, and project memory. In embodied agents, it can maintain persistent observations, navigation routines, battle or manipulation specialists, and online refinement procedures.

The harness is therefore more than a prompt. It is an information-flow and control-flow layer whose behavior changes the effective policy executed by a fixed model. “Meta-Harness: End-to-End Optimization of Model Harnesses” defines it as “a stateful program that wraps a language model and determines what context the model sees at each step” [2603.28052]. “AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents” instead emphasizes the model–harness–environment system:

$$
C_{\mathrm{system}} =
F(C_{\mathrm{model}},C_{\mathrm{harness}},C_{\mathrm{environment}},T).
$$

This formulation does not make model capability irrelevant. Rather, it asserts that latent model capability is converted into reliable behavior only through an appropriate runtime interface. A model may possess the competence to solve a problem yet fail because it receives irrelevant context, lacks a required tool, misinterprets tool feedback, forgets earlier decisions, cannot preserve state, or declares completion without requirement-level evidence [2605.13357].

The term co-design has several meanings in the literature. In its strictest form, it denotes joint optimization of model parameters and harness structure. HarnessForge represents an agent as a coupled harness–policy pair,

$$
\mathcal{G}=(\mathcal{H},\mathcal{R}_{\delta}),
$$

where the harness contains planning, action, and memory layers, while $\mathcal{R}_{\delta}$ is a harness-specific policy adapter attached to a frozen base reasoner [2606.01779]. HarnessX similarly treats model parameters and a symbolic runtime program as variables with different optimization geometries: model parameters are updated by reinforcement learning, whereas harnesses are modified through typed code-level substitutions [2606.14249].

In a weaker but still important sense, harness-only optimization is a form of partial co-design. Meta-Harness, Agentic Harness Engineering, Continual Harness, HarnessCompass, and Harness-R1 hold the task model fixed while adapting the surrounding runtime. These systems still optimize the interface in response to a model’s strengths and failure modes. The resulting harness is conditional on the model, task distribution, and environment, even when no model-weight update occurs.

The scope of the concept extends beyond LLM software agents. The earlier many-core co-design framework integrates application complexity, architecture, programming language, simulator, optimizer, and workload-generation spaces into an automated binding process [1806.01104]. A3C3 generalizes the same principle to AI algorithms and accelerators, jointly searching model architecture, quantization, mapping, scheduling, memory behavior, and hardware generation [2606.20869]. These systems differ in implementation domain but share the same abstraction: behavior emerges from the interaction between a computational model and the execution substrate through which it is realized.

## 2. Harness responsibilities and architectural layers

The responsibilities assigned to a harness vary by domain, but several recurring layers have emerged.

### Information and context management

A harness determines what the model sees. This includes system prompts, tool descriptions, retrieved evidence, conversation history, memory records, environment observations, and task state. Meta-Harness treats context construction as including retrieval, filtering, deduplication, reranking, truncation, contrastive presentation, and adaptive retrieval depth [2603.28052]. RHI identifies agent roles, instructions, contracts, and workflow hops as the main textual specification of a multi-agent loop; it argues that contracts and hops determine inter-agent information flow more directly than role descriptions alone [2607.15524].

HarnessCompass distinguishes structural and guidance components. Guidance includes the system prompt, tool descriptions, skills, and memory; structural components include tool implementations, middleware, and sub-agents [2608.01918]. The distinction is operational rather than merely semantic. A prompt can advise the model to verify a patch, whereas middleware or a tool implementation can automatically perform or enforce verification.

### Tools and action interfaces

Tools expose the action space. Tool descriptions determine how the model interprets affordances, while implementations determine actual behavior. Tool schemas, argument formatting, routing, approval policies, timeouts, and recovery behavior are therefore co-design variables. In ALFWorld, harness informativeness changes whether the model receives valid-tool information, carrying state, preconditions, and tool interactions. Under a strong tool-environment shift, a model trained with minimal descriptions emits obsolete tool names, while a richer harness produces substantially more valid-format calls [2606.25447].

Coding-agent harnesses may expose shell execution, repository inspection, patch validation, environment bootstrapping, file protection, and test execution. Agentic Harness Engineering demonstrates that timeout-aware execution, background-job guidance, and hard protection of verified outputs can correct failures that natural-language instructions do not reliably prevent [2604.25850]. Harness-R1 provides lifecycle hooks for episode initialization, pre-decision hints, pre-action mediation, and post-feedback recovery, enabling the harness to block, rewrite, or force actions where supported [2608.02276].

### Memory and task state

Memory preserves project knowledge, observations, strategies, prior failures, and reusable procedures. Task state maintains hypotheses, inspected files, open questions, decisions, and next steps. These mechanisms are particularly important for long-horizon and partially observable tasks, where the model must retain information across thousands of actions.

Continual Harness represents memory as a persistent component containing observations, facts, strategies, and lessons. It also evolves executable skills and sub-agents, allowing successful routines to persist across runs [2605.09998]. The paper reports that inherited mature harnesses use memory more actively than from-scratch systems, indicating that memory existence alone is insufficient; retrieval and reuse policies are also part of co-design.

### Control flow, planning, and orchestration

The harness determines how tasks are decomposed, how subgoals are represented, when replanning occurs, how retries are allocated, and when execution terminates. Harnesses for inference-time alignment distinguish task decomposition from guided execution: decomposition defines what should happen next, whereas guidance reshapes the local action or trajectory distribution [2605.21516].

Over-decomposition can impose milestones smaller than the model’s controllable progress scale, producing overshoot, premature completion, or boundary failures. Under-decomposition can create stages too large for the model to complete reliably. The same work shows that partial workflows may outperform fully specified workflows when early scaffolding reduces uncertainty but later scaffolding interferes with adaptive execution.

RHI similarly treats workflow hops, recall triggers, acceptance gates, fallback rules, and termination criteria as part of the harness rather than as incidental prompt text [2607.15524]. Its experiments suggest that structured multi-agent orchestration can outperform simply concatenating role instructions into one context.

### Observability, attribution, and verification

Modern harnesses increasingly treat observability as a capability. AI Harness Engineering defines eleven responsibilities, including observability, failure attribution, verification, permissions, entropy auditing, and intervention recording [2605.13357]. The associated H0–H3 ladder progressively exposes tools, project memory, task state, deterministic checks, reproduction logs, attribution records, and verification reports.

Agentic Harness Engineering introduces three matched observability mechanisms: component observability, experience observability, and decision observability. Editable components are represented as versioned files; millions of trajectory tokens are distilled into layered evidence; and every edit is paired with a prediction that is later checked against task outcomes [2604.25850].

The distinction between success and verified success is consequential. A task can produce a seemingly correct patch while lacking sufficient evidence, leaving regressions undiscovered or maintenance damage unrecorded. Verification should bind requirements to evidence, test both desired and preserved behavior, and report limitations such as regression timeouts rather than silently treating them as success.

### Maintenance, permissions, and entropy

A harness can improve immediate task success while damaging the surrounding system. Entropy auditing captures maintenance burden from stale documentation, unnecessary dependencies, debug files, weakened tests, residue, and architectural violations [2605.13357]. Permissions and sandboxing constrain destructive operations and provide approval gates. Intervention recording treats human assistance as evidence of a harness gap: instructions about which file to inspect indicate context deficiencies, human interpretations of test failures indicate observability or attribution deficiencies, and human cleanup indicates entropy deficiencies.

## 3. Co-design mechanisms and optimization loops

Model–harness co-design systems differ primarily in what they optimize, how they generate candidates, and how they attribute consequences.

### Outer-loop harness search

Meta-Harness searches over executable harness code while keeping the task model fixed. An agentic proposer inspects the source code, scores, and execution traces of prior candidates through a filesystem, proposes new harnesses, and repeats evaluation over multiple iterations [2603.28052]. The harness objective is:

$$
H^* =
\arg\max_H
\mathbb{E}_{x,\tau}
\left[r(\tau,x)\right].
$$

Accuracy and context cost are evaluated using Pareto dominance rather than necessarily a single scalar objective. The system’s principal distinction from prompt optimization is its access to raw source code and uncompressed traces, allowing structural changes to memory, retrieval, tool orchestration, state machines, and verification.

Agentic Harness Engineering adds explicit prediction contracts and rollback granularity. Its evolution loop runs the current harness, collects multiple rollouts, cleans traces, attributes previous edits, distills experience, asks an Evolve Agent to modify components, records predicted fixes and risks, commits the change, and retains the best configuration [2604.25850]. The reported attribution precision for predicted fixes is substantially better than random, while regression prediction is weak, illustrating the asymmetry between recognizing beneficial changes and anticipating collateral damage.

HarnessCompass constrains this outer-loop search. Its Generalization Gate prohibits task identifiers, private symbols, exact test names, and task-specific memory. Structural and guidance components are evolved in separate tracks, then consolidated using R$^3$ integration—revision, recombination, and refinement—to preserve useful synergy while reducing cross-component interference [2608.01918]. On SWE-bench Verified, this procedure improves Pass@1 from 54% to 66% in five evolution iterations and obtains 60.4% on held-out tasks, compared with 54.7% for AHE.

### Online and reset-free refinement

Continual Harness adapts the harness during a single ongoing embodied episode. The harness is represented as

$$
\mathcal{H}=(p,\mathcal{G},\mathcal{K},\mathcal{M}),
$$

where $p$ is the prompt, $\mathcal{G}$ the sub-agent set, $\mathcal{K}$ reusable skills, and $\mathcal{M}$ persistent memory [2605.09998]. After a warm-up interval, a Refiner analyzes recent trajectories and applies edits:

$$
\mathcal{H}_{t+1}=\mathcal{H}_t\oplus
(\Delta p,\Delta\mathcal{G},\Delta\mathcal{K},\Delta\mathcal{M}).
$$

The emulator state is not reset. This allows late-game failures, navigation loops, tool failures, and battle strategies to be repaired without discarding earlier progress. The approach differs from reset-based prompt optimization, which generally completes an episode, edits a prompt, resets the environment, and retries.

The method also supports model–harness co-learning. An open-source policy acts through the evolving harness, a process reward model scores transitions, a frontier teacher relabels low-reward windows, and soft supervised fine-tuning updates model weights. The updated model then acts through a further-refined harness, creating an adaptive data distribution.

### Learned harness editing

Harness-R1 trains a dedicated harness engineer to convert target-agent failure packets into executable runtime patches. The target agent remains frozen during harness training. If $P=H_\theta(s_B)$ is a patch generated from a batch failure packet, its reward is the realized before–after batch improvement:

$$
\Delta_B(P)=
\frac{1}{n}\sum_{i=1}^{n}(R_i^P-R_i^0).
$$

A patch is evaluated on all tasks in the batch, including initially successful tasks, so that regressions reduce its reward [2608.02276]. Cold-start supervised fine-tuning initializes the engineer, after which online GRPO selects among multiple candidate patches generated for the same failure evidence. The engineer is rewarded for downstream behavioral utility, not syntactic plausibility.

HASE integrates task solving and harness editing into one model’s multi-turn action space. The policy can generate a solution or inspect and modify prompts, retrieval, memory, factor-pool presentation, or selected evaluators [2607.03935]. Guidance edits receive proof-of-concept credit only when they improve downstream task performance. Evaluation-harness edits are more restricted and require disagreement between a local evaluator and an immutable real-world evaluator.

### Joint harness–policy evolution

HarnessForge explicitly co-evolves a harness and a policy adapter. Its population contains executable pairs:

$$
\mathbb{G}^{(r)}
=
\left\{
(\mathcal{H}_i^{(r)},\mathcal{R}_{\delta_i}^{(r)})
\right\}.
$$

Each round first evolves the harness through fault-guided tailoring, then aligns a policy adapter to the selected child harness using successful trajectories [2606.01779]. Compatibility checks require importable code, valid provider interfaces, parseable tool schemas, defined memory keys, and successful smoke tests.

The key empirical result is that a stronger harness can underperform when paired with a policy that has not learned its conventions. On API-Bank, matched harness–policy pairs improve across rounds, whereas final harnesses paired with earlier policies and final policies paired with earlier harnesses perform worse. This indicates that the adaptation unit is the executable pair rather than either component in isolation.

HarnessX provides a related typed substitution algebra. Its harness configuration maps lifecycle hooks to typed asynchronous processors, with singleton groups, execution-order metadata, and soft dependencies. Variants can replace or remove processors without forking the entire codebase. AEGIS evolves these configurations through digestion, planning, candidate generation, critique, and deterministic gating [2606.14249]. The operational mirror treats harness evolution as an MDP over symbolic configurations, but explicitly does not claim convergence guarantees because the action space is open-ended code.

### Algorithm–accelerator co-design

A3C3 extends the same logic to model implementation. The joint space is $\{A,I\}$, where $A$ denotes algorithm or model choices and $I$ denotes implementation choices [2606.20869]. Bundle representations bind computational subgraphs to tiling, parallelism, precision, memory, and scheduling decisions. Search may use evolutionary algorithms, reinforcement learning, particle swarm optimization, or differentiable architecture and implementation search.

SkyNet searches channel expansion and pooling positions jointly with FPGA mapping. EDD jointly optimizes architecture parameters, implementation parameters, and quantization. Medusa changes both LLM decoding and accelerator execution by adding speculative decoding heads, candidate trees, and parallel verification. SnapKV changes memory retention and attention workload through dynamic per-layer and per-head cache selection. In each case, the execution substrate is treated as an active determinant of model behavior rather than as a passive deployment target.

## 4. Evaluation methodology and empirical findings

Evaluation must measure both task quality and the operational consequences of harness design. Reported metrics across the literature include success or pass rate, token usage, latency, context cost, tool validity, path completion, utilization, energy, memory traffic, verification autonomy, no-action turns, regression rate, and human intervention.

### Harness effects in LLM agents

Meta-Harness reports a 7.7-point improvement over a state-of-the-art context-management system on online classification while using approximately four times fewer additional context tokens. On retrieval-augmented mathematical reasoning, a harness discovered using GPT-OSS-20B improves accuracy by 4.7 points on 200 IMO-level problems averaged across five held-out models [2603.28052]. Its classification strategies include draft verification, contrastive challengers, label coverage, query-dependent retrieval, and local decision-boundary construction.

AHE improves Terminal-Bench 2 Pass@1 from 69.7% to 77.0%, exceeding Codex’s 71.9%, ACE’s 68.9%, and TF-GRPO’s 72.3% overall [2604.25850]. On SWE-bench Verified, the frozen evolved harness raises success from 75.2% to 75.6% while reducing token usage from 526k to 461k. Cross-model transfer gains range from 2.3 to 10.1 percentage points across alternate model families, although the harness remains partly tuned to the model used during evolution.

Continual Harness reduces button-press cost on Pokémon Red and Emerald relative to a minimalist interface and recovers a majority of the gap to an expert harness for capable models. For Gemini 3 Pro on Emerald, from-scratch Continual Harness reaches 100% of milestones at a median cost of \$130, compared with 98% at \$215 for the minimalist baseline. Effects are capability-dependent: Gemini 3 Flash shows high variance, while Gemini 3 Flash-Lite can perform worse with the richer harness, establishing a capability floor for exploiting tools, context, and delegation [2605.09998].

### Tool-interface and post-training interaction

On ALFWorld, richer harnesses improve zero-shot performance and generally preserve their advantage after post-training. GPT-5 Mini rises from 28.1% under a low-information harness to 68.3% under a high-information harness. For Qwen2.5-7B with GRPO, training under a high-information harness reaches 77.9%, whereas applying that harness only after training under the low-information interface yields 55.4%, a 22.5-point gap [2606.25447].

Under a strong tool-schema shift, Qwen2.5-7B with GiGPO achieves 69.6% using the high-information harness but only 33.2% using the low-information harness. The high-information system generates 95.7% valid-format calls, compared with invalid-format rates of 75.1% and 81.2% for the medium- and low-information systems. These results show that harness design is part of the post-training distribution, not merely an evaluation-time decoration.

### Task-generalization and harness generalization

HarnessCompass improves held-out SWE-bench Verified Pass@1 from 51.6% for the seed to 60.4%, compared with 54.7% for AHE [2608.01918]. The Generalization Gate contributes substantially to this result, while proactive feedback alone improves search performance but can reduce held-out performance through overfitting. R$^3$ integration restores held-out performance and reduces the number of required turns.

Harness-R1 improves vanilla Qwen3.5-9B from 44.3% to 53.6% across WebShop, ALFWorld, and DBBench, and target-agent fine-tuning followed by target-specific harness adaptation increases performance from 59.2% to 64.2% [2608.02276]. Across 20 unseen target configurations, the learned editing policy yields a benchmark-averaged gain of 7.06 percentage points, although transfer is performed using new failure evidence rather than by blindly reusing one patch.

HASE raises Qwen3-8B Symptom2Disease accuracy from 59.4% with ordinary five-shot prompting to $86.98\pm0.38$, slightly exceeding the reported GPT-OSS-120B Meta-Harness result of 86.8% [2607.03935]. In alpha factor mining, HASE Qwen3-8B reports an AER of 14.35% and IR of 1.70, compared with 7.89% and 0.80 for GPT-OSS-120B. In geometric discovery, evaluator repair converts invalid proxy optimization into valid optimization, while weight co-evolution improves Heilbronn performance from 0.021880 with a frozen policy to 0.036082.

### Efficiency and hidden scaffold effects

The evaluation unit is not necessarily the model. The Scaffold Effect study runs Qwen 3.6 Plus and MiniMax M2.5 through Goose, OpenCode, and OpenHands-SDK on 50 Terminal-Bench Pro tasks [2607.22585]. Paired pass-rate differences remain within 0–8 percentage points in most comparisons, but tokens per solved task differ by approximately 40 times. Goose is substantially more token-efficient than OpenHands-SDK and OpenCode, while each harness exhibits a reproducible failure fingerprint:

- Goose: REASON-dominated voluntary stopping.
- OpenHands-SDK: VERIFY and MAX_TURNS failures.
- OpenCode: TIME and HANG failures, with many idle turns.

The study reports that no-action turns are approximately ten times more frequent in OpenCode than Goose. This makes them simultaneously a token, latency, API-spend, waiting, and oversight cost. The result supports evaluating harness–model pairs by pass rate under token and latency budgets rather than by pass rate alone.

### No universally optimal discovery harness

Automated Discovery Has No Universally Superior Harness decomposes OpenEvolve-style and TTT-Discover-style search across 12 model–problem pairs, 30 configurations, repeated trials, and more than 3.1 million LLM rollouts [2607.18235]. The study finds no fixed harness that is reliably superior across evaluated pairs. Larger archives, MAP-Elites inspiration, multiple islands, exploration constants, breadth–depth allocations, and multiple-parent expansion all show model- and problem-dependent effects.

An adaptive allocation strategy that begins with multiple harnesses, prunes weak partial runs, and reallocates computation to stronger survivors achieves 85.75%, compared with 84.35% for five Sequential Best-of-$N$ runs and 84.54% for an unpruned five-harness portfolio. The strongest schedule is $12\to5\to2\to1$. This supports online harness selection guided by early performance rather than commitment to a fixed recipe.

## 5. Theoretical principles and failure modes

The literature identifies several recurring principles.

### Match structure to model capability

Workflow granularity should match the model’s controllable progress scale. Inference-time trajectory-alignment theory defines a recoverable prefix as one from which some continuation can still reach the correct result. A harness is beneficial when it increases the probability of remaining in this recoverable region; it is harmful when it rewards locally compliant but globally unrecoverable trajectories [2605.21516].

The same theory distinguishes guidance strength from guidance quality. If $\Gamma$ is the retention gap between recoverable and non-recoverable trajectories, positive $\Gamma$ improves reliability, while negative $\Gamma$ amplifies harmful behavior as guidance strength increases. This explains over-pruning and hallucinated execution: stronger instructions can increase unsupported confidence, verbosity, or formal compliance without increasing evidence-grounded success.

### Prefer enforceable mechanisms over prose

If a behavior must always hold, it should be implemented as an executable invariant rather than entrusted to a prompt. Prompt instructions can explain desired behavior; skills can encode reusable workflows; middleware can detect cross-step patterns; tools can enforce state and execution constraints; and verifiers can independently establish success [2604.25850].

The distinction appears in multiple failure cases. A coding agent that creates a correct file and deletes it during cleanup may not be corrected reliably by a prompt rule, but a shell-level guard can prevent deletion. A model that repeatedly invokes an invalid tool schema requires interface validation or schema normalization rather than additional descriptive text. A model that validates a proxy rather than the official evaluator requires middleware or verification gates that expose the relevant contract.

### Separate components to manage interference

Harness components interact non-additively. In Agentic Harness Engineering, memory, tools, and middleware each improve performance individually, while the full combination improves less than the sum of their gains because redundant verification and overlapping closure rules consume turns [2604.25850]. HarnessCompass addresses this through structural and guidance tracks followed by R$^3$ consolidation. HarnessX uses typed processor substitution and variant isolation. HarnessForge attaches distinct policy adapters to distinct harness lineages.

Variant isolation is particularly important under heterogeneous task distributions. A global harness can be forced into incompatible compromises, whereas multiple variants can preserve specialized strategies and route tasks according to estimated success. HarnessX reports that ensemble variants improve GAIA performance relative to a global single-harness strategy while reducing token consumption [2606.14249].

### Optimize the deployment objective

Pass rate alone can conceal major differences in token usage, latency, retries, no-action turns, verification burden, and human oversight. A deployment-oriented comparison should consider success under token and latency constraints, or report a Pareto frontier over performance, cost, latency, and safety.

A harness can improve accuracy while reducing context through selective retrieval and structured presentation. Conversely, an apparently persistent harness can spend enormous resources pursuing an unrecoverable trajectory. The appropriate objective is therefore task- and deployment-dependent.

### Preserve generalization and provenance

Harnesses can overfit through prompts, memory, retrieval corpora, evaluator quirks, or task-specific tool rules. HarnessCompass prohibits task-specific identifiers and requires conditional, transferable principles. HarnessOpt-Bench evaluates optimizers using a held-out test partition that remains inaccessible during search and meters target-agent resources in a trusted execution environment [2608.06301]. RHI reports a related risk: a harness optimized for current pairwise evaluator feedback may generate traces that are verbose, evaluator-specific, or pedagogically misleading for future model training [2607.15524].

Reproducibility requires recording harness versions, model identifiers, prompts, tool schemas, context policies, random seeds, runtime configurations, evaluator versions, traces, candidate histories, and failure classifications. Without these artifacts, model comparisons confound model and scaffold effects.

### Common failure modes

The principal failure modes recur across domains:

- **Over-decomposition**: subgoals are too fine for the model’s action scale, causing overshoot and boundary loss.
- **Over-pruning**: guidance removes actions needed for recoverable execution.
- **Hallucinated execution**: the harness rewards textual compliance unsupported by observations.
- **Schema fragility**: generated tools or calls are logically plausible but syntactically unusable.
- **Destructive cleanup**: the model reaches a correct state and then removes or resets it.
- **Verification failure**: the agent checks a proxy rather than the named evaluator.
- **Memory poisoning or staleness**: persistent context preserves incorrect or obsolete assumptions.
- **Component interference**: prompts, middleware, tools, and memory produce redundant or contradictory behavior.
- **Regression blindness**: an edit repairs observed failures while damaging previously successful tasks.
- **Capability-floor effects**: a weaker model cannot exploit a richer harness and may perform worse with additional structure.
- **Evaluator hacking**: an incomplete local evaluator rewards invalid solutions or exploitable formatting.
- **Search over-exploration**: archives, islands, and non-elite parents dilute computation when the model cannot produce useful alternatives.
- **No-action or idle looping**: the agent repeatedly consumes inference rounds without changing the environment.
- **Lack of rollback**: continuous self-modification accumulates harmful components without transactional recovery.

## 6. Research directions and assessment

The current evidence supports a broad systems principle but does not establish a universal co-design theory. Harness-only methods demonstrate that runtime structure can substantially improve fixed-model behavior. Joint methods demonstrate that model adaptation and harness adaptation are coupled, with compatibility effects that cannot be explained by independent component scores. Accelerator studies demonstrate that the same logic applies when the harness includes compilers, mappers, memory systems, schedulers, cost models, and generated hardware.

Several distinctions remain essential. HarnessForge and HarnessX perform explicit harness–policy co-evolution, but the model adaptation is lightweight or adapter-based rather than unrestricted retraining. HASE updates model weights and selected harness components through one unified agentic process, but evaluator repair relies on oracle feedback and remains bounded by whitelisted interfaces. Continual Harness performs reset-free harness adaptation and online model–harness co-learning in embodied tasks, but does not provide general rollback or convergence guarantees. HarnessCompass and Harness-R1 improve harness engineering around fixed target models, while Meta-Harness optimizes executable harnesses through an external proposer. HarnessOpt-Bench evaluates the capability of LLMs to perform this optimization rather than proposing a single universal algorithm [2608.06301].

The most significant open problem is a robust objective for the complete system. A future formulation may need to jointly account for model parameters, harness structure, task success, token and latency cost, verification confidence, safety, maintainability, and the value of generated trajectories for future training. These variables have different optimization geometries: neural parameters are continuous and gradient-updated, whereas harnesses are discrete programs with typed interfaces and non-differentiable execution effects.

A second problem is causal attribution. Trace-based evidence, self-declared predictions, before–after reruns, and compatibility matrices improve interpretability, but they do not establish a complete causal decomposition of model, harness, and environment effects. Stronger evaluations require factorial designs over models, harness levels, task families, tool interfaces, and training conditions; held-out models and tasks; counterfactual rollouts; and explicit measurements of component interactions.

A third problem is generalization. Automated Discovery Has No Universally Superior Harness shows that search policies are conditional on model, task, and budget. HarnessCompass shows that constrained evolution improves transfer, while Harness-R1 shows that editing policies can adapt across target models when supplied with new failure evidence. These results suggest that future systems should learn model- and task-conditioned harness selection rather than search for a single globally optimal scaffold.

A fourth problem is safe self-modification. Harnesses may change tools, evaluators, memory, permissions, prompts, and execution policies. Necessary safeguards include typed interfaces, sandboxing, immutable verifiers or oracle-gated changes, deterministic smoke tests, held-out validation, regression detection, provenance recording, rollback, entropy auditing, and human governance for security-sensitive modifications.

A fifth problem is trace quality. RHI argues that harnesses are data-generating components whose organization affects the quality of trajectories available for future model training [2607.15524]. This suggests a recursive objective in which a harness is optimized not only for immediate task performance but also for the informativeness, diversity, modularity, and downstream training utility of its traces. The proposed information-theoretic analysis—rewarding task information in contracts and hops while penalizing task-conditional redundancy—remains a hypothesis rather than a validated training objective.

The principal methodological conclusion is that model evaluation, harness evaluation, and system evaluation should be separated and then recombined. A fair report should specify the model–harness pair, the environment, tool schemas, context and memory policies, control flow, verification protocol, resource budgets, failure categories, and complete execution traces. For coding agents, tokens per solved task, no-action turns, latency, verification outcomes, and oversight burden should accompany pass rate. For discovery systems, repeated-trial null distributions, budget matching, early-progress analysis, and adaptive allocation should accompany best-of-run scores. For model–accelerator systems, accuracy, latency, throughput, energy, memory traffic, utilization, and hardware feasibility should be measured jointly.

Model-Harness Co-design therefore denotes a shift from model-centric optimization to system-centric optimization. The model is not an isolated source of intelligence, and the harness is not incidental middleware. Together with the environment, they define the observable state, action space, information flow, execution trajectory, verification process, and operational cost of an agent. The strongest systems treat harnesses as explicit computational artifacts: composable, typed, observable, versioned, evolvable, and evaluated as part of a matched model–harness pair.

Source: https://www.emergentmind.com/topics/model-harness-co-design