Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions

Abstract: Self-evolving frameworks usually optimize task solutions while treating the surrounding harness as fixed. We introduce Harness-Aware Self-Evolving (HASE), an agentic reinforcement-learning framework in which a single model can generate task solutions or edit selected harness components in a multi-turn action space. HASE enables a single Qwen3-8B model to match the text-classification performance of a GPT-OSS-120B model that uses Claude Code as the harness proposer. In alpha factor mining, HASE outperforms the reported GPT-OSS-120B baseline. HASE also repairs imperfect evaluation components and converges to state-of-the-art performance in circle-packing algorithm discovery. These results show that HASE improves the harness and the solution through one unified agentic process.

• The paper presents a novel method that co-evolves model weights, harness guidance, and evaluator repairs in a unified RL framework.
• It demonstrates improved performance in tasks like circle packing, Heilbronn triangles, text classification, and equity alpha mining through dynamic harness updates.
• The approach leverages phase-level review and mismatch auditing to prevent reward hacking, ensuring safe and effective evolution of agent capabilities.

Harness-Aware Self-Evolving: Co-Evolution of Model Weights, Harness, and Task Solutions

Introduction

The paper presents Harness-Aware Self-Evolving (HASE), an agentic RL framework capable of co-evolving the weights of a single model, object-level solutions, and selected harness components, including both guidance and evaluation modules. In contrast to typical pipeline or two-agent systems, HASE consolidates solution generation and harness modification into one unified action space, operationalized via an 8B parameter Qwen3 model. The framework’s central premise is that the harness—prompts, memory manipulation, retrieval policy, context construction, and particularly local evaluators—should not be treated as static, since their limitations can bound the agent’s effective competence and induce reward hacking. HASE introduces a phase-level review protocol that anchors harness edits to real-world feedback when evaluation is mutable and commits guidance evolution through proof-of-concept and validation-centric criteria.

Figure 1: HASE pipeline, contrasting with conventional pipelines by integrating harness and solution edits within a single model, eliminating the need for an external code proposer.

The HASE Methodology

Dual-Surface Harness Evolution

HASE identifies two categories of harness components: guidance (prompts, retrieval logic, memory/context construction) and evaluation (local scoring, validation logic). Guidance evolution is treated as an unconstrained agentic process—trajectories that leverage guidance edits to improve downstream solution rewards are candidates for phase-level review and eventual persistent harness updates. Evaluation edits, by contrast, are strictly audited: local evaluator rewrites are adopted only in response to explicit proxy-oracle reward divergences, and are validated against a growing mismatch set $\mathcal{M}$ containing previously detected harness failures. This ensures that reward hacking cannot drive unmoored evaluator redefinition.

Unified Action Space and Agentic RL Dynamics

HASE operationalizes a multi-turn action space where a single model can freely choose, per step, to propose an object-level solution, perform harness inspection (e.g., via sandboxed file read/write), or patch guidance/evaluator code:

• Guidance edits: Adopted when phase review shows task reward improvement, with proof-of-concept (PoC) credit to filter rolling harness mutations.
• Evaluation repairs: Triggered only by observed reward divergences; committed if and only if mismatch set coverage is increased without regression.
• Persistent harness updates: At the end of each phase, the most stable and performant candidate harness is merged, available to all subsequent rollouts.

Evaluation Harness Evolution: Reward Alignment and Repair

Circle Packing

The system’s efficacy in evaluator repair is demonstrated via the circle packing experiment, where the agent begins with a deliberately incomplete local evaluator. In early phases, the agent exploits missing boundary and overlap guards to generate candidates with high proxy score yet zero oracle score, registering concrete mismatches. HASE then patches the evaluator, first by reinstating basic geometric constraints, realigning the proxy with oracle feedback, and then subsequently by refining numerical tolerances only after new exploits emerge. After resolving both classes of deviation, the agent’s composite system optimizes to state-of-the-art performance ($2.635983$), matching specialized baselines such as AlphaEvolve.

Figure 2: Alignment dynamics and staged code evolution in circle packing, showing two exploit events, their detection, and subsequent evaluator repairs resulting in oracle-proxy synchronization and optimal solution scores.

Heilbronn Triangles

In the continuous optimization setting of Heilbronn triangles, the agent repeats the pattern: initial policy exploits missing geometric validation to achieve a high proxy but zero oracle reward. Upon detection, HASE edits the evaluator to include barycentric strictness, closing the reward gap. Critically, when policy weights are co-evolved alongside evaluator repairs, agent performance continues to rise post-mismatch-closure, whereas frozen-weight baselines stagnate at suboptimal levels. This empirically establishes that co-evolution—not evaluator repair alone—is essential for internalizing constraints and maximizing objective fitness.

Figure 3: Consequences of co-evolving policy and evaluator versus freezing weights after evaluator repair; only co-evolution enables exploitation closure and further learning.

Guidance Harness Evolution: Impact on Text and Alpha Domains

Online Text Classification

HASE is directly compared to Meta-Harness on the Symptom2Disease benchmark, with matching train/val/test splits and harness edit interface. Despite using Qwen3-8B in both solution and harness roles—with no test-time editing, retry, or retrieval over test examples—the HASE-deployed system achieves $86.98 \pm 0.38$ test accuracy, matching the GPT-OSS-120B + Claude Code (Meta-Harness) regime while outperforming few-shot and classic context engineering methods. Agent behaviors such as tool co-evolution and targeted harness rewrites are observed to emerge, yielding both better retrieval and disambiguation via agent-authored code.

Alpha Factor Mining

HASE’s guidance search protocol is deployed in large-scale equity alpha mining, using an 8B model for both alpha code generation and harness optimization. The harness evolves from recency-based retrieval toward context window-adaptive, information-coefficient (IC) prioritized retrieval, and explicit prompt engineering, all managed agentically. Under the full CogAlpha protocol—complete with strict leak-detection and external auditing—the HASE system achieves AER $14.35\%$ and IR $1.70$, exceeding GPT-OSS-120B ($7.89\%$, $0.80$) and all other baselines on real-world costed simulation metrics. The improvements are obtained entirely via harness-aware self-play with no human-authored code proposals, underscoring the harness surface’s role as a critical optimization axis under compute and context constraints.

Theoretical and Practical Implications

HASE unifies agentic optimization across solutions, harness guidance, and local evaluators without decomposing the agent into a powerful LLM proposer and a weaker solver, as in recent pipeline paradigms. This design unlocks solution strategies and harness rewrites that would otherwise require external intervention or more powerful code synthesis models. The proof that smaller models can bootstrap their harness to match or exceed the performance of baseline systems with much larger parameter count is a source of both theoretical and systems-level interest, pointing toward adaptive agentic systems that tightly couple environment feedback, code evolution, and RL weight adaptation.

The approach’s safety is managed by hard separation of guidance edits (unconstrained by oracle reward) and evaluator modifications (strictly mismatch-gated and interface-anchored), preventing ungrounded reward lofting while allowing iterative self-debugging reflective of real-world developer workflows. The methodology provides a reproducible blueprint for multi-modal self-evolution, suitable for code, text, and algorithmic discovery domains.

Limitations and Future Directions

While the paper demonstrates robust, general harness-aware self-evolution across four domains, the experiments stop short of a full scaling study or unconstrained environment. Editable surfaces are whitelisted and evaluation edits remain under phase-level audit, potentially limiting the spectrum of discovered modifications. The protocol’s safety-centric constraints are conservative, but adjusting selection mechanisms, expanding the modifiable surface, and incorporating more aggressive search (potentially with static/dynamic analysis) are straightforward next research steps. Identifying general principles for automated harness optimization in agentic settings is central for future application in high-stakes or unstructured domains.

Conclusion

HASE offers a comprehensive framework for integrating solution generation, guidance evolution, and evaluator repair within a single agent RL paradigm. The system attains strong empirical results, demonstrating that harness-aware self-evolution allows smaller models to rival the performance of systems previously reliant on much larger models and external code proposers. The separation between guidance and evaluator evolution, anchored by real-world feedback, provides both effectiveness and safety. HASE sets a precedent for future research in agentic RL systems and automated code/harness co-design, and defines a new baseline for unified agentic iteration in complex, open-ended problems.