Harness Continual Learning: Continual Adaptation Beyond Model Parameters

Abstract: Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.

• The paper introduces Harness Continual Learning, a framework that updates prompts, memories, capabilities, and routing around a frozen foundation model while using guarded proposal–evaluation–commitment to preserve prior behavior.
• Experiments across ALFWorld, Minecraft, reasoning, and multimodal streams show substantial gains, including 62.98% on ALFWorld, completion of all 50 Minecraft tasks, and up to 68.92% on multimodal tasks.
• The results show that evaluator tolerance controls the stability–plasticity trade-off, with intermediate retention budgets often outperforming unrestricted updates, although finite anchors cannot fully prevent unmeasured forgetting.

Motivation and problem formulation

Continual learning research has been overwhelmingly model-centric: the state that changes with sequential experience is the set of model parameters, representations, or architectural components. This paper argues that modern LLM agents admit a second locus of adaptation — the harness of prompts, memories, tool and skill registries, and routing rules that surrounds a foundation model. Because these artifacts persist across interactions and jointly shape execution, a harness update can disrupt previously reliable behavior even when the model is frozen. The authors formalize this as Harness Continual Learning (HCL): sequentially updating harness state $H_n$ around a frozen foundation model $F_\theta$ to acquire new behavior while retaining behavior that was reliable before each update, with the resulting regression termed harness-level forgetting.

The setting is distinct from conventional harness optimization (prompt search, workflow synthesis), which judges a single update against a current objective. HCL instead studies a sequence of deployed harnesses and requires that retention be an explicit condition of adaptation. Formally, at interaction step $n$ the harness transforms a raw interaction $\mathbf{u}_n$ into structured form $\mathbf{i}_n$, assembles an execution context $\mathbf{z}_n$ from selected memory and capabilities, executes it to produce $\mathbf{y}_n$, and receives feedback $\mathbf{f}_n$. A Continual Optimizer proposes a candidate harness $\widetilde{H}_{n+1}$ from this evidence; a binary commitment decision $G_n$ determines whether the deployed state advances or remains $F_\theta$0. A candidate therefore affects later interactions only upon commitment.

Harness state and guarded evolution

The mutable harness state is organized into four jointly versioned components:

• Task Interface ($F_\theta$1): prompts, task templates, and parsing/normalization rules that map raw inputs into a unified representation of input, objective, and constraints.
• Experience Memory ($F_\theta$2): Raw Memory storing concrete $F_\theta$3 triples for replay, plus LLM-summarized Abstract Memory entries carrying scoped guidance.
• Capability Map ($F_\theta$4): outer capabilities provided by the runtime (APIs, tools, environment actions) and inner skills abstracted from Abstract Memory, with explicit invocation conditions.
• Adaptive Router ($F_\theta$5): routing prompts, selection criteria, and workflow templates that retrieve experience, select capabilities, and assemble the execution context.

All proposed changes are treated as one complete candidate harness rather than independently edited artifacts, since component changes interact in execution.

The central methodological contribution is guarded harness evolution, which separates update generation from state commitment. The Continual Optimizer generates up to $F_\theta$6 alternatives per component in a predefined order, retaining the highest-scoring admissible alternative as the basis for subsequent revisions. The Continual Evaluator then applies three criteria under identical model, decoding, tool, environment, and seed conditions:

1. Current improvement: $F_\theta$7 on current validation cases.
2. Historical retention: the historical loss $F_\theta$8, counting anchors solved by $F_\theta$9 but failed by the candidate, must satisfy $n$0. Anchors are drawn at task boundaries from previously successful and failed cases and are unavailable during candidate generation.
3. Validity: syntax, output-schema compliance, legal tool use, and environment consistency checks.

Passing candidates are ranked by a composite score; if none passes, $n$1 remains deployed. The tolerance $n$2 directly parameterizes the stability–plasticity trade-off: $n$3 defines Stability-HCL, $n$4 defines Plasticity-HCL. Conceptually, HCL maps classical continual-learning families onto harness mechanisms — Experience Memory realizes replay, the Capability Map realizes reusable representations/architectures, and the Evaluator's gate plays the role of regularization-constrained updates — but unifies them within a single system-level objective.

Experimental results

Experiments span open-world interaction (ALFWorld, Minecraft) and controlled textual-reasoning and multimodal-perception streams, using different frozen models across settings (Qwen3.5-9B, Qwen3.6-27B, DeepSeek-V4-Flash, Qwen3.5-4B) to test generality.

On ALFWorld over six task categories, both HCL profiles exceed memory-based baselines (RAG, MemP, MemRL): Plasticity-HCL reaches 62.98% final average versus 55.56% for RAG, solving all Two-object episodes, while Stability-HCL attains 61.74% with substantially lower forgetting (2.64 vs. 10.94). Since the two profiles differ only in $n$5 under a frozen model, this comparison directly demonstrates evaluator-controlled trade-off adjustment. On Minecraft, HCL completes all 50 curriculum tasks where the Static Harness plateaus at 15, while using fewer cumulative environment actions (83) than MemRL (88) and MemP (91).

The controlled streams quantify harness-level forgetting directly. On the textual stream (MuSiQue → ProofWriter → GSM8K → HotpotQA), Plasticity-HCL improves the final average from 45.50% (zero-shot) to 64.70% with only 0.07 average forgetting, while Stability-HCL achieves zero measured forgetting at 52.20%. On the multimodal stream (COCO detection → captioning → RefCOCO grounding → VQAv2), Stability-HCL reaches 68.92% final average against 42.73% for DGG, a relative gain exceeding 60%, with detection improving from 4.27 to 65.34 and grounding from 43.00 to 91.60. VQAv2 is the sole task where zero-shot remains stronger, indicating that harness evolution adds little when the frozen model already handles direct image–question answering well.

A controlled sweep varying only $n$6 shows forgetting rising monotonically from 0.39 to 3.45, but final performance peaking at intermediate tolerance ($n$7: 63.46%) rather than at unrestricted plasticity (60.13%). The authors attribute this to unconstrained locally beneficial updates overwriting reusable harness contents, weakening both retention and the resources available for later tasks. Notably, residual forgetting at $n$8 persists because the retention constraint covers only a finite anchor set, so behavior on historical cases not represented by anchors is not guaranteed.

Component ablations on the multimodal stream show Full HCL at 63.41% final average, with disabling Interface or Memory updates causing the largest declines; removing Memory updates also raises forgetting to 0.83, suggesting evolving memory supports both acquisition and retention. Disabling Capability updates has minimal effect here, which the authors attribute to this stream's limited reliance on long-horizon executable skills. Several ablations exhibit lower forgetting than Full HCL simply because restricted editability limits adaptation — lower forgetting alone is not evidence of a better harness.

Limitations and open questions

The paper concedes several constraints. Retention evaluation requires rerunning anchor cases under every candidate, an overhead whose cost is not analyzed and which the authors list among unresolved challenges alongside harness-content consolidation and evaluation over longer interaction streams. Anchor-based retention provides only partial coverage: preserving all currently solved anchors does not guarantee unchanged behavior on unrepresented historical cases, as evidenced by nonzero forgetting at $n$9. The Minecraft evaluation retains skill-level rather than full task-level anchors and does not systematically replay completed tasks after every update, so task-level forgetting in that setting is not directly measured. Thresholds for minimum improvement, format compliance, and anchor composition are chosen heuristically, and the explanation offered for the non-monotonic performance–tolerance relationship remains a hypothesis rather than a demonstrated mechanism. Whether the four-component state decomposition is canonical, or how results scale beyond four-task streams and 50-task curricula, remain open.

Conclusion

This paper reframes continual learning around harness state rather than model parameters, identifies harness-level forgetting as a distinct retention problem arising from coupled execution components, and operationalizes acquisition-with-retention through a proposal–evaluation–commitment gate whose historical-loss tolerance explicitly controls the stability–plasticity trade-off. Across reasoning, perception, and open-world interaction, frozen-model agents accumulate capabilities and recover from failures with relative gains exceeding 10% over baselines in multiple settings, while exhibiting measurable, tunable forgetting. The framework establishes harness state as a legitimate object of continual learning study and leaves concrete questions about efficient retention evaluation and long-stream dynamics for subsequent work.