MetaClaw: Continual Meta-Learning for LLM Agents

• MetaClaw is a continual meta-learning framework that integrates gradient-free skill evolution with gradient-based policy optimization for deployed LLM agents.
• It employs a proxy-based architecture that strictly separates support and query data to prevent stale-reward contamination during continuous learning.
• The system achieves rapid on-the-fly skill updates and opportunistic RL fine-tuning, significantly enhancing accuracy and robustness in live deployments.

MetaClaw is a continual meta-learning framework designed for deployed LLM agents that require persistent adaptability without service interruption. Developed primarily for environments such as OpenClaw, where agent workloads span over 20 heterogeneous channels, MetaClaw unifies two complementary adaptation mechanisms—fast, gradient-free skill distillation, and slower, gradient-based policy optimization—within a production-scale, proxy-based architecture. This system facilitates joint evolution of a base LLM policy and its library of reusable behavioral skills, ensuring agents remain responsive to evolving user task distributions with zero service downtime (Xia et al., 17 Mar 2026).

1. Architectural Overview

MetaClaw maintains a meta-model $M = (\theta, S)$, where $\theta$ represents the LLM policy weights (e.g., GPT-5.2, Kimi-K2.5) and $S = \{s_1, \ldots, s_K\}$ is a dynamic library of “skills”—natural-language behavioral instructions. Task inference proceeds by retrieving relevant skills via $\text{Retrieve}(S, \tau)$ for the current task $\tau$, followed by sampling actions $$a \sim \pi_\theta(\cdot | \tau,\ \text{Retrieve}(S, \tau))$$.

MetaClaw’s proxy-based infrastructure intercepts all agent–tool–user exchanges: it logs trajectories, dispatches failures to the skill evolver, and buffers successful executions for reinforcement learning (RL)-based updates. A key architectural principle is trajectory versioning. Every trajectory is tagged with an index $g$ that encodes the generation of skills $S_g$ present at execution. This supports a strict split between:

• Support data $D^{\text{sup}}_{(g)}$: Pre-adaptation failures under $S_g$, consumed exclusively during skill synthesis and then discarded.
• Query data $\theta$0: Post-adaptation (i.e., after skill evolution) trajectories, eligible for reward estimation and gradient-based policy updates.

Whenever the skill library transitions from $\theta$1 to $\theta$2, all RL buffer data with indices $\theta$3 are flushed, preventing contamination from stale reward signals.

2. Skill-Driven Fast Adaptation

MetaClaw leverages a skill evolver $\theta$4—an LLM-driven prompting pipeline—to synthesize new skills from failure trajectories. Upon accumulation of a threshold minibatch of $\theta$5, $\theta$6 produces 1–$\theta$7 new skills, each specified in JSON schema:

$S = \{s_1, \ldots, s_K\}$6

Examples include skills such as "backup-before-modify" and "ISO8601-timezone-format." Skills reside in the discrete, natural language prompt space; thus, their injection is gradient-free, requires no model weight adjustment, and introduces no downtime—the new skills are instantly available.

Pseudo-code for Skill Evolution (excerpt):

$S = \{s_1, \ldots, s_K\}$7

This mechanism enables rapid plasticity directly at the prompt level, allowing MetaClaw to adapt on-the-fly to emergent or unmet behavioral requirements.

3. Opportunistic Policy Optimization

Longer-term policy improvements are achieved via gradient-based fine-tuning using RL and LoRA-adapter techniques on cloud endpoints. The system employs an RL buffer $\theta$8 storing tuples $\theta$9, where $S = \{s_1, \ldots, s_K\}$0 is scored by a process reward model (PRM). Policy optimization updates are scheduled by the Opportunistic Meta-Learning Scheduler (OMLS), which detects idle windows (continuous inactivity above a threshold, sleep hours, or conflicts with calendar meetings).

The policy meta-update is:

$S = \{s_1, \ldots, s_K\}$1

Updates are implemented with Gradient-based Reinforcement Policy Optimization (GRPO) over LoRA adapters. OMLS ensures that hot-swapping of model weights and remote fine-tuning only trigger during non-disruptive user-inactive periods, preventing service lapses. Batch thresholds are also enforced to mitigate gradient variance, formally training only if both $S = \{s_1, \ldots, s_K\}$2 and $S = \{s_1, \ldots, s_K\}$3 are satisfied.

4. Mutual Reinforcement of Adaptation Loops

The two adaptation mechanisms—fast skill evolution and policy optimization—operate in mutually reinforcing cycles:

• As $S = \{s_1, \ldots, s_K\}$4 improves, low-level errors decrease; subsequently, more challenging, higher-level failure cases emerge, informing skill synthesis with richer data.
• As $S = \{s_1, \ldots, s_K\}$5 expands with high-value skills, average process reward increases and RL buffer variance decreases, resulting in more stable and efficient policy training.

Empirical learning curves exhibit an initial phase dominated by prompt-level skill-driven gains (notable within the first 8 days), followed by a secondary phase of RL-facilitated improvements, consistent with the hypothesized complementary timescales of the two learning loops.

5. Empirical Evaluation

MetaClaw’s empirical performance has been assessed on two testbeds: MetaClaw-Bench and AutoResearchClaw.

MetaClaw-Bench features 934 CLI questions over 44 simulated workdays, divided into two parts:

• Part I: 346 questions (30 days), focusing on end-to-end file-check and multi-choice tasks with persistent state.
• Part II: 588 questions (14 days), emphasizing high-frequency procedural compliance.

Metrics: mean per-question accuracy and file-check completion rate.

AutoResearchClaw tests MetaClaw within a 23-stage autonomous research pipeline, measuring stage retry rate, refine cycle count, pipeline completion, and a composite robustness score.

Key Results

Model & Condition	Part I: Acc. (%)	Part I: Compl. (%)	Part II: Acc. (%)	Part II: Compl. (%)
GPT-5.2 Baseline	41.1	14.7	44.9	58.4
GPT-5.2 + Skills	44.0 (+7.1%)	17.1	49.1 (+9.4%)	67.5
Kimi-K2.5 Baseline	21.4	2.0	21.1	18.2
Kimi-K2.5 + Skills	28.3 (+32.2%)	2.0	26.9 (+27.5%)	33.8
Kimi-K2.5 + Full Pipeline	40.6	16.5	39.6	51.9

Metric	Baseline	+Skills	Δ (%)
Stage retry rate	10.5%	7.9%	–24.8%
Refine cycles per stage	2.0	1.2	–40.0%
Pipeline completion	18/19	19/19	+5.3%
Composite robustness	0.714	0.845	+18.3%

Skill-driven adaptation alone yields substantial absolute and relative gains. For instance, Kimi-K2.5 accuracy increased from 21.4% to 28.3% (+32.2%) in Part I, and full pipeline adaptation raised accuracy to 40.6%, nearly matching state-of-the-art model levels. Similar trends appear for composite robustness in AutoResearchClaw.

6. Distinctive Contributions and Comparisons

MetaClaw distinguishes itself from prior approaches by integrating both skill distillation and weight optimization within a unified, continual meta-learning framework. Notable contributions include:

• Strict separation of support and query data via trajectory versioning, fully mitigating stale-reward contamination.
• Opportunistic scheduling of RL fine-tuning, aligning optimization exclusively with periods of inactivity to preserve continuous service.
• Complete reliance on a proxy front-end and cloud-based LoRA adapters, eliminating the need for local GPU resources and seamlessly scaling to production-class LLMs.
• Workflow that leverages mutual reinforcement between prompt-level skill learning and deep RL-based representation adaptation.

MetaClaw demonstrates that agents can persistently “learn to learn” from live deployment, enhancing both heuristic (prompt-level) and representational (policy weight) performance over time (Xia et al., 17 Mar 2026).