MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild

Abstract: LLM agents are increasingly used for complex tasks, yet deployed agents often remain static, failing to adapt as user needs evolve. This creates a tension between the need for continuous service and the necessity of updating capabilities to match shifting task distributions. On platforms like OpenClaw, which handle diverse workloads across 20+ channels, existing methods either store raw trajectories without distilling knowledge, maintain static skill libraries, or require disruptive downtime for retraining. We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a library of reusable behavioral skills. MetaClaw employs two complementary mechanisms. Skill-driven fast adaptation analyzes failure trajectories via an LLM evolver to synthesize new skills, enabling immediate improvement with zero downtime. Opportunistic policy optimization performs gradient-based updates via cloud LoRA fine-tuning and Reinforcement Learning with a Process Reward Model (RL-PRM). This is triggered during user-inactive windows by the Opportunistic Meta-Learning Scheduler (OMLS), which monitors system inactivity and calendar data. These mechanisms are mutually reinforcing: a refined policy generates better trajectories for skill synthesis, while richer skills provide higher-quality data for policy optimization. To prevent data contamination, a versioning mechanism separates support and query data. Built on a proxy-based architecture, MetaClaw scales to production-size LLMs without local GPUs. Experiments on MetaClaw-Bench and AutoResearchClaw show that skill-driven adaptation improves accuracy by up to 32% relative. The full pipeline advances Kimi-K2.5 accuracy from 21.4% to 40.6% and increases composite robustness by 18.3%. Code is available at https://github.com/aiming-lab/MetaClaw.

• The paper introduces a dual adaptation approach combining immediate skill-driven improvements with opportunistic RL-based optimization, significantly boosting task accuracy and robustness.
• It employs a reusable skill library to instantly correct failures and a strict support-query data partition to prevent gradient contamination during RL updates.
• Experimental results show notable performance gains, including up to 32% relative accuracy improvements and drastic reductions in refinement cycles on autonomous pipelines.

MetaClaw: Continual Meta-Learning and Adaptive Evolution for Deployed LLM Agents

Introduction and Motivation

LLM agents exhibit strong proficiency in complex, multi-stage reasoning tasks, yet their practical deployment is hindered by static training paradigms that do not accommodate the evolving nature of real-world user task distributions. Once deployed, these agents remain largely immutable, becoming increasingly misaligned with user expectations as operational contexts and requirements drift over time. MetaClaw proposes a unified, continual meta-learning framework that enables deployed agents to self-improve by extracting and reusing operational knowledge from both recent failures and ongoing user interactions, without incurring service downtime or data contamination—the two principal obstacles in applied agent adaptation systems.

Figure 1: The MetaClaw architecture showing dual adaptation loops: an immediate, skill-driven mechanism that enriches the behavioral instruction library on failure, and a slower, RL-based weight optimization triggered opportunistically when the user is inactive.

Architecture and Methodology

MetaClaw models the agent as a tuple $\mathcal{M} = (\theta, \mathcal{S})$, comprising the base LLM parameters $\theta$ and a dynamic skill library $\mathcal{S}$ of reusable, prompt-injected behavioral instructions. Adaptation proceeds along two axes:

Skill-Driven Fast Adaptation

Upon identifying failure trajectories, an LLM-based skill evolver distills new, task-agnostic skill instructions. These skills are injected immediately into the agent’s system prompt, yielding instant operational improvements without requiring any parameter updates. The skill library thus serves as both an accumulating behavioral memory and an adaptation substrate, selected at inference via embedding-based retrieval. MetaClaw enforces strict support-query trajectory separation, ensuring that only post-adaptation (query) data informs subsequent RL optimization, preventing gradient pollution from stale reward assignments.

Opportunistic Policy Optimization

Model parameter updates ($\theta$) are executed via RL (utilizing a process reward model) but only during user-inactive intervals determined by the Opportunistic Meta-Learning Scheduler (OMLS). OMLS monitors user sleep schedules, system idle times, and calendar occupancy to identify optimal RL update windows. Policy gradient updates employ only trajectories generated after skill adaptations, further enforced via skill generation versioning metadata attached to all interaction logs.

This bifurcated adaptation strategy ensures rapid, zero-downtime behavioral improvements through skill synthesis, while longer-term, data-intensive model enhancements are carried out in a manner that is unobtrusive from a user experience perspective and robust to validity concerns in the data stream.

Experimental Results

MetaClaw is evaluated on two principal platforms:

• MetaClaw-Bench: A continual agentic benchmark simulating 44 multi-day user sessions with 934 total CLI and conceptual tasks, emphasizing both structured file operations and procedural knowledge generalization.
• AutoResearchClaw: A 23-stage autonomous research pipeline demanding robust multi-domain reasoning, experimentation, and technical writing.

Key quantitative findings:

• For Kimi-K2.5, MetaClaw’s full pipeline elevates task accuracy from 21.4% to 40.6% and file-check completion from 2.0% to 16.5% (+8.25×), nearly matching the GPT-5.2 baseline (41.1%) (see Figure 2). Skill-driven fast adaptation alone accounts for up to +32% relative accuracy gains on weaker models, whereas GPT-5.2, starting with higher baseline capabilities, shows more modest relative improvements.
• In the AutoResearchClaw pipeline, skills-only adaptation achieves an 18.3% improvement in composite robustness, 24.8% reduction in retry rate, and 40% reduction in refinement cycles—all without requiring gradient-based updates.

Figure 2: Mean per-day task accuracy for MetaClaw variants and backbones. Skill-driven and full-pipeline MetaClaw considerably improve mid-phase performance where procedural knowledge accumulation is pivotal.

Analysis and Insights

The experiments substantiate several critical assertions:

• Skills as Immediate, Transferable Heuristics: The LLM evolver rapidly identifies and encodes high-utility behavioral patterns (e.g., automated file-backup procedures, ISO-8601 timestamping, naming conventions) as universal, reusable skills. These are injected contextually, allowing immediate correction of repeated failure modes.
• RL Optimization as a Complement, Not Replacement: Gradient-based updates, while lagging due to data availability and scheduling constraints, are essential for robust procedural execution—especially in long-horizon or interdependent CLI operations where skill instructions alone lack the authority to enforce zero-defect task completion.
• Validation of Support-Query Separation: The inclusion of versioning and strict buffer partitioning ensures that only valid, post-adaptation trajectories inform RL optimization, sidestepping the “stale reward” contamination that undermines naïve continual RL approaches.
• Generalization Across Domains: Skill-based mechanisms transfer beyond the original CLI task family, demonstrating robust performance improvements in a multi-stage, research-focused pipelined environment without any additional domain-specific engineering.

Implications and Future Directions

MetaClaw serves as a blueprint for the next generation of continually adaptive agents, integrating iterative behavioral distillation and weight-level optimization without downtime or reward contamination. This hybrid regime is particularly valuable for production scenarios constrained by hardware (no local GPU) or organizational policy (service uptime guarantees). Implicitly, it demonstrates that behavioral memory—in the form of dynamically versioned skill libraries—can substantially compensate for underlying model deficits and improve agent robustness, especially in non-stationary or under-specified task regimes.

Potential avenues for future work include:

• Extending opportunistic training to finer-grained, event-driven scheduling based on more nuanced user engagement analytics.
• Enhancing the expressivity of skill retrieval through context-awareness and hierarchical prompting.
• Investigating the theoretical limits of skill-library saturation and potential diminishing returns in infinite adaptation regimes.
• Exploring direct integration of skill evolution mechanisms into the agent’s architecture as differentiable memory modules or in-context learnable components.

Conclusion

MetaClaw demonstrates that continual meta-learning—leveraging asynchronous, skill-driven adaptation and opportunistic RL—enables LLM agents to evolve organically in the wild. By maintaining strict data versioning and buffer separation, the framework circumvents persistent technical obstacles (reward contamination, downtime, user experience disruption) of practical, interactive agent systems. These results position MetaClaw as a scalable, pragmatic solution for dynamic, production-grade LLM deployment, while providing a testbed for further advances in continual meta-learning and memory-augmented agent architectures (2603.17187).