MicroSkill Architecture

• MicroSkill Architecture is a modular system for representing, retrieving, and updating procedural knowledge using externalized, versioned skill packages.
• It employs a reflective read–write loop with memory-based reinforcement learning to dynamically select and optimize skills based on real-time feedback.
• Empirical evaluations demonstrate significant accuracy gains on generalist agent benchmarks by continually evolving skills without modifying core LLM parameters.

The MicroSkill Architecture defines a modular, persistent, and continually-evolving system for representing, retrieving, and updating procedural knowledge within LLM-based agents. Rather than relying on monolithic prompts or static agent definitions, MicroSkills externalize behaviors as structured skill packages—typically realized as folders containing declarative markdown specifications, prompt templates, and helper code. At runtime, an agent's core logic is decomposed into a Read–Write reflective loop: a behavior-trainable skill router selects the optimal MicroSkill in response to the present state and query, executes the relevant skill in a frozen LLM context, and then updates or generates new skills based on feedback and experience. All adaptation is mediated by skill library evolution and externalized memory, with the underlying LLM parameters remaining immutable throughout agent lifetime. The resulting system can autonomously design, adapt, and optimize its agentic capabilities, as evidenced by substantial empirical gains on generalist agent benchmarks (Zhou et al., 19 Mar 2026).

1. Structural Representation of MicroSkills

Each MicroSkill is encapsulated as a distinct, versioned package within a central skill library (denoted $M_t$ at time $t$). The canonical structure for a single MicroSkill includes:

• Declarative Specification (SKILL.md): Provides human- and machine-readable metadata (e.g., name, description, version, params) outlining the skill's semantic contract.
• Prompt Template (prompt.txt): Encodes the stateful LLM prompt, specifying exactly how the underlying LLM should be cued for this skill.
• Helper Code (run.py or similar): Implements the actual actions or subroutines to be called by the agent, ensuring operational consistency and side-effect control.

The agent constructs execution-time prompts by concatenating user/system context, the identified skill, and the inlined prompt template. For instance, an elementary web search skill would define all operational logic in these three artifacts, facilitating both runtime selection and post-hoc introspection.

2. Memory-Based RL and Skill Routing

The skill execution pipeline is formulated as a one-step Markov Decision Process, where:

• State $q$: Comprises the current user query and the evolving "tip memory" ($T_t$), a buffer capturing failure traces and explanatory guidance from previous runs.
• Action $c$: Selection of a MicroSkill $c \in M_t$.
• Reward $r(q, c)$: Binary feedback, set to $1$ on correct skill invocation, $0$ otherwise.

Skill selection employs a behavior-trainable router, which jointly embeds both the user query and the available skill metadata using a learned encoder. The Q-function is implemented as $s(d, q) = e(d)^\top u(q)$, with $t$0 and $t$1 denoting the embedding of a skill and query, respectively. The routing policy applies a Boltzmann distribution over these Q-values, enabling stochastic sampling or argmax choice.

The router is trained using synthetic positive and hard-negative query-skill pairs via a multi-positive InfoNCE loss, effectively performing a one-step offline RL update per interaction. Theoretical guarantees ensure loop convergence to an optimal retrieval policy $t$2 in the limit (Zhou et al., 19 Mar 2026).

3. The Read–Write Reflective Learning Mechanism

MicroSkill evolution is orchestrated by an explicit Read–Write loop:

1. Observe: Process the current query and update context.
2. Read (Skill Selection): Route to the skill $t$3 best matching context and tip memory.
3. Execute: Run the LLM with the constructed prompt and invoke the associated external code.
4. Feedback: Collect reward signal from a "Judge" module.
5. Write (Reflective Update):
   • (a) Utility Update: Policy evaluation via empirical success rate.
   • (b) Tip Memory: Extend $t$4 with failure explanations when errors occur.
   • (c) Skill Library Update: Discover or optimize new skills when policies underperform.
   • (d) Retry: Iterate up to $t$5 retries on modified skills if correctness is not achieved.

All learning—policy evaluation and improvement—operates on external representations; no weights within the LLM are updated.

Pseudocode Sketch

$t$6 (Zhou et al., 19 Mar 2026)

4. Continual Learning Without LLM Parameter Modification

All adaptation in the MicroSkill Architecture is realized externally to the frozen LLM. When a skill fails under a new scenario or error mode, the agent either:

• Updates the prompt or code artifact in place via an LLM-based rewriter, or
• Synthesizes a new skill for the failure trajectory, gating inclusion by auto-generated unit tests.

This purely externalized evolution allows complete, continual adaptation of agent behaviors without any modification to core model parameters. Over time, the skill library grows to densely cover the agent’s operational envelope, supporting persistent and progressive capability accumulation.

5. Case Analyses and Emergent Skill Organization

Concrete trajectories illustrate the architecture’s iterative process:

• Web-Search Skill: Initially solves only direct fact queries; upon failure (e.g., ambiguity), the optimizer rewrites the prompt for disambiguation handling, raising future success rates.
• Terminal Operation Skill: Covers basic bash execution; adapts over failures (such as misstyped filtering) by inserting guards, and over multiple episodes spawns new sub-skills for specialized terminal tasks.

Emergent organization—observable in skill embedding spaces—clusters related skills (e.g., "Search/Web" and "Math/Chemistry") as the library evolves through repeated cycles on benchmarks like Humanity’s Last Exam.

6. Empirical Validation

Evaluation on challenging agentic agent benchmarks demonstrates significant relative improvements using the MicroSkill Architecture: 26.2% and 116.2% relative gains in overall accuracy on the General AI Assistants and Humanity's Last Exam benchmarks, respectively. These improvements are directly attributed to the evolving skill library, the read–write loop, and dynamic skill discovery and optimization capabilities. Code and system implementation are publicly available (Zhou et al., 19 Mar 2026).

7. Significance and Implications

The MicroSkill Architecture operationalizes continual learning, modularity, and efficient adaptation for LLM-based agents by externalizing behavioral knowledge in composable skill artifacts. Its read–write RL loop, externalized memory, and routable skill selection provide theoretical and empirical foundations for agent self-improvement without direct model retraining. Through explicit skill library growth, tip-memory-guided reflection, and compositional specialization, it enables autonomous construction and persistent evolution of agentic workflows across a broad range of environments and tasks (Zhou et al., 19 Mar 2026).