SKILL0: LLM Skill Internalization

• The paper introduces SKILL0, a paradigm using curriculum-based reinforcement learning to internalize external skills, eliminating runtime injection and reducing context costs.
• The methodology employs dynamic curriculum staging, vision-encoded inputs, and a PPO-style composite loss to transition from full skill support to zero-shot operation.
• Empirical results demonstrate enhanced memory efficiency and superior zero-shot performance on embodied and search-augmented benchmarks compared to retrieval-based methods.

SKILL0 refers to a family of paradigms for training LLM agents to internalize external skills—structured, modular packages encoding procedural knowledge, tool usage, or high-level strategies—so that at inference time agents require little or no context injection of such skills and can operate in a “zero-skill” (purely autonomous) fashion. The SKILL0 framework is most associated with "SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization" (Lu et al., 2 Apr 2026), though the concept is both a target of recent RL curricula and the archetype against which hybrid and lifecycle skill paradigms are benchmarked. Unlike memory-augmented methods (SkillRL, SkillOS, SkillX), which dynamically retrieve and load skills at decision time, SKILL0 seeks to fully absorb the value of external skills into the agent’s parametric memory through curriculum-structured reinforcement learning, thus removing reliance on expensive and noisy runtime retrieval.

1. Motivation: From Inference-Time Skill Augmentation to Skill Internalization

Skill augmentation has become central to agentic LLM systems: at each decision step, the agent retrieves task-relevant skill files (procedures, tool patterns) from a “SkillBank” and appends them to its prompt. This retrieve-then-prompt pattern, driving systems such as ToolNet and SkillRL, improves reasoning and tool-use but has inherent limits (Lu et al., 2 Apr 2026):

• Retrieval noise: Imperfect retrieval introduces irrelevant or misleading context, degrading reasoning quality.
• Token overhead: Skills injected each step bloat the context window, slowing inference and risking context drop.
• Lack of true learning: The policy simply “follows” injected skills; the parametric weights never capture the strategies themselves.

Skill internalization is the process of training the agent so that, after curriculum withdrawal of skill context, it can execute formerly skill-dependent behaviors with no runtime skills present—operating in a strictly zero-shot “SKILL0” mode. This aims to deliver both memory efficiency (sub-kilobyte context costs) and genuine autonomous generalization (Lu et al., 2 Apr 2026).

2. Methodology: Dynamic Curriculum for Skill Internalization

SKILL0 introduces agentic reinforcement learning with a curriculum that transitions from full skill support to total withdrawal. Several related innovations underlie the methodology:

2.1 Agentic RL Setup and Vision Encoding

• Agent Loop: The agent operates in a Markov Decision Process (MDP): at each time $t$, history $h_t$ and current skill subset $\mathcal{S}$ are rendered as a color-coded RGB image, input to a vision encoder $\mathrm{Enc}(\cdot)$ forming $\mathcal{V}_t$.
• Policy: $\pi_\theta(a_t, c_t \mid I, \mathcal{V}_t)$, where $c_t\in(0,1]$ is a compression factor, enabling both action and skill-context compression to be learned (Lu et al., 2 Apr 2026).

2.2 Curriculum Staging

• Phased Budgeting: Training proceeds over $N_S$ curriculum stages, with a skill budget $M^{(s)}$:

$$M^{(s)} = \left\lceil N\,\frac{N_S-s}{N_S-1}\right\rceil$$

where $h_t$0 is the initial number of skill files. Early on, agents have access to all skills in context; at each stage, fewer skill files are available, enforcing progressive dependence on internalized knowledge.

• Dynamic Skill Filtering: Every $h_t$1 training steps, for each skill file $h_t$2 grouped by semantic category, SKILL0 measures

$h_t$3

on associated sub-tasks. Only the top $h_t$4 skills with $h_t$5 are retained. Skills the agent no longer benefits from are withdrawn, forcing policy internalization.

2.3 Composite RL Objective

A PPO-style loss is used with the following composite reward:

$h_t$6

where $h_t$7 is the usual task step-wise reward and $h_t$8 ($h_t$9 the context compression factor), penalizing unnecessary context usage and encouraging compact representations (Lu et al., 2 Apr 2026).

3. Offline SkillBank Construction and Organization

SKILL0 relies on a SkillBank $\mathcal{S}$0, typically partitioned by high-level skill categories (e.g., "pick_and_place.md", "multi_hop_reasoning.md"). For ALFWorld and Search-QA, this often involves $\mathcal{S}$1 skill files, each encoding either atomic actions or high-level strategies. Skill grouping is performed offline by analyzing trajectories for sub-task dependencies, with unique skill files allocated per major functional requirement (Lu et al., 2 Apr 2026).

Both the recent interaction history and the current subset $\mathcal{S}$2 are rendered as images, reducing the token context cost to $\mathcal{S}$3k tokens/step (ALFWorld: 0.38k; Search-QA: 0.18k, compared to $\mathcal{S}$42.2k for SkillRL) (Lu et al., 2 Apr 2026).

4. Zero-Shot Operation and Efficiency

At the end of the curriculum (stage $\mathcal{S}$5), the context skill set $\mathcal{S}$6: the agent receives only the environment and history, with no external skills provided. This enables:

• Memory footprint minimization: Compared to previous methods’ multi-kilobyte prompts, SKILL0’s internalized agents operate on compressed vision-encoded history alone.
• Elimination of retrieval noise: No skill files injected at inference, so there is no risk of guidance conflict.
• Autonomous, parameter-based execution: All critical skill knowledge is integrated into the policy weights.

5. Empirical Results, Ablations, and Limitations

SKILL0 demonstrates state-of-the-art (relative to SkillRL and memory-augmented baselines) performance on embodied (ALFWorld) and search-augmented (NaturalQuestions, HotpotQA, Bamboogle) benchmarks in a purely zero-shot regime (Lu et al., 2 Apr 2026):

• ALFWorld: 87.9% success rate (Qwen2.5-VL-3B), a +9.7% gain over AgentOCR.
• Search-QA: 40.8 F1 (Qwen2.5-VL-3B), a +6.6% gain over AgentOCR.
• Context cost: $\mathcal{S}$70.5k tokens/step (vs. 2.2k for SkillRL).

Ablations reveal:

• Smooth curriculum transitions (“rise-then-fall” in $\mathcal{S}$8) correspond to skill internalization in training curves.
• Skill grouping is essential: without grouped withdrawal, stagewise performance degrades.
• Visual context encoding is necessary for sub-kilobyte context operation.

Limitations:

• SKILL0 as implemented is evaluated in text-based and vision-encoded RL environments, with mid-scale models (3B/7B parameters); extension to large-scale, multi-modal settings remains an open area.
• All skill filtering is performed offline; joint, online skill construction and internalization is left for future work (Lu et al., 2 Apr 2026).

6. Relation to Hybrid and Lifecycle Skill Frameworks

SKILL0 represents one axis of the broader skill-lifecycle design space (Shen et al., 11 May 2026):

• SkillRL (Xia et al., 9 Feb 2026): Persistent, hierarchical skill augmentation with recursive co-evolution, balancing external skill retrieval and policy learning.
• SkillOS (Ouyang et al., 7 May 2026), SkillX (Wang et al., 6 Apr 2026): Autonomous, RL-driven skill curation and knowledge base construction, with modular, plug-and-play libraries.
• Skill0.5 (Zhu et al., 27 May 2026): Joint internalization (general skills) and context-based utilization (task-specific skills) with dynamic, difficulty-aware routing—bridging full-internalization (SKILL0) and persistence (SkillRL).
• SLIM (Shen et al., 11 May 2026): Lifecycle management with dynamic “retain–retire–expand” rules; strictly generalizes SKILL0 and SkillRL as end-points of a non-monotonic skill-lifecycle spectrum.

A plausible implication is that while SKILL0 achieves maximum memory efficiency and robust zero-shot transfer, hybrid approaches (Skill0.5, SLIM) may offer superior adaptability in settings with long-tail or compositional skill requirements.

7. Implementation and Reproducibility

The canonical SKILL0 codebase is available at https://github.com/ZJU-REAL/SkillZero. Models are trained with PPO, curriculum parameters ($\mathcal{S}$9, validation interval $\mathrm{Enc}(\cdot)$0), batch size 16×8 (ALFWorld) or 128 (Search-QA), and require $\mathrm{Enc}(\cdot)$112 hours (4×NVIDIA H800). SkillBank initialization uses the distilled set from SkillRL; prompt lengths are 3,072–4,096 tokens. The code includes scripts for reproducing all reported results, and full details on data preparation, hardware, and training hyperparameters appear in the published supplementary materials (Lu et al., 2 Apr 2026).