Collective Skill Tree Search (CSTS)

• The paper introduces CSTS, a framework that decomposes complex tasks into subtasks and uses multi-model collective generation to overcome skill fragmentation and limited transferability.
• CSTS is a tree-search-based method that constructs an ordered, reusable compositional skill path from diverse candidate skill nodes generated and assessed in parallel.
• By integrating skill-augmented supervision with collective skill reinforcement learning, CSTS significantly enhances long-horizon task performance and procedural guidance.

Collective Skill Tree Search (CSTS) is a tree-search-based skill construction framework for agentic LLMs in which a complex task is decomposed into ordered subtasks, each subtask is associated with a layer of candidate skills, and a path through the resulting tree defines a compositional skill path for solving the original task. In the formulation introduced with OpenClaw-Skill, CSTS is designed to address three specific deficiencies in prior skill-learning pipelines: skill fragmentation, limited skill diversity, and limited skill transferability. Its core mechanism is explicitly collective: multiple models generate candidate skill nodes for each subtask, and multiple models also participate in judging their quality and transferability across backbones. The selected skill paths are then used to construct skill-augmented supervised data and to support a reinforcement-learning stage that samples across multiple candidate skills rather than committing to a single procedural pattern (Lin et al., 15 Jun 2026).

1. Definition, scope, and formal structure

CSTS begins from an ordered task decomposition

$T \rightarrow (t_{1},t_{2},\ldots,t_{M}),$

where each $t_m$ is a subtask. This decomposition induces the depth of the skill tree: layer $m$ corresponds to subtask $t_m$. A node at layer $m$ is a candidate skill for $t_m$, and a complete path is an ordered composition of subtask-level skills rather than a single monolithic skill artifact (Lin et al., 15 Jun 2026).

For subtask $t_m$, each participating model $M_n$ produces a trajectory $\tau_{m,n}$, and a shared skill synthesizer converts that trajectory into a reusable skill node: $s_{m,n}= \Phi_{\mathrm{skill}}(t_{m},\tau_{m,n}).$ The candidate set for layer $t_m$0 is

$t_m$1

The selected task-level output is then

$t_m$2

which the framework treats as an ordered composition of local skills for the full task, not as one flattened skill description (Lin et al., 15 Jun 2026).

Within this formulation, a skill is a procedural abstraction distilled from an execution trace. The description attached to a skill includes applicable context, required inputs, recommended actions, expected outputs, verification criteria, and recovery strategies. Structure, diversity, and generalizability are built into the representation itself. Structure comes from the ordered layering by subtasks; diversity comes from generating multiple candidate nodes per layer from heterogeneous models; generalizability comes from explicitly preferring skills that remain useful when transferred to other models (Lin et al., 15 Jun 2026).

A common misconception is to read CSTS as a conventional action-level tree-search algorithm over immediate environment actions. In OpenClaw-Skill, the search objects are skills, not primitive actions, and the tree is aligned to task decomposition rather than to every intermediate action token. A second misconception is to interpret the “tree” as merely a flat skill bank with labels. The defining feature is precisely that a path through the tree specifies how local skills are selected and ordered across subtasks (Lin et al., 15 Jun 2026).

2. Collective Skill Node Generation

The first major phase is Collective Skill Node Generation (CSN-Gen). Let the participating model set be

$t_m$3

For a fixed subtask $t_m$4, each model $t_m$5 attempts the same subtask and produces its own trajectory. These trajectories are not ensembled directly. Instead, each trajectory is summarized independently into a candidate skill node, producing the layer-local candidate set $t_m$6 (Lin et al., 15 Jun 2026).

Operationally, CSN-Gen performs the following transformation. It selects the current subtask, broadcasts that same subtask to all models, runs all models in parallel, collects reasoning states, actions, observations, and final outcomes from each trajectory, and applies a shared skill synthesizer uniformly across models. The output is one candidate skill node per model trajectory. The design assumption is that different backbones expose different procedural habits, failure modes, and verification routines; collective generation broadens the local search space by preserving those variations instead of collapsing them prematurely (Lin et al., 15 Jun 2026).

The “collective” attribute is therefore literal rather than metaphorical. Diversity is not produced by temperature sweeps over a single model but by cross-model procedural heterogeneity. This distinguishes CSTS from single-model skill distillation pipelines, and it also distinguishes CSTS from methods that rewrite one incumbent skill iteratively without maintaining a layer of parallel alternatives (Lin et al., 15 Jun 2026).

At the same time, the OpenClaw-Skill formulation is more constrained than classical online tree policies such as branch-and-bound or backtracking MCTS. The described CSTS procedure is primarily a layerwise generate-assess-select pipeline over subtasks. The paper does not specify Monte Carlo rollouts, UCB/PUCT-style online branch selection, or revisiting earlier layers after later-layer feedback. This suggests that its “tree search” is best understood as structured search over a tree-shaped candidate skill space rather than as a full online search policy over partial paths (Lin et al., 15 Jun 2026).

3. Collective Skill Node Assessment

Collective Skill Node Assessment (CSN-Assess) assigns each candidate skill node two scores: a collective quality score and a collective transferability score. The quality score averages independent judgments from multiple judge models: $t_m$7 where $t_m$8 is the score assigned by judge $t_m$9 to skill $m$0. The paper characterizes these judgments in terms of whether the skill is clear, executable, complete, and relevant to the target subtask (Lin et al., 15 Jun 2026).

Transferability is evaluated more directly. If $m$1 was distilled from model $m$2, the framework shares that skill with every other model $m$3, $m$4, and asks those models to solve the same subtask while conditioned on that skill: $m$5 Let $m$6 denote the verification score of those transferred rollouts. Then the transferability score is

$m$7

This mechanism is distinctive because it does not assume that a well-written skill is automatically portable; it measures whether other models can in fact use it effectively (Lin et al., 15 Jun 2026).

The final node score is

$m$8

and the selected skill for subtask $m$9 is

$t_m$0

This scoring rule gives equal formal status to intrinsic textual quality and empirically verified cross-model usefulness (Lin et al., 15 Jun 2026).

The assessment stage also clarifies what CSTS is not. It is not merely majority voting over skill descriptions, and it is not solely a reward-model ranking step. The quality term aggregates evaluator judgments, whereas the transferability term is rollout-based. A plausible implication is that CSTS prefers skills that are both procedurally legible and operationally reusable, even if one of those properties alone would have ranked a candidate highly (Lin et al., 15 Jun 2026).

This assessment machinery is computationally expensive. It requires multi-model rollout generation, multi-judge quality evaluation, and transferability rollouts from all non-source models for each candidate skill. The paper’s efficiency note is limited to parallel generation and shared synthesis; it does not provide a separate complexity analysis (Lin et al., 15 Jun 2026).

4. Skill-augmented supervision and Collective Skill Reinforcement Learning

Once a task-level path $t_m$1 has been selected, CSTS turns it into supervised training data. For each task $t_m$2, it constructs an instance

$t_m$3

where $t_m$4 is the selected compositional skill path and $t_m$5 is the demonstration trajectory assembled from the rollouts that produced the selected skills. The supervised objective is

$t_m$6

The conditioning variable is therefore richer than in ordinary imitation learning: the model is trained on task plus explicit skill path, not on task alone (Lin et al., 15 Jun 2026).

OpenClaw-Skill then adds Collective Skill Reinforcement Learning (CSRL). The stated motivation is that supervised fine-tuning on selected skill paths does not explicitly teach the model how to compare multiple relevant skills for the same subtask. For a given subtask $t_m$7 and candidate skill set $t_m$8, the old policy samples $t_m$9 rollouts for each skill: $m$0 All such rollouts form a collective skill-conditioned group

$m$1

This construction is central: the RL signal is defined over competition among skills within the same subtask layer, not only over trajectories generated under one chosen skill (Lin et al., 15 Jun 2026).

Each rollout receives reward

$m$2

and advantages are normalized across the full collective group: $m$3 with $m$4 and $m$5 computed over all rollouts in $m$6. This means a rollout is judged relative to trajectories produced under competing skills for the same subtask. The paper interprets this as broadening exploration, avoiding entrapment by a single skill, and reducing homogeneous or suboptimal solution patterns (Lin et al., 15 Jun 2026).

A useful distinction emerges here. CSTS constructs the skill space; CSRL teaches the policy how to exploit that space under competitive, cross-skill relative feedback. A plausible implication is that CSTS without CSRL yields structured procedural supervision, whereas CSTS with CSRL yields both procedural supervision and an exploration scaffold over alternative skills (Lin et al., 15 Jun 2026).

5. Empirical performance and ablation evidence

OpenClaw-Skill evaluates CSTS on QwenClawBench and PinchBench using four backbones: Qwen3-4B, Qwen3-8B, Qwen3.5-4B, and Qwen3.5-9B. The implementation section states that CSTS collects multi-agent rollouts, synthesizes skills, constructs skill-augmented trajectories, and then produces 2K high-quality SFT examples. Fine-tuning is performed for 2 epochs on 8 H100 GPUs with learning rate $m$7 (Lin et al., 15 Jun 2026).

On QwenClawBench, the reported overall scores are as follows.

Backbone	Base	OpenClaw-Skill	Gain
Qwen3-4B	7.0	12.8	+5.8
Qwen3-8B	11.5	15.8	+4.3
Qwen3.5-4B	31.5	41.2	+9.7
Qwen3.5-9B	34.5	44.9	+10.4

The strongest category-level gains are concentrated in long-horizon, tool-use-heavy settings. For OpenClaw-Skill 9B, SVM rises from 33.2 to 70.9 and CS rises from 30.2 to 78.4. For OpenClaw-Skill 4B, RIR rises from 24.4 to 54.1. The paper interprets these improvements as evidence that CSTS-generated skills and CSRL improve procedural guidance following, intermediate verification, and error recovery (Lin et al., 15 Jun 2026).

On PinchBench, the Qwen3.5-9B to OpenClaw-Skill 9B transition improves the 23-task original version from 67.5 to 72.8 best success rate and from 53.8 to 58.9 average success rate. On the 123-task expanded version, it improves from 61.1 to 68.2 best score and from 47.1 to 53.6 average score. Qwen3.5-4B to OpenClaw-Skill 4B improves on the 123-task setting from 60.9 to 61.4 best and from 45.9 to 47.6 average. Smaller backbones also improve: Qwen3-4B rises from 13.6 to 20.8 average, and Qwen3-8B rises from 18.3 to 22.5 average on the 123-task setting (Lin et al., 15 Jun 2026).

The ablation on Qwen3.5-9B over QwenClawBench isolates the three major stages:

• base model: 34.5
• $m$8 CSN-Gen: 39.8
• $m$9 CSN-Gen $t_m$0 CSN-Assess: 42.8
• full OpenClaw-Skill: 44.9

These numbers support a layered interpretation. Collective generation provides the largest first-stage gain, assessment adds a second gain by filtering with quality and transferability, and CSRL adds a further gain by learning across skill-conditioned rollout groups (Lin et al., 15 Jun 2026).

6. Position in the literature, misconceptions, and limitations

CSTS is most directly associated with OpenClaw-Skill, but it sits within a broader family of structured search methods over skills, reasoning traces, and agentic procedures.

System	Search object	Relation to CSTS
OpenClaw-Skill	Subtask-layered skill nodes	Canonical CSTS formulation
MASA	Task-specific skill rewrites	Hierarchical, model-aware, not unified collective tree
SkillFuzz	Skill compositions	Safety-oriented search over collective skill sets
SkillDAG	Typed skill graph	Graph alternative to strict tree structure
CoMCTS	Reasoning nodes	Collective tree search over reasoning paths

MASA, or Model-Aware Skill Alignment, turns skill adaptation into a hierarchical search problem over general and task-specific skills. Its task-specific stage uses UCB-driven tree search over skill rewrites, but the trees are independent per task type and per model rather than constituting a unified collective tree (Yu et al., 29 May 2026). SkillFuzz formulates implicit-intent discovery in open skill marketplaces as a fuzzing problem over skill compositions and uses contract-guided Monte Carlo Tree Search to prioritize risky combinations, making it a safety-oriented search over collective skill states rather than a framework for constructing transferable task-solving skills (Hu et al., 2 Jul 2026). SkillDAG argues that large-scale skill selection is better represented as a typed directed graph with depends_on, specializes, composes_with, similar_to, and conflicts_with relations, suggesting that graph-structured retrieval may be preferable to a strict tree when library scale and cross-cutting dependencies dominate (Bai et al., 2 Jun 2026). CoMCTS, introduced for multimodal reasoning, constructs a shared reasoning tree from multiple models and is therefore close in collective search spirit, but its nodes are reasoning states rather than reusable skill procedures (Yao et al., 2024).

These comparisons help clarify several misconceptions. CSTS is not synonymous with any Monte Carlo tree search over prompts or reasoning steps. In its defining formulation, the search objects are reusable procedural skills aligned to subtasks. Nor is “collective” reducible to selecting multiple skills at inference time. In CSTS, collectivity first enters through multi-model generation and assessment, and only later, in CSRL, through multi-skill rollout groups (Lin et al., 15 Jun 2026).

The limitations stated or implied by the OpenClaw-Skill formulation are substantial. CSTS is computationally expensive because it requires multiple models for candidate generation, multiple judges for quality scoring, and cross-model rollouts for transferability verification. Its quality signal may still be noisy if judges disagree or share common biases. It assumes that tasks can be decomposed into an ordered subtask sequence; brittle or incorrect decomposition can therefore produce poor skill trees. The concrete search procedure is also less flexible than the term “tree search” might suggest: the paper does not describe explicit backtracking, revising earlier layers after later evidence, or global optimization over partial paths (Lin et al., 15 Jun 2026).

Within those bounds, CSTS can be defined precisely. It is a structured, collective, subtask-layered procedure for building a tree of reusable skills, selecting one transferable skill per subtask, composing those selections into skill paths, and then turning the resulting procedural space into both supervised data and reinforcement-learning structure. Its distinctiveness lies less in any one search primitive than in the combination of tree-structured skill representation, multi-model node generation, multi-model assessment, and downstream learning over competing skill-conditioned trajectories (Lin et al., 15 Jun 2026).