Corpus2Skill: Hierarchical RAG Framework

• Corpus2Skill is a compile-then-navigate framework that organizes raw documents into a hierarchical skill directory for precise, multi-hop inference.
• It compiles corpora offline using iterative clustering and LLM summarization, creating a filesystem of SKILL.md and INDEX.md files for progressive retrieval.
• Empirical evaluations on WixQA show significant improvements in retrieval quality metrics, underlining its effectiveness for enterprise QA tasks.

Searching arXiv for the cited Corpus2Skill-related papers to ground the article in current sources. Corpus2Skill is a compile-then-navigate framework for retrieval-augmented generation (RAG) that transforms a raw document corpus into a hierarchical skill directory which a LLM agent actively explores at query time, rather than treating retrieval as a single opaque top-$k$ lookup step (Sun et al., 16 Apr 2026). In the narrower sense, the term refers to the enterprise QA architecture introduced in "Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG" (Sun et al., 16 Apr 2026). In a broader sense, related contemporaneous work explicitly recasts trajectory distillation, test-time skill synthesis, and heterogeneous knowledge compilation as instances of a general "Corpus2Skill" problem: transforming corpora of documents, trajectories, manuals, logs, or mixed records into reusable procedural guidance for agents (Ni et al., 26 Mar 2026, Wang et al., 16 May 2026, Pan et al., 8 Jun 2026).

1. Definition and conceptual basis

Corpus2Skill replaces runtime dependence on a vector index with offline compilation of a corpus into a visible hierarchy of topic clusters and summaries (Sun et al., 16 Apr 2026). The resulting artifact is a filesystem of SKILL.md and INDEX.md files that exposes corpus organization to the agent. At serve time, the agent begins from one-line skill descriptions, drills into progressively finer summaries, and retrieves full documents by identifier only when needed. The framework is therefore two-phase: an offline compile phase and an online serve phase.

The central claim of the framework is that flat retrieval gives the model no visibility into the overall corpus structure: it cannot see what topics remain unexplored, cannot backtrack systematically, and cannot combine evidence across separate branches (Sun et al., 16 Apr 2026). By contrast, a hierarchical directory affords progressive disclosure, explicit backtracking, and cross-branch synthesis. The navigation cost is characterized as logarithmic: with branching factor $p$ and $N$ documents, the depth is $L=\lceil \log_p N\rceil$, and only $O(p\log_p N)$ summaries are inspected instead of $O(N)$ (Sun et al., 16 Apr 2026).

This formulation treats the hierarchy itself as procedural scaffolding. The agent is not only given evidence; it is given a navigable map of where evidence is likely to reside. A plausible implication is that Corpus2Skill shifts part of the burden of inference from late-stage answer synthesis to earlier-stage search planning, which distinguishes it from conventional dense retrieval pipelines.

2. Offline compilation into a skill hierarchy

The offline pipeline begins with a corpus $\mathcal{D}=\{d_1,\dots,d_N\}$ and builds a hierarchical skill forest in four stages (Sun et al., 16 Apr 2026). First, raw documents such as .md, .txt, .json, and .jsonl are read, assigned deterministic IDs by content hash, truncated to a maximum length, and embedded with a sentence-embedding model such as Qwen3-Embedding-0.6B to produce vectors $v_i\in\mathbb{R}^d$.

Second, the system performs iterative hierarchical clustering. At level $\ell$, the current node embeddings $\{x_1,\dots,x_n\}$ are normalized to unit norm, partitioned into $p$0 clusters with K-Means using Euclidean distance $p$1, and summarized cluster by cluster (Sun et al., 16 Apr 2026). Because the vectors are unit-norm, this Euclidean objective is equivalent to cosine similarity for ranking purposes. For each cluster $p$2, the compiler selects up to $p$3 representative texts, uses an LLM summarization prompt emphasizing topic area, question types, and key terms, embeds the resulting summary $p$4 into a new vector $p$5, and repeats until the number of top clusters is at most $p$6. The hierarchy depth is given as

$p$7

Third, each non-leaf node is labeled by an LLM prompt that outputs a filesystem-safe directory name of 2–5 words, such as wix-payments-ecosystem (Sun et al., 16 Apr 2026). Fourth, the hierarchy is materialized on disk as a forest of $p$8 skills. Root clusters become skill directories with SKILL.md; sub-clusters become subdirectories with INDEX.md; and leaf documents are stored in a global documents.json referenced by document ID. The markdown files use YAML frontmatter containing fields such as name, description, level, and num_documents, which supports the progressive disclosure mechanism (Sun et al., 16 Apr 2026).

The compile phase is therefore not merely summarization. It is a corpus-to-filesystem compilation process whose output is directly consumable by an agentic runtime. This distinguishes Corpus2Skill from systems that summarize clusters only to reinsert them into a conventional retriever.

3. Serve-time navigation and decision process

At serve time, the agent has two tools: view(path) to read SKILL.md or INDEX.md, and get_document(doc_id) to load a full document (Sun et al., 16 Apr 2026). Initially, only skill names and one-line descriptions are preloaded, approximately 200 tokens in total. Content is loaded only through explicit view calls.

A typical query proceeds in stages. The agent first takes a bird’s-eye view of the top-level skill descriptions and selects one or two relevant skills. It then views the corresponding SKILL.md, inspects the $p$9 subgroup summaries, drills down to INDEX.md, selects one to three document IDs at the leaves, retrieves the full documents, and finally synthesizes a grounded answer (Sun et al., 16 Apr 2026). The agent can return to higher levels if a branch proves unproductive, and it can visit multiple branches if evidence must be combined across topic areas.

The paper gives an idealized decision rule for selecting the next child node:

$N$0

where relevance may be instantiated by embedding similarity,

$N$1

or approximated by the LLM’s own reasoning (Sun et al., 16 Apr 2026). The important point is not the exact scoring function but the visibility of alternatives: the agent knows which subgroups remain unexplored and can backtrack explicitly.

A common misconception is that Corpus2Skill eliminates retrieval entirely. The more precise characterization is that it eliminates vector or graph database retrieval at runtime while preserving explicit document retrieval by identifier and relying on embeddings during offline compilation (Sun et al., 16 Apr 2026). It is therefore a different serve-time retrieval regime, not an absence of retrieval.

4. Empirical evaluation on WixQA

The principal evaluation of Corpus2Skill is on WixQA, an enterprise customer-support benchmark for RAG (Sun et al., 16 Apr 2026). The corpus contains 6,221 Wix support articles, and the query set contains 200 expert-written customer-support questions with gold answers and gold document IDs. With $N$2 and $N$3, the compilation produces a 3-level hierarchy with 6 top-level skills, 665 navigation files, a 13 MB document store, and an approximate compile time of 6.5 minutes (Sun et al., 16 Apr 2026).

The baselines include BM25, Dense retrieval using Qwen3 embeddings and FAISS, a Hybrid method based on reciprocal-rank fusion of BM25 and Dense, RAPTOR using UMAP, GMM, and BIC with collapsed-tree retrieval, and Agentic RAG with up to 10 iterative retrieval rounds across BM25, Dense, and Hybrid (Sun et al., 16 Apr 2026). Evaluation uses lexical metrics including Token F1, BLEU, ROUGE-1, and ROUGE-2; LLM-judged Factuality and Context Recall normalized from 1 to 5; and cost metrics including input tokens and dollars per query.

Corpus2Skill achieves the highest quality across all metrics, albeit at higher cost (Sun et al., 16 Apr 2026). Relative to the best flat baseline, the reported improvements are +27% Token F1, with 0.460 versus 0.363; +85% BLEU, with 0.137 versus 0.074; and +45% Context Recall, with 0.652 versus 0.450. The cost tradeoff is explicit: \$N$40.012 and Agentic RAG at \$0.098 (Sun et al., 16 Apr 2026).

The failure analysis isolates where the system breaks down. Among the 31% of queries with low Factuality $N$5 or Context Recall $N$6, 61% are attributed to top-level navigation misses, 31% to partial navigation in which the gold document is retrieved but diluted by distractors, 5% to synthesis errors, and 3% to cases where no documents are retrieved (Sun et al., 16 Apr 2026). An ablation reports that narrower top clusters reduce routing errors. This indicates that hierarchy design, especially top-level partitioning, is a dominant determinant of performance.

5. Relation to the broader corpus-to-skill literature

Related 2026 work broadens the meaning of Corpus2Skill from navigable document hierarchies to a general family of corpus-to-procedure compilation methods (Ni et al., 26 Mar 2026, Wang et al., 16 May 2026, Pan et al., 8 Jun 2026). These systems differ in what counts as the corpus, how skills are represented, and whether adaptation happens offline, online, or at test time.

System	Source corpus	Runtime use
Corpus2Skill (Sun et al., 16 Apr 2026)	Document corpus	Navigate SKILL.md / INDEX.md, then retrieve full docs
Trace2Skill (Ni et al., 26 Mar 2026)	Labeled execution traces	Load a single evolved skill folder, no retrieval index
SkillTTA (Wang et al., 16 May 2026)	Training trajectory pool	Retrieve top-$N$7 trajectories and synthesize a temporary SKILL.md
Anything2Skill (Pan et al., 8 Jun 2026)	Manuals, logs, trajectories, dialogues, mixed records	Dual retrieval of passages and skills from a SkillBank

Trace2Skill distills a large pool of trajectories into a single, conflict-free skill directory via a three-stage pipeline: trajectory generation, parallel patch proposal by success and error analysts, and hierarchical consolidation under programmatic guardrails (Ni et al., 26 Mar 2026). The skill representation is a standard SKILL.md plus auxiliary resources, and the evolved skill is deployed with no retrieval index and no parameter updates. The framework reports gains in spreadsheet, VisionQA, and math reasoning, including a +57.65 percentage point cross-model transfer result on WikiTableQuestions and 3–5 percentage point gains on math reasoning benchmarks (Ni et al., 26 Mar 2026).

SkillTTA moves the corpus-to-skill operation to test time (Wang et al., 16 May 2026). Its training corpus is

$N$8

where $N$9 is the instruction, $L=\lceil \log_p N\rceil$0 metadata, $L=\lceil \log_p N\rceil$1 the full execution trajectory, and $L=\lceil \log_p N\rceil$2 the success or failure label. For a test task $L=\lceil \log_p N\rceil$3, the method retrieves $L=\lceil \log_p N\rceil$4 nearest neighbors by cosine similarity over metadata embeddings produced by text-embedding-3-large, synthesizes a temporary textual skill $L=\lceil \log_p N\rceil$5 using a second LLM, and prepends it to the fixed solver prompt. No parameter updates are permitted at test time (Wang et al., 16 May 2026). On SpreadsheetBench, task-specific skills improve Pass@1 from 0.397 to 0.505 relative to static Trace2Skill under GPT-5.5 synthesis; on BigCodeBench, Pass@1 improves from 0.517 to 0.651; and on ALFWorld, the method reaches 0.872 success with 8.88 steps, matching a heavier memory-learning baseline within four points while producing the shortest successful trajectories among reported methods (Wang et al., 16 May 2026).

Anything2Skill generalizes further to heterogeneous corpora $L=\lceil \log_p N\rceil$6 and compiles them into a persistent SkillBank $L=\lceil \log_p N\rceil$7 under a skill-tree prior $L=\lceil \log_p N\rceil$8 (Pan et al., 8 Jun 2026). Each record is broken into evidence windows, candidate skills are produced by a plan-and-expand operator, overlapping drafts are canonically compiled, and registry-level reconciliation predicts lifecycle actions such as Create, Strengthen, Revise, Merge, Split, Unchanged, or Discard. At inference, the agent performs dual retrieval:

$L=\lceil \log_p N\rceil$9

and outputs $O(p\log_p N)$0 (Pan et al., 8 Jun 2026). On qsv and GitHub-CLI, Anything2Skill combined with RAG reaches 98.85% and 94.10% success rates, respectively, outperforming RAG-only agents (Pan et al., 8 Jun 2026).

Taken together, these systems suggest that "Corpus2Skill" now denotes a design space rather than only a single implementation. The common thread is the conversion of latent procedural information in corpora into explicit, agent-readable skills.

6. Limitations, misconceptions, and open problems

The original Corpus2Skill framework is explicit about its limitations (Sun et al., 16 Apr 2026). Per-query cost is dominated by navigation token load, making the method better suited to high-value, multi-hop queries than to cheap, high-throughput retrieval. API constraints such as a maximum of 8 skills and 200 files per skill restrict tree shape. Hard single-path clustering means that documents covering multiple topics appear in only one branch, which can create blind spots. The system also lacks incremental updates: adding documents requires full recompilation.

These limitations frame several open problems. The paper proposes prompt caching or delta-updates to reduce repeated navigation costs, soft or multi-parent clustering to handle multi-topic documents, evaluation on larger and more diverse enterprise corpora, and tighter integration with evolving Skills APIs as file-count and size limits grow (Sun et al., 16 Apr 2026). Related work points to adjacent extensions: semantic or AST-based conflict resolution for richer skill formats, human review of high-support merges, learned taxonomies, and retrieval hybrids for rare edge cases (Ni et al., 26 Mar 2026, Pan et al., 8 Jun 2026).

Another misconception is that skill compilation necessarily replaces declarative retrieval. The broader literature indicates multiple coexistence patterns. Corpus2Skill emphasizes navigable summaries plus document retrieval (Sun et al., 16 Apr 2026); Trace2Skill removes runtime retrieval in favor of a single consolidated skill (Ni et al., 26 Mar 2026); SkillTTA performs one retrieval and one synthesis step per task (Wang et al., 16 May 2026); and Anything2Skill couples compiled procedural skills with conventional RAG passages at inference (Pan et al., 8 Jun 2026). The controversy is therefore not whether retrieval should exist, but where procedural abstraction should occur: offline in a persistent hierarchy, offline in a consolidated skill file, or online as a temporary task-specific skill.

In that broader sense, Corpus2Skill marks a shift in agent design from retrieving isolated evidence snippets toward compiling and exposing reusable procedural structure. The specific enterprise QA system shows that making the corpus organization visible can improve grounded answer quality on WixQA (Sun et al., 16 Apr 2026), while the adjacent trajectory- and knowledge-compilation systems show that the same corpus-to-skill principle extends to agent traces, manuals, logs, and mixed knowledge bases (Ni et al., 26 Mar 2026, Wang et al., 16 May 2026, Pan et al., 8 Jun 2026).