Language-Guided Skill Scaling

• Language-guided skill scaling is a framework where language organizes, acquires, and analyzes skills via methods such as compiling textual instructions into latent parameters.
• It integrates textual instructions with parameter adaptation, segmentation, and compositional routing to tackle scaling bottlenecks like context overhead and catastrophic forgetting.
• Empirical results show significant performance improvements and efficiency gains, while theoretical models link emergent behavior to structured skill composition and scaling laws.

Language-guided skill scaling denotes a family of research programs in which language is used not only to specify tasks, but to organize how skills are represented, acquired, selected, composed, and analyzed as systems scale. In the cited literature, the expression refers to several distinct but related mechanisms: compiling textual skills into latent LoRA adapters in weight space; using language to plan, segment, and label robot trajectories; guiding token sharing in lifelong imitation learning; evolving natural-language skill prompts at test time; synthesizing compositionally challenging data from a skill taxonomy; and explaining capability emergence through scaling laws over skill-text structure (Yu et al., 4 Jun 2026, Ha et al., 2023, Fu et al., 2024, Zhang et al., 2 Aug 2025, Du et al., 20 May 2026, Wei et al., 7 Jan 2026, Arora et al., 2023, Liao et al., 2024).

1. Conceptual scope and problem setting

Across these works, the common problem is that useful skills must scale along with task diversity, horizon length, and library size, but naive scaling produces distinct bottlenecks. Prompt-based agent skills incur repeated context overhead, prefill cost, and plaintext exposure; lifelong policies face catastrophic forgetting and linear parameter growth; robot systems require large amounts of labeled behavior without human demonstrations; large skill libraries create routing ambiguity and “black-hole skills”; and compositional generalization is limited by long-tailed distributions of multi-skill combinations (Yu et al., 4 Jun 2026, Zhang et al., 2 Aug 2025, Ha et al., 2023, Chen et al., 15 May 2026, Wei et al., 7 Jan 2026).

The cited literature therefore treats language as an engineering interface for capability growth. In some cases language is compiled into parameters; in others it acts as a planner, vocabulary-expansion mechanism, segmentation prior, routing substrate, or formal object of theoretical analysis. A recurring theme is that scaling is not only a matter of increasing model size. Several papers instead make the scaling variable the skill representation itself: adapter modules, parameter tokens, primitive labels, task-local skill prompts, profile specifications, or hierarchically organized skill compositions (Yu et al., 4 Jun 2026, Wang et al., 23 Jun 2026, Wu et al., 12 Mar 2026).

Setting	Language signal	Scaling substrate
LatentSkill	textual skill document	LoRA adapter in weight space
T2S	natural-language task instruction	tokenized atomic skills
InSight	primitive labels and VLM plans	expandable primitive vocabulary
Trace2Skill	natural-language skill prompt	evolvable policy artifact
STEPS	skill taxonomy and synthesized instructions	hard skill combinations

This variety matters because “language-guided” does not imply prompt-only control. In the surveyed work, language can remain in context, be compiled into weights, index reusable tokens, define structured JSON agent specifications, or supervise downstream decomposition and evaluation (Yu et al., 4 Jun 2026, Zhang et al., 2 Aug 2025, Wu et al., 12 Mar 2026).

2. From textual skills to latent parameters

A prominent formulation appears in LatentSkill, where a textual agent skill is compiled once into a latent LoRA adapter rather than reinserted into the prompt at every decision step. The framework uses a frozen backbone LLM $M_\theta$, instantiated as Qwen3-8B, together with a Transformer-based hypernetwork $G_\phi$ that maps a skill document $s$ to LoRA parameters, $\Delta_s = G_\phi(s)$. Inference then proceeds with the adapted backbone,

$$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$

so the skill knowledge is stored in weight space rather than context space (Yu et al., 4 Jun 2026).

The pretraining regime is explicitly document-centric. The hypernetwork is pretrained on about 171K deduplicated skill documents from GitHub, totaling roughly 300M tokens, using reconstruction and completion tasks in which information must be mediated through the generated adapter while the backbone remains frozen. A second stage tunes the compiler on teacher trajectories paired with skill documents, mounting the same generated adapter across all trajectory steps so that the adapter encodes skill-level policy behavior rather than ad hoc step-wise conditioning. At the module level, the compiler outputs standard LoRA updates $\Delta W_s^{(m)} = B_s^{(m)} A_s^{(m)}$, mounted as

$W' = W + \frac{\alpha}{r} B_s A_s.$

An ablation identifies attn_o and mlp_down as the most discriminative injection sites, and using only these two positions retains most of the performance while using only 2 of 7 injection positions (Yu et al., 4 Jun 2026).

The scaling variable in this framework is the LoRA coefficient $\alpha$. Empirically, $\alpha$ yields a continuous control axis rather than a binary switch. On ALFWorld, average success on the seen split rises from 43.57% at $\alpha=0$ to 74.29% at $G_\phi$0, then falls to 22.86% at $G_\phi$1; on the unseen split it peaks at 70.90% at $G_\phi$2, then falls to 8.21% at $G_\phi$3. The paper therefore characterizes scaling as useful only within a stable range. It further reports that LatentSkill achieves 74.3% average success on ALFWorld seen and 69.4% on unseen, improving over the in-context skill baseline by +21.4 and +13.4 points while reducing prefill overhead by 64.1%; on Search-QA it reaches 35.6 EM, +3.0 over in-context skill, with 72.2% lower skill-token overhead (Yu et al., 4 Jun 2026).

The same paper also studies parameter-space composition. Cached adapters can be added directly,

$G_\phi$4

but naive addition can fail when whole skills share components. The proposed alternative is component-level composition, in which semantic components $G_\phi$5 are compiled separately and only aligned components are added,

$G_\phi$6

In the Look + Pick experiment, Component Merging reaches 84.6% success on seen and 77.8% on unseen. The same analysis argues that generated skill LoRAs inhabit a structured semantic geometry: MDS reveals domain clustering, inter-cluster distance drops from 0.0887 to 0.0704 after SFT, within-domain similarity is higher than cross-domain similarity, and the updates have stable rank around 2.35–2.40 before SFT and 2.17–2.23 after SFT, versus 837.87 for a random LoRA of the same shape (Yu et al., 4 Jun 2026).

A related parameter-centric formulation appears in T2S, where transformer parameters are tokenized and the linear map is replaced by token-parameter attention,

$G_\phi$7

Natural-language task instructions are encoded into embeddings that select relevant tokens from a global pool, so language guides token sharing and expansion across tasks rather than indexing isolated task IDs. On the three LIBERO suites, T2S reports average NBT = 1.0%, average FWT = 77.7%, and only 8.0% trainable tokens on average in lifelong tasks; an ablation on LIBERO-GOAL shows that language guidance exceeds task-ID selection, with 84.0 ± 1.0 FWT and 70.0 ± 2.0 AUC for language versus 75.7 ± 1.0 and 67.2 ± 2.0 for task IDs (Zhang et al., 2 Aug 2025).

3. Embodied acquisition, primitive discovery, and temporal abstraction

In robot learning, language-guided skill scaling is used to scale data generation, temporal abstraction, and primitive reuse. “Scaling Up and Distilling Down” proposes a two-stage pipeline in which an LLM-based hierarchical planner decomposes language-described tasks into a task tree, grounds leaf tasks into 6DoF exploration primitives, and also generates a Python-like success-condition code snippet for each task or subtask. The success function enables failure detection, retrying without resetting the scene until success, and automatic success/failure labeling. The resulting successful trajectories are distilled into a multi-task language-conditioned diffusion policy. The paper introduces a benchmark with 18 tasks across 5 domains and reports an average absolute success-rate improvement of 33.2%, from 32.2% average for Distill No Retry to 67.0% average for Distill Ours; it also shows that verify-retry raises data-generation success from 8.8% average with 6DoF robot utilities alone to 33.8% average (Ha et al., 2023).

InSight addresses a different limitation: VLAs can only perform behaviors already present in their training data. Its first stage segments human teleoperation demonstrations into primitive-labeled trajectories by matching a VLM-generated primitive plan with gripper transitions, end-effector pose, motion magnitudes, and a dominant-axis tag over $G_\phi$8. These segments fine-tune a pretrained VLA with LoRA, using primitive labels as language prompts and a learned progress channel in $G_\phi$9 as a primitive-level termination signal. The second stage is a VLM-guided data flywheel: for a novel task, the VLM decomposes the task into a primitive sequence, identifies any primitive gap $s$0, proposes a single-axis motion for the missing primitive, executes it, uses a VLM oracle to judge success from before/after images, and stores successful rollouts for retraining. In simulation, the system improves block flipping to 75% success after 246 acquired primitive rollouts; for drawer closing it produces 70 successful close-drawer primitives across 82 episodes and reaches 100% success on 25 trials while retaining drawer opening; on hardware it reaches 92% on twisting, 96% on pouring, and 80% on the longer twist-then-pour composition, and succeeds in 5/5 sweeping trials (Wang et al., 23 Jun 2026).

LAST uses language to make temporal skill discovery tractable on long trajectories. An LLM first proposes a fine-grained segmentation based only on the goal description and an action sequence described in language. These initial segments are short, typically 1–5 actions, and are accompanied by natural-language summaries. The hierarchical variational model then merges adjacent LLM segments rather than searching over all segmentations, subject to the constraint

$s$1

so the final segmentation cannot be finer than the LLM segmentation. Reusability is encouraged by an MDL auxiliary objective, and the learned low-level policies are later frozen while a high-level SAC policy selects skills and termination decisions online. The paper reports that LAST achieves the highest average success rate among baselines in the zero-shot setting on both BabyAI and ALFRED, that SAC from scratch fails on the selected downstream ALFRED tasks, and that varying the MDL weight in ALFRED yields libraries ranging from about 4 to 20 skills, with the best zero-shot transfer around 11 skills (Fu et al., 2024).

Taken together, these robot and sequential-learning papers treat language as a mechanism for reducing search complexity. In one case it generates plans and executable success criteria; in another it names and expands primitive vocabularies; in another it constrains the segmentation space of latent skills. A plausible implication is that the scaling bottleneck in long-horizon control often lies less in raw policy capacity than in how trajectories are partitioned, labeled, and reassembled (Ha et al., 2023, Wang et al., 23 Jun 2026, Fu et al., 2024).

4. Skill libraries, routing, and system-level composition

As skill libraries grow, the problem shifts from acquiring skills to exposing them without inducing semantic crowding. “The Scaling Laws of Skills in LLM Agent Systems” studies this directly across 15 frontier LLMs, 1,141 real-world skills, and over 3 million routing or execution decisions. Holding tasks fixed while varying exposed library size $s$2, it identifies a routing law

$s$3

with $s$4 for all models and roughly 3 percentage points lost per doubling of library size. The failure progression moves from local skill competition to cross-family drift and then capture by overly general “black-hole skills”; under normal descriptions, 94% of local-competition errors remain inside the correct cluster, and the most confusing region is intermediate similarity roughly in $s$5. Richer descriptions reduce the slope $s$6, with the appendix summarizing $s$7 for name-only descriptions and $s$8 for richer, boundary-aware descriptions. A law-guided library manager then raises held-out routing accuracy from 71.3% to 91.7%, reduces hijack from 22.4% to 4.1%, and improves mean pass rate from 49.3% to 61.6% on ClawBench and from 28.4% to 34.5% on ClawMark (Chen et al., 15 May 2026).

Trace2Skill addresses a related system-level problem in long-context EDA agents, but changes the optimization target. Instead of updating model weights, it treats the natural-language skill as an evolvable policy artifact. For each task, repeated rollouts are mined into dense diagnostics, an oracle summarizes task-local lessons, a mutator proposes child skills, and a selector chooses survivors using pass rate plus proxy scores such as AgentProgressQ and SkillQ. The framework also supports bounded-runtime dense verifier feedback through verify_feedback, which returns sanitized signals such as partial test count, failure phase, and next-focus hints without revealing hidden harnesses or reference solutions. On the hard subset of eight CVDP tasks, the paper compares four conditions: baseline sparse verifier only (0.0%, 0/8, AgentQ 0.34), baseline plus dense feedback (12.5%, 2/8, AgentQ 0.42), Trace2Skill with sparse reward (27.0%, 3/8, AgentQ 0.523), and Trace2Skill plus dense feedback (33.6%, 6/8, AgentQ 0.566) (Du et al., 20 May 2026).

STEPS moves the scaling locus to post-training data synthesis. Its premise is that atomic skills are abundant but multi-skill combinations follow a long-tailed, power-law distribution, so compositional generalization is bottlenecked by sparse exposure to hard combinations. The method induces a hierarchical skill taxonomy by recursively merging skill communities to reduce structural entropy, then synthesizes skill tuples by maximizing marginal structural information while preserving semantic coherence through a recursive, taxonomy-guided search. On AlpacaEval 2.0, MT-Bench, and WildBench, using base and instruct variants of Qwen-2.5-7B, Llama-3-8B, and Mistral-7B-v0.3, the paper reports that STEPS beats Alpaca52k and Instruct-SkillMix using only 4K synthetic examples. It also studies tuple depth: with 500 samples for each $s$9 plus 4K atomic instructions, WildBench rises from -22.75 at $\Delta_s = G_\phi(s)$0 to 23.91 at $\Delta_s = G_\phi(s)$1, peaks locally at 27.19 for $\Delta_s = G_\phi(s)$2, and reaches 31.52 with a mixture over $\Delta_s = G_\phi(s)$3 (Wei et al., 7 Jan 2026).

A further system-level variant appears in educational agents. “Scaling Laws for Educational AI Agents” argues that capability scales with the structural richness of an AgentProfile rather than only with base-model size. AgentProfile is a JSON-based specification with fields for name, description, details, agent_template, skills, tools, and subagents, and EduClaw operationalizes this through a three-stage pipeline of profile generation, skill resolution, and agent instantiation. The reported implementation includes 330+ agent profiles and 1,100+ skill modules across K–12 subjects. The empirical evidence is described as observational: richer role definitions, more core dimensions, and stronger output-format specifications correlate with more pedagogically appropriate responses; role definitions of 50+ words outperform generic ones under 20 words; profiles usually have 3–7 core dimensions; and profiles with 2–4 complementary skills outperform single-skill profiles (Wu et al., 12 Mar 2026).

These papers converge on a system-level claim: the scalability of an agent depends on library geometry, description quality, and exposure policy, not only on the underlying model. This suggests that “skill scaling” is partly a problem of interface design: what is exposed, how it is delimited, and how later decisions are conditioned on earlier realized state (Chen et al., 15 May 2026, Du et al., 20 May 2026, Wu et al., 12 Mar 2026).

5. Theoretical accounts of skill emergence and skill-dependent scaling

Several papers attempt to formalize why skills appear to emerge under scale. “A Theory for Emergence of Complex Skills in LLMs” models language understanding through a skill-text bipartite graph and interprets cross-entropy as entropy plus excess entropy. The central bridge is the Cloze Sufficiency Assumption: average prediction loss on cloze questions closely tracks the excess cross-entropy of next-word prediction. Under random-graph mixing, if model loss drops from $\Delta_s = G_\phi(s)$4 to $\Delta_s = G_\phi(s)$5, the performance curve for $\Delta_s = G_\phi(s)$6-tuples of skills in the larger model is identical to the performance curve for individual skills in the smaller model. The paper informally terms this “slingshot generalization,” and its headline claim is that competence at executing tasks involving $\Delta_s = G_\phi(s)$7-tuples of skills emerges essentially at the same scaling and same rate as competence on the elementary skills themselves (Arora et al., 2023).

“A Mathematical Theory for Learning Semantic Languages by Abstract Learners” makes the graph structure explicit. A semantic language is $\Delta_s = G_\phi(s)$8, where each text $\Delta_s = G_\phi(s)$9 depends on a skill set $$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$0, inducing a random bipartite graph between texts and skills. Training is modeled as Successive Cancellation of Novel Skills, mathematically parallel to iterative decoding in LDPC codes and IRSA. For the 1-skill learner, density evolution yields fixed-point equations for the residual unlearned probability and the learned-skill fraction, with testing error

$$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$1

The paper derives an emergence threshold as the training-text-to-skill ratio $$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$2 increases and a giant-component criterion for the learned skill association graph,

$$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$3

It further extends the analysis to hierarchical skills, multiple classes of skills and texts, and semantic compression (Liao et al., 2024).

A different theoretical angle appears in “Compute Optimal Scaling of Skills: Knowledge vs Reasoning,” which asks whether compute-optimal scaling is skill-dependent. The paper defines a skill-conditioned compute optimum relative to a dataset that measures a target skill and shows that the optimal parameter/token trade-off differs across skill families. Knowledge-based QA tasks are reported to be consistently capacity-hungry, while code tasks are data-hungry, and this remains true even after datamix ablations that vary the proportion of skill-relevant data. On the smallest compute scale, the inferred optimal parameter count can vary by almost 50% across datamixes and validation choices, and for the canonical datamix there is over 30% difference across validation sets at that scale (Roberts et al., 13 Mar 2025).

Taken together, these theories do not propose a single mechanism. One emphasizes excess entropy and random mixing over skill tuples, another treats learning as graph decoding with threshold behavior, and another shows that optimal scaling decisions depend on the skill family being targeted. A plausible synthesis is that “skill scaling” has at least three analytically distinct layers: emergence of competence from lowered uncertainty, emergence of connectivity in the latent skill graph, and emergence of different compute optima for different kinds of skills (Arora et al., 2023, Liao et al., 2024, Roberts et al., 13 Mar 2025).

6. Limitations, misconceptions, and open directions

A common misconception is that skill scaling is monotonic: more skill signal, more exposed skills, or larger scaling coefficients should always help. The surveyed papers reject that view. In LatentSkill, the control curve over $$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$4 is inverted-U shaped and $$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$5 is catastrophic on ALFWorld; composition fails when shared components are double-counted or when concatenated skill texts become out-of-distribution inputs to the hypernetwork. The same paper also notes that evaluation is limited to ALFWorld and Search-QA, both with Qwen3-8B and a fixed LoRA configuration, and explicitly calls for broader validation on web browsing, software engineering, and multi-agent settings (Yu et al., 4 Jun 2026).

Another misconception is that language-guided scaling eliminates the need for environmental structure or privileged supervision. The robot literature makes the opposite point. “Scaling Up and Distilling Down” depends on privileged simulation state for success-condition code, existing 3D assets and physics environments, manually designed evaluation success functions, and root-task evaluation rather than reusable subskills. InSight currently restricts primitive acquisition to single-axis motions, filters for successful rollouts only, requires human environment resets between rollouts, and identifies incorrect axis selection as a dominant failure mode. LAST reports that without initial segmentation, merge constraints, language annotations, or MDL regularization, performance degrades markedly, indicating that language guidance alone is insufficient without carefully structured inference objectives (Ha et al., 2023, Wang et al., 23 Jun 2026, Fu et al., 2024).

System-level scaling also introduces new fragilities. Large natural-language skill libraries can degrade routing approximately logarithmically with library size; vague prompts and broad skills jointly create attractor routing; and dense verifier feedback in Trace2Skill is most effective when coupled to skill evolution rather than used as a tool for a fixed policy. In lifelong imitation learning, T2S still depends on manually designed token-pooling and masking with the hyperparameter $$p_{\theta,\phi}(y_t \mid h_t, s) = p_{\theta \oplus \alpha \Delta_s}(y_t \mid h_t),$$6, and performance remains sensitive to task characteristics and order. In educational agents, the evidence for profile structural richness is observational rather than a controlled quantitative benchmark study (Chen et al., 15 May 2026, Du et al., 20 May 2026, Zhang et al., 2 Aug 2025, Wu et al., 12 Mar 2026).

The theoretical papers also set explicit boundaries on their claims. The slingshot-generalization framework is intentionally conservative, gives lower bounds on competence, assumes a cloze insertion mechanism, and does not model out-of-distribution generalization in detail. The graph-decoding theory relies on stylized random-graph assumptions, Poisson degrees, and abstract learners rather than transformer-specific mechanisms. This suggests that the current theory is strongest as a structural explanation of why emergence can occur, not as a full predictive account of the within-cluster rate of emergence or of fine-grained mechanistic behavior (Arora et al., 2023, Liao et al., 2024).

The literature therefore presents language-guided skill scaling not as a single algorithm, but as a research area organized around a recurring design choice: language can be the substrate through which skills are compressed, expanded, decomposed, routed, and theoretically modeled. What changes across papers is the object being scaled—adapter weights, primitive vocabularies, reusable latent skills, parameter tokens, natural-language libraries, synthesized task distributions, or structured agent profiles—and the dominant failure mode each method tries to control (Yu et al., 4 Jun 2026, Wei et al., 7 Jan 2026, Chen et al., 15 May 2026).