---
title: 'Skill Neologisms: Extending Capabilities'
url: https://www.emergentmind.com/topics/skill-neologisms
type: topic
---

# Skill Neologisms: Extending Capabilities

Skill neologisms are newly introduced skill-bearing units that extend, label, or stabilize capabilities. In the most explicit formulation, they are soft tokens integrated in a model’s vocabulary and optimized so that providing them in context enhances performance on a specific skill without updating base-model weights [2605.04970]. Closely related mechanisms appear in trainable neologism tokens used as promptable behavioral handles such as `~short` and `~kidmode` [2512.18551], in multi-concept controllability and self-verbalization through new input-only tokens [2510.08506], and in externalized, named skill artifacts derived from interaction traces, as in AutoSkill’s versioned skill cards [2603.01145]. Taken together, these lines of work suggest that skill neologisms form a broader research program at the intersection of continual learning, controllability, personalization, prompt engineering, and the linguistic study of how new lexical items emerge and diffuse [1111.6843][2001.07740].

## 1. Core definition and conceptual scope

A pretrained LLM \(M\) with parameters \(\theta_{\mathrm{LLM}}\) and a target skill \(S\) admits a skill neologism as a set of learnable soft tokens \(\theta_S \in \mathbb{R}^{d_{\mathrm{model}} \times l}\) that minimize loss over an \(S\)-centered dataset \(D(S,\Sigma)\), with an insertion function \(\phi_S\) placing the tokens into text [2605.04970]:

$$
\theta_S^* = \arg\min_{\theta_S} \mathbb{E}_{t \sim D(S,\Sigma)}\left[
\mathcal{L}\big(M(\theta_{\mathrm{LLM}}, \theta_S, \phi_S(t))\big)
\right].
$$

This definition fixes three properties. First, the base model weights remain frozen. Second, the new capability is localized in vocabulary-level parameters rather than distributed across updated internal weights. Third, the learned object is tied to a skill rather than to a single task instance. The same paper frames skill-based continual learning through three desired properties: **P1** learning new skills without modifying existing model parameters, **P2** compositional transfer to out-of-distribution skill mixtures, and **P3** zero-shot composition of independently trained skills [2605.04970].

Related work defines a neologism more narrowly as a new token added to a model’s vocabulary, with all parameters frozen except the embedding of that token, trained on data exemplifying its meaning [2512.18551]. In that setting, the token functions as a discrete steering handle: prompts such as “Give me a `~short` answer” or “Give me a `~kidmode` answer” selectively activate the learned behavior while leaving the model’s default behavior available when the token is absent [2512.18551]. A further extension treats multiple added tokens as controllable concept handles for text length, flattery, refusal, incorrect answers, and more complex AxBench concepts, while constraining generation to the original vocabulary so that the neologisms act as input-only controls [2510.08506].

A broader, external-memory interpretation appears in AutoSkill. The paper does not use the term “neologism,” but it explicitly “can be read as a concrete mechanism for skill neologisms”: the system coins skills from experience, stabilizes them in a shared representation, refines them over time, and reuses them as named capabilities [2603.01145]. In this reading, a skill neologism is not only a learned embedding but also a persistent lexical entry in an agent’s capability vocabulary.

## 2. Representational forms

Skill neologisms currently appear in two main representational forms. The first is the vocabulary-integrated soft token. If the original embedding matrix is \(E \in \mathbb{R}^{d_{\mathrm{model}} \times |\mathcal{V}|}\), then a new skill neologism of length \(l\) extends it to \(E' = [E \;\; s_1 \;\; \dots \;\; s_l]\), with corresponding new vocabulary items \(\langle S_1\rangle,\dots,\langle S_l\rangle\) [2605.04970]. In the single-token variant, a new token \(c\) adds one trainable column \(E_c \in \mathbb{R}^d\); for Mistral-7B Instruct, this means training only \(d=4096\) parameters for one neologism [2512.18551]. This representation is continuous, parameter-efficient, and directly promptable.

The second form is the explicit skill artifact. AutoSkill defines a skill as the tuple
\[
s = (n, d, p, \tau, \gamma, \xi, v),
\]
where \(n\) is the skill name, \(d\) the description, \(p\) the prompt body, \(\tau\) the trigger set, \(\gamma\) the tag set, \(\xi\) the example set, and \(v\) the version [2603.01145]. The tuple is materialized as a **SKILL.md** artifact, optionally with extra files. The paper’s skill cards use fields such as `ID`, `Version`, `Description`, `Tags`, `Triggers`, and a structured `Prompt` block containing `Role & Objective`, `Communication & Style Preferences`, `Operational Rules / Constraints`, optional workflow sections, `Anti-Patterns`, and sometimes `Output Format` [2603.01145]. In that representation, the neologism is both a named concept in a capability vocabulary and a machine-usable policy.

These two forms are not equivalent. Vocabulary-integrated tokens encode skills as learned directions in embedding space; skill artifacts encode them as inspectable, editable, versioned text-plus-metadata objects. A plausible implication is that the former favors compact, low-latency activation, whereas the latter favors auditability, transfer, and human intervention. Current work explores both rather than collapsing them into a single formalism.

## 3. Acquisition, naming, and self-evolution

The learning dynamics differ sharply across representations. In soft-token approaches, acquisition is optimization over the new embedding alone. One line uses preference-style datasets
\[
\mathcal{D} = \{(x, y^{(c)}, y^{(r)})_j\}_{j=1}^M
\]
where \(x\) appends an instruction containing the neologism token, \(y^{(c)}\) is a chosen response exemplifying the target behavior, and \(y^{(r)}\) is a rejected response exhibiting an opposed behavior [2512.18551]. Training minimizes an Anchored Preference Optimization objective while keeping all base parameters frozen. Under a matched setup, a single neologism trains 4,096 parameters, whereas the LoRA baseline trains 3,407,872 parameters; the neologism runs also complete over 50% faster than the respective LoRA runs [2512.18551]. A parallel framework likewise freezes transformer weights and existing token embeddings, adds one or more new tokens, and optimizes only the new embeddings with APO-up, showing that new words can control concepts such as flattery, incorrect answers, text length, and multiple jointly learned concepts [2510.08506].

AutoSkill acquires skills by extraction rather than by gradient updates to a token embedding. For a user \(u\), dialogue history is represented as \(\mathcal{X}_u = \{x_1,\dots,x_T\}\), with \(x_t = (q_t, r_t)\), but skill extraction uses only user queries \(\mathcal{Q}_u^t = \{q_1,\dots,q_t\}\) as primary evidence so that skills reflect stable user requirements rather than assistant hallucinations [2603.01145]. The extraction prompt instructs the model to identify durable, reusable constraints, policies, workflows, or templates; to avoid one-shot requests, stale constraints, and assistant-invented details; and to produce structured outputs containing `name`, `description`, `prompt`, `triggers`, `tags`, `examples`, and `confidence` [2603.01145]. This is the point at which new skill names are coined.

A candidate skill is then compared with the existing skill bank using a hybrid dense-plus-BM25 similarity, followed by a judge model that decides among `add`, `merge`, and `discard` [2603.01145]. The skill-bank update rule is explicit:
\[
\mathcal{B}_u^{t+1} =
\begin{cases}
\mathcal{B}_u^t \cup \{z_t\}, & a_t = add, \\
(\mathcal{B}_u^t \setminus \{s_t^\ast\}) \cup \{s_t'\}, & a_t = merge, \\
\mathcal{B}_u^t, & a_t = discard.
\end{cases}
\]
Merge performs semantic union rather than raw concatenation, preserves the capability identity, removes case-specific entities, deduplicates sections, and bumps the version \(v\) [2603.01145]. The frequently refined `professional_text_rewrite` skill reaches version `0.1.34`, whereas other skills remain at `0.1.0`, which the paper interprets as qualitative evidence of continual refinement rather than duplication [2603.01145].

This acquisition pipeline makes the naming aspect literal. Skills such as `professional_text_rewrite`, `selenium_automation_workflow_generator`, `顶级心理咨询师`, or `猫娘角色扮演对话` become stable lexical entries in a growing skill bank [2603.01145]. In external-memory systems, then, skill neologisms are not latent only in parameters; they are overtly named artifacts with version histories.

## 4. Composition, controllability, and evaluation

The central empirical claim of the skill-neologism literature is that skills can be learned in modular form and then composed. In the direct soft-token setting, skill-centered datasets are built so that each example requires the target skill together with varying numbers of already mastered skills, which pushes the learned representation toward compositional rather than purely task-specific behavior [2605.04970]. On synthetic digit-sequence transformations, skill neologisms for `[SHIFT]` and `[INV-POL]` achieve near-perfect in-distribution performance and maintain high accuracy on out-of-distribution compositions with held-out skills, whereas LoRA degrades markedly on those out-of-distribution mixtures and prompt tuning falls in between [2605.04970]. The same paper shows that independently trained skill neologisms can be composed zero-shot, outperforming an in-context learning baseline even when the latter receives up to 100 examples per skill [2605.04970].

Capacity is not monotone. When neologism length \(l\) is varied from 1 to 200, in-distribution accuracy saturates quickly, but out-of-distribution accuracy rises only up to a point and then declines, peaking around \(l \approx 20\) in the reported ablation [2605.04970]. Training on more complex compositions also improves compositional transfer: higher \(k_{\max}\) in the training mixtures yields better out-of-distribution generalization, especially for `[INV-POL]` [2605.04970]. These results motivate the view that limited token capacity acts as an inductive bias toward abstract skill representation rather than memorization.

The broader controllability literature reaches compatible conclusions. One study shows that new tokens can encode concepts such as long-text, short-text, single-sentence, use-like, flattery-answer, refusal-answer, and wrong-answer, with neologism control closing 92% of the training gap on average across these concepts [2510.08506]. The same work demonstrates joint learning of multiple concepts in multiple words and shows that, for the difficult “likely” concept, a neologism reaches a goal score of 0.667 versus 0.281 for few-shot prompting; for the three-way combination short+numerical+likely, the neologism reaches 0.482 versus 0.387 for few-shot prompting [2510.08506]. Another matched comparison against LoRA finds that `~short` reaches 95.5% gap closure and `~kidmode` 69.2%, outperforming the corresponding fine-tuned models under the reported setup [2512.18551].

Evaluation of composition predates the term itself. Skill-Mix samples random subsets of \(k\) skills from a list of \(N\) skills, combines them with a topic, and grades whether a model can produce coherent text instantiating the entire subset [2310.17567]. Because the number of subsets grows as \(\binom{N}{k}\), the evaluation aims to probe combinations that the model probably did not encounter in training. GPT-4’s reasonable performance at \(k=5\) is described as suggestive of going beyond “stochastic parrot” behavior, and the resulting framework supplies a natural benchmark backdrop for later skill-neologism work, which treats zero-shot skill composition as a core requirement [2310.17567].

## 5. Linguistic emergence, diffusion, and operationalization

Skill neologisms also belong to the ordinary linguistic history of neologisms. Research on Blip.pl models tags as locally invented lexemes that spread through a directed social network, with popularity following a power-law distribution and the distribution of adoption thresholds being strongly right-skewed with median \(\approx 0.11\) [1111.6843]. Most users adopt tags when only about 11% of their neighbors, or fewer, have already used them, whereas more popular tags tend to be adopted at higher exposure. The paper explicitly links such processes to self-organization, social contagion, and the emergence of institutionalized vocabulary from local invention and imitation [1111.6843]. For skill terminology, this implies that new skill labels can stabilize socially in much the same way as other technical jargon.

Distributional-semantic work reaches a complementary conclusion: neologisms tend to emerge in semantic neighborhoods that are somewhat sparse and, more strongly, in neighborhoods whose existing words have high frequency growth over time [2001.07740]. In diachronic English corpora, both semantic sparsity and the average growth rate of semantic neighbors are predictive of word emergence, though the study finds more support for the growth-rate hypothesis [2001.07740]. This suggests that skill neologisms are especially likely to arise where discourse around related technologies, roles, or practices is already expanding.

Operationalizing such emergence at scale remains difficult. A Chinese web pipeline over the BAAI CCI 3.0 corpus of 267,655,159 documents uses tokenizer-independent character \(n\)-grams, dictionary anchoring, a well-formedness skill, and a three-way semantic classification skill to produce 226,959 classified candidates including 4,853 labeled neologisms [2606.08715]. Its recall decomposition identifies Stage 1 candidate coverage \(R_1=41.53\%\) and Stage 4B semantic classification \(R_{4B}=59.97\%\) as the main bottlenecks; Stage 3 well-formedness is near-lossless at \(R_3=97.72\%\), and its length-stratified results show structural judgment to be length-invariant at \(\ge 96.9\%\) while semantic novelty classification drops from 65.6% to 59.0% to 44.1% across 2-, 3-, and 4-character candidates [2606.08715]. A separate English Reddit pipeline reduces 124.6 million unique token types to 1,021 candidates by combining rule-based filtering with multi-LLM classification; manual verification confirms that 599 of the 1,021 are genuine lexical innovations [2605.06426].

These detection results matter directly for skill neologisms because newer capability labels can be missed, mistranslated, or misclassified even when their morphology is transparent. NEO-BENCH shows that model performance is nearly halved in machine translation when a single neologism is introduced into an otherwise near-identical sentence, and models with later knowledge cutoffs yield lower perplexities and perform better on downstream tasks involving recent neologisms [2402.12261]. A plausible implication is that skill vocabularies will drift faster than static LLM training cycles can fully absorb, making ongoing detection and adaptation necessary rather than optional.

## 6. Limits, failure modes, and open directions

Current methods remain bounded by the base model’s latent competence. Skill-neologism work explicitly assumes that the new skill is “not too far” from the pretraining distribution and that the model already contains circuitry that can be steered or activated by the new embedding [2605.04970]. The same paper notes potential saturation and interference when many neologisms share the same embedding space, and its length ablation shows that larger neologisms can overfit training compositions and hurt out-of-distribution generalization [2605.04970]. Parameter efficiency therefore does not eliminate representational limits.

Composition is not uniformly robust. In one study, `~short` and `~kidmode` work well individually, but the combined `~short + ~kidmode` prompt drops the kid-friendliness score to 4.1/10, only 23.4% of the training gap closed, and the dual-neologism run scores worst on capability preservation among the reported variants [2512.18551]. Another study finds that multi-concept control can work, but also introduces norm regularization and multi-template training to improve robustness under composition and negation [2510.08506]. These results indicate that modularity is empirical rather than guaranteed.

Interpretability introduces its own complications. Self-verbalization experiments show that models can often explain what a learned token means, but they can also generate machine-only synonyms. One paper reports “lack” functioning as a behaviorally effective synonym for a single-sentence concept, and another documents invented descriptors such as “Mutexpoitary” and “Poornessily” when the model attempts to explain a neologism [2510.08506][2512.18551]. The same line of work notes that a benign-looking natural word could function as an effective handle for an undesirable skill, which raises auditing and governance problems for learned capability tokens [2510.08506].

Externalized skill banks have different limits. AutoSkill mitigates conflicts through per-skill anti-patterns and a meta-instruction that retrieved skills may be irrelevant and should be ignored unless they directly match the current intent; it does not implement explicit conflict logic, explicit numerical skill-quality metrics, or pruning policies for rarely used skills [2603.01145]. This leaves prioritization and conflict resolution largely to the LLM at inference time.

Human-facing skill-neologism support is also incomplete. In a cross-cultural communication study, AI Explanation yields the largest gains over no support in NS-rated communicative competence when non-native speakers learn and use English neologisms, but contextual appropriateness judgments show indifference across support conditions, self-reported perceptions overestimate NS ratings, and a significant gap remains between NNS- and NS-produced writing [2604.23842]. The result suggests that even when AI improves production, it does not automatically confer reliable meta-pragmatic judgment.

Taken together, the literature treats skill neologisms less as a single method than as a family of mechanisms for creating, stabilizing, composing, and auditing new capability labels. Some methods learn them as embedding-space handles; others externalize them as versioned skill artifacts; still others study their spread and detect them as ordinary lexical innovations in social corpora. Across these settings, the same research agenda recurs: how to add new skills without weight updates, preserve old capabilities, expose the new capability in a reusable lexical form, and keep the resulting vocabulary interpretable and socially grounded [2605.04970][2603.01145].

Source: https://www.emergentmind.com/topics/skill-neologisms