Skill Smells in LLM Agent Artifacts

• Skill smells are recurring quality issues in skill artifacts that indicate mismanaged specialization and degraded runtime performance.
• They manifest as skill leakage in global configuration files (AGENTS.md/CLAUDE.md), causing inflated prompts, increased token usage, and maintenance challenges.
• In SKILL.md and Scratch contexts, these smells signal violations of best practices and potential deficits in computational thinking, highlighting the need for structured remediation.

Skill smells are recurring quality problems in skill-related artifacts for LLM agents, especially when specialized operational knowledge is authored, stored, or maintained in ways that degrade runtime efficiency, clarity, safety, or maintainability. In the recent literature, the term has two closely related but distinct uses. In coding-agent configuration research, skill smells refer primarily to mistakes in AGENTS.md or CLAUDE.md where task-specific guidance is inappropriately embedded in the global configuration rather than isolated in on-demand skill files; the canonical instance is Skill Leakage (Santos et al., 14 Jun 2026). In research on Agent Skills proper, skill smells are defined more broadly as violations of best practices in mandatory SKILL.md files, with empirical evidence indicating that over 99% of analyzed SKILL.md files contain at least one such smell (Hong et al., 1 Jul 2026). A third, looser usage appears in computing-education research, where “skill smells” is an interpretive label for code-smell patterns that act as proxies for deficits in computational-thinking dimensions in novice Scratch programs (Aragón et al., 23 Jul 2025). Across these contexts, the unifying concern is that smell-like anomalies serve as actionable indicators of impaired separation of concerns, weakened specification quality, and accumulating maintenance debt.

1. Conceptual scope and terminological variants

The most precise current definition comes from the study of coding-agent configuration files. There, a configuration smell is defined as “a recurring content or structure problem in agent configuration files (AGENTS.md or CLAUDE.md) that impairs coding agents’ performance by inflating context, duplicating tool-enforced rules, creating ambiguity, or embedding non-essential, task-specific instructions in the global prompt” (Santos et al., 14 Jun 2026). Within that catalog, skill smells are the subset of configuration mistakes tied to skills, meaning errors in how task-specific instructions are distributed between global configuration and dedicated skill files. The paper identifies Skill Leakage as the primary skill-related smell.

In the study of Agent Skills, the unit of analysis shifts from AGENTS.md and CLAUDE.md to SKILL.md, the mandatory file that defines each skill. Here, “skill smells” are defined as violations of best practices for authoring SKILL.md files (Hong et al., 1 Jul 2026). This usage is broader than Skill Leakage: it encompasses incompleteness, ambiguity, contradiction, stale documentation, structural weaknesses, and policy omissions in the skill artifact itself.

A further extension appears in the Scratch literature, where the term is not introduced by the paper as formal nomenclature, but can be read as an implied interpretation: design-level code smells act as proxies for weaknesses in particular computational-thinking skills (Aragón et al., 23 Jul 2025). This suggests a useful cross-domain generalization: a skill smell is not only a defect in a skill artifact, but also a smell whose presence signals impaired skill acquisition, deployment, or operationalization.

2. Skill Leakage in AGENTS.md and CLAUDE.md

In coding-agent configuration research, Skill Leakage occurs when “specific, rarely used, or highly context-dependent instructions are placed in the AGENTS.md/CLAUDE.md file rather than in dedicated skill markdown files (e.g., skills.md) that are fetched only when relevant” (Santos et al., 14 Jun 2026). The defect is architectural rather than merely stylistic. It violates a separation between project-wide guidance, which belongs in the global prompt, and specialized procedures, which should be loaded only for relevant tasks.

The paper identifies three direct causes. First is the practice of centralizing all instructions in AGENTS.md or CLAUDE.md rather than separating task-specific content. Second is treating specialized procedures, such as detailed testing workflows, scaffolding steps, or infrastructure-specific commands, as if they were always relevant. Third is failure to adopt a skills-on-demand pattern, in which additional task instructions are fetched only when needed (Santos et al., 14 Jun 2026).

The consequences are concrete. Skill Leakage produces larger prompts and higher token usage and cost during sessions. It reduces adherence to critical project-wide rules because non-essential instructions compete for attention. It also increases maintenance difficulty, since buried specialized guidance is harder to update and more likely to become outdated. The paper explicitly links this to co-occurring smells such as Context Bloat (Santos et al., 14 Jun 2026).

Typical manifestations include sections for niche or infrequent workflows, long step-by-step routines relevant only to a small subset of tasks, and overly detailed testing or review checklists that are not universal across tasks. The paper’s paraphrased example is a section titled “Adding a new OS to quickget,” containing detailed steps for switch cases, edition functions, architecture functions, and download URL logic. Because most sessions do not involve adding operating systems, this material inflates the global configuration unnecessarily. The proposed mitigation is to move the procedure into a dedicated skill markdown file such as skills/quickget-os-addition.md, leaving only a concise pointer in AGENTS.md or CLAUDE.md instructing the agent to load that skill when performing OS setup tasks (Santos et al., 14 Jun 2026).

3. Relation to the broader catalog of configuration smells

Skill Leakage is part of a six-smell catalog for coding-agent configuration files (Santos et al., 14 Jun 2026). The full catalog clarifies both what makes Skill Leakage distinctive and why it often appears with other defects.

Smell	Brief definition
Context Bloat	Excessively long configuration file, e.g., ≥200 lines
Skill Leakage	Specialized instructions placed in the global file instead of skill files
Lint Leakage	Restatement of style or formatting rules already enforced by tools
Blind References	External references without contextual description
Init Fossilization	Init-generated file persists unchanged and accumulates noise
Conflicting Instructions	Contradictory rules in the same configuration

Skill Leakage is closely related to Context Bloat, but the two are not identical. Context Bloat is a size-oriented condition: files with LOC(f) ≥ 200 are flagged by the paper’s heuristic. Skill Leakage is a semantic condition: it concerns whether the content is globally appropriate, regardless of line count. A file can therefore leak specialized knowledge without exceeding 200 lines, or be bloated without containing skill-specific content.

Skill Leakage also interacts with Conflicting Instructions. When many niche procedures are embedded into one global file, contradictory rules can emerge across task contexts. The paper reports that Context Bloat, Skill Leakage, and Conflicting Instructions often appear together (Santos et al., 14 Jun 2026). This suggests that a global prompt overloaded with specialized fragments is not merely inefficient; it is structurally prone to ambiguity and contradiction.

A further relation exists with Lint Leakage. A configuration file that already centralizes too much information may also restate deterministic style rules already enforced by linters or formatters, consuming further instruction budget. This suggests that skill smells in configuration files are often part of a broader anti-pattern of over-specification in the global prompt.

4. Detection, formalization, and empirical prevalence

The coding-agent configuration study formalizes smell detection with binary indicators $I_S(f) \in \{0,1\}$, prevalence $P(S) = n_S / N$, and co-occurrence $C_{i,j} = n_{i \wedge j} / N$ (Santos et al., 14 Jun 2026). For Skill Leakage, detection is LLM-based: the prompt describes the smell, provides operational guidelines, and asks the model to return exact lines or “NO SMELL.” The study used gemini-3.1-flash-lite with temperature 0, followed by manual verification.

The structural cues targeted by the Skill Leakage prompt are explicitly described: the presence of sections or lines defining narrow, task-specific procedures in testing, workflow, scaffolding, infrastructure, or architecture that are not universally needed in every session. By contrast, Context Bloat uses a threshold heuristic, and Init Fossilization uses a number-of-commits heuristic, with files flagged when commits(f) = 1 (Santos et al., 14 Jun 2026).

The dataset consisted of N = 100 popular open-source repositories containing either AGENTS.md or CLAUDE.md in the root folder, of which 39 used AGENTS.md and 61 used CLAUDE.md. Among the six smells, Lint Leakage was most prevalent at 62%, followed by Context Bloat at 42% and Skill Leakage at 35%. For Skill Leakage specifically, the paper reports 35 detected instances, 29 confirmed cases, 82% precision, and 6 false positives. Overall, at least one smell appeared in 91 files, leaving only 9 smell-free repositories (Santos et al., 14 Jun 2026).

The co-occurrence analysis uses Apriori association rules with minimum support 0.05, reported through support, confidence, and lift. Selected rules show the structural embedding of Skill Leakage within larger smell constellations. The rule Conflicting Instructions + Skill Leakage → Context Bloat has support 0.06, confidence 0.83, and lift 1.81. The rule Skill Leakage → Lint Leakage has support 0.24, confidence 0.76, and lift 1.19. The rule Context Bloat + Skill Leakage → Lint Leakage has support 0.10, confidence 0.75, and lift 1.18 (Santos et al., 14 Jun 2026). These values indicate that Skill Leakage is not an isolated anomaly but part of recurrent joint failure modes in real repositories.

5. Skill smells in SKILL.md artifacts

The study of SKILL.md broadens the discussion from misplaced skill content to quality problems within the skill artifact itself (Hong et al., 1 Jul 2026). The abstract states that the authors qualitatively analyzed 238 real-world skills, derived a taxonomy with 13 higher-level and 44 lower-level semantic components, conducted a multivocal literature review of 29 sources, and developed an automated detector. The detector found that over 99% of SKILL.md files contain at least one skill smell, and that once introduced, skill smells rarely disappear as skills evolve.

Within the provided material, only the abstract-level findings are directly attributable to the paper. The detailed smell catalog is not included in the excerpt, so exact categories cannot be stated as paper claims. However, the provided summary explicitly characterizes skill smells in this context as violations of best practices for authoring SKILL.md files, analogous to code smells: not necessarily bugs, but actionable indicators of increased risk, degraded usability, or unreliable agent behavior (Hong et al., 1 Jul 2026).

The same excerpt states that each Agent Skill is defined by a mandatory SKILL.md file containing metadata and an unstructured Markdown body whose contents are left entirely to the author. This structural openness helps explain why smell prevalence is so high. When authoring is unconstrained, omissions, contradictions, vague scope boundaries, undocumented dependencies, and malformed structure can proliferate. The longitudinal observation that smells rarely disappear suggests a documentation-debt dynamic: once low-quality content enters a skill artifact, it tends to persist across revisions rather than being systematically remediated (Hong et al., 1 Jul 2026).

A plausible implication is that the research on SKILL.md and the research on AGENTS.md/CLAUDE.md expose two levels of the same governance problem. One level concerns where specialized instructions should live; the other concerns how those instructions should be authored once they have been placed in a skill file.

6. Skill smells as indicators of skill deficits in Scratch

In the Scratch registered report, the term “skill smells” is not part of the paper’s formal vocabulary, but the provided synthesis explicitly interprets the study in that way (Aragón et al., 23 Jul 2025). The paper examines the relation between computational-thinking proficiency and design-level code smells in novice Scratch projects. It operationalizes nine computational-thinking dimensions through DrScratch scores on a 0–4 scale: Abstraction, Parallelism, Logic, Synchronisation, Flow Control, User Interactivity, Data Representation, Math Operators, and Motion Operators.

The study mines approximately 2 million public Scratch projects, using DrScratch and LitterBox to derive nine CT scores and smell indicators. The tooling yields 33 detectable smells, although the abstract mentions “40 indicators” (Aragón et al., 23 Jul 2025). Because the work is a registered report, no empirical effect sizes or confirmed associations are yet reported. What is provided are hypotheses and planned tests.

The interpretive bridge to skill smells is explicit in the supplied synthesis: concrete design-level smells can function as indicators of CT skill issues. Examples include duplication and long scripts as signals of weak abstraction or decomposition, busy-waiting and polling as signals of weak flow control or synchronization, unused variables or parameters as signals of weak data representation and procedural abstraction, and naming issues as signals of weak communication of intent (Aragón et al., 23 Jul 2025).

This usage differs from the agent-engineering literature in that the smell does not reside in a documentation artifact. Instead, it resides in executable novice code and serves as a proxy for latent skill weaknesses. Even so, the conceptual family resemblance is strong. In both settings, the smell is a recurring, observable anomaly that indicates a deeper problem in specification, decomposition, or coordination. This suggests that “skill smell” may be productively understood as a cross-domain diagnostic notion linking artifact quality to competence quality.

7. Remediation strategies, methodological limits, and significance

The most concrete remediation guidance currently available concerns Skill Leakage in coding-agent configuration. The recommended strategy is to place task-specific instructions such as testing playbooks, scaffolding procedures, infrastructure runbooks, and architecture sub-guidelines in dedicated skill markdown files with self-descriptive names, and to load those skills only when relevant. AGENTS.md and CLAUDE.md should remain focused on core, project-wide guidance (Santos et al., 14 Jun 2026). Related recommendations include keeping configuration files under 200 lines, avoiding repetition of linter-enforced rules, adding contextual explanation for external references, periodically reviewing files to prevent fossilization, and removing contradictory rules.

The paper also offers structuring advice by document function. Under Architecture, it recommends summarizing high-level constraints and linking to detailed documents with short contextual descriptions. Under Workflows, it recommends providing only broadly applicable core steps while delegating niche routines to skills. Under Coding conventions, it recommends referencing tool-enforced standards instead of restating them. Under Testing practices, it recommends keeping general strategy and commands in the global file while moving detailed or rare playbooks into skills (Santos et al., 14 Jun 2026).

The methodological limitations are also explicit. In the AGENTS.md/CLAUDE.md study, threats to validity include manual annotation bias in smell extraction, limited Google search coverage in the grey literature review, and LLM-based detection variability and non-determinism, mitigated through sanity checks, expanded keywords, temperature 0, and manual verification (Santos et al., 14 Jun 2026). In the SKILL.md study, the provided excerpt notes that exact catalog details, detector architecture, and quantitative metrics beyond the headline findings are not available in the excerpt, limiting precise reconstruction of smell categories (Hong et al., 1 Jul 2026). In the Scratch study, findings are still prospective because the work is a registered report; the notion of skill smells there is interpretive rather than formally defined by the authors (Aragón et al., 23 Jul 2025).

Taken together, the current literature indicates that skill smells are best understood as artifact-level indicators of mismanaged specialization. In AGENTS.md and CLAUDE.md, they arise when specialized instructions pollute the global prompt. In SKILL.md, they arise when the skill artifact itself violates authoring best practices. In educational code analysis, they appear as design antipatterns that reveal deficits in underlying computational-thinking skills. The common thread is separation of concerns: when knowledge, constraints, or procedures are poorly partitioned, smell-like anomalies emerge, and these anomalies are measurable, recurrent, and operationally consequential.