Skill Neurons in Neural Models

• Skill neurons are sparse, highly predictive hidden units in neural models that track and causally drive specific skills or behaviors.
• They are identified through methods that measure activation predictivity, intervention effects, and activation shifts across various architectures and tasks.
• Manipulating skill neurons enables targeted model control, improved interpretability, and robust performance tuning with applications from network pruning to clinical insights.

Skill neurons are sparse, highly predictive hidden units within large neural models whose activations track and causally drive the implementation of specific skills or behaviors. Originally identified in Transformer feed-forward networks, and extended to both recurrent and spiking architectures, skill neurons exhibit structured, task-specific activity patterns and are directly implicated in the model's performance on targeted subtasks. Their identification, manipulation, and theoretical characterization reveal new facets of model interpretability, robustness, and targeted control.

1. Formal Definitions and Operational Criteria

In Transformer-based models, a skill neuron is formally defined as an individual feed-forward network (FFN) unit whose activation, when exposed to carefully designed prompts, reliably predicts the presence or outcome of a downstream task or skill (Wang et al., 2022, Ackermann et al., 2023). For a frozen model with a trainable soft prompt $P = \{p_1,\ldots,p_l\}$, the activation $a(N, t, x)$ of neuron $N$ at token $t$ and input $x$ is compared to its baseline (mean across all training examples), yielding a per-token binary predictor. The neuron’s "predictivity" is the maximal accuracy or anti-accuracy over prompt tokens and multiple prompt initializations:

$$\mathrm{Pred}(N) = \frac{1}{R}\sum_{r=1}^R \max_{p \in P_r} \max \left( \mathrm{Acc}(N, p), 1 - \mathrm{Acc}(N, p) \right)$$

A neuron is declared a skill neuron for the task if it ranks among the top-K by Pred(N).

In alternative contexts, such as skill "unlearning" or probing, other operationalizations exist:

• By comparing per-neuron pre-activation means on skill (“forget”) vs. non-skill (“retain”) datasets, neurons with the largest $\Delta\mu^{(i)} = |\mu_f^{(i)} - \mu_r^{(i)}|$ are designated as skill neurons for the forgotten behavior (Li et al., 27 Mar 2025).
• Using neuron–metric correlation (Pearson or Spearman) between neuron activation and external labels or model-internal metrics (confidence, loss) isolates skill neurons in multi-skill or open-ended setups (Zhao et al., 26 Nov 2025).
• In spiking RNNs, “skill neurons” are structural units (e.g., Von Economo neurons, VENs) that provide a dedicated direct pathway critical for stable and reliable acquisition of social or classification tasks (Keskin, 17 May 2026).

2. Empirical Methodologies for Identification

The canonical methodology involves:

1. Freeze all model weights except the soft prompt; tune on the downstream task.
2. Aggregate per-neuron activations across a labeled dataset: compute baseline and per-class means.
3. Score neurons by predictivity or activation shifts: for classification, predictivity; for skill-unlearning, mean/variance shifts; for open-ended skills, activation–metric correlations.
4. Rank and select the top-performing neurons: typically the top 1–2% by score.
5. Validate via intervention: perturbations (e.g., zeroing, adding noise, shifting activations) to skill neurons produce marked decrements in measured skill, while random neuron perturbations do not (Wang et al., 2022, Li et al., 27 Mar 2025, Hiraoka et al., 2024).

Specialized approaches include:

• Auxiliary-metric probing: Rather than restricting to binary classification, correlate neuron activations with any external or internal metric, allowing analysis of multi-way, continuous, or latent skills (Zhao et al., 26 Nov 2025).
• Global linear controllability (Neuron Empirical Gradient, NEG): For each neuron–output pair, measure the functional gradient (NEG) of output probability w.r.t. input activations, allowing fine-grained skill neuron identification via linear regression or the NeurGrad estimator (Zhao et al., 2024).

The robustness of these methods is confirmed across architectures (RoBERTa, T5, Llama, Pythia), tasks (classification, arithmetic, social learning), and languages.

3. Functional and Causal Role

Skill neurons encode relevant decision boundaries or subtask logic. Key findings include:

• Causality: Ablating top skill neurons, measured by predictivity or other metrics, sharply degrades performance, demonstrating that these neurons are necessary, not merely correlated (Wang et al., 2022, Hiraoka et al., 2024, Doan et al., 10 Jul 2025).
• Task specificity and similarity: Correlations of skill neuron rankings are high between similar tasks (Spearman’s $\rho \approx 0.8$ for SST-2/IMDB), low across unrelated tasks ($\rho \approx 0.2$ SST-2/QNLI) (Wang et al., 2022).
• Robustness: The consistency of skill neuron activation under adversarial shift is associated with model robustness. T5 maintains activation of its skill neurons under adversarial data (Spearman $\rho$ up to 0.84), while RoBERTa does not (Ackermann et al., 2023).
• Depth distribution: In Transformer models, skill neurons often cluster in upper layers (task execution) with secondary peaks in intermediate layers (task detection); the same principles hold for repetition and induction subcircuits (Hiraoka et al., 2024, Doan et al., 10 Jul 2025).
• Structural necessity: In spiking RNNs, VENs—only 2% of the population—enable reliable gradient flow through a direct path, scaffolding acquisition and causing stochastic learning failure when removed (Keskin, 17 May 2026).

4. Circuitry and Interactions

Skill neurons operate in compositional or collaborative circuits:

Component	Mechanism	Functional Role
Deep-layer skill neurons	Direct influence	Executing class label, enforcing output (e.g., repetition)
Mid-layer skill neurons	Detection features	Recognizing n-gram echoes (e.g., pattern detection, recursion)
Attention heads (“induction”)	Copy/match input	Feeding pattern context to skill neurons
Projection (VENs in SNNs)	Non-recurrent path	Providing robust learning gradients, bypassing recurrence

Ablation analyses consistently show that late-layer skill neurons and top attn heads are necessary for pattern recall, with joint ablation leading to near-total task loss (Doan et al., 10 Jul 2025). The interdependency of skill neurons is supported by findings that tree-based probe classifiers leveraging NEG features outperform majority-vote baselines, reflecting complex coordinative effects (Zhao et al., 2024).

5. Skill Neuron Manipulation and Unlearning

Recent work introduces methods for surgical manipulation of skills:

• Neuron Adjust: At inference time, shift or zero activations of skill neurons (identified by mean pre-activation shift) to “forget” target skills (e.g., math-solving, coding) without retraining (Li et al., 27 Mar 2025).
• Key Space Detection: Monitor query post-activation (“key”) space for skill-specific clusters; abstain or abort output if detected, selectively removing skills with high specificity and minimal collateral loss.
• Performance: Both approaches yield over 80% drop in designated skill with <10% drop on unrelated skills and general knowledge (MMLU), vastly outperforming random or pruning-based ablations.

6. Applications in Model Control and Interpretability

Characteristic applications of skill neurons include:

• Network pruning: Retaining only top skill neurons per task enables aggressive parameter reduction (down to 66.6% of model size) with only modest accuracy loss and up to 1.4× speedup (Wang et al., 2022).
• Transferability estimation: Overlap of top skill neurons between tasks is a superior predictor of prompt transfer performance compared to full-neuron overlap metrics (Wang et al., 2022).
• Knowledge editing: By intervening only on the subset of skill neurons involved with a target skill, one can perform targeted model editing with considerably less collateral damage than weight-level approaches (Zhao et al., 2024).
• Interpretability and auditing: NEG and auxiliary-metric analysis offer interpretable “fingerprints” for tasks, heuristics, and shortcuts; neurons distilled by these methods often uncover latent or shortcut behaviors that were not explicitly labeled (Zhao et al., 26 Nov 2025, Zhao et al., 2024).

7. Theoretical and Clinical Implications

Skill neurons ground mechanistic hypotheses of learning and robustness:

• Origins: Multiple studies indicate that skill neurons are present in pre-trained models, not created de novo by prompt- or adapter-tuning, as evidenced by their persistence under tuning-free prompts and across distinct finetuning paradigms (Wang et al., 2022).
• Gradient flow in RNN/SNNs: Structural skill neurons (VENs) resolve the vanishing/exploding gradient problem near the critical boundary of recurrence, akin to providing “acquisition scaffolds” (Keskin, 17 May 2026).
• Clinical prediction: The VEN-account predicts that developmental reduction of VENs causes stochastic failure in skill acquisition (mimicking autistic heterogeneity), and that adult VEN lesions have variable acute impact (reflected in bvFTD phenotypes).
• Emergence and distribution: While a concentrated top-K subset holds principal skill control, knowledge and capability are broadly distributed, with sizable performance recapitulated even by less-predictive neuron groups (Zhao et al., 2024).

A plausible implication is that extending skill neuron methodology to other parameter classes (attention heads, layer norms) and modalities (vision, code) will yield new control and interpretability tools, while tracing the emergence of these structures may further elucidate the developmental dynamics of intelligent systems.

---

References:

• (Wang et al., 2022) Finding Skill Neurons in Pre-trained Transformer-based LLMs
• (Ackermann et al., 2023) On the Relationship between Skill Neurons and Robustness in Prompt Tuning
• (Hiraoka et al., 2024) Repetition Neurons: How Do LLMs Produce Repetitions?
• (Zhao et al., 2024) Neuron Empirical Gradient: Discovering and Quantifying Neurons Global Linear Controllability
• (Li et al., 27 Mar 2025) Effective Skill Unlearning through Intervention and Abstention
• (Doan et al., 10 Jul 2025) Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning
• (Zhao et al., 26 Nov 2025) Auxiliary Metrics Help Decoding Skill Neurons in the Wild
• (Keskin, 17 May 2026) Von Economo neurons enable reliable social skill acquisition in recurrent spiking neural networks: a computational account with clinical predictions