RCNs: Critical Neurons in Deep Models

• RCNs are sparse, critical neurons identified by contrasting activation patterns between correct and incorrect reasoning traces in deep models.
• They are extracted using polarity-aware mean-difference methods and contrastive trace analysis that quantify how neuron activity predicts reasoning success.
• Targeted interventions on RCNs improve model inference reliability, explanation quality, and cross-task transfer without needing retraining.

Reasoning-Critical Neurons (RCNs) are defined as the small, sparse subset of neural units in deep models—particularly LLMs and vision networks—whose activation patterns exhibit the strongest predictive correlation with the correctness or faithfulness of model reasoning. Recent research demonstrates that these units can be systematically identified, that their activity is selectively aligned with correct multi-step inference, and—crucially—that targeted interventions on this subset can improve model inference reliability and explanation quality without retraining or architectural modification (Dong et al., 27 Jan 2026, Yamauchi et al., 8 Dec 2025, Tezuka et al., 21 Sep 2025).

1. Formal Definition and Identification Criteria

RCNs are those neurons whose mean activation across reasoning trajectories most robustly discriminates between (a) correct and (b) incorrect outputs. In the context of LLMs, given a dataset of prompts with correct answers $D=\{(x_i, y_i)\}$, the model $\mathcal{M}$ produces a reasoning trace culminating in response $a_i$, with indicator $c_i = \mathbb{I}[a_i = y_i]$. For each neuron $(l, i)$, define the averaged token activation over a trajectory $p$ as

$$\mu(a_i^l, p) = \frac{1}{|p|} \sum_{t \in p} a_i^l(t)$$

where $a_i^l(t)$ is the activation at token position $t$. The raw importance score is the polarity-aware mean-difference between correct ($p_k^+$) and incorrect ($p_k^-$) traces: $$S(l,i) = \mathbb{E}_{k} [ \mu(a_i^l, p_k^+) - \mu(a_i^l, p_k^-) ]$$ Neurons are retained as RCNs if their correct/incorrect means have opposite sign: $$\mathbb{E}_k[\mu(a_i^l, p_k^+)] \times \mathbb{E}_k[\mu(a_i^l, p_k^-)] < 0$$ The top-$K$ neurons per layer (or globally) are selected for intervention or interpretability analysis (Dong et al., 27 Jan 2026).

In image classifiers, RCNs are the subset of hidden units at a specific layer whose contribution—measured via Integrated Gradients or similar techniques—is maximally positive toward the predicted logit, reflecting the “reason” for the prediction (Yamauchi et al., 8 Dec 2025).

2. Practical Methodologies for Extraction and Validation

Identification of RCNs is grounded in systematic comparison of activation statistics between successful and unsuccessful reasoning episodes. In LLMs, this involves:

• Sampling contrastive trace pairs—correct and incorrect—on identical prompts at varied sampling temperatures.
• Computing per-neuron polarity-aware activation scores and filtering by sign consistency.
• Training or employing simple probes (e.g., logistic regression over last-token activations) to quantify the predictivity of neuron clusters for downstream correctness (e.g., AUROC ≈ 0.76 in Qwen3-1.7B on AIME) (Dong et al., 27 Jan 2026).

For vision models, the TEXTER approach leverages Integrated Gradients to assign each neuron at a candidate layer a contribution score toward the predicted class, selecting the TopK as RCNs. A TopK sparse autoencoder can be trained to render features more disentangled and thereby the explanation more interpretable (Yamauchi et al., 8 Dec 2025).

3. Functional Role and Experimental Evidence

RCNs have been empirically demonstrated to encapsulate a neural “reasoning correctness” signal:

• In AdaRAS, steering activations of identified RCNs during inference—only when a shallow gate predicts likely failure—substantially raises reasoning reliability on mathematics and coding benchmarks, with accuracy improvements exceeding 13 percentage points on AIME-24/AIME-25 versus the baseline (Dong et al., 27 Jan 2026).
• The effect is specific: enhancements are observed when intervening on high-score, polarity-filtered neurons, but not when randomly ablating or steering the same-sized subsets.
• In vision models (ResNet, ViT), explanations conditioned on RCN activity (rather than global features) yield textual rationales more faithful to the actual decision mechanism, as verified by semantic alignment and classifier reconstruction metrics (Yamauchi et al., 8 Dec 2025).
• In multilingual LLMs, “transfer neurons” (a functional analogue to RCNs) are shown to be essential for aligning language-specific representations into a shared “reasoning” latent space, with their ablation resulting in decreased multilingual QA accuracy and latent misalignment (Tezuka et al., 21 Sep 2025).

4. Architectural and Theoretical Insights

A consistent finding is that RCNs are sparsely distributed and layer-dependent. For LLMs:

• RCNs are most concentrated in mid-to-late layers, implicating deep, feed-forward subspaces in carrying global logical signals.
• Polarity filtering reveals that very few neurons exhibit robust, task-general sign flips tied to reasoning success or failure, supporting the hypothesis of a compact “correctness detector.”
• Interventions do not degrade performance on already-correct outputs, due to selective gating, but have a positive impact when initialized on likely-failure cases (Dong et al., 27 Jan 2026).

In vision models, application of sparse autoencoders to hidden activations (prior to RCN extraction) increases both faithfulness and interpretability of resultant explanations, as tangled neuron representations are made more axis-aligned and modular (Yamauchi et al., 8 Dec 2025).

5. Applications to Reliability, Interpretability, and Transfer

The practical upshot of RCN identification is twofold:

• Reliability: AdaRAS, a test-time activation-steering framework, uses RCNs to adapt internal model computation mid-inference, increasing the proportion of correct answers without retraining, additional sampling, or distorting model behavior elsewhere. Gains are robust across benchmarks and model scales (Dong et al., 27 Jan 2026).
• Interpretability: In vision-language settings, isolating RCNs enables generation of “concept images”—synthetic stimuli activating only the decision-critical units—whose semantic projection into CLIP space yields compact, faithful rationales matching the classifier's internal evidence (Yamauchi et al., 8 Dec 2025).
• Transferability: RCNs identified with one dataset (e.g., mathematics benchmarks) can be reused to steer model inference or probe correctness signals on other tasks, with observed cross-dataset improvements (Dong et al., 27 Jan 2026). In multilingual models, certain “transfer neurons” among RCNs are language-agnostic, facilitating alignment into a common reasoning space (Tezuka et al., 21 Sep 2025).

6. Limitations and Distinctions from Related Notions

No formal notion of “Reasoning-Critical Neurons” appears in prior concept-construction networks such as Blazek & Lin’s Essence Neural Network framework (Blazek et al., 2020). There, unit types are defined by their logical roles (differentia, subconcept, concept neurons), and no polarity or mean-difference analysis of reasoning trace activations is performed; all claims and ablations are at the cell-type or network level, not for a separate RCN class.

While “decision-critical” features and “transfer neurons” serve analogous functional roles to RCNs in other modalities or tasks, only recent LLM studies provide explicit, polarity-based mean-difference and activation steering methodology for post hoc reliability or interpretability enhancement.

7. Broader Implications and Future Directions

Ongoing research on RCNs refines mechanistic understanding of deep model cognition and offers practical tools for both inference control and explanation. The framework is agnostic to training paradigm, model scale, or data domain, as long as model activations reflecting reasoning success can be probed. A plausible implication is that RCN-based intervention and analysis could generalize to any architecture where global logical consistency depends on a small set of identifiable, correct-path-aligned neural features. Future directions include systematic evaluation for adversarial robustness, finer-grained temporal (per-token) analysis, and integration with model editing or feature-attribution pipelines.