Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

Abstract: Post-training LLMs without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain. Recent annotation-free self-evolution methods address this by using the model's own outputs as supervision signals, constructing a teacher via additional context and aggregating predictions across multiple rollouts through majority voting to produce pseudo-labels. However, these approaches are not without drawbacks: SFT- and GRPO-based variants suffer out-of-domain performance degradation, while reward-based on-policy RL inflates calibration error. In this paper, we propose Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric framework for annotation-free self-distillation that leverages internal neuron activations to guide both training-data selection and teacher context construction. The model is then trained via on-policy distillation from the teacher distribution, requiring no ground-truth labels at any stage. Across specialized-domain benchmarks, Neuron-OPSD improves in-domain task performance while preserving cross-domain generalization and mitigating calibration collapse over prior annotation-free baselines. This framework is particularly relevant to settings where online interaction or external supervision is costly or infeasible, and is conceptually distinct from offline RL approaches that rely on logged, reward-labeled trajectories.

• The paper introduces Neuron-OPSD, a novel, annotation-free self-distillation method that leverages internal neuron activations for guiding model improvement.
• It employs neuron consensus for reliable sample selection and neuron overlap for constructing few-shot teacher contexts, enhancing token-level supervision.
• Empirical results show improved calibration, cross-domain generalization, and accuracy over traditional reward-based self-training approaches.

Neuron-OPSD: A Neuron-Aware, Annotation-Free LLM Self-Distillation Framework

Introduction

Post-training LLMs to adapt to domain-specific tasks in the absence of external feedback or human-annotated data introduces significant challenges, particularly in specialized domains where expert supervision is expensive or infeasible. Prior annotation-free self-evolution methods, including strategies based on Supervised Fine-Tuning (SFT), Generalized Reward Policy Optimization (GRPO), and on-policy RL with rewards derived from model self-consistency, have partially mitigated this problem. However, these approaches exhibit out-of-domain generalization deficiencies and can fail to maintain calibration, amplifying spurious reasoning and hallucinations.

The paper "Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation" (2607.02460) introduces Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric, annotation-free self-distillation framework that leverages internal model neuron activations for both training sample selection (Neuron Consensus) and few-shot context construction in the teacher (Neuron Overlap). This framework operates through on-policy distillation, guiding the student model using the teacher distribution—conditioned solely on the model's own retrieved context—without reliance on any ground-truth labels.

Figure 1: Overview of the Proposed Neuron-OPSD method. Neuron consensus and neuron overlap mechanisms guide annotation-free data selection and teacher construction, respectively.

Neuron-OPSD Framework

Neuron-OPSD adopts a four-stage pipeline:

1. Neuron Activation Extraction: For each sample in the unlabeled pool, compute a set of "active" neurons whose early-unembedding contributions reflect the neurons most responsible for generating the output, using top-K selection across MLP layers.
2. Sample Selection via Neuron Consensus: Data reliability is estimated using neuron consensus, defined as the cardinality $s(x) = |\mathcal{N}(x)|$ of the activated-neuron set. Samples with lower $s(x)$ (stronger consensus) are inferred to be less prone to hallucination, as empirically validated by their stronger correlation with correct predictions across SciKnowEval domains.
3. Context Construction via Neuron Overlap: For each training query, the few-shot teacher context is built by selecting neighbor samples with minimal Jaccard distance between their neuron activation patterns and the query—encoding similar reasoning paths.
4. On-Policy Self-Distillation: On-policy reverse KL minimization is performed between the student (zero-shot) and teacher (context-augmented, EMA-updated) distributions. This token-level, dense supervision facilitates more stable learning than reward-based RL approaches typical in annotation-free regimes.

Empirical Analysis of Neuron-Based Signals

Neuron Consensus as a Hallucination Signal

Analysis on SciKnowEval reveals a robust, monotonic relationship between normalized neuron activation count and prediction correctness. Queries that activate more neurons tend to be incorrect and likely hallucinated. This property, validated across scientific domains, enables neuron consensus to serve as an effective, annotation-free diagnostic for filtering unreliable samples.

Figure 2: Relational regression between normalized neuron activation count and correctness (ΔAcc) across SciKnowEval domains.

However, the utility of this signal for training is non-monotonic. While high-consensus (low activation count) samples provide reliable supervision, their marginal contribution to self-improvement is limited. In contrast, low-consensus (high activation count) samples, while less reliable, can offer higher learning value but risk amplifying noise and hallucinations in an annotation-free regime.

Neuron Overlap for Teacher Context Construction

To construct context for the on-policy teacher, neuron-overlap retrieval identifies few-shot examples exhibiting maximum neuron set similarity to each query. This retrieval is most effective in domains where samples are processed through distinct neural pathways—effectively clustering reasoning modes. In Chem, retrieved examples are thematically coherent and tightly clustered in activation space; in Phys, neuron-overlap retrieval offers little benefit relative to random selection due to homogeneous neuron usage across diverse problems.

Comparative Evaluation

Across SciKnowEval, Edu-Feedback, and MMLU-Pro, Neuron-OPSD is compared against LMSI (SFT-based), TTRL (GRPO-based), and Intuitor (entropy/confidence-based RL) baselines.

Performance

• In-domain: Neuron-OPSD yields significant accuracy improvements in domains where the neuron-overlap teacher context sharpens the token distribution (Mat., Phys., Edu.), matching or exceeding TTRL.
• Cross-domain: Neuron-OPSD preserves performance better than reward-based methods, which often degrade generalization (see LMSI, Intuitor).
• Calibration: Neuron-OPSD consistently maintains or improves Expected Calibration Error (ECE), particularly on cross-domain transfer and difficult validation sets, in contrast to TTRL and Intuitor, which inflate ECE.

Ablations and Analysis

• Sample Selection: Training on bottom-20% (high consensus) and top-20% (low consensus) subsets shows that neither extreme is universally optimal, highlighting the trade-off between reliability and learning utility.
• Context Retrieval: Neuron-Jaccard retrieval outperforms random demonstration selection in domains with diverse reasoning patterns, confirming the utility of neuron-level similarity as a retrieval signal.
• Token-Level Teacher-Student Gap: The degree of entropy reduction (teacher sharper than student) aligns with the magnitude of self-improvement, highlighting that effective teacher-student divergence—achieved through neuron-overlap context—is necessary for improvement.

Theoretical and Practical Implications

Theoretical Implications: Neuron-OPSD exemplifies a paradigm shift from output-based, reward-or-uncertainty-driven data selection toward leveraging synaptic-level model interpretability as a signal for intrinsic reliability and structural similarity. This neuron-level basis for annotation-free distillation connects interpretability research (e.g., model utility laws and neuron agreement (Cao et al., 10 Apr 2025, Chen et al., 30 Oct 2025)) to practical self-improvement, and offers a mechanistic understanding of how LLMs internalize knowledge during post-training.

Practical Implications: This approach can be deployed in high-value domains where annotation is expensive or infeasible, enabling continuous self-improvement of LLMs without access to external labels or reward signals. It offers a robust alternative to reward-based approaches, mitigating calibration collapse while preserving cross-domain generalization—critical for model deployment at scale.

Limitations and Future Directions:

• Gains are not uniform across all domains; teacher-student token-level entropy gap is critical, and improvements vanish when teacher distributions add no information.
• Neuron Consensus, while correlating with reliability, is insufficient as a singular rule for data selection.
• The discriminative power of neuron-overlap retrieval is domain dependent and may collapse in homogeneous reasoning spaces.
• The proposed framework has been evaluated only on 4B-parameter models; scaling to larger architectures and more complex tasks requires further investigation.

Conclusion

Neuron-OPSD introduces a neuron-aware, annotation-free self-distillation framework that sets a new direction for data selection and context construction in LLM self-training. By tightly coupling model interpretability with self-supervised post-training, the method demonstrates state-of-the-art in-domain gains, robust cross-domain preservation, and strong calibration—in sharp contrast with prior reward-based annotation-free approaches. This work highlights the potential for mechanistically-informed intrinsic signals to drive reliable, scalable, and data-efficient self-improvement in large-scale LLMs.