AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals

Abstract: Self-distillation enables LLMs to learn on-policy from their own trajectories by using the same model as both student and teacher, with the teacher being conditioned on privileged information unavailable to the student. Such information can come in different types or views, such as solutions, demonstrations, feedback, or final answers. This setup provides dense token-level feedback without relying on a separate external model, but creates a fundamental asymmetry: the teacher may rely on view-specific information that the student cannot access at inference time. Moreover, the best type of privileged information is often task-dependent, making it difficult to choose a single teacher view. In this work, we address both these challenges jointly by introducing AVSD (Adaptive-View Self-Distillation), a novel method of self-distillation with multiple privileged-information views, which reconstructs token-level supervision by separating stable cross-view consensus from view-specific residual signals. AVSD identifies the consensus signal shared across views, which provides a reliable update direction, and then selectively adds the view-specific residual signal to adjust the update magnitude when it both aligns with the consensus direction and remains proportionate to the consensus signal. Experiments on math competition benchmarks (AIME24, AIME25, and HMMT25) show that AVSD consistently outperforms both single-view self-distillation baselines and GRPO, achieving average Avg@8 gains of 3.1% and 2.2% over the strongest baselines on Qwen3-8B and Qwen3-4B, respectively. Moreover, on code-generation benchmarks (Codeforces, LiveCodeBench v6) using Qwen3-8B, AVSD outperforms the single-view self-distillation baseline by 2.4% on average.

• The paper introduces a multi-view self-distillation method that adaptively integrates consensus and residual signals from diverse privileged teacher views.
• It employs gated residual integration to combine geometric consensus and arithmetic marginal targets for stable token-level gradient signals.
• Empirical results on math and code benchmarks show consistent performance gains and enhanced reliability in token credit assignment.

Adaptive-View Self-Distillation: Balancing Consensus and View-Specific Privileged Information for On-Policy LLM Training

Introduction

The "AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals" paper (2605.20643) presents a principled approach for on-policy self-distillation in LLMs by leveraging multiple privileged teacher views. Unlike conventional self-distillation, which utilizes a single fixed form of privileged information during training, AVSD recognizes the heterogeneity and task-dependence of privileged signals (full solutions, partial reasoning, final answers, algorithmic hints, or execution feedback). The method employs a multi-view teacher aggregation framework to construct stable and informative token-level gradient signals, aiming to mitigate both information leakage and view-dependent bias.

Background and Motivation

Reinforcement learning with verifiable rewards (RLVR) is increasingly employed for post-training LLMs on challenging tasks in mathematics and code. RLVR approaches, while robust in terms of alignment, suffer from sparse and outcome-level rewards, requiring significant sampling for rare successes. To complement this, token-level distillation methods offer denser signals, but standard approaches typically face off-policy generalization gaps. On-policy distillation, specifically self-distillation using model rollouts, addresses this issue, but heavily depends on the selection of a single form of privileged information as teacher input. Empirical evidence demonstrates that no single privileged-view teacher is optimal across all tasks and benchmarks.

Figure 1: AVSD consistently yields superior performance by adaptively integrating multiple privileged teacher views, compared to using any single privileged view.

Recent research demonstrates that fixed-view privileged teachers can induce mutual information gaps between teacher and student and may even degrade reasoning performance due to mismatched or excessively specific guidance. Therefore, there is a strong incentive to systematically exploit the diversity of available privileged teacher views, aggregating their signals in a manner that maximizes reliability and minimizes overfitting.

Methodology

Multi-View Teacher Aggregation

AVSD defines a set $\mathcal{V}(r)$ of $M$ teacher views $r^{(m)}$, each corresponding to a different type of privileged information. Each teacher, realized with the same model architecture but conditioned on a different privileged view, produces its own token-level distribution $q_t^{(m)}(v)$ for each student prefix $h_t = (x, y_{<t})$.

AVSD synthesizes these into two key aggregate distributions:

• Geometric Consensus Target ($q_t^G$): Emphasizes cross-view intersection; a token receives high probability only if all teachers support it.
• Arithmetic Marginal Target ($q_t^A$): Aggregates marginal support across views; a token with strong support from at least one teacher can receive high probability.

From these, the consensus advantage $A_t^G(v)$ and arithmetic advantage $A_t^A(v)$ are defined, enabling decomposition into consensus and residual signals. The cross-view residual $J_t(v) = \log q_t^A(v) - \log \tilde{q}_t^G(v)$ quantifies how much extra support arises from outlier views.

Figure 2: AVSD's aggregation pipeline, including the separation of consensus and residual signals and the gating mechanism that ensures robust learning.

Gated Residual Integration

The central algorithmic contribution of AVSD is a gated residual integration: the training signal is primarily determined by the consensus direction $M$0 but can be adaptively modified by the residual $M$1, subject to a gate $M$2. The gate incorporates two terms:

• Alignment Component ($M$3): Measures agreement on update direction among teacher views.
• Magnitude Component ($M$4): Ensures residual amplification is proportionate to consensus strength; prevents residual from overwhelming or reversing consensus.

Thus, the final token-level advantage becomes

$M$5

The reconstructed teacher target $M$6 is produced by reweighting the geometric consensus with the gated residual. Training proceeds with a reverse-KL distillation objective using $M$7 for all student-generated prefixes, ensuring on-policy sample efficiency and credit assignment.

Empirical Evaluation

Experimental Setup

Experiments are conducted on multiple LLM backbones (Qwen3-4B, Qwen3-8B, and DeepSeek-R1-Distill-Qwen-7B) with evaluation on challenging math (AIME24, AIME25, HMMT25) and code generation (Codeforces, LiveCodeBench) benchmarks. For each (math or code) instance, three to four types of privileged views are constructed—e.g., for math: full solution, partial solution, final answer; for code: reference implementation, algorithmic hint, execution feedback.

Main Results

AVSD consistently improves Avg@8 across all tested domains and models relative to SFT, GRPO, and single-view On-Policy Self-Distillation (OPSD) baselines. For instance, on Qwen3-4B, AVSD achieves a 5.5% improvement over base models and outperforms the strongest baseline (OPSD) by 2.2%. On code benchmarks, AVSD yields 2.4% average gains on Qwen3-8B versus the strongest baseline.

Figure 3: AVSD achieves the best average performance across all problem sets compared to single-view and partial aggregation baselines.

View Scaling

AVSD demonstrates monotonic improvements as the number of privileged teacher views increases, with diminishing returns once the primary sources of information are represented—demonstrating robustness to inclusion of diverse privileged signals.

Figure 4: Math reasoning performance increases as more privileged views are included in AVSD's aggregation.

Figure 5: Complementary effect of scaling privileged code views; more views result in better final code reasoning performance.

Token-Level Analysis

Detailed token-level credit diagnostics show that AVSD provides more reliable, directionally correct feedback on student rollouts—especially important for erroneous trajectories. For instance, AVSD reduces the fraction of high-impact tokens receiving incorrect-sign updates to only 2.9%, compared with 12.3% for OPSD and 29.6% cases with no feedback for GRPO.

Figure 6: AVSD sharply reduces the prevalence of incorrect or missing token-level credit assignments, improving gradient signal reliability.

Ablation Study

Ablating consensus-only and arithmetic-only variants verifies that neither standalone pooling yields optimal results. The gated integration mechanism is essential; consensus-only targets are overly conservative, arithmetic-only variants are prone to overfitting to privileged artifacts. Only AVSD's integrated approach achieves uniform superiority.

Theoretical Perspective and Related Work

AVSD's multi-view aggregation framework is motivated by the geometry of KL aggregation and classic opinion pooling (arithmetic/logarithmic pooling, weighted KL barycenters). Its bounded-residual gating addresses known pitfalls of privileged information distillation, such as view-induced information asymmetry and leakage, as identified by (Yang et al., 3 Apr 2026) and (Kim et al., 25 Mar 2026). The approach stands in contrast to both traditional RLVR (sparse rewards) and standard ensemble/logit-averaging methods, which either under-utilize available dense privileged feedback or are insufficiently sensitive to cross-view disagreement.

Implications and Future Directions

Practically, AVSD substantially enhances reasoning performance on tasks where multiple forms of training-time privileged information exist—even if no single one is optimal. The consistent gains in math and code reasoning, reliability of token-level credit assignment, and modest computational overhead (only a 13% increase over single-view self-distillation) position AVSD as a robust post-training protocol for LLMs in verifiable and open-ended reasoning domains.

Theoretically, AVSD offers a blueprint for robust aggregation in self-distillation pipelines, motivating further study into:

• Extension to broader modalities (multimodal, tool-use, or interactive agents) where privileged views underpin divergent reasoning strategies.
• Alternative divergence objectives and non-uniform (reliability-weighted) view integration.
• Formal analysis of mutual-information minimization under information asymmetry.

Conclusion

AVSD formalizes a method for multi-view self-distillation by strictly separating consensus from residual signals and adaptively gating their contribution to the student update. Empirical and analytic results confirm that this approach systematically outperforms fixed-view or simple pooled baselines, leading to more reliable and effective LLM training on complex reasoning tasks. The results have both immediate practical applicability in math and code LLMs, as well as strong theoretical implications for the design of privileged-information learning in future generalist LLM pipelines.