Privileged-Context Self-Distillation

• Privileged-context self-distillation (PCSD) is a training technique where a teacher, conditioned on added context like solution traces or multimodal rationales, provides token-level guidance to an on-policy student.
• It leverages various privileged inputs—such as verified answers, evidence, or safety constitutions—to align the teacher’s supervision with the states the student encounters during inference.
• Recent studies highlight that PCSD enhances efficiency, robustness, and safety across domains including math reasoning, document QA, and affective computing.

to=arxiv_search.search 公众号天天中彩票 ,超碰 json {"query":"\"privileged context\" self-distillation on-policy 2026", "max_results": 10} to=arxiv_search.search 天天购彩票 ӡамാത്ര to=arxiv_search.search 彩神争霸提现 _天天啪 สำนักเลขานุการองค์กร json {"query":"OPSD privileged context self-distillation 2026", "max_results": 10} to=arxiv_search.search ಪ್ರಜಾವjson {"query":"OmniOPSD rationale-privileged on-policy self-distillation affective computing", "max_results": 5} Privileged-context self-distillation (PCSD) is a training paradigm in which a model learns on its own on-policy trajectories while a teacher, typically the same backbone conditioned on training-time-only privileged context, provides dense token-level supervision. The privileged context may be a verified solution trace, a concise hint, a relevant document, a multimodal rationale, a safety constitution, a question construction path, or other signals unavailable at deployment. Across recent work, the central design choice is constant: the student is optimized under inference-time conditions, while the teacher is strengthened only by conditioning, not by a separate deployment interface (Zhao et al., 26 Jan 2026, Cheng et al., 14 Jun 2026).

1. Definition, scope, and lineage

Within the recent literature, PCSD is the broader family name for on-policy distillation methods in which privileged context is visible only to the teacher. OPSD is the canonical answer-conditioned instance: the student sees only the original problem or prompt, while the teacher sees the same prefix plus privileged information and supervises the student along the student’s own rollout. Several papers explicitly place this formulation in the LUPI tradition, but adapt it to autoregressive token-level alignment rather than classical feature regression (Huang et al., 27 May 2026, Cheng et al., 14 Jun 2026).

The idea predates the current LLM wave in weaker or non-autoregressive forms. In learning-to-rank, privileged features distillation trains a teacher on regular and privileged features and then distills into a student that only uses regular features; the reported behavior is non-monotone, because increasingly predictive privileged features can eventually increase student variance and hurt test performance (Yang et al., 2022). In multimodal expression recognition, MT-PKDOT transfers training-only modalities through a multi-teacher privileged distillation pipeline with entropy-regularized optimal transport and centroid alignment, again separating train-time information from test-time availability (Aslam et al., 2024).

Representative privileged contexts used in recent PCSD systems are summarized below.

Paper	Privileged context	Student task
"Self-Distilled Reasoner" (Zhao et al., 26 Jan 2026)	Verified solution trace	Math reasoning
"OmniOPSD" (Cheng et al., 14 Jun 2026)	Evidence-aware multimodal rationale	Affective computing
"GATES" (Stein et al., 24 Feb 2026)	Relevant document	Document-grounded QA
"PTD-PO" (Xiang et al., 5 Jun 2026)	Structured privileged hints	Multimodal policy optimization
"$\pi$-Play" (Zhang et al., 15 Apr 2026)	Question construction path	Search agents
"Constitutional On-Policy Safe Distillation" (Wen et al., 2 Jun 2026)	Safety constitution	Safety alignment

This range of contexts shows that PCSD is not tied to answer supervision alone. It spans answer-bearing traces, evidence-bearing hints, asymmetric documents, environment feedback, constitutions, and latent artifacts produced by self-play.

2. Canonical training formulation

The standard OPSD formulation uses two conditionals over the same sampled prefix: a student policy $p_S(\cdot \mid x, y_{<t})$ and a privileged teacher policy $p_T(\cdot \mid x, c, y_{<t})$. The student samples an on-policy rollout $y \sim p_S(\cdot \mid x)$, and training minimizes a token-level divergence along that rollout:

$$J(\theta) = \mathbb{E}_{x \sim \mathcal{D},\, y \sim p_S(\cdot \mid x)} \left[ \sum_{t=1}^{T} D\!\big( p_T(\cdot \mid x,c,y_{<t}) \;\|\; p_S(\cdot \mid x,y_{<t}) \big) \right].$$

In "Self-Distilled Reasoner", teacher and student share parameters and differ solely in conditioning; gradients flow only through the student, and the teacher is a stop-gradient target (Zhao et al., 26 Jan 2026).

Several implementations specialize this template. OmniOPSD constructs two prompts per example, $c_i^S=\mathrm{Prompt}_S(m_i,x_i)$ and $c_i^T=\mathrm{Prompt}_T(m_i,x_i,r_i)$, samples $\hat y_i$ from the student, and aligns student and teacher with a generalized Jensen–Shannon divergence over completion tokens. It also allows an optional reward-grounded hybrid term,

$$L_{\mathrm{OmniOPSD}}=L_{\mathrm{distill}}+\alpha L_R,$$

and updates the teacher by EMA,

$$\bar{\theta}_{k+1}\leftarrow \mu \bar{\theta}_k + (1-\mu)\theta_{k+1}.$$

In the reported experiments, $p_S(\cdot \mid x, y_{<t})$0, $p_S(\cdot \mid x, y_{<t})$1, $p_S(\cdot \mid x, y_{<t})$2, and $p_S(\cdot \mid x, y_{<t})$3 (Cheng et al., 14 Jun 2026).

Other systems modify the divergence or the decomposition of the update. SDPG makes the self-distillation term an exact full-vocabulary reverse KL from student to privileged teacher, then combines it with verifier-grounded group-relative advantages and a reference-policy KL anchor (Liu et al., 2 Jun 2026). GATES combines off-policy trajectory imitation from a privileged tutor with an on-policy advantage-weighted term, but gates both by online tutor consensus rather than assuming tutor correctness (Stein et al., 24 Feb 2026). GeoSD replaces KL-style matching with a Hellinger pull and a Fisher–Rao proximal term, then performs a natural-gradient update in the geometry of predictive distributions (Jukić et al., 7 Jul 2026).

The shared rationale for on-policy supervision is explicit across these papers: training on the student’s own visitation distribution avoids the exposure bias of fixed reference trajectories and aligns the teacher signal with the states the student will actually encounter at inference (Zhao et al., 26 Jan 2026, Cheng et al., 14 Jun 2026).

3. Privileged context as answers, evidence, boundaries, and world feedback

A first major axis of variation is what counts as privileged context. In mathematical reasoning, the canonical choice is a verified solution trace or final answer. "Self-Distilled Reasoner" uses ground-truth solution traces as privileged context; HDPO appends the verified final answer to cliff prompts and filters privileged generations with $p_S(\cdot \mid x, y_{<t})$4; SDPG conditions the teacher on the correct final answer and a worked solution path generated by Gemini 2.5 Pro (Zhao et al., 26 Jan 2026, Ding, 25 Mar 2026, Liu et al., 2 Jun 2026).

A second family uses evidence-bearing but answer-free context. PTD-PO constructs structured privileged hints from spatial attention guidance and intermediate textual reasoning steps under a zero-spoiler rule, then routes distillation only to failed trajectories in RLVR training (Xiang et al., 5 Jun 2026). OmniOPSD uses GPT-4o-generated evidence-aware multimodal rationales as teacher-side privileged evidence rather than as student imitation targets; the student never imitates rationale text and never requires rationales at inference (Cheng et al., 14 Jun 2026). This separation between evidence enrichment and target imitation is one of the most distinctive features of the recent multimodal literature.

A third family uses asymmetric context that is not an answer at all. GATES gives the tutor a relevant document during training and distills full tutor reasoning trajectories into a document-free student (Stein et al., 24 Feb 2026). COPSD conditions the teacher on a safety constitution, where the privileged signal defines boundaries to avoid rather than a correct target response (Wen et al., 2 Jun 2026). The predictive-law paper studies privileged context constructed from world feedback, expert preambles, peer hints, own solution plus feedback, and peer solution plus feedback, and reports a linear relationship between the initial student–self-teacher gap and the final OPSD improvement across these context types (He et al., 28 May 2026).

A fourth family uses privileged context that is generated by the training loop itself. In $p_S(\cdot \mid x, y_{<t})$5-Play, self-play yields a question construction path, and that reverse-solution artifact becomes the training-only context for the teacher (Zhang et al., 15 Apr 2026). SGSD weakens the privileged-information assumption further: teachers are conditioned on retrieved skill–mistake pairs rather than trusted answers, and their stance is validated by a verifier before any token-level signal is used (Huang et al., 27 May 2026). EDGE-OPD uses personas, private facts, or worked solutions as privileged context, but shows that support injection and evidence masking are necessary when the privileged information changes local token preferences more than intended (Lazaridis et al., 22 May 2026).

4. How recent methods modulate teacher pressure

Recent work increasingly treats vanilla OPSD as insufficiently selective. One line of research keeps the privileged teacher, but anchors direction in world feedback rather than in the teacher alone. RLSD computes a token-level log-probability difference

$p_S(\cdot \mid x, y_{<t})$6

builds a weight

$p_S(\cdot \mid x, y_{<t})$7

clips it, and uses it only to modulate the magnitude of a verifier-grounded advantage, not its sign (Yang et al., 3 Apr 2026). HDPO reaches a similar hybridization from the opposite direction: it activates privileged self-distillation only on cliff prompts where GRPO provides no learning signal, and uses privileged rollouts filtered by correctness (Ding, 25 Mar 2026). SDPG similarly combines verifier-grounded policy gradients, privileged reverse-KL self-distillation, and reference-policy regularization (Liu et al., 2 Jun 2026).

A second line filters or gates teacher influence locally. EDGE-OPD uses guided rollouts so that rare privileged behavior actually appears in on-policy data, then applies a hard evidence mask $p_S(\cdot \mid x, y_{<t})$8 so that only positions positively supported by the privileged context are updated (Lazaridis et al., 22 May 2026). GATES gates entire questions by tutor consensus and only distills trajectories whose final answer agrees with the modal tutor answer (Stein et al., 24 Feb 2026). SGSD aggregates multiple skill-conditioned teachers, validates each teacher’s polarity against the verifier outcome, and uses a bounded gated loss so that harmful or uncertain teacher stances are reversed or ignored rather than imitated (Huang et al., 27 May 2026).

A third line uses uncertainty or geometry to decide whether the student should move toward or away from the privileged teacher. EGRSD multiplies reward-grounded direction and teacher–student likelihood ratio by a teacher-entropy confidence gate,

$p_S(\cdot \mid x, y_{<t})$9

while CL-EGRSD replaces instantaneous entropy with a causal lookahead minimum to rescue transient pivots (Ke et al., 13 May 2026). DASD goes further and makes the sign itself entropy-routed:

$p_T(\cdot \mid x, c, y_{<t})$0

so that low-entropy scaffolding tokens are attracted toward the privileged teacher and high-entropy forks are repelled from it (Zhang et al., 21 May 2026). AntiSD formalizes the same failure mode through conditional PMI and replaces descent of a student–teacher divergence with ascent of Jensen–Shannon divergence, producing a token-level term $p_T(\cdot \mid x, c, y_{<t})$1 and disabling it when teacher entropy collapses (Shen et al., 12 May 2026). GeoSD addresses unsupported teacher pulls with a Hellinger loss, then controls cumulative drift with a Fisher–Rao proximal term to a recent checkpoint (Jukić et al., 7 Jul 2026).

A plausible implication is that recent PCSD work is shifting from unconditional teacher matching toward mechanisms that decide where, when, and even in which direction the privileged teacher should matter.

5. Empirical results across domains

Reported gains are domain-specific, and the strongest papers emphasize different axes: accuracy, token efficiency, coverage, robustness, or safety–helpfulness trade-off.

Domain	Representative method	Reported result
Math reasoning	"Self-Distilled Reasoner" (Zhao et al., 26 Jan 2026)	4–8x token efficiency vs. GRPO; Qwen3-8B average 52.2 vs. GRPO 51.3
Affective computing	"OmniOPSD" (Cheng et al., 14 Jun 2026)	MER-UniBench mean 84.19; +4.71 over AffectGPT-R1
Document-grounded QA	"GATES" (Stein et al., 24 Feb 2026)	In-domain 46.0% to 62.0%; public benchmark average 20.2 to 35.4
LVLM RLVR	"PTD-PO" (Xiang et al., 5 Jun 2026)	2B: 61.21 vs. GRPO 50.63; 4B: 71.23 vs. 68.06; 8B: 71.86 vs. 68.78
Search agents	"$p_T(\cdot \mid x, c, y_{<t})$2-Play" (Zhang et al., 15 Apr 2026)	2–3× improvement in evolutionary efficiency; Qwen3-4B-Instruct-2507 total EM 277.2 vs. Dr.Zero 266.3
OOD reasoning	"GeoSD" (Jukić et al., 7 Jul 2026)	OOD accuracy +5.7–8.6 over base; forward KL drops OOD by 4.6–8.1

Efficiency and coverage are often as important as end accuracy. AntiSD reaches the GRPO baseline’s accuracy in 2 to 10x fewer training steps and improves final accuracy by up to 11.5 points across five math-reasoning models from 4B to 30B (Shen et al., 12 May 2026). HDPO improves pass@4 by +0.8–1.1% and pass@8 by +0.4–1.7% on OpenMathInstruct-2 while maintaining greedy accuracy, explicitly targeting cliff prompts rather than average-case prompts (Ding, 25 Mar 2026). EGRSD and CL-EGRSD advance the accuracy-length frontier among the compared trainable methods; on Qwen3-8B, CL-EGRSD reaches Avg 69.32, and on Qwen3-4B, EGRSD reaches the highest reported token efficiency among the trainable methods in that comparison (Ke et al., 13 May 2026).

The empirical record is therefore not a single monotone trend. Some methods improve average correctness, some improve OOD retention, some improve pass@$p_T(\cdot \mid x, c, y_{<t})$3 or evolutionary efficiency, and some primarily reduce the safety tax on general reasoning.

6. Failure modes, controversies, and open problems

A central controversy is whether privileged teacher matching is well posed at all. RLSD argues that direct privileged distribution matching contains an irreducible mutual-information gap,

$p_T(\cdot \mid x, c, y_{<t})$4

so the student cannot remove the part of the teacher distribution that genuinely depends on hidden privileged information. In that analysis, privileged information leakage and unstable long-term training are structural rather than incidental (Yang et al., 3 Apr 2026).

A second debate concerns strong thinking models. "Rethinking On-Policy Self-Distillation for Thinking Models" reports that privileged-context distillation causes a relative drop of up to 17% in avg@16 accuracy across five Qwen3 and OLMo thinking models, with harm concentrated at long rollout budgets and high-entropy forks. The reported mechanism is suppression of exploration, verification, backtracking, and hedging markers. AntiSD reaches a closely related conclusion through a PMI analysis: verified solutions inflate teacher confidence on structural connectives and verifiable claims while deflating it on deliberation tokens such as “Wait,” “Let,” and “Maybe” (Kaur et al., 6 Jul 2026, Shen et al., 12 May 2026).

A third debate is specific to safety alignment. COPSD argues that constitutional conditioning contracts the teacher distribution toward short and overly conservative responses, and that Reverse KL amplifies this contraction into reduced expressiveness. The paper formalizes the effect as geometric leakage in a non-orthogonal semantic space, then uses Cross-SFT teacher calibration to recover a stronger safety–helpfulness trade-off (Wen et al., 2 Jun 2026). AMR-SD reports a related failure mode in reasoning RL: direct exposure to raw privileged context can induce over-conditioned teacher distributions, implicit answer leakage, and late-stage collapse, which motivates its reflection bottleneck and asymmetric CIG gating (Wei et al., 18 May 2026).

A fourth issue is privileged-context quality. OmniOPSD states directly that guidance depends on the quality and impartiality of frontier-generated rationales, and that biased or incorrect evidence may misdirect teacher scoring; GATES warns about spurious consensus when tutor rollouts are systematically biased; SGSD emphasizes that retrieved skills can be irrelevant, stale, or misleading, so teacher hypotheses must be validated rather than imitated (Cheng et al., 14 Jun 2026, Stein et al., 24 Feb 2026, Huang et al., 27 May 2026).

Taken together, these results suggest that PCSD is most reliable when privileged context is informative but not overpowering, when teacher pressure is routed or bounded rather than applied uniformly, and when deployment-time autonomy of the student is preserved. The open problem is no longer merely how to add privileged context, but how to convert it into token-level supervision without collapsing exploration, diversity, or expressiveness.