Teacher-Side Defense Template

• Teacher-side defense templates are interventions that modify a teacher's outputs and supervisory workflows to control the quality, structure, and admissibility of information passed to students.
• They encompass techniques such as defensive distillation, bias correction, output perturbation, and selective answer sampling to improve adversarial robustness and model performance.
• In educational settings, these templates also include reflective tools and structured professional development programs to enhance pedagogical fidelity and teacher accountability.

A teacher-side defense template is a class of intervention that moves protection upstream to the teacher: the defense operates on the supervisory signal, the teacher’s output distribution, the teacher’s intervention policy, or the teacher-facing workflow that governs how supervision is produced and justified. Across the cited literature, this idea appears in at least two closely related forms. In machine learning, it denotes methods that alter what a teacher model exposes to a student or attacker, including distillation chains, bias-corrected soft targets, output perturbation, answer selection, pseudo-label generation, and value-based intervention (Mandal et al., 2023, Kim, 23 Jun 2025, Allouah et al., 21 May 2026, Hu et al., 26 May 2026). In educational research, it denotes structured teacher-facing instruments that help teachers justify, reflect on, and improve practice, including reflective diaries, worked-solution design frameworks, professional-development programs, and benchmarks for pedagogical fidelity and teacher-specific safety (Ukrop et al., 2018, Lin et al., 2016, Fies et al., 2021, Jiang et al., 10 Nov 2025).

1. Conceptual scope and theoretical basis

The unifying premise is that the teacher is not a neutral source of labels. In adversarial-robustness theory, robustness is presented as “a relative concept conditioned on a certain teacher,” and adversarial examples are said to exist because “the student cannot obtain sufficient information of the teacher from the training data” (Ma et al., 2020). This shifts attention from the student alone to the information channel between teacher and student. Under that view, a teacher-side defense does not merely harden a final classifier; it changes the supervisory geometry that the student is allowed to absorb.

This perspective reappears in distillation research under several different names. Long-Tailed Knowledge Distillation treats class imbalance as a pathology in teacher supervision rather than only a classification-loss problem, and Student-Centric Answer Sampling argues that even among verified correct answers, the highest-performing teacher may not provide the best supervision for a given student (Kim, 23 Jun 2025, Hu et al., 26 May 2026). In anti-distillation work, the same logic is inverted: the released teacher is deliberately shaped to suppress outputs that are most useful for imitation while staying close to a reference teacher under a fidelity constraint (Allouah et al., 21 May 2026).

A plausible implication is that teacher-side defense is best understood as control over the quality, structure, and admissibility of supervision. The control variable may be feature disclosure, soft-label temperature, groupwise reweighting, answer selection, pseudo-label filtering, or refusal style, but the underlying design principle is the same: the teacher decides not only what is correct, but how much, in what form, and for whom that correctness is transmitted.

2. Supervision shaping for robustness and distillation

A canonical teacher-side robustness mechanism is defensive distillation. In its ordinary form, a teacher and student of the same architecture are trained sequentially with a temperature-scaled softmax,

$q_i = \frac{\exp(z_i/T)}{\sum_j \exp(z_j/T)},$

and the student learns from the teacher’s softened outputs rather than from one-hot labels (Mandal et al., 2023). “Improving Defensive Distillation using Teacher Assistant” extends this by inserting an intermediate assistant $F^a$ between teacher $F$ and final student $F^d$, again using the same architecture and the same temperature during training. The paper evaluates targeted Carlini-and-Wagner-style $L_2$, $L_0$, and $L_\infty$ attacks on MNIST and CIFAR10, using attack-generated minimum perturbation over all 10,000 test samples as the principal robustness metric (Mandal et al., 2023).

Its reported pattern is teacher-side rather than student-side: robustness generally increases with training temperature, the assistant-mediated student usually has higher robustness than either the teacher or the assistant, and multi-step distillation can further improve robustness with “very little impact on model accuracy” (Mandal et al., 2023). The baseline clean accuracy for the teacher at $T=1$ is reported as $99.38\%$ on MNIST and $77.72\%$ on CIFAR10; the maximum variation relative to the teacher is about $F^a$0 on MNIST and about $F^a$1 on CIFAR10 (Mandal et al., 2023). The paper’s mechanism is explicitly teacher-side: the same labeled data are reused, but labels are progressively replaced by soft labels generated by the previous model in the chain.

The active-teacher formulation in “Achieving Adversarial Robustness Requires An Active Teacher” generalizes the same idea. Instead of assuming the student should infer the teacher’s rule from examples alone, it argues that the teacher should actively provide information such as “the features used, invariances, sparsity, or the structure of the model” (Ma et al., 2020). In the paper’s synthetic examples, robustness improves when the teacher tells the student that only $F^a$2 and $F^a$3 matter, or that classification should depend on shape rather than texture, allowing the student to restrict its hypothesis class or preprocessing accordingly (Ma et al., 2020). This suggests that teacher-side defense can also take the form of structural disclosure rather than only output smoothing.

3. Correcting biased or mismatched teacher supervision

Teacher-side templates are equally important when the teacher is biased or pedagogically mismatched. Long-Tailed Knowledge Distillation starts from the observation that in long-tailed data, standard KD transfers teacher head-class bias directly into the student. It decomposes

$F^a$4

into an inter-group term over head, medium, and tail groups and intra-group conditional KL terms weighted by teacher group mass (Kim, 23 Jun 2025). The paper’s diagnosis is that two teacher-bias channels coexist: the teacher’s marginal group distribution is skewed, and the tail-group conditional KL is downweighted because $F^a$5 is small.

LTKD corrects both channels by recalibrating teacher group probabilities with batch-wise scaling and by replacing teacher-mass-weighted intra-group transfer with a uniform intra-group loss (Kim, 23 Jun 2025). On CIFAR-100-LT in the homogeneous setting, ResNet32$F^a$64 $F^a$7 ResNet8$F^a$84 at $F^a$9, tail accuracy improves from $F$0 to $F$1 and overall accuracy from $F$2 to $F$3; on VGG13 $F$4 VGG8 at $F$5, tail accuracy rises from $F$6 to $F$7 and overall accuracy from $F$8 to $F$9 (Kim, 23 Jun 2025). The key point is that the teacher is left fixed, but the teacher-derived supervision is not trusted monolithically.

Student-Centric Answer Sampling pushes the same logic further: even when multiple teachers provide correct answers to the same question, the strongest teacher is not necessarily the best teacher for a given student (Hu et al., 26 May 2026). SCAS assumes a pool of verified correct answers $F^d$0, computes a forward-only student-centric score

$F^d$1

from token NLLs and final-layer hidden states, ranks candidates by this score, partitions them into $F^d$2 groups, and samples uniformly from the lowest-score group $F^d$3 (Hu et al., 26 May 2026). Across 30 teacher models, 6 student base models, and 8 tasks, SCAS consistently improves student performance; for Qwen2.5-0.5B the average score is $F^d$4 against $F^d$5 for GRACE, for Qwen2.5-3B it is $F^d$6 against $F^d$7, and for Qwen2.5-7B it is $F^d$8 against $F^d$9 (Hu et al., 26 May 2026). The broader implication is that a teacher-side defense template may need to defend the student not only from incorrect supervision, but also from correct yet badly matched supervision.

4. Anti-distillation and output-level protection

In anti-distillation research, teacher-side defense becomes an explicit confidentiality mechanism. “The Distillation Game” formulates a minimax game between a utility-constrained released teacher $L_2$0 and an adaptive student $L_2$1, yielding the teacher best response

$L_2$2

where $L_2$3 is an example-value function for distillation (Allouah et al., 21 May 2026). Under the likelihood-gap proxy,

$L_2$4

this becomes Product-of-Experts: $L_2$5 implemented as a forward-pass-only weighted combination of teacher and proxy-student token log-probabilities (Allouah et al., 21 May 2026). Empirically, adaptive evaluation reveals a large passive–adaptive gap. On GSM8K at representative operating points, ADS yields teacher accuracy $L_2$6, passive student accuracy $L_2$7, and adaptive student accuracy $L_2$8, while PoE yields $L_2$9, $L_0$0, and $L_0$1, respectively; PoE is also cheaper, with time cost $L_0$2 against $L_0$3 for ADS (Allouah et al., 21 May 2026).

“What Does It Mean to Break a Distillation Defense?” argues that these defenses cannot be evaluated in the abstract. It formalizes attacker capability as a tuple

$L_0$4

where $L_0$5 is query budget, $L_0$6 is data budget, and $L_0$7 is interface profile, and states that “only teacher API queries and input prompts constitute the attacker’s cost,” while local compute, filtering, and post-processing are treated as free (Libon et al., 23 Jun 2026). In that framework, a defense is meaningful only relative to a concrete attacker. The paper’s ADS case study shows that increasing $L_0$8, allowing local post-processing, or exposing response prefill can largely neutralize the apparent protection (Libon et al., 23 Jun 2026).

DistillGuard systematizes output-level defenses into output perturbation, data poisoning, and information throttling, and shows that most output-level defenses are weak against even a naive black-box distillation attacker (Jiang, 8 Mar 2026). On MATH-500, only chain-of-thought removal substantially impairs reasoning distillation, reducing the distilled student from $L_0$9 to $L_\infty$0, but code generation remains unaffected and the average distillation cost of CoT removal is $L_\infty$1; for the teacher on MATH specifically, the paper reports $L_\infty$2 (Jiang, 8 Mar 2026). “Adversarial Sparse Teacher” pursues the same output-space objective through teacher retraining: it aligns clean outputs to sparse versions of adversarial outputs via the AST loss and the Exponential Predictive Divergence

$L_\infty$3

reporting strong reductions in distilled student accuracy on CIFAR-100 while largely preserving teacher accuracy (Yilmaz et al., 2024). Yet “Distilling the Undistillable” shows that Nasty Teacher-style defenses can be bypassed by High Temperature Composition and Sequence of Contrastive Models, recovering substantial learning from the defended teacher (Jandial et al., 2022). A direct implication is that teacher-side anti-distillation must be evaluated against adaptive transformations of the teacher outputs, not only against vanilla KD.

5. Teacher-guided intervention in adaptation and reinforcement learning

Teacher-side defense also appears in settings where the teacher is not a static label generator. In unsupervised cross-domain object detection, Adversarial Defense Teacher uses an EMA teacher to generate pseudo-labels on weakly augmented target images, then attacks the student-side target input with PGD using those pseudo-labels as supervision, and finally updates the teacher only by EMA (Wang et al., 2024). The attack is positive-only: the attack-specific loss includes pseudo-labeled positives in the RPN and ROI terms, avoiding background regions because pseudo-labels have high precision but lower recall (Wang et al., 2024). On Foggy Cityscapes All, ADT reaches $L_\infty$4 [email protected], beating CMT by $L_\infty$5 mAP; on BDD100K day-to-night adaptation, it reaches $L_\infty$6, beating 2PCNet by $L_\infty$7 (Wang et al., 2024). The generic teacher-side template extracted by the paper is explicit: generate pseudo-labels $L_\infty$8 craft adversarial student examples using those pseudo-labels $L_\infty$9 train the student under attack $T=1$0 update the teacher by EMA (Wang et al., 2024).

In reinforcement learning, Teacher-Student Shared Control treats the teacher as a guarding controller that selectively intervenes during student training (Xue et al., 2023). The mixed behavior policy is

$T=1$1

and intervention is triggered not by action mismatch, but by teacher-value superiority or uncertainty: $T=1$2 if the ensemble mean teacher value exceeds the student’s by more than $T=1$3, or if the ensemble variance on the student’s value exceeds $T=1$4 (Xue et al., 2023). The main guarantee is

$T=1$5

and in safety-critical settings cumulative cost satisfies

$T=1$6

(Xue et al., 2023). Here the teacher need not be optimal; the paper explicitly develops TS2C to incorporate “arbitrary teacher policies with modest or inferior performance” (Xue et al., 2023). That is a distinct teacher-side defense pattern: the teacher blocks clearly worse or risky behavior without forcing imitation.

6. Teacher-facing reflective, professional, and pedagogical templates

Outside model training, teacher-side defense template refers to structured teacher-facing mechanisms for planning, reflection, and bounded pedagogical behavior. The “Reflective Diary for Professional Development of Novice Teachers” introduces a paper-based, semi-structured diary for novice CS teaching assistants. It is a 48-page, A6, paper diary with 14 double-page weekly spreads, each divided into “Before the lesson,” “After the lesson,” and a notes page with guiding prompts; it also includes a rubric on 14 teaching skills and a reference handbook (Ukrop et al., 2018). Piloted on 25 novice teaching assistants, it produced “overwhelmingly positive responses” and was later used by about 30 teachers in total (Ukrop et al., 2018). The design principle is teacher-side in the literal sense: regular, near-immediate reflection structures how the teacher interprets and defends practice.

“TA-designed vs. research-oriented problem solutions” provides a parallel template for worked-solution design. It studies 24 first-year graduate TAs and finds that while many value expert-like solution features, they primarily notice “surface features” and rarely implement deeper reasoning structure (Lin et al., 2016). All 23 TA solutions collected included a diagram, but only 3 of 23 provided some outline of the sub-problems, only 6 of the 23 provided any justification for the principles used, and no TA performed an answer check (Lin et al., 2016). The paper’s practical lesson is that a teacher-side defense of worked solutions must make expert decision-making visible: initial analysis, sub-problem choice, explicit principles, explicit reasoning, and final-result checking.

“Interdisciplinary Teams for Teacher Professional Development” expands the same logic to a broader PD architecture. The SATTAA program integrates content experts, STEM pedagogy experts, and informal education partners, and is organized around a theory of change that “an increased confidence level will increase student retention in astronomy and related STEM fields” (Fies et al., 2021). On a 15-item astronomy content measure, the paper reports pretest $T=1$7, posttest $T=1$8, with $T=1$9, $99.38\%$0, $99.38\%$1, and effect size $99.38\%$2 (Fies et al., 2021). A teacher-side defense template in this setting is not a refusal mechanism but a support system that integrates content, pedagogy, technology, feedback, and alumni community.

EduGuardBench adds a safety-focused teacher-facing benchmark for LLMs as simulated teachers (Jiang et al., 10 Nov 2025). Its dual-component design measures professional fidelity through the Role-playing Fidelity Score,

$99.38\%$3

and adversarial safety through Attack Success Rate and a three-tier refusal-quality rubric of flimsy, standard, and educational refusal (Jiang et al., 10 Nov 2025). The benchmark reports a strong negative correlation between ASR and Educational Refusal,

$99.38\%$4

and identifies an “Educational Transformation Effect,” in which the safest models convert harmful requests into teachable moments (Jiang et al., 10 Nov 2025). Claude-3.7 is reported at $99.38\%$5 educational transformation, against $99.38\%$6 for DeepSeek-V3 (Jiang et al., 10 Nov 2025). In this pedagogical sense, a teacher-side defense template is a policy that preserves fidelity, blocks academic misconduct, and redirects harm into education.

7. Evaluation discipline, limitations, and unresolved questions

A consistent conclusion across these works is that teacher-side defense is useful but rarely sufficient on its own. The defensive-distillation-with-assistant paper explicitly states that “modern adversarial attacks are very effective and for most cases distillation cannot provide substantial defense,” and its evaluation remains limited to MNIST, CIFAR10, relatively shallow CNNs, and selected CW-style attacks (Mandal et al., 2023). The anti-distillation literature reaches a similar conclusion from a different angle: strong distillation remains difficult to stop, and output-level defenses can look far stronger under passive evaluation than under adaptive evaluation (Allouah et al., 21 May 2026, Jiang, 8 Mar 2026).

Evaluation discipline is therefore central. The threat-model framework based on $99.38\%$7 argues that a defense cannot be said to work without explicit specification of query budget, data budget, and interface profile, and that “fully exhausting the attacker’s power within the assumed threat model is thus necessary to make any robustness claim” (Libon et al., 23 Jun 2026). DistillGuard reinforces the same point empirically: paraphrasing-based perturbation barely degrades distilled student quality, data poisoning primarily impairs conversational fluency while leaving task-specific capabilities intact, and only CoT removal substantially impairs mathematical reasoning while imposing high user cost (Jiang, 8 Mar 2026). In the pedagogical domain, EduGuardBench makes an analogous point: teacher safety is not exhausted by refusal; incompetence remains the dominant failure mode across most models, and mid-sized models can be the most vulnerable, producing a “counterintuitive scaling paradox” (Jiang et al., 10 Nov 2025).

What these results suggest is not a single settled template, but a recurring design rule. Teacher-side defense is strongest when it is explicit about the teacher’s role, explicit about the attack or failure channel, explicit about what the teacher is allowed to reveal, and explicit about how effectiveness is measured. It is weakest when the teacher is treated as a passive source of labels, when evaluation assumes a non-adaptive student or attacker, or when a defense preserves appearance while leaving recoverable structure intact.