Contrastive Student-Forcing Loss (CSFL)

• CSFL is a loss function for autoregressive models that aligns student-forced predictions with teacher references to reduce exposure bias and train-test mismatch.
• It employs pairwise contrastive supervision to stabilize learning by anchoring student outputs to trusted teacher trajectories in both visual and language settings.
• Empirical results show CSFL improves performance metrics like FID in visuals and win rates in language models, demonstrating enhanced stability and convergence.

Contrastive Student-Forcing Loss (CSFL) is a loss function designed to address the challenges of model alignment in autoregressive generation tasks, with key applications spanning both large-scale visual and language generation models. CSFL explicitly contrasts the model's predictions under "student forcing"—i.e., conditioning on its own sampled predictions—with either teacher-forced outputs or teacher distributions, thereby promoting stable learning and mitigating exposure bias and train-test mismatch. As demonstrated independently in visual autoregressive refinements (Zhou et al., 6 Dec 2025) and LLM distillation (Ko et al., 10 Mar 2025), CSFL leverages a pairwise alignment or contrastive principle to drive student outputs toward reliable reference trajectories while discouraging pathological self-reinforcement or hallucination.

1. Motivation and Background

Conventional autoregressive (AR) models are trained with teacher forcing (TF), where the target at each timestep is predicted using ground-truth context. At inference, however, models must generate recursively from their own prior predictions (student forcing, SF), which magnifies initial errors and causes distributional shift between training and deployment. This exposure bias is particularly acute in scale-wise AR generation (e.g., next-scale prediction in image models) and sequence-to-sequence tasks (e.g., LLM distillation), where early mistakes propagate and semantic drift cannot be corrected downstream (Zhou et al., 6 Dec 2025, Ko et al., 10 Mar 2025).

Naïve attempts to train AR models directly under self-generated contexts (minimizing the loss between student predictions and ground truth under SF) frequently result in instability and divergence, as repeated errors are amplified through the model's own feedback. In LLM distillation, standard approaches that treat teacher- and student-generated data with identical loss terms (e.g., forward KL divergence) typically fail to optimize for both diversity and correctness, suffering either mode collapse or insufficient learning on long-tailed targets (Ko et al., 10 Mar 2025). CSFL is devised to reconcile these issues by supplying an adaptive, contrastive supervision signal for student-forced trajectories.

2. Mathematical Formulations

CSFL admits distinct mathematical realizations in visual AR and LLM settings, both enforcing pairwise alignment between model outputs under SF and a reference "teacher" trajectory—either a latent prediction or a probability distribution.

Visual Autoregressive Generation

Given notations:

• $f_i \in \mathbb{R}^{h_i \times w_i \times d}$: ground-truth latent at scale $i$
• $\hat f^{(T)}_i = g_\theta(f_{1:i-1})$: teacher-forced prediction
• $$\tilde f^{(T)}_i = \mathrm{Upsample}(\hat f^{(T)}_i)$$: upsampled TF prediction (input to SF)
• $\hat f^{(S)}_i = g_\theta(\tilde f^{(T)}_{1:i-1})$: student-forced prediction

The loss terms are:

$$\mathcal{L}_{\mathrm{TF}} = \sum_{i=1}^N \ell(\hat f^{(T)}_i,\,f_i)$$

$$\mathcal{L}_{\mathrm{CSF}} = \sum_{i=2}^N \ell(\hat f^{(S)}_i,\,\hat f^{(T)}_i)$$

The overall objective in Self-Autoregressive Refinement (SAR) is:

$$\mathcal{L}_{\mathrm{SAR}} = \mathcal{L}_{\mathrm{TF}} + \gamma\,\mathcal{L}_{\mathrm{CSF}}$$

where $\gamma > 0$ balances supervision.

LLM Distillation

Let $p(\cdot|x)$ denote the teacher distribution, $q_\theta(\cdot|x)$ the student, $\hat y_T \sim p(\cdot|x)$ (teacher-generated output), and $\hat y_S \sim q_\theta(\cdot|x)$ (student-generated output). Define skew mixing:

• $$\tilde q^{(\alpha)}_\theta = \alpha p + (1-\alpha) q_\theta$$
• $$\tilde p^{(\alpha)} = (1-\alpha) p + \alpha q_\theta$$

The sequence-level skew-KL and reverse skew-KL are:

$$D_{\mathrm{SKL}}^{(\alpha)}(x, \hat y_T) = \sum_{t=1}^T p(y_t|x,y_{<t}) \log \frac{p(y_t|x,y_{<t})}{\tilde q^{(\alpha)}_\theta(y_t|x,y_{<t})}$$

$$D_{\mathrm{SRKL}}^{(\alpha)}(x, \hat y_S) = \sum_{t=1}^T q_\theta(y_t|x,y_{<t}) \log \frac{q_\theta(y_t|x,y_{<t})}{\tilde p^{(\alpha)}(y_t|x,y_{<t})}$$

The CSFL is then:

$$\mathcal{L}_{\mathrm{CSFL}}(x) = \frac{1}{2}\Big[ (1-\beta) D_{\mathrm{SKL}}^{(\alpha_T)}(x, \hat y_T) + \beta D_{\mathrm{SRKL}}^{(\alpha_S)}(x, \hat y_S) \Big]$$

where $\alpha_T, \alpha_S \in [0,1]$ and $\beta$ is linearly annealed.

3. Integration with Training Algorithms

In SAR for visual AR models, CSFL is coupled with Stagger-Scale Rollout (SSR). Each batch involves two forward passes:

• A teacher pass computes $\hat f^{(T)}_{1:N}$ on ground-truth latents.
• An SSR pass shifts and upsamples the TF outputs, then performs one-step SF to obtain $\hat f^{(S)}_{2:N}$.

CSFL ties these by retrospectively aligning SF outputs with their TF counterparts, thus maintaining consistency along student-generated contexts (Zhou et al., 6 Dec 2025). Only one extra forward pass per batch is required, resulting in roughly double the number of function evaluations (NFE), yet the overall training accounts for less than $5.5\%$ of the original pretraining cost.

In LLM distillation (DistiLLM-2 (Ko et al., 10 Mar 2025)), training proceeds via batch on-policy sampling of both teacher and student outputs, with CSFL forming the sole distillation objective:

1. Batched sampling yields $(x, \hat y_T, \hat y_S)$ triplets.
2. $\alpha$ parameters are adapted per-batch via a curriculum scheme: "easy" samples (where $p$ and $q_\theta$ are close) get smaller skew, "hard" samples receive larger skew.
3. The $\beta$ weight increases linearly over epochs to gradually emphasize the student-forced reverse KL component.
4. Parameter updates occur via stochastic gradient descent on the CSFL objective.

4. Theoretical Properties and Intuition

CSFL leverages contrastive alignment between student- and teacher-forced model outputs:

• Stable Reference: In visual AR, anchoring the student trajectory to the teacher-forced manifold prevents deviation into semantically inconsistent regions, avoiding the drift or collapse characteristic of naïve student forcing.
• Gradient Consistency: The loss propagates stable, reliable gradients through the student-forced pathway, thus regularizing the model to behave consistently even under self-generated error accumulation.
• Pairwise Contrast and Alignment: Though CSFL is not a classic InfoNCE loss, it realizes a similar "attraction" between desired (teacher or ground-truth) and actual (student) representations, facilitating robust alignment without introducing temperature or margin parameters (Zhou et al., 6 Dec 2025, Ko et al., 10 Mar 2025).
• Token-Wise Effect in Sequence Models: The decomposition of CSFL over tokens enables granular adjustment: on teacher outputs, the student increases likelihoods where the teacher is confident; on student outputs, the student suppresses tokens it tends to hallucinate.

In DistiLLM-2, the use of forward and reverse skewed KL terms addresses known pathologies: standard forward KL "mode-averages," while reverse KL "mode-collapses." The joint contrastive setup balances both effects, yielding improved stability and accuracy across model scales (Ko et al., 10 Mar 2025).

5. Implementation Considerations

Visual AR Models

• Optimizer: AdamW with $\beta_1=0.9$, $\beta_2=0.95$, weight decay $0.05$, learning rate $10^{-4}$
• Training Regimen: 10-epoch post-training on pretrained models; total epochs are comparable to baselines
• SSR Sampling: Best performance with stochastic sampling and classifier-free guidance (CFG scale $2.5$, top-$k=900$, top-$p=0.95$). Deterministic sampling degrades recall and generation fidelity.
• $\gamma$ Hyperparameter: Typically set to $1.0$, no significant need for tuning
• Compute Overhead: Slight, with only one extra forward pass (≈2× NFE per batch), under $5.5\%$ of original training cost (Zhou et al., 6 Dec 2025)

LLM Distillation

• Batch Selection: On-policy generation of both teacher- and student-generated sequences for each prompt
• Curriculum $\alpha$ Update: Per-batch adaptation equalizes gradient weights across samples
• $\beta$ Schedule: Linear increase from $\beta_0$ to $1.0$ during training
• Loss Application: No added MLE or standard teacher-forcing loss; all supervision comes from CSFL
• Automatic Stability: No margin or temperature tuning required (Ko et al., 10 Mar 2025)

6. Empirical Results and Ablations

Visual Autoregressive Models

CSFL incorporated via SAR and SSR yields consistent improvements in Fréchet Inception Distance (FID) across scales:

• FlexVAR-d16: FID 3.05 → 2.89 (–5.2%)
• FlexVAR-d20: 2.41 → 2.35 (–2.5%)
• FlexVAR-d24: 2.21 → 2.14 (–3.1%)

Naïve student forcing or hybrid TF/SF schedules significantly worsen FID (3.83 → 16.56), confirming the necessity of the contrastive approach. The addition of CSFL stabilizes training, accelerates convergence, and lowers final error (see Figure 1(a) in (Zhou et al., 6 Dec 2025)).

LLM Distillation

CSFL in DistiLLM-2 consistently increases win rates and accuracy on instruction following, code generation, and math benchmarks:

• Qwen2-7B→1.5B: +2.34% average win-rate over next best
• Mistral-7B→Danube2-1.8B: +1.95% gain
• HumanEval+MBPP: 64.00 → 67.79% on DS-Coder

Ablations indicate additive effects: pure CSFL (+2.06% win-rate); β anneal (+1.19%); curriculum α (+1.67%); full DistiLLM-2 (+4.95% over baseline). Importantly, performance monotonically scales with teacher capacity, where other distillation methods degrade (Ko et al., 10 Mar 2025).

Empirical Table: CSFL Effects

Setting	Metric	CSFL Improvement
FlexVAR-d16	FID	–5.2%
Qwen2-7B→1.5B	Win-rate	+2.34%
DS-Coder (HumanEval+MBPP)	Pass@1	+3.79%

7. Broader Impact and Applicability

CSFL’s alignment methodology extends naturally to various autoregressive generative tasks, covering both visual and textual modalities. Its simple, $\ell$-based or token-wise contrastive structure enables easy implementation and tuning, incurring minimal additional cost relative to standard training schedules.

Its information-rich, stability-inducing effect positions CSFL as an effective and scalable solution for post-training refinement (visual models) and high-fidelity knowledge distillation (LLMs), especially under regimes with exposure bias and divergent teacher/student behaviors (Zhou et al., 6 Dec 2025, Ko et al., 10 Mar 2025).