CEFR LLM Alignment Drift

• Alignment drift in CEFR-prompted LLMs is the gradual convergence of output difficulty levels during multi-turn dialogues, compromising language proficiency alignment.
• The study employs a teacher–student self-chat protocol with linear mixed-effects models to quantify drift using metrics such as readability, structural complexity, and surprisal.
• Empirical results reveal that drift undermines tutoring reliability, urging the implementation of periodic re-anchoring, external difficulty monitors, and dynamic prompt learning.

Alignment drift in CEFR-prompted LLMs denotes the gradual loss of adherence to an intended text difficulty level, as specified by system prompts based on the Common European Framework of Reference (CEFR) for languages, during multi-turn interactive dialogues. In the context of adaptive Spanish language tutoring, this refers to the tendency of LLMs, initially constrained by precise CEFR-level instructions (A1, B1, C1), to revert to unconstrained output styles as conversation proceeds. This convergence across CEFR levels undermines the reliability of automated proficiency-aligned tutoring systems using LLMs (Almasi et al., 13 May 2025).

1. Formalization and Quantification of Alignment Drift

Alignment drift is defined as the reduction in separation between the text difficulty metrics corresponding to different CEFR levels over successive conversation turns. Let $m_\ell(t)$ represent a difficulty metric (such as readability, structural complexity, or surprisal) for CEFR level $\ell \in \{\text{A1}, \text{B1}, \text{C1}\}$ at turn $t$. Then, for the A1–C1 boundary, initial separation is

$$\Delta_{1-3}(0) = m_{\text{A1}}(0) - m_{\text{C1}}(0)$$

and alignment drift by turn $t$ is

$$\text{drift}(t) = \Delta_{1-3}(0) - [m_{\text{A1}}(t) - m_{\text{C1}}(t)]$$

A positive value of $\text{drift}(t)$ implies convergence of the output difficulty between levels.

Statistically, a linear mixed-effects model is fit for each metric:

$$m_{i,j} = \beta_0 + \beta_1 \cdot I[\text{level} = \text{B1}] + \beta_2 \cdot I[\text{level} = \text{C1}] + u_j + \epsilon_{i,j}$$

with $i$ indexing tutor message, $j$ indexing dialogue, $u_j \sim N(0, \sigma_u^2)$ (random intercept), and $\beta_1, \beta_2$ quantifying mean A1–B1 and A1–C1 differences. Alignment drift manifests as a reduction in these fixed-effect coefficients over dialogue turns (Almasi et al., 13 May 2025).

2. Experimental Methodology

Experiments utilize leading instruction-tuned, open-source LLM architectures:

Model	Parameter Count	Vendor
Llama-3.1-8B-Instruct	8B	Meta
Gemma-3-12B-IT	12B	Google
Mistral-7B-v0.3-Instruct	7B	Mistral AI
Qwen-2.5-7B-Instruct	7B	Alibaba

In the teacher–student self-chat protocol, a single model instance alternately assumes the roles of "tutor" and "student," with segregated chat histories. Each tutor turn is governed by a CEFR-level-specific system prompt (defined in English, instructing output in Spanish only). The initial student input is fixed ("Hola") to standardize initiation.

Each dialogue consists of nine alternating turns, replicated 30 times per level per model, yielding 360 dialogues overall. Tutor responses containing English or Mandarin are automatically regenerated to enforce linguistic control. Preprocessing includes emoji removal and language detection filtering.

Automated evaluation employs:

• Readability: Fernández Huerta, Szigriszt–Pazos, Gutiérrez de Polini indices via textstat
• Structural complexity: token count, mean dependency distance (MDD) via TextDescriptives and spaCy es_core_news_md
• Surprisal: normalized negative log-probability under EuroBERT (minicons)
• Statistical analysis: linear mixed-effects models in R, with Bonferroni-corrected $p$-values (Almasi et al., 13 May 2025)

3. Empirical Findings and Drift Manifestation

At the initial turn ($t = 1$), models show clear metric separation between CEFR levels. For example, Fernández Huerta readability scores (higher = easier) cluster near A1 ≈ 95, B1 ≈ 88, C1 ≈ 80. By turn $t = 9$, values converge (A1 ≈ 90, B1 ≈ 88, C1 ≈ 86), indicating substantial drift.

Mixed-effects coefficients reflect this trend:

• For Fernández Huerta, B1 vs A1: $\beta_1 \approx -4$ to $-9$ ($p < 0.001$)
• C1 vs A1: $\beta_2 \approx -12$ to $-17$ ($p < 0.001$) However, the magnitude of these coefficients declines in later turns, quantifying diminishing separation.

Structural metrics, such as token length and MDD, initially form a "sandwich" pattern with clear separation, but overlap increases with each turn. Surprisal-based measures exhibit the weakest and least reliable separation; for some models (e.g., Qwen), distinctions are statistically negligible.

Descriptive statistics from all models highlight this convergence:

Turn	A1 (mean±SD)	B1 (mean±SD)	C1 (mean±SD)
1	93.5 ± 4.2	86.7 ± 5.1	79.8 ± 5.5
9	90.1 ± 4.7	88.3 ± 5.0	85.9 ± 5.2

Example dialogue excerpts further illustrate qualitative drift. At turn 1 (A1), a tutor may say: “¿Cómo estás? ¿Qué tal tu día?” By turn 9, under a C1 prompt but after drift, outputs escalate in complexity: “Me alegra saber que trabajaste todo el día. ¿Cómo hubieras manejado mejor tu tiempo si tuvieras más recursos?” This reflects use of past subjunctive forms and abstract vocabulary, exceeding intended CEFR boundaries (Almasi et al., 13 May 2025).

4. Causal Mechanisms and Brittleness

Alignment drift is attributed to several factors:

• Lack of persistent constraint from static system prompts across long dialogs.
• Instruction-tuned LLMs reverting toward pre-training priors favoring more complex, multi-lingual text patterns.
• System prompts issued in English may exert weaker control over Spanish text generation, diminishing their rhetorical force.

The single-turn effectiveness of prompting is thus insufficient for sustained, level-appropriate output. Brittleness emerges most acutely in contexts necessitating persistent, strict control over text complexity across session duration (Almasi et al., 13 May 2025).

5. Implications for Adaptive Tutoring with LLMs

Initial prompting can effectively constrain LLM outputs for short exercises or isolated responses, making it an appealing, low-cost method for rapid development. However, for multi-turn interactive sessions central to adaptive tutoring, reliability is compromised by alignment drift. The ensuing unpredictability of output difficulty undermines pedagogical safety and the assurance of proficiency-appropriate support (Almasi et al., 13 May 2025).

For system designers, several mitigation strategies have been identified:

• Periodic Re-anchoring: Resend system prompts or inject concise CEFR reminders at fixed intervals.
• External Difficulty Monitors: Employ auxiliary classifiers to detect drift and trigger corrective measures.
• Dynamic Prompt Learning: Adapt prompt content in response to observed drift, reinforcing or intensifying level-specific instructions.
• Multi-modal Feedback: Integrate in-context exemplars at the target CEFR level every few turns to bolster alignment.

A plausible implication is that multi-pronged, closed-loop control mechanisms—rather than prompting alone—are necessary for robust, long-term proficiency alignment.

6. Open Questions and Future Directions

Critical questions for future research include:

• Which prompt refresh intervals or mechanisms most cost-effectively minimize drift?
• Does employing monolingual prompts (in Spanish) reduce code-switching or improve constraint fidelity?
• How can live, human-in-the-loop validation complement automated metric pipelines to ensure sustained alignment, safety, and pedagogical appropriateness?

Further exploration is needed into dynamic prompt engineering, continuous monitoring, and selective fine-tuning, with the goal of achieving robust, scalable, and cost-effective adaptive tutoring using LLMs within the CEFR proficiency framework (Almasi et al., 13 May 2025).