Correct-Answer Extinction Window in TTRL

• Correct-Answer Extinction Window is a transient phase in label-free TTRL where the correct answer, though initially present, becomes a competitive minority before majority voting entrenches an incorrect label.
• The process involves a sequence of competition, suppression, and lock-in, quantified by metrics like Match Rate and Flip Rate that signal the shift from correct to erroneous pseudo-labeling.
• Guarded interventions like TTRL-Guard—using strategies such as Flip-Rate-Aware Reward Scaling and Minority-Preserving Sampling—demonstrate improved accuracy and mitigated degradation in test-time reinforcement learning.

Searching arXiv for the most relevant papers on “Correct-Answer Extinction Window” and closely related formulations. Correct-Answer Extinction Window is a term introduced for label-free test-time reinforcement learning (TTRL) to denote a brief early-training phase in which the correct answer still survives as a competitive minority before majority-vote reinforcement locks onto an incorrect answer and the corresponding corrective signal is lost (Lin et al., 19 May 2026). In that original formulation, the phenomenon is not a hyperparameter but an empirical regime in the dynamics of pseudo-label formation. A broader, interpretive use has emerged around adjacent problems in reasoning and assessment: correct answers may be behaviorally recoverable before they become extractable, may be overwritten by later decoding, or may become pedagogically invisible once correctness prematurely terminates diagnosis. Those broader usages are analogous rather than identical to the formal TTRL definition (Wang et al., 8 Apr 2026).

1. Origin and formal definition

The canonical setting is an unlabeled problem set

$\mathcal{Q}=\{q_i\}_{i=1}^{N},$

where at training step $t$ the model samples $K$ responses $\{a_i^{(k)}\}_{k=1}^K$ for each problem $q_i$, then forms a pseudo-label by majority vote:

$$\hat{y}_i^t = \arg\max_{a}\,\sum_{k=1}^{K}\mathbf{1}[a_i^{(k)}=a].$$

The reward target is

$r(a,\hat{y}_i^t)=\mathbf{1}[a=\hat{y}_i^t].$

Within this loop, the Correct-Answer Extinction Window is the interval in which the correct answer still appears often enough to remain a competitive minority before majority voting becomes stably wrong and self-reinforcing (Lin et al., 19 May 2026).

The paper operationalizes the state of this competition with Match Rate and Flip Rate:

$$\mathrm{MR}_i^t = \frac{\max_a\sum_k\mathbf{1}[a_i^{(k)}=a]}{K},$$

$$\mathrm{FR}_i^t = \frac{1}{W}\sum_{s=t-W+1}^{t} \mathbf{1}[\hat{y}_i^s \neq \hat{y}_i^{s-1}],$$

using $W=5$ by default. It also defines

$t$0

and

$t$1

In this framework, high FR indicates that the pseudo-label is still contested; low FR after a period of competition indicates consolidation, which may be either benign convergence or harmful lock-in (Lin et al., 19 May 2026).

A central definitional point is that “extinction” does not mean literal impossibility in all future rollouts. It means operational irrecoverability under the given TTRL dynamics: once majority vote is stably wrong, the reward function no longer carries usable signal favoring the correct answer, so subsequent updates continue to reinforce the wrong pseudo-label (Lin et al., 19 May 2026).

2. Consensus dynamics and the extinction mechanism

The mechanism is a three-phase process of competition, suppression, and lock-in. Early in training, the correct answer and one or more plausible wrong answers may both appear in the sampled responses. During this competition phase, the correct answer can remain “alive” even when it is not yet the majority. The paper reports that on aggregated degraded problems, around 30% still have wrong-answer match rate below 0.5 early in training, so the correct answer still has a realistic chance to become the majority in later steps (Lin et al., 19 May 2026).

Suppression begins whenever an incorrect answer wins majority vote. Because the reward is winner-takes-all, that step increases probability mass on the wrong answer and decreases relative support for alternatives, including the correct one. The process is cumulative: wrong-answer victories make future wrong-answer victories more likely. The authors characterize this as a “Scissor Effect,” in which the wrong answer’s dominance rises while the correct answer’s survival space collapses (Lin et al., 19 May 2026).

Lock-in occurs once the wrong answer becomes sufficiently dominant that pseudo-label flips largely stop and correct-answer samples become too rare to influence the vote. The paper states that “once the wrong answer's match rate exceeds $t$2, the pseudo-label locks irreversibly.” The irreversibility is methodological rather than metaphysical: in the absence of external supervision, the training loop has lost its self-correction channel (Lin et al., 19 May 2026).

This mechanism is strongest in capability-difficulty mismatch regimes. If the model is already strong on a task, majority vote is usually correct and TTRL mainly sharpens existing competence. If the model is extremely weak, there may be little correct signal to preserve. The most hazardous regime is intermediate: the model can generate plausible wrong answers often enough for them to win the vote, while still occasionally producing the true answer as a minority that is vulnerable to extinction (Lin et al., 19 May 2026).

3. Empirical signatures and measurement

The empirical case for the extinction window relies on per-problem trajectory tracking rather than aggregate pass@1 alone. The paper defines Initial Label Accuracy (ILA) as mean label accuracy over the first 3 checkpoints and Final Label Accuracy (FLA) as mean label accuracy over the last 5 checkpoints, then assigns each problem to one of six fates: Stable Always Right, Degraded, Learned, Marginal Degraded, Marginal Stable, and Always Wrong (Lin et al., 19 May 2026).

The headline quantitative result is that most apparent TTRL gains reflect sharpening rather than genuine acquisition. A representative breakdown reports 44.5% of problems already solvable before training as Stable Always Right, only 0.7% genuinely Learned, and 21.6% Degraded. The resulting asymmetry is summarized as problems corrupted from correct to incorrect outnumbering truly learned ones by 31× (Lin et al., 19 May 2026).

The degradation signal is particularly stark in weaker-model settings. On MATH-500, Llama-3.2-3B shows 53.8% Degraded and 0.2% Learned; Qwen2.5-7B shows 13.8% Degraded and 0.6% Learned; Qwen3-4B shows 0.8% Degraded and 0.4% Learned, with the paper attributing the smaller degradation in the last case to high initial solvability, specifically 74.4% Stable AR at initialization (Lin et al., 19 May 2026).

Flip Rate functions as a leading indicator because it changes before pass@1 clearly diverges. The paper reports that for Llama-3.2-3B on MATH-500, the correct answer wins majority vote 41.1% of the time during the high-FR phase, but only 3.3% after FR collapses. This supports the interpretation that the extinction window is the high-competition phase in which the correct answer is still sampled often enough to be recoverable through intervention (Lin et al., 19 May 2026).

The paper also proposes the Learned/Degraded ratio,

$t$3

to quantify whether apparent gains come from new problem acquisition or silent corruption. It reports $t$4 for Llama-3.2-3B under TTRL, indicating severe asymmetry between learning and degradation (Lin et al., 19 May 2026).

4. Guarded intervention in test-time reinforcement learning

TTRL-Guard is proposed as a lightweight intervention framework targeted specifically at the extinction window. Its three components are Flip-Rate-Aware Reward Scaling (FRS), Minority-Preserving Sampling (MPS), and Risk-Conditioned Sparse Updating (RCSU) (Lin et al., 19 May 2026).

FRS rescales the reward

$t$5

with

$t$6

where

$t$7

$t$8

$t$9

The triggers are

$K$0

$K$1

The weight is clipped below by $K$2 (Lin et al., 19 May 2026).

MPS preserves competitive minority answers during high-FR periods. It activates when

$K$3

and defines

$K$4

These answers receive a small auxiliary reward

$K$5

and the objective becomes

$K$6

with $K$7 scaling linearly with $K$8 (Lin et al., 19 May 2026).

RCSU targets post-window lock-in. A problem is marked high-risk when it previously experienced competition, has at least $K$9 steps of history, and currently satisfies

$\{a_i^{(k)}\}_{k=1}^K$0

Such problems are skipped with probability $\{a_i^{(k)}\}_{k=1}^K$1, capped at 25% of problems per step. The full guarded loss is

$\{a_i^{(k)}\}_{k=1}^K$2

where $\{a_i^{(k)}\}_{k=1}^K$3 is the high-risk set (Lin et al., 19 May 2026).

Empirically, TTRL-Guard achieves the best average pass@1 on Qwen2.5-7B-Instruct and Qwen3-4B, and improves relatively over TTRL by +54% on AIME 2025. On Qwen2.5-7B-Instruct, TTRL gives 15.6 on AIME 2025 while TTRL-Guard gives 24.1; on Qwen3-4B, average pass@1 rises from 57.6 under TTRL to 59.7 under TTRL-Guard. On Llama-3.2-3B-Instruct, TTRL gives 24.1 average, SCOPE gives 29.6, and TTRL-Guard gives 29.2, with the paper emphasizing that TTRL-Guard is the only method with consistent gains on AIME 2025 in that low-capacity regime (Lin et al., 19 May 2026).

5. Related phenomena in reasoning-model decoding and representation

Outside TTRL, the phrase is not standard, but several neighboring results support an operationally similar picture in which correct-answer signal is transiently available and later becomes inaccessible or weaker. In “The Detection–Extraction Gap,” the answer is behaviorally recoverable from partial prefixes far earlier than forced extraction succeeds. Across five model configurations, 52–88% of chain-of-thought tokens are produced after the answer is recoverable, free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases, and verified overwrite cases show full-CoT rollouts initially producing the correct answer before replacing it later (Wang et al., 8 Apr 2026). This suggests an extinction-like regime of prompt-conditioned extractability rather than the majority-vote lock-in studied in TTRL.

A related training-time view appears in S-GRPO, which samples multiple exit positions from a single reasoning trajectory and rewards earlier correct exits more than later ones. Across GSM8K, AIME 2024, AMC 2023, MATH-500, and GPQA Diamond, S-GRPO reduces sequence length by 35.4%–61.1% while improving accuracy by 0.72%–6.08% (Dai et al., 12 May 2025). A plausible implication is that many reasoning traces contain a substantial suffix after correctness is already preserved, so the onset of correctness and the final emitted answer are not temporally coincident.

At the representation level, forced-completion probing shows a distinct suppression regime under incorrect-answer forcing. In models at or above roughly 1.6B parameters, correct and incorrect trajectories diverge in middle layers, while the commitment ratio for the correct token under incorrect forcing collapses late, reaching $\{a_i^{(k)}\}_{k=1}^K$4 at normalized depths 0.85 in LLaMA-2 13B and 0.91 in Mistral 7B; the phenomenon is absent in Qwen2 1.5B (Marín, 25 Feb 2026). This is not the TTRL extinction window, but it is a close analogue of late-stage active suppression rather than mere non-recovery.

Efficiency-RL work provides a second training analogue. Under GRPO efficiency optimization, any tested nonzero continuous penalty on incorrect answers leads to collapse, while correct-only rewards avoid the primary structural failure but still permit a stochastic collapse in which correct traces become over-compressed. The paper calls this a sharp phase boundary at $\{a_i^{(k)}\}_{k=1}^K$5 and documents a collapsed $\{a_i^{(k)}\}_{k=1}^K$6 seed whose MATH-500 accuracy falls from 87.8 at step 400 to 55.8 at step 1000 as mean tokens compress from 2778 to 984 (Lee et al., 21 Jun 2026). This is not framed as extinction of a minority correct label, but it is another setting where correct-answer behavior disappears after a transient regime.

6. Broader analogues, limits, and interpretive boundaries

The term also has instructive analogues outside autoregressive reasoning optimization. In educational assessment, the “correct answer trap” describes cases where a student reaches the correct answer through flawed reasoning, making the misconception invisible to correctness-only systems. In a dataset of 20,964 Eedi responses, hidden misconceptions occur at 1.6% prevalence; fine-tuned classifiers detect only 57.4% of them, while a reasoning model detects 83.6% but yields roughly 8 false alarms per genuine detection at realistic prevalence (Imran et al., 22 Jun 2026). A plausible interpretation is that correctness can create a pedagogical extinction window in which the underlying misconception becomes extinct from observation, not from the student’s reasoning.

In video understanding, absent-answer detection creates a structurally related regime by removing the true answer from the candidate set. Across VideoMME and EgoSchema, multimodal LLMs overwhelmingly select plausible distractors rather than detecting answer absence; unprompted detection rates are near zero, and the failure is more pronounced in temporal reasoning tasks and worsens with denser frame sampling (Wang et al., 6 Jun 2026). Here the extinct element is not a minority correct label in training but the correct candidate in the answer space itself.

These broader analogues delimit the concept’s interpretive boundaries. The original paper’s term is specific to label-free TTRL with majority-vote pseudo-labeling (Lin et al., 19 May 2026). It does not mean any temporary answer uncertainty, any non-monotone accuracy trajectory, or proof of literal erasure from latent state. Indeed, adjacent papers explicitly caution against stronger claims: behavioral recoverability “upper-bounds latent commitment” in the detection–extraction work, and forced-completion probing measures token-level suppression under incorrect forcing rather than full internal-state disappearance (Wang et al., 8 Apr 2026). The safest encyclopedic usage therefore reserves Correct-Answer Extinction Window for the TTRL phenomenon while recognizing a broader family of extinction-like regimes in which correct-answer signal is briefly present, then rendered inaccessible by decoding, training, or evaluation dynamics.