Training Re-evaluation Curve (TREC)

• TREC is a diagnostic tool that re-evaluates training batches with the final model to reveal how well they are retained across training.
• It identifies optimal data insertion periods by detecting minima in the loss curve where the final model best aligns with past training batches.
• Developed via AdamW-based EMA coefficients and drift modeling, TREC supports curriculum design and continual pre-training improvements.

Searching arXiv for the cited TREC-related papers to ground the article in current literature. Search "Training Re-evaluation Curve TREC (Bergsma et al., 29 Sep 2025)" Training Re-evaluation Curve (TREC) is a diagnostic for LLM training defined by retrospectively evaluating each original training batch with the final model parameters. In the formulation introduced in "Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs" (Bergsma et al., 29 Sep 2025), the curve is $L(t) := \mathcal{L}(\mathcal{B}_t; \theta_T)$ for training steps $t=1,\dots,T$, where $\mathcal{B}_t$ is the batch seen at step $t$ and $\theta_T$ is the fully trained parameter vector. The resulting time-indexed loss profile characterizes how well the final model retains training data as a function of when that data was encountered, and its low points are proposed as preferred insertion regions for scarce high-quality data. The acronym is distinct from the classic information retrieval usage of TREC as Text REtrieval Conference and TREC-style evaluation methodology (Zhou et al., 2022).

1. Definition and interpretive framework

TREC is constructed after training finishes. Instead of recording the loss of $\mathcal{B}_t$ at the moment it was seen, the final model is frozen and re-evaluated on the training batches in their original order. The horizontal axis is training time, represented either by the discrete batch index $t$ or the normalized training fraction $t/T \in [0,1]$. The vertical axis is the retrospective loss of the final model on the batch seen at that time, typically cross-entropy loss, plotted either in absolute form or normalized/min-max scaled form when the emphasis is on shape rather than magnitude (Bergsma et al., 29 Sep 2025).

The central empirical observation is that the curve is typically not flat. It often exhibits a valley: a region of training whose batches receive lower final loss than batches from much earlier or later in training. Low TREC loss at time $t$ indicates that the final model is especially well aligned with batches from that period; high TREC loss indicates that those batches are less well retained, more forgotten, or less aligned with the final-state optimization geometry. A TREC minimum is therefore treated as the region where injected high-quality data should have the greatest impact.

The proposal is explicitly curricular rather than merely descriptive. The paper defines high-quality data operationally: given training distribution $D$, a distribution $t=1,\dots,T$0 is high-quality for a task if replacing $t=1,\dots,T$1 samples with $t=1,\dots,T$2 samples improves task performance. Under that definition, code-heavy or math-heavy mixtures are high-quality only relative to the corresponding target validation tasks. The framework also assumes homogeneous i.i.d. training data for defining and analyzing TREC shape, so that the curve reflects training dynamics rather than content heterogeneity (Bergsma et al., 29 Sep 2025).

A key conceptual distinction is between training loss and re-evaluation loss. The standard heuristic that high-quality data should be placed at the very end relies on low instantaneous training loss late in training, but the paper reports that TREC can rise near the end under decay-to-zero or step-drop schedules. This means late placement can be suboptimal even when training loss is low. This suggests that TREC is intended to measure retained usefulness under the final model rather than instantaneous ease of fitting.

2. Mathematical formulation and optimizer-level explanation

The optimizer-level account begins with AdamW:

$t=1,\dots,T$3

where $t=1,\dots,T$4 is learning rate, $t=1,\dots,T$5 is weight decay, $t=1,\dots,T$6 and $t=1,\dots,T$7 are bias-corrected first- and second-moment estimates, and $t=1,\dots,T$8 is the numerical stabilizer. Following the paper’s formulation, this can be interpreted as an exponential moving average of updates,

$t=1,\dots,T$9

with $\mathcal{B}_t$0, $\mathcal{B}_t$1, and

$\mathcal{B}_t$2

This yields an optimizer memory timescale of roughly $\mathcal{B}_t$3 and, for single-epoch LLM pretraining, the normalized AdamW timescale

$\mathcal{B}_t$4

where $\mathcal{B}_t$5 is batch size in tokens, $\mathcal{B}_t$6 is total number of training tokens, and $\mathcal{B}_t$7 is the number of optimization steps (Bergsma et al., 29 Sep 2025).

For time-varying learning rates, the smoothing parameter becomes $\mathcal{B}_t$8, and the unrolled EMA for the final parameters induces stepwise contribution coefficients

$\mathcal{B}_t$9

The continuous function $t$0 over normalized training fraction is interpreted as the retrospective weighting of updates from each training time into the final model. These coefficients are not themselves the TREC, but they are the first major ingredient in its prediction.

The predictive approximation for normalized TREC shape is

$t$1

where $t$2 is the normalized AdamW EMA coefficient for the final weights, $t$3 controls the strength of the EMA contribution, and $t$4 is a training-fraction exponent capturing drift or decay in the usefulness of earlier gradients. The experiments fix $t$5 and focus on fitting or predicting $t$6 (Bergsma et al., 29 Sep 2025).

The justification for the extra $t$7 term comes from a simplified quadratic analysis. If batch-specific local minimizers drift as training progresses, then EMA weighting alone overstates the influence of early updates. In the appendix, the paper writes the retrospective loss using the final parameters and shows that low TREC loss requires both high optimizer weight $t$8 and good alignment between the update at time $t$9 and the final batch-specific minimizer. This is why the curve is governed by both optimizer persistence and minimizer drift rather than by learning-rate schedule alone.

The paper also derives a small-weight-decay limit in which $\theta_T$0 becomes proportional to $\theta_T$1, so the TREC shape approaches a learning-rate-shaped limit as $\theta_T$2. This provides the analytical basis for the reported transition toward learning-rate-dominated behavior at very small weight decay (Bergsma et al., 29 Sep 2025).

3. Empirical structure across model size, schedule, and scaling regime

The empirical study spans models from 111M to 3.9B parameters, 578 TREC computations and 41 placement-trained models, and datasets from 20 to 1280 TPP, using GPT2-style LLMs with AdamW, context lengths 2048 or 8192, and optimizer settings $\theta_T$3, $\theta_T$4 (Bergsma et al., 29 Sep 2025). Across this range, the paper reports that TREC shape is largely controlled by the AdamW timescale $\theta_T$5 for a fixed learning-rate schedule.

For 610M models at 80 TPP, sweeping learning rate $\theta_T$6, weight decay $\theta_T$7, and batch size $\theta_T$8 while matching

$\theta_T$9

produces very similar normalized TREC shapes. As $\mathcal{B}_t$0 increases, the TREC valley shifts earlier in training. When models are trained at fixed TPP and similar $\mathcal{B}_t$1, TRECs align from 111M to 3.3B, despite about 1000× difference in training FLOPs. At larger TPP, the valley shifts slightly later and the absolute depth of the TREC drop shrinks. The authors interpret this as reduced memorization or reduced plasticity in overtrained regimes.

The paper further reports that normalized TREC shape is stable across batch sizes up to around critical batch size but diverges at very large batches far beyond it. Sweeping $\mathcal{B}_t$2 shows that normalized TREC shape is mostly insensitive to momentum settings, with the weight-decay timescale dominating (Bergsma et al., 29 Sep 2025).

Prediction accuracy is evaluated at the level of shape rather than absolute scale. The exponent $\mathcal{B}_t$3 is fit using 111M models and transferred across scales. The reported fit $\mathcal{B}_t$4 for predicted $\mathcal{B}_t$5 is about 99% at 111M, declining to 77% at 3.3B, while predicted TREC shape remains highly accurate, with Pearson correlation around 98% across scales. Schedule transfer is also reported: using a decay-to-zero fit on $\mathcal{B}_t$6 decay runs gives $\mathcal{B}_t$7, and schedule-specific fitting improves this to $\mathcal{B}_t$8. This suggests that exact estimation of $\mathcal{B}_t$9 is less critical than preservation of the overall curve geometry.

4. Curriculum placement and continual pre-training

The core curricular claim is direct: place high-quality data at low points of the TREC (Bergsma et al., 29 Sep 2025). In the paper’s mid-training placement study, 610M models are trained on 50B tokens (82 TPP) consisting of 45B “general blend” tokens and 5B “code blend” high-quality tokens, with code validation as the target. The code blend upweights GitHub and StackExchange relative to the general blend. For each learning-rate schedule, 10 models are trained, each placing the 5B high-quality code tokens into a different 10% segment of training.

Across tested schedules—step-drop schedules, linear decay-to-zero, and linear decay-to-$t$0 variants—the best validation performance occurs when high-quality data is placed at the lowest point of the TREC. The paper also reports that optimal placement beats uniform mixing of the same high-quality data throughout training. A complementary ablation evaluates general-blend validation after replacing some general-blend data with code-blend data; the largest degradation occurs when code displaces general-blend data at the TREC minimum, implying that those positions are also where the baseline data was most valuable.

The framework is also used to reinterpret published training recipes. The paper states that Llama-3 405B likely saw no gain from late annealing on GSM8k/MATH because the final 40M tokens over batch size 16M amount to only about 3 optimizer steps at vanishing learning rate. It further reports that the best onset for the high-quality phase in Feng et al. aligns with the predicted TREC bottom, that OLMo-2 13B places its high-quality blend during the final 5.7% in a way that aligns with a narrow TREC dip, and that Pangu-Ultra 135B aligns reasoning and annealing phases well with the predicted TREC valley, though perhaps slightly too late (Bergsma et al., 29 Sep 2025).

The largest demonstration concerns continual pre-training. A 3.9B model is pretrained on 900B tokens (about 234 TPP) and then continued for an extra 18B tokens (about 5 TPP) during continual pre-training. A 1.3B-token high-quality math blend that upweights OpenWebMath and UltraTextBooks is inserted either halfway through continual pre-training, where TREC is low, or at the very end, where TREC returns to baseline. Placing the high-quality data at the TREC minimum outperforms end placement across all tested continual-pre-training learning rates. At the same time, the paper reports an important caveat: across different continual-pre-training learning rates, deeper TREC valleys do not always imply better validation, so TREC is reliable within a schedule but not always for choosing among schedules (Bergsma et al., 29 Sep 2025).

5. Practical use and methodological limits

The paper presents two operational modes (Bergsma et al., 29 Sep 2025). In retrospective analysis mode, one saves or reconstructs the sequence of training batches or coarse training segments, evaluates the final model $t$1 on those batches in original order, plots $t$2 against training fraction, identifies valleys or minima, and uses those regions for data placement in a retraining run. In predictive curriculum-design mode, one specifies the planned learning-rate schedule $t$3, weight decay $t$4, total training tokens $t$5, batch size $t$6, total steps $t$7, tokens-per-parameter, and a schedule-specific fit or estimate for $t$8; computes

$t$9

forms

$t/T \in [0,1]$0

and places scarce high-quality data into windows around the minima of the predicted curve.

Several methodological constraints are explicit. Observed TREC is optimizer-agnostic, but the predictor is AdamW-specific. The predictive model targets normalized shape, not absolute retrospective cross-entropy values. The main evidence is for 20 TPP and above, i.e. mostly compute-optimal and overtrained regimes rather than very data-scarce undertrained settings. The formal derivation assumes i.i.d. homogeneous batches, although the paper reports that TREC-informed placement still transfers to heterogeneous inserted datasets. It also notes that observed TRECs require explicit re-evaluation on training data and can therefore be costly, whereas predicted TRECs require only schedule arithmetic plus a pre-fit exponent model.

A further limitation concerns schedule comparison. The paper explicitly distinguishes intra-schedule placement, where TREC is reliable, from cross-schedule selection, where absolute TREC values may fail to predict validation performance. This suggests that TREC is best viewed as a placement diagnostic conditioned on a training recipe rather than as a universal objective for choosing the recipe itself.

6. Acronym ambiguity and related usages

The term TREC is polysemous in arXiv literature. In "Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective" (Zhou et al., 2022), TREC refers to the classic information retrieval paradigm of Text REtrieval Conference, TREC-style pooling, and judgment-set construction rather than to a training diagnostic. That paper imports TREC methodology into knowledge graph completion by treating triple questions as IR queries and recommends TREC-style pooling and IR-like macro metrics to mitigate incomplete judgments. Its use of the acronym is therefore unrelated to Training Re-evaluation Curve.

A separate usage appears in "Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023" (Parry et al., 2024), where TREC denotes the TREC 2023 Deep Learning Track. That work studies adaptive re-ranking as a budgeted iterative process and reports effectiveness and ranking-correlation curves as re-ranking budget increases, but it does not define a training re-evaluation curve. Its convergence analysis is relevant only in the looser sense of measuring system behavior as additional evaluation effort is spent.

Within this landscape, Training Re-evaluation Curve designates a specific optimizer-aware diagnostic for LLM training. Its novelty lies in combining a retrospective loss curve over original training batches, the claim that TREC minima indicate optimal insertion windows for scarce high-quality data, and a predictive approximation derived from AdamW EMA coefficients and a drift term (Bergsma et al., 29 Sep 2025). This suggests that careful disambiguation is necessary whenever the acronym TREC appears in machine learning and information retrieval research.