D³: Dynamic Directional Scheduling for LLMs

• D³ is a dynamic, directional data scheduling framework that treats training batches as non-exchangeable, stressing the importance of order in LLM training.
• It models pairwise batch influences via a dynamic directed graph using first-order and curvature-aware second-order estimates to inform batch ordering.
• The framework employs scalable techniques like chunking and gradient compression, achieving significant perplexity reductions and improved performance on reasoning tasks.

Searching arXiv for the specified paper and closely related scheduling work to ground the article. arxiv_search(query="(Xu et al., 29 May 2026)", max_results=5, sort_by="submittedDate") arxiv_search(query="D^3 Dynamic Directional graph-constrained Data scheduling for LLM training", max_results=10, sort_by="relevance") D$^3$, short for Dynamic, Directional graph-constrained Data scheduling, is a data scheduling framework for LLM training that treats training data as non-exchangeable: the utility of a batch depends not only on its intrinsic properties but also on when it is presented relative to other batches. The framework is motivated by the claim that real-world training samples exhibit asymmetric, evolving interactions, so that one batch may lower the future loss of another without the reverse being equally true. D$^3$ models these interactions as a dynamic directed influence graph, converts the resulting pairwise preferences into a constrained ordering problem, and executes training according to an approximate solution that is recomputed online during training (Xu et al., 29 May 2026).

1. Conceptual basis and motivation

Most prior data scheduling or reweighting methods are described as focusing on the overall distribution of training data, such as domain proportions or sample difficulty, while neglecting interactions between training units. D$^3$ is built on the opposing premise that the order of presentation is itself a primary optimization variable because a batch seen earlier can “prepare” the model in a way that changes the effect of a later batch. In this formulation, the central question is: given the current model state, which batches should be trained first so the whole epoch is most effective? (Xu et al., 29 May 2026)

The operational principle is explicit: if training batch $i$ now significantly lowers the future loss of batch $j$, then $i$ should precede $j$. This shifts the problem from static mixture design to graph-structured scheduling over evolving pairwise dependencies. The framework therefore departs from static curricula and fixed mixture weights by assuming that directional interactions are both asymmetric and state-dependent. A plausible implication is that scheduling quality cannot be reduced to a scalar notion of batch hardness or importance; it depends on relational structure among batches at the current parameter state.

A common misconception is to treat D$^3$ as a reweighting method. The formulation indicates otherwise: the method does not primarily change how much data from a source is used, but which batch comes before which other batch. This distinction is substantive because the core signal is not a marginal score assigned to an individual batch, but a directional preference induced by pairwise influence.

2. Training influence prediction and asymmetry

D$^3$ defines the interaction between two batches through a look-ahead loss change. Given model parameters $\theta^{(t)}$, the method considers a hypothetical gradient step on batch $^3$0: $^3$1 where $^3$2 is a look-ahead step size used only for influence estimation. The influence of batch $^3$3 on batch $^3$4 is then

$^3$5

If $^3$6, updating on $^3$7 makes $^3$8 easier later (Xu et al., 29 May 2026).

Using a second-order Taylor expansion, the paper derives

$^3$9

The interpretation of this expansion is central. The first-order term is symmetric,

$^3$0

because it is a gradient inner product. By contrast, the second-order term is generally asymmetric,

$^3$1

because it depends on the batch-specific Hessian. D$^3$2 therefore argues that the decisive scheduling signal arises from curvature-aware asymmetry, not merely from gradient similarity.

This framing is significant because it identifies why static or first-order-only heuristics can be insufficient. If pairwise interactions were effectively symmetric, ordering would carry less information than weighting. The explicit asymmetry of the second-order term instead supports the claim that training order matters in a genuinely directional sense.

3. Dynamic influence graph and constrained ordering

At each step $^3$3, D$^3$4 constructs a directed weighted graph

$^3$5

whose nodes correspond to batches $^3$6. An edge $^3$7 exists if $^3$8, with weight $^3$9, and the graph is encoded by

$i$0

Because $i$1 depends on $i$2, the graph changes during training; the framework is therefore explicitly dynamic (Xu et al., 29 May 2026).

To translate pairwise influences into ordering preferences, the method defines the directional advantage matrix

$i$3

so that

$i$4

If $i$5, then training $i$6 before $i$7 is preferable. The corresponding dominance edge set is

$i$8

The schedule is represented by a permutation $i$9, and D$j$0 seeks

$j$1

This is a violation-cost minimization objective. Whenever $j$2, the preferred order is $j$3 before $j$4; if the realized order reverses that relation, the schedule incurs a penalty of $j$5. If the dominance graph is acyclic, a topological ordering yields zero cost. If cycles exist, the schedule is a compromise that breaks weaker directional constraints.

A plausible implication is that D$j$6 treats scheduling not as rank aggregation from independent batch scores, but as a constrained linearization of a directed relational structure. That distinction explains why the framework emphasizes graph construction and permutation optimization rather than only batch scoring.

4. Approximation, Random-Swap Refinement, and scalable implementation

Because exact permutation search is factorial in $j$7, D$j$8 uses a heuristic solver called Random-Swap Refinement (RSR). The first stage computes, for each batch $j$9,

$i$0

The initial ordering sorts batches in descending $i$1, where a larger row sum indicates more global advantage as an early batch. The second stage performs up to $i$2 random swap trials. For a swap of positions $i$3, with batches $i$4 and $i$5, the change in cost is

$i$6

where $i$7 is the set of batches between $i$8 and $i$9. The swap is accepted if $j$0. The reported complexity is $j$1 for row-sum sorting, $j$2 worst case per swap, and $j$3 in total (Xu et al., 29 May 2026).

Scalability is addressed through two approximations. First, D$j$4 applies chunk-wise internal reordering: instead of scheduling all $j$5 batches at once, it samples a chunk $j$6 of size $j$7, builds the influence graph only within that chunk, and reorders only those batches. This reduces per-step cost from $j$8 to roughly $j$9. Second, it uses gradient compression and curvature approximation. Batch gradients are projected into a low-dimensional random subspace,

$^3$0

and the first-order influence is approximated via preserved inner products: $^3$1

For second-order information, the framework uses a Hutchinson-style diagonal Hessian estimate and compresses curvature into a scalar

$^3$2

leading to the efficient approximation

$^3$3

The paper states that a small number of Hutchinson probes, typically $^3$4, is used to balance speed and fidelity.

The online pipeline is correspondingly direct: sample $^3$5 batches, compute compressed gradients and curvature estimates, build the directional advantage matrix $^3$6, solve for an order using RSR, and perform SGD updates in that order. The method is therefore a dynamic online scheduler, not a preprocessing step performed once before training.

5. Theoretical analysis and interpretive scope

The theoretical analysis is presented in a simplified linear regression setting. There, the loss reduction from two samples depends on their interaction, and the difference between the losses obtained by the two possible orders is

$^3$7

This expression shows that the order of two examples can change the resulting loss and that the effect depends on both sample similarity and residuals (Xu et al., 29 May 2026).

The paper further proves a convergence-acceleration style result: an influence-aware policy can achieve lower expected loss than random ordering by exploiting pairwise differences. However, the stated theorem holds in a regression proxy setting, not for full Transformer training. The theory is therefore presented as motivation and illustration rather than a complete guarantee for large-scale LLM optimization.

This boundary matters for interpretation. The formal argument supports the idea that ordering effects are structurally real, but it does not eliminate the gap between proxy analysis and production-scale autoregressive training. A plausible implication is that the theory should be read as validating the mechanism—interaction-sensitive ordering—rather than fully characterizing its behavior in contemporary foundation-model regimes.

6. Empirical performance, ablations, and limitations

The empirical results are reported for both pre-training and post-training. On GPT-2 Medium trained on 100B tokens from SlimPajama, D$^3$8 achieves lower perplexity than all baselines, including Uniform Sampling, Dynamic Loss, RegMix, Data Mixing Law, DoReMi, and DoGE. The paper reports a 4.2% relative PPL reduction over the strongest baseline and an increase in aggregated commonsense reasoning accuracy to 42.56, compared to the strongest baseline around 41.26, with especially strong improvements on OBQA and COPA. On a 1.1B Llama-based model trained on 100B tokens from The Pile, D$^3$9 again outperforms all baselines, reaching PPL 2.53, a 4.1% improvement over the strongest baseline, while also improving commonsense reasoning on most tasks and again showing particularly strong gains on OBQA and COPA (Xu et al., 29 May 2026).

In supervised fine-tuning, the framework is evaluated on Llama-3-8B with datasets such as CodeAlpaca, GSM8K-RFT, and Alpaca-GPT4. It is reported to surpass static curricula such as easy-to-hard and hard-to-easy, multi-task learning baselines, and specialization ceiling baselines. The reported gains are GSM8K: +3.1%, MATH: +11.4%, HumanEval: +8.0%, MBPP: +6.6%, with an average improvement: +6.7%. The paper’s explicit interpretation is that dynamic scheduling based on current influence is better than fixed curricula.

The ablation results indicate that first-order-only influence is cheap but weak because it misses asymmetry; Fisher-based curvature helps somewhat but remains less effective than batch-specific diagonal Hessian estimation; and the diagonal Hessian / Hutchinson approach provides the best performance-efficiency tradeoff, while still adding overhead. The best setting is reported to use projection dimension $^3$0 for GPT-style models and chunk size $^3$1 for better scheduling quality. In solver comparisons, random ordering is worst, row-sum sorting and greedy insertion are better, and RSR gives the best perplexity, though with some extra GPU hours.

The paper is explicit about limitations. Exact pairwise influence estimation over all batches is infeasible for LLMs; second-order estimation increases overhead, though within a practical range; and chunking together with random projection is essential for scalability. Additional stated limitations are that resource cost could be further reduced with better gradient compression or layer-selective methods, the theory is proven only in a simplified linear regression proxy, the method relies on random projection and Hutchinson diagonal Hessian estimates together with local influence computed at the current $^3$2, and it assumes batch-level or sample-level scheduling within an epoch rather than solving the full global combinatorial scheduling problem exactly. These caveats position D$^3$3 as a principled but approximate approach to influence-aware ordering rather than an exact solution to large-scale scheduling.