Lie-Bracket Tournament

• Lie-Bracket Tournament is a geometric approach that models order dependence in sequential learning using Lie-bracket commutators of gradient update fields.
• The method efficiently computes pairwise curriculum preferences via Hessian-vector products and dot products without constructing a full O(N²) matrix.
• Empirical results demonstrate high short-horizon prediction accuracy and scalable performance, though accuracy gradually degrades as the fine-tuning horizon lengthens.

The Lie-Bracket Tournament is a curriculum-planning method for sequential learning in which order effects are modeled through the Lie-bracket commutator of gradient update fields. In the formulation of "The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order" (Sweeney, 23 Jun 2026), sequential learning is order-dependent because, from Pile-style next-token domain adaptation to instruction-SFT and DPO, $N$ candidate sources induce $N!$ possible curricula. The method assigns a local pairwise score indicating whether $A\to B$ or $B\to A$ is preferable for a target domain, and then lifts these pairwise preferences to a many-domain ranking by a weighted tournament construction using Borda or row-sum scores. Its reported computational form uses one Hessian-vector product per source, $O(N)$ dot products, and an $O(N\log N)$ sort, without materializing the $O(N^2)$ edge matrix (Sweeney, 23 Jun 2026).

1. Geometric formulation of order dependence

The method views model parameters $\theta\in\mathbb R^p$ as a point on a smooth manifold. Each training domain $D$ defines a loss function

$L_D(\theta): \mathbb R^p \to \mathbb R$

and its gradient field

$N!$0

A small gradient-descent update on $N!$1 with step size $N!$2 is

$N!$3

or, equivalently, the vector field

$N!$4

whose flow for time $N!$5 moves $N!$6 along the manifold (Sweeney, 23 Jun 2026).

Within this framework, order dependence is identified with non-commutativity of update operators. For two domains $N!$7, one generally has $N!$8, so the order of sequential fine-tuning matters. The operational question is local and target-specific: whether $N!$9 or $A\to B$0 yields lower loss on a held-out target domain $A\to B$1. The Lie-Bracket Tournament supplies a geometric diagnostic for that question by measuring the commutator of the two update fields.

2. Lie-bracket commutator and pairwise prediction

For smooth vector fields $A\to B$2, the Lie bracket is

$A\to B$3

Specialized to gradient-flow fields $A\to B$4 and $A\to B$5, the bracket becomes

$A\to B$6

where $A\to B$7 is the Hessian. The corresponding bracket vector at $A\to B$8 is

$A\to B$9

A second-order Baker-Campbell-Hausdorff expansion gives

$B\to A$0

Projecting this displacement onto the target gradient yields the directional score

$B\to A$1

so that

$B\to A$2

Its sign predicts the better order: if $B\to A$3, then $B\to A$4 in target loss (Sweeney, 23 Jun 2026).

To reduce $B\to A$5 errors in $B\to A$6, the method evaluates the target gradient at the shared-drift reference

$B\to A$7

The deployable Trotter estimate is

$B\to A$8

The same construction defines stakes $B\to A$9 and a normalized confidence $O(N)$0. Computationally, $O(N)$1 requires only the two Hessian-vector products $O(N)$2 and $O(N)$3, each computed by Pearlmutter’s trick without forming $O(N)$4 explicitly (Sweeney, 23 Jun 2026).

3. Tournament construction for many-domain curricula

For $O(N)$5 source domains $O(N)$6 and one target $O(N)$7, the method forms a weighted tournament graph with vertex set $O(N)$8. Fixing a shared reference gradient $O(N)$9, each ordered pair $O(N\log N)$0 receives the score

$O(N\log N)$1

Here $O(N\log N)$2 means “$O(N\log N)$3 before $O(N\log N)$4” (Sweeney, 23 Jun 2026).

The key simplification uses Hessian symmetry. Defining

$O(N\log N)$5

one obtains

$O(N\log N)$6

This permits a Borda or row-sum ranking. The classical Borda score of $O(N\log N)$7 is

$O(N\log N)$8

Sorting domains in descending $O(N\log N)$9 yields a full curriculum.

The same algebra yields a linear-time score computation: $O(N^2)$0 Hence the procedure is: compute $O(N^2)$1 once; for $O(N^2)$2, compute $O(N^2)$3 and $O(N^2)$4; form $O(N^2)$5 and $O(N^2)$6; then compute each $O(N^2)$7 by two dot products and sort. The reported overall cost is $O(N^2)$8 gradients, $O(N^2)$9 HVPs, and $\theta\in\mathbb R^p$0 sorting; the detailed complexity is $\theta\in\mathbb R^p$1 for gradients, $\theta\in\mathbb R^p$2 for HVPs, $\theta\in\mathbb R^p$3 for dot-products and sums, and $\theta\in\mathbb R^p$4 for sorting (Sweeney, 23 Jun 2026).

4. Empirical performance

The reported experiments cover post-training, pretraining-domain adaptation, diffusion, long-horizon generalization, and many-domain scheduling (Sweeney, 23 Jun 2026).

Regime	Setting	Reported result
Post-training	Qwen2.5-1.5B, $\theta\in\mathbb R^p$5	98.1% sign accuracy (SFT), 98.9% (DPO)
Post-training	Same predictor, evaluated at $\theta\in\mathbb R^p$6	73.1% (SFT), 72.2% (DPO)
Long-horizon	Llama-3.2-1B, predictor computed at $\theta\in\mathbb R^p$7	93.0% at $\theta\in\mathbb R^p$8, 94.3% at $\theta\in\mathbb R^p$9, 89.4% at $D$0, 81.5% at $D$1, 65.3% at $D$2
Exact $D$3 scheduling	Best of all $D$4 orders	87.5% top-1, 96.0% top-2, mean Spearman 0.929
Borda $D$5 scheduling	Top-1 recovery	79.0%
Large-$D$6 scheduling	MMLU subjects, $D$7	99.0–99.6th sampled percentile
Large-$D$8 scheduling	Stack languages $D$9 Python, $L_D(\theta): \mathbb R^p \to \mathbb R$0	99th sampled percentile
Large-$L_D(\theta): \mathbb R^p \to \mathbb R$1 scheduling	Dolly summarization, $L_D(\theta): \mathbb R^p \to \mathbb R$2	99.8th sampled percentile

For pretraining-domain adaptation and diffusion, the paper reports results over $L_D(\theta): \mathbb R^p \to \mathbb R$3 source pairs and $L_D(\theta): \mathbb R^p \to \mathbb R$4 seeds. The Trotter accuracy is $L_D(\theta): \mathbb R^p \to \mathbb R$5 for Qwen2.5-1.5B with Acc@25% $L_D(\theta): \mathbb R^p \to \mathbb R$6, $L_D(\theta): \mathbb R^p \to \mathbb R$7 for Llama-3.2-1B with Acc@25% $L_D(\theta): \mathbb R^p \to \mathbb R$8, $L_D(\theta): \mathbb R^p \to \mathbb R$9 for Llama-3.1-8B with Acc@25% $N!$00, $N!$01 for SmolLM3-3B with Acc@25% $N!$02, and $N!$03 for a DDPM diffusion UNet with Acc@25% $N!$04 (Sweeney, 23 Jun 2026).

The same study reports regret reductions $N!$05 in the post-training setting and $N!$06 even at $N!$07 in the long-horizon setting. For sampled-percentile evaluation against $N!$08 random curricula, the many-domain planner reaches $N!$09–$N!$10th percentile on MMLU with $N!$11, $N!$12th percentile on Stack with $N!$13, and substantially exceeds the reported descending gradient-norm baseline, which is listed as $N!$14 on MMLU subjects with $N!$15, $N!$16–$N!$17 on MMLU with $N!$18, $N!$19th on Stack languages $N!$20 Python with $N!$21, $N!$22th on Stack with $N!$23, and $N!$24th on Dolly summarization with $N!$25 (Sweeney, 23 Jun 2026).

5. Mathematical status of the “Lie bracket” terminology

In the Lie-Bracket Tournament, the Lie bracket is the commutator of gradient update fields on parameter space, and its role is predictive: it captures the leading-order non-commutativity of two learning steps. This use of Lie brackets is geometric and local, rather than a definition of a fixed algebraic bracket on a static vector space (Sweeney, 23 Jun 2026).

That distinction matters because the same terminology has a different technical meaning in other arXiv work. In "Cup product, Frölicher-Nijenhuis bracket and the derived bracket associated to Hom-Lie algebras" (Baishya et al., 2024), the brackets $N!$26, $N!$27, $N!$28, and $N!$29 are graded Lie brackets on cochain spaces attached to a multiplicative Hom-Lie algebra $N!$30. There, the cup-product bracket is used for deformations of Hom-Lie morphisms, the Hom–Nijenhuis–Richardson bracket controls Hom-Lie algebra brackets themselves, the Frölicher–Nijenhuis bracket characterizes Nijenhuis operators, and the derived bracket gives a DGLA whose Maurer-Cartan equation is exactly the Rota-Baxter identity of weight $N!$31 (Baishya et al., 2024).

Likewise, in "The bracket of the exceptional Lie algebra E8" (Kollross, 23 Apr 2025), the bracket is the explicit Lie algebra product on the compact real form of $N!$32, written in the Barton-Sudbery decomposition

$N!$33

with triality and the oct-octonion $N!$34-product entering directly into the formula. The shared phrase “Lie bracket” therefore spans several mathematically distinct settings: geometric commutators of update fields, graded brackets on cochain complexes, and explicit structure brackets of finite-dimensional simple Lie algebras (Kollross, 23 Apr 2025).

6. Interpretation, scope, and limitations

The reported interpretation is that the score $N!$35 captures the leading-order geometric non-commutativity of two fine-tuning steps and predicts pairwise order effects with very high accuracy at $N!$36. Through the Trotter reference, the method corrects $N!$37 drift and provides stakes and confidence. Embedded in a Borda tournament, the pairwise scores yield a fully automated curriculum over $N!$38 domains at cost $N!$39 HVPs and $N!$40 sorting (Sweeney, 23 Jun 2026).

At the same time, the paper does not present the score as an exact long-horizon optimizer. The construction is explicitly local, being derived from a second-order expansion around $N!$41, and the reported long-horizon results show that accuracy decays as $N!$42 increases: from $N!$43 at $N!$44 to $N!$45 at $N!$46 in the Llama-3.2-1B setting. A common misunderstanding would be to read the tournament ranking as a guarantee of globally optimal ordering for arbitrary horizons; the reported evidence instead supports strong local prediction, graceful degradation with horizon, and near-optimal behavior in the evaluated curriculum settings (Sweeney, 23 Jun 2026).

This suggests a precise conceptual summary. The Lie-Bracket Tournament reframes sequential transfer as a geometric tournament problem in which local commutator information is aggregated into a scalable curriculum planner. A plausible implication is that its usefulness is greatest when short-horizon order effects dominate, while its empirical diffusion and many-domain results indicate that the same geometric primitive can remain informative well beyond the two-domain case.