Determine the useful learning-rate range and instability threshold for FP-to-ternary adaptation

Determine how far the stage-2 learning-rate sweep for the 90/10 full-precision-then-ternary training schedule can be extended before optimization becomes unstable, and characterize the useful learning-rate range beyond 3×10−3.

Background

The 90/10 schedule was evaluated with a stage-2 learning-rate sweep that improved monotonically through the largest tested value, 3×10−3. Since the sweep ended before saturation, the paper does not identify an optimum or the point at which the ternary adaptation becomes unstable.

Establishing this range is necessary for interpreting the apparent advantage of FP-initialized ternary training and for comparing it fairly with from-scratch ternary training.

References

The sweep had not saturated at its upper end, so 3×10−3 is a lower bound on the useful rate, not an estimated optimum; we did not extend it further, and where the recipe becomes unstable is untested.

— Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters  (2609.29397 - Veldanda, 24 Sep 2026) in Section 6.4, “A 90/10 FP-then-ternary schedule helps, at a learning rate 10× the pre-training peak”

3×10−3 is the optimal stage-2 LR sweep stops there not established

— Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters  (2609.29397 - Veldanda, 24 Sep 2026) in Section 6.7, Table 6, “What the experiments do and do not establish”