Rank-1 Trajectory Extrapolation in RLVR

• Rank-1 trajectory extrapolation uses a single dominant direction to forecast model weight changes, optimizing computer resources in RLVR.
• Key methods, RELEX and NExt, synthesize weights at advanced training stages, reducing time and preserving model performance.
• Empirical studies show extrapolation maintains accuracy while drastically cutting RLVR training steps by up to 80%.

Rank-1 trajectory extrapolation encompasses a family of parameter-efficient techniques for predicting the future evolution of model weights during Reinforcement Learning with Verifiable Rewards (RLVR), primarily in LLMs. It exploits the empirical observation that the optimization trajectory of model parameters under RLVR is dominated by a single, predictable low-rank direction. By fitting and extrapolating this rank-1 subspace, checkpoint weights at advanced training stages can be synthesized with minimal training, sharply reducing compute requirements while preserving or surpassing downstream performance.

1. Geometry of Parameter Trajectories under RLVR

Let $\theta_0$ denote the pretrained model parameters, and let $\theta_t$ be the model after $t$ steps of RLVR. The relative parameter delta is defined as

$\Delta\theta_t = \theta_t - \theta_0,$

with the stepwise delta given by $\delta\theta_t = \theta_t - \theta_{t-1}$ for $t \ge 1$. For practical purposes, all deltas are measured relative to $\theta_0$.

By stacking the flattened tensors $\Delta\theta_1, \dots, \Delta\theta_T$ into a matrix

$$M = \begin{pmatrix} (\Delta\theta_1)^\top\ (\Delta\theta_2)^\top\ \vdots\ (\Delta\theta_T)^\top \end{pmatrix} \in \mathbb{R}^{T\times d},$$

empirical analysis reveals that the singular value spectrum of $M$ is extremely concentrated: the top singular value accounts for the vast majority of the variance in each tensor's trajectory. This phenomenon persists across diverse model architectures and tasks under RLVR, and is further intensified through the use of low-rank adaptation such as LoRA (Wei et al., 20 May 2026, Chen et al., 13 Apr 2026).

2. Core Algorithms: RELEX and NExt

Two principal algorithmic frameworks exemplify rank-1 trajectory extrapolation: RELEX and NExt.

RELEX

RELEX (REinforcement Learning EXtrapolation) is a compute-efficient, training-free method for extrapolating weight updates along the dominant rank-1 subspace (Wei et al., 20 May 2026). The algorithm operates as follows:

1. Rank-1 SVD: For each tensor, form the trajectory matrix $\theta_t$0 from the first $\theta_t$1 RLVR checkpoints. Compute the top right singular vector $\theta_t$2 from the truncated SVD

$\theta_t$3

2. Projection: Project the observed deltas onto $\theta_t$4 to obtain scalar coefficients

$\theta_t$5

3. Linear Model Fitting: Fit a linear model $\theta_t$6 using ordinary least squares, with

$\theta_t$7

4. Extrapolation: For any target step $\theta_t$8, compute the extrapolated coefficient $\theta_t$9 and reconstruct weights by

$t$0

No gradient descent or model training is needed beyond the initial SVD and linear regression.

NExt

NExt (Nonlinear Extrapolation of low-rank Trajectories) extends rank-1 extrapolation to scenarios where the subspace itself evolves in a nonlinear manner (Chen et al., 13 Apr 2026). The methodology is summarized as:

1. Rank-1 Extraction: At multiple RLVR checkpoints, LoRA-based RLVR updates $t$1 are decomposed by SVD:

$t$2

These triplets $t$3 are collected for several intervals.

1. Predictor Training: Each triplet is concatenated into vectors representing both global and local deltas. These serve as input/output pairs for a multilayer perceptron (MLP) that predicts future delta triplets in the low-rank space.
2. Predict-and-Extend: At the last observed checkpoint, use the trained MLP to generate the next-step rank-1 triplet, reconstruct the predicted delta, and extend forward by a chosen scalar factor $t$4:

$t$5

where $t$6.

Ablation studies highlight the importance of LoRA-specific deltas and both global/local trajectory information. In contrast to RELEX, the nonlinear predictor handles situations where the rank-1 trajectory does not exhibit perfect linearity.

3. Ablation Studies and Minimalist Sufficiency

Systematic ablation confirms that the rank-1/linear approach has optimal bias–variance tradeoff in practice:

• Rank selection: Increasing the rank above 1 (e.g., to 5 or 10) provides no extrapolation benefit; higher components are noisy and non-monotonic, accounting for marginal variance.
• Model complexity: Neural-predictor or polynomial curve fits yield no consistent gains versus the linear-in-subspace approach, and high-degree fits fail catastrophically outside the observation window.
• Raw weights vs. subspace: Direct linear prediction in raw weight space underperforms by 0.5–1.0 points of task accuracy compared to SVD-projected subspace extrapolation.
• Subspace denoising: Empirical direction analysis demonstrates that SVD projection onto the leading singular vector acts as a spectral “low-pass filter,” removing stochastic optimization noise and stabilizing extrapolation in both RELEX and NExt (Wei et al., 20 May 2026).

4. Quantitative Performance and Extrapolation Range

Empirical benchmarks on Qwen2.5-Math-1.5B, Qwen3-4B-Base, and Qwen3-8B-Base yield the following key outcomes (Wei et al., 20 May 2026):

• RELEX achieves in-domain accuracy (e.g., 71.6% on MATH) matching or exceeding full RLVR with as little as 15–20% of the training steps.
• Out-of-domain accuracy is either preserved or improved, with consistent gains over baselines such as Weight/Logits Extrapolation, ExPO, and AlphaRL.
• Stable extrapolation is demonstrated for 10–20× the observed window (e.g., first 50 RLVR steps extrapolated to 1,000 steps), maintaining or improving target accuracy without additional RLVR training.
• NExt reduces RLVR compute by approximately 37.5% while remaining compatible with various RLVR algorithms and tasks, outperforming linear baselines by 5–10% absolute accuracy on held-out checkpoints (Chen et al., 13 Apr 2026).

5. Implementation Details and Pseudocode

Both RELEX and NExt are summarized by succinct, reproducible routines per tensor or parameter block:

Algorithm	Subspace Modeling	Extrapolation Method	Additional Training
RELEX	SVD (rank-1, top-$t$7)	Linear regression	None
NExt	SVD per interval	Neural predictor (MLP)	Yes (MLP fit)

• RELEX: Only two SVDs and OLS regression per tensor are needed; see the detailed pseudocode in (Wei et al., 20 May 2026).
• NExt: MLP is trained on concatenated SVD triplets and predicts future deltas; see Algorithm 1 and associated equations in (Chen et al., 13 Apr 2026).

Key hyperparameters include the number of RLVR steps (typically $t$8–$t$9 of full RLVR), SVD rank (fixed at 1), and LoRA configuration details for NExt.

6. Limitations, Scope, and Extensions

Analysis across multiple LLMs confirms that, under RLVR and LoRA, the model parameter trajectory is overwhelmingly governed by a single rank-1 direction, facilitating faithful extrapolation. However, exceptions arise:

• NExt observes that in some instances, the rank-1 subspace can drift nonlinearly, necessitating nonlinear predictors.
• More than 50% of entries in $\Delta\theta_t = \theta_t - \theta_0,$0 may show poor linear least-squares predictability to the next checkpoint in certain models (Chen et al., 13 Apr 2026).
• The reported approaches are benchmarked on LLMs fine-tuned for reasoning tasks, particularly mathematical problem solving; generalization to radically different RLVR settings or tasks merits further empirical evaluation.

A plausible implication is that the spectral structure of optimization trajectories enables substantial amortization of RLVR training costs, and that future methods may explore hybrid, adaptive, or uncertainty-aware extrapolation schemes within this framework.

7. Relationship to Broader Low-Rank and Efficient Optimization

Rank-1 trajectory extrapolation builds on and extends a broader literature on low-rank model adaptation, trajectory modeling, and efficient RLVR optimization in large models. LoRA-specific deltas, high energy ratios in leading singular values, and denoising projections are recurring techniques in both RELEX and NExt (Wei et al., 20 May 2026, Chen et al., 13 Apr 2026).

The operational simplicity, closed-form extrapolation, and in some cases, improved downstream accuracy make rank-1 trajectory extrapolation a practical paradigm for reducing RLVR training latency in modern LLMs. As RLVR emerges as the dominant paradigm for LLM reasoning, these methods are likely to play a central role in scalable model deployment.