---
title: 'LongStraw: Execution Stack for Long-Context RL'
url: https://www.emergentmind.com/topics/longstraw
type: topic
---

# LongStraw: Execution Stack for Long-Context RL

Searching arXiv for the LongStraw paper and closely related long-context work to ground the article in current literature.
LongStraw denotes an architecture-aware execution stack for long-context reinforcement-learning post-training under a fixed GPU budget, introduced for Group Relative Policy Optimization (GRPO) and designed to make prompts at and beyond 2 million tokens executable during RL updates without keeping the full prompt-plus-responses autograd graph live [2607.14952]. In adjacent long-context literature, the name has also been used informally for a distinct perspective centered on long-short alignment, namely the consistency of output distributions across sequences of varying lengths, measured by a Long-Short Misalignment metric and optimized through a training-time regularizer [2506.11769]. The primary usage in the 2026 paper is therefore infrastructural and systems-oriented, whereas the 2025 alignment work is methodological and output-distribution-oriented. Both usages address the widening gap between nominal inference context length and effective long-context behavior, but they operate at different levels of the stack.

## 1. LongStraw as a long-context RL execution stack

The 2026 formulation defines LongStraw as an execution strategy for RL post-training workloads in which the shared prompt is evaluated once without autograd, only model-specific prompt state needed by later tokens is retained, and short response branches are replayed one at a time under autograd [2607.14952]. The motivating problem is that inference systems are approaching million-token contexts, while RL post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. This gap is especially salient for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long trajectories [2607.14952].

Within a grouped GRPO update, the long prompt \(x_{1:P}\) is shared across multiple sampled responses \(\{y_i\}_{i=1}^G\). Standard autograd would keep prompt and response activations live across the group, making GPU memory the critical bottleneck. LongStraw changes the autograd boundary. Its grouped update consists of prompt capture without gradients, pre-step scoring under old and reference policies, group-normalized advantage construction, policy replay on one short response suffix at a time, and a single optimizer step after all member-specific backwards have completed [2607.14952].

The resulting memory-time trade-off is explicit. The live memory is approximated as
\[
M_{\text{live} \;\approx\;  M_{\text{fixed} + M_{\text{prefix-state}(P) + M_{\text{grad} + \max_i M_{\text{branch}(R_i) + M_{\text{score}\!\left(\sum_i R_i\right),
\]
while the update time is roughly
\[
T_{\text{update} \;\approx\; T_{\text{prefix}(P) + \sum_{i=1}^{G} T_{\text{score+replay}(R_i).
\]
This means group size primarily increases wall-time rather than peak memory, because only one short response branch remains live at a time [2607.14952].

The stack is explicitly architecture-aware. It is instantiated for Qwen3.6-27B, which combines hybrid recurrent GDN layers with full self-attention, and for GLM-5.2, which combines MLA/DSA compressed sparse attention with MoE feed-forward blocks [2607.14952]. A plausible implication is that LongStraw is not a model-agnostic wrapper in the narrow sense; its retained prompt state, replay schedule, and device placement policy depend on architectural details.

## 2. GRPO under extreme context length

LongStraw is instantiated with GRPO, a PPO-style RL objective that groups several responses for the same prompt, normalizes rewards within the group, and uses a reference policy for KL regularization without a learned critic [2607.14952]. For response \(i\) and timestep \(t\), the importance ratio is
\[
\rho_{i,t}(\theta)= \exp\!\left( \log\pi_\theta(y_{i,t}\mid x_{1:P},y_{i,<t})- \log\pi_{\mathrm{old}(y_{i,t}\mid x_{1:P},y_{i,<t}) \right),
\]
and the clipped policy loss is
\[
\mathcal{L}_{\mathrm{policy} = -\frac{1}{G}\sum_{i=1}^{G}\frac{1}{R_i}\sum_{t=1}^{R_i} \min\!\left( \rho_{i,t}A_i, \operatorname{clip}(\rho_{i,t},1-\epsilon,1+\epsilon)A_i \right),
\]
with an additional token-wise KL term to a reference policy with weight \(\beta\) [2607.14952].

At long context, the difficulty is not the GRPO objective itself but the execution of its conditioning. Old-policy, current-policy, and reference-policy log-probabilities all depend on the same very long prompt state. LongStraw therefore freezes the prompt-conditioned state during scoring, computes old and reference scores before any in-group optimizer step, and then replays each response suffix under autograd against stored prompt state \(\bar z_P\) [2607.14952].

This does not alter GRPO semantics at the objective level. The paper states that LongStraw’s contribution is to restructure how log-probabilities and gradients are executed, without changing the objective itself [2607.14952]. The system-level novelty lies in making grouped RL scoring and response backward feasible at 2.1M positions for Qwen on eight H20 GPUs, with groups of 2 and 8, and validating the end-to-end execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2 on 32 H20 GPUs [2607.14952].

## 3. Architecture-specific state retention and replay

LongStraw’s defining systems idea is selective state retention. For Qwen3.6-27B, the retained prompt state consists of a compact recurrent state and short convolution tail for each GDN layer, whose size is fixed with respect to prompt length, together with sharded KV pages for each full-attention layer, whose storage grows linearly with prompt length \(P\) [2607.14952]. Everything else from the prompt forward is released, including FFN activations, attention scratch, and intermediate hidden states [2607.14952].

For Qwen at prompt length \(P=2{,}088{,}960\), page size \(64\), CP world size \(8\), and 16 full-attention layers, the paper gives
\[
N_{\text{page} = \frac{2{,}088{,}960}{64} = 32{,}640, \quad
N_{\text{page/rank} = \frac{32{,}640}{8} = 4{,}080,
\]
and estimates retained KV storage per rank as approximately \(15.94\) GiB [2607.14952]. The global dense-attention forward on sharded KV is then reconstructed through stable log-sum-exp composition:
\[
m_t = \max_r \ell_{r,t},\quad a_t = \sum_r e^{\ell_{r,t}-m_t},\quad n_t = \sum_r e^{\ell_{r,t}-m_t} o_{r,t},
\]
followed by
\[
o_t = \frac{n_t}{a_t}, \quad \ell_t = m_t + \log a_t.
\]
The paper states that this yields partition-correct global dense attention over the sharded KV, modulo BF16 merge [2607.14952].

Qwen response replay is blockwise. The long prompt is processed in 510 chunks of 4,096 tokens without gradients, while each response is split into 4 blocks of 2,048 tokens. For each member, LongStraw first runs a no-grad suffix pass, recording GDN state at block boundaries and appending KV pages, then traverses response blocks in reverse order under whole-layer checkpointing, backpropagating block by block and popping block-specific KV after use [2607.14952].

For GLM-5.2, the retained state is different. Each rank stores MLA latent pages for all 78 layers and DSA indexer-key pages for the 21 index-compute layers. With page size \(64\) and prompt length \(2{,}097{,}152\), the prompt state is stored on CPU; the paper computes per-rank CPU prompt state as
\[
B_{\text{CPU/rank} = 78\times 72\text{ MiB} + 21\times 16\text{ MiB} = 5{,}952\text{ MiB} = 5.8125\text{ GiB},
\]
or roughly \(186\) GiB across 32 ranks [2607.14952]. During replay, layer-specific prompt pages are staged from CPU to GPU, used to rebuild sparse attention and the short response path, and then released [2607.14952].

GLM also exposes an MoE-specific activation bottleneck. On one CP shard of \(65{,}536\) tokens, top-8 routing expands to
\[
65{,}536 \times 8 = 524{,}288
\]
expert rows, and one BF16 hidden buffer for this routed set is estimated at approximately \(6.44\) GiB [2607.14952]. The paper presents this as evidence that sparsity alone does not resolve long-context autograd memory pressure; prompt detachment and whole-layer checkpointing over the short response are required [2607.14952].

## 4. Reported execution capacity

The paper presents execution-capacity results rather than full training-validation results. For Qwen3.6-27B on eight H20 GPUs, grouped GRPO scoring and response backward complete at total context length \(2{,}097{,}152 = 2^{21}\) tokens for both \(G=2\) and \(G=8\) [2607.14952]. The reported measurements are as follows:

| Configuration | Wall time | Peak allocated memory |
|---|---:|---:|
| Qwen, 2.1M context, \(G=2\) | 5198.780 s | 97.503 GB |
| Qwen, 2.1M context, \(G=8\) | 6785.225 s | 97.711 GB |
| Qwen, group-size increase \(2 \rightarrow 8\) | +1586.445 s | +0.208 GB |

The increase of only \(0.208\) GB, or \(0.213\%\), when moving from group size \(2\) to \(8\) is used to support the claim that group size mainly scales time rather than peak memory under member-serial replay [2607.14952]. The same paper reports a separate stress test reaching \(4{,}456{,}448\) positions on the same eight-H20 budget, and a prefix-frozen response-only run performing eight optimizer steps over the same \(4.25\)M context with peak \(83.894\) GB [2607.14952].

For GLM-5.2 on 32 H20 GPUs, the paper validates the end-to-end LongStraw execution path for a \(2.1\)M-token prompt across all 78 layers with \(G=2\), though the main table reports peak GPU memory as not reported because the PyTorch counter was read between prefix capture and replay rather than across the whole transaction [2607.14952]. During prompt capture, rankwise `max_memory_allocated` ranged from \(112.571\) GB to \(145.148\) GB, indicating non-uniform allocation patterns [2607.14952].

These results establish execution capacity rather than complete training correctness. The paper is explicit that the experiments show that one can capture \(2\)M+ prompts, run grouped GRPO scoring and response backwards, and issue optimizer calls without OOM, but do not yet demonstrate exact gradient equivalence to conventional full-sequence training [2607.14952].

## 5. Correctness limits, detached prompts, and incomplete distributed gradients

A central limitation is that LongStraw detaches the captured prompt state. If \(z_P(\theta)\) denotes the full model state after the prompt, full-sequence training would require
\[
\nabla_\theta \ell(\theta, z_P(\theta)) =
\left.\frac{\partial \ell}{\partial \theta}\right|_{z_P}
+
\frac{\partial \ell}{\partial z_P}
\frac{\partial z_P}{\partial \theta}.
\]
LongStraw stores \(\bar z_P = \text{stopgrad}(z_P(\theta))\) and computes only
\[
\left.\frac{\partial \ell}{\partial \theta}\right|_{\bar z_P},
\]
which the paper describes as the response-conditional gradient [2607.14952]. The missing second term means the method is not equivalent to full-sequence training, and prompt state must be recaptured after each optimizer step because parameter updates make the stored state stale [2607.14952].

The paper also identifies unfinished distributed gradient composition. In Qwen, forward CP8 global attention composition is correct, and query gradients are summed across ranks,
\[
dQ = \sum_{r=0}^{7} dQ_r,
\]
but K/V gradients remain local to each shard. Since LoRA K/V projection weights are replicated across ranks, the required reductions
\[
\nabla W_K = \sum_r \nabla W_K^{(r)},\quad
\nabla W_V = \sum_r \nabla W_V^{(r)}
\]
are missing, so eight independent AdamW optimizers may produce inconsistent adapter updates [2607.14952].

For GLM, the historical acceptance run used DSA sparse attention locally per CP shard rather than globally across shards, and the custom training loop bypassed `finalize_model_grads()`, so CP-replicated non-expert adapters were updated using unreduced local gradients [2607.14952]. The paper further notes that no full gradient parity against a short-context reference implementation was performed for either Qwen or GLM [2607.14952].

These caveats are not incidental. They define the paper’s scope. LongStraw demonstrates that the execution path fits in memory and traverses all intended operations at million-token scale, but it does not yet establish optimizer-correct distributed RL training under the same execution schedule [2607.14952].

## 6. Relation to long-short alignment and other long-context methods

The term “LongStraw” is also associated, in a different 2025 line of work, with long-short alignment: the idea that good long-context behavior is largely about making a model’s output distributions invariant to irrelevant changes in input length [2506.11769]. In that formulation, an autoregressive model \(g_\theta\) should produce similar next-token distributions for suffixes \(\mathbf{x}_{[-l_1:]}\) and \(\mathbf{x}_{[-l_2:]}\) when the relevant contextual information is effectively the same [2506.11769].

The corresponding discrepancy is measured by Symmetric Cross-Entropy,
\[
\mathcal{L}_{\mathrm{SCE}(\mathbf{y}, \mathbf{y}') = -\left(\left<\mathbf{y}',\log(\mathbf{y})\right>+\left<\mathbf{y},\log(\mathbf{y}')\right>\right),
\]
and Long-Short Misalignment is defined as
\[
\mathcal{L}_{\mathrm{misalign}(g_{\theta}) = \mathbb{E}_{\mathbf{x},l_1, l_2}\Big[\mathcal{L}_{\mathrm{SCE}\big(g_{\theta}(\mathbf{x}_{[-l_1:]}),\, g_{\theta}(\mathbf{x}_{[-l_2:]})\big)\Big].
\]
The training objective adds this term to standard LM loss,
\[
\mathcal{L}^*_{\mathrm{train}(g_{\theta}) = \mathcal{L}_{\mathrm{train}(g_{\theta}) + \alpha\cdot\mathcal{L}_{\mathrm{misalign}(g_{\theta}),
\]
with best performance reported for \(\alpha \in [0.1,0.3]\) and modest overhead of approximately \(3\)–\(5\%\) [2506.11769].

This alignment-based usage differs sharply from the 2026 LongStraw stack. The 2025 method is a training-time regularizer that does not modify positional encodings or architecture, whereas the 2026 method is an execution stack for RL post-training that also does not modify the underlying GRPO objective [2506.11769; 2607.14952]. The connection is conceptual rather than procedural: both respond to the inadequacy of relying solely on nominal context length. One addresses output-distribution consistency during training; the other addresses execution feasibility during RL updates at million-token scale.

Related long-context work also includes training-free length extrapolation methods such as Dimension-Wise Positional Embeddings Manipulation (DPE), which operates by measuring effective relative distance per RoPE dimension group and remapping only key dimensions at inference time [2504.18857]. DPE is described as belonging to the “train short, test long” family and is evaluated on RULER, multi-needle NIAH, and InfiniteBench, which target retrieval, variable tracing, aggregation, and QA over long contexts [2504.18857]. This suggests that the broader LongStraw landscape spans at least three levels: output-space regularization [2506.11769], inference-time positional manipulation [2504.18857], and systems-level RL execution restructuring [2607.14952].

## 7. Significance and open directions

LongStraw, in its 2026 sense, shifts attention from nominal context-window support to the systems question of what can actually be executed during RL post-training on a fixed hardware budget [2607.14952]. Its key contribution is to frame state lifetime and physical ownership as the controlling variables for million-token RL, rather than treating long-context feasibility as only a matter of attention FLOPs or sequence parallelism [2607.14952]. This is particularly relevant for agents, where prompts are long trajectories rather than single static documents.

At the same time, the paper’s own framing is deliberately cautious. The demonstrated result is execution capacity, not complete training correctness, downstream task improvement, or parity with conventional full-sequence gradients [2607.14952]. The workloads are synthetic, rewards are deterministic functions of group index, old and reference policies are identical at the first step so KL is zero and clipping is inactive, and no multi-step RL training curve or environment-integrated evaluation is reported [2607.14952].

Open directions are therefore concrete. The paper identifies the need to restore missing distributed reductions, validate full-gradient parity at short context, and extend the execution stack from capacity receipts to correct large-scale RL optimization [2607.14952]. A plausible implication is that LongStraw’s long-term importance may depend less on the current detached-prefix implementation than on whether its prompt-capture and serial-replay design can be combined with exact distributed gradient composition.

In the broader literature, the coexistence of two “LongStraw” usages underscores a persistent theme in long-context research: effective long-context behavior is jointly limited by statistical generalization, positional or architectural representation, and systems-level training executability. Long-short alignment addresses the first of these by regularizing output consistency across lengths [2506.11769]. DPE addresses the second by dimension-wise RoPE manipulation at inference time [2504.18857]. The LongStraw execution stack addresses the third by making grouped RL response backward feasible beyond 2M tokens under fixed GPU budgets [2607.14952].

Source: https://www.emergentmind.com/topics/longstraw