Complete-Group Pipelining (CGP) in GRPO Systems

Updated 6 July 2026

Complete-Group Pipelining (CGP) is defined around GRPO’s group-level dependency, treating a complete set of K responses as the fundamental trainable unit.
CGP dispatches trainable complete groups via FIFO queues with dynamic micro-batching to overlap rollout and training in disaggregated architectures.
Empirical evidence shows CGP reduces rollout-to-train-end times by up to 42.3% and lowers trainer idling, highlighting its performance benefits.

Searching arXiv for the cited work and closely related pipelining papers to ground the article. arxiv_search query: "RolloutPipe Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning (Chen et al., 25 Jun 2026)" Complete-Group Pipelining (CGP) is a training-side pipelining mechanism introduced in the RolloutPipe framework for disaggregated, on-policy GRPO/RLVR systems. Its defining idea is that, in GRPO, the natural trainable unit is not an individual response but a fully materialized prompt-level group of $K$ sampled responses generated under the same prompt. CGP therefore reduces synchronization granularity from the full rollout round to the complete-group boundary: as soon as a group becomes trainable, it is appended to a FIFO and dispatched to the trainer in units of $U = \frac{B}{K}$ groups, allowing training to begin before rollout completion while maintaining on-policy correctness (Chen et al., 25 Jun 2026).

1. Formal object of pipelining

CGP is defined around the GRPO dependence on prompt-level groups. In the RolloutPipe setting, a group means the $K$ sampled responses generated for one prompt in a rollout round. GRPO computes rewards and normalizes them within the group, so training depends on all $K$ responses together rather than on any single finished response. This is the immediate reason CGP pipelines complete groups rather than token streams, individual requests, or partially finished prompt instances (Chen et al., 25 Jun 2026).

The paper distinguishes three related objects.

Term	Meaning
Group	The $K$ responses for one prompt
Complete group	A group whose $K$ responses have all finished generation
Trainable complete group	A complete group that has also finished reward computation, verifier processing, group-statistics computation, sample-record conversion, filtering, and any loss-mask/advantage attachment

The transition from complete group to trainable complete group is called group materialization. This boundary is central: CGP does not dispatch on serving completion alone. A group becomes eligible only after all $K$ responses have completed generation and the control side has finished reward/verifier work and trainable-record construction. The paper also distinguishes serving-complete from trainable-complete, and CGP dispatches on the latter notion, not the former (Chen et al., 25 Jun 2026).

A common misconception is to treat CGP as sample-level pipelining. That is not the mechanism described. The trainable granularity is group-level because GRPO group statistics require all $K$ responses under one prompt. This suggests that CGP is best understood as a semantics-aware pipeline boundary derived from the optimization rule itself, rather than as a generic throughput optimization layered on top of arbitrary rollout records.

2. Disaggregated architecture and execution context

CGP is defined for a disaggregated rollout/training architecture in which rollout generation and policy training run on separate GPU pools. The system described in RolloutPipe is built over Slime, SGLang, Megatron-LM, and Ray, and exposes three logical nodes: a Rollout node that admits requests and runs generation in the serving engine, a Control node that collects responses and materializes trainable groups, and a Training node that consumes trainable groups, runs forward/backward, accumulates gradients, performs optimizer steps, and publishes updated weights (Chen et al., 25 Jun 2026).

A rollout round contains $R$ prompt groups. Each prompt group triggers $K$ sampled responses under a fixed policy snapshot for that round. Responses return to the control side, which gathers all $U = \frac{B}{K}$ 0 responses for a group, runs reward/verifier logic, computes group-level statistics, and builds the trainable representation. Only then is the group eligible for training. In serial systems such as Slime, the trainer receives data only after the entire rollout of $U = \frac{B}{K}$ 1 groups has finished, even though groups finish at staggered times. The paper characterizes the resulting inefficiency as early-completed groups remaining outside the trainer FIFO until the last group completes (Chen et al., 25 Jun 2026).

CGP is motivated by the mismatch between this whole-round synchronization barrier and GRPO’s true trainable unit. If groups finish at staggered times, then a full-round barrier artificially delays trainer startup. CGP removes that barrier without relaxing the round-level fixed-weight assumption. This point matters because the same disaggregated architecture also makes naive asynchronous overlap unsafe: if rollout weights were updated while later groups were still being generated, later samples could come from a newer policy than earlier ones, creating mixed-policy data and breaking on-policy semantics.

3. Dispatch mechanism, queues, and update granularity

The operational rule of CGP is FIFO dispatch of trainable complete groups. Once group materialization finishes, CGP appends the group to Pending Complete Groups in the ActorGroup. A Feasible Batch Selector extracts the largest prefix of whole groups that fits the per-GPU token budget and forwards it into a U-group Ready Queue. Whenever the ready queue contains at least $U = \frac{B}{K}$ 2 complete groups, the earliest $U = \frac{B}{K}$ 3 groups are dispatched to the trainer. The trainer consumes those $U = \frac{B}{K}$ 4 groups through a pipelined train RPC and a Gradient Accumulator; after every $U = \frac{B}{K}$ 5 consumed groups, one optimizer step is performed; after all $U = \frac{B}{K}$ 6 groups of the rollout are consumed, a Weight Publisher publishes updated weights for the next rollout (Chen et al., 25 Jun 2026).

The logical update width is defined as

$U = \frac{B}{K}$ 7

where $U = \frac{B}{K}$ 8 is the sample-level global batch size and $U = \frac{B}{K}$ 9 is the number of responses per prompt. One logical update therefore consumes $K$ 0 complete groups. In the reported experiments, $K$ 1, $K$ 2, and therefore $K$ 3 (Chen et al., 25 Jun 2026).

The dispatch conditions are explicit. A group is dispatched only after materialization; training occurs only when at least $K$ 4 groups are ready; optimizer stepping occurs only after $K$ 5 groups have been consumed; weight refresh occurs only after all $K$ 6 groups from the round have been consumed. The paper’s compact pseudocode summarizes this as $K$ 7: compute a feasible batch from the FIFO, move it to the ready queue, and dispatch the earliest $K$ 8 groups to the trainer once the queue length reaches $K$ 9 (Chen et al., 25 Jun 2026).

CGP operates on whole groups only. The Feasible Batch Selector never splits a group into partial requests. If even a single group exceeds the budget, that group is still taken alone and handled through dynamic micro-batching. This is a practical implementation constraint: the complete-group boundary is not merely conceptual but enforced in queue management and token-budget admission.

4. On-policy correctness and separation from asynchronous training

CGP is designed to overlap rollout and training without introducing stale data. The key invariant is that all groups in a round share the same fixed rollout weights, and weight refresh happens only after the trainer drains all logical updates for the current rollout. In consequence, CGP pipelines only the handoff and trainer consumption of already materialized groups from a fixed-weight rollout; it does not interleave weight changes with ongoing generation (Chen et al., 25 Jun 2026).

The paper’s timing formulation makes the recovered overlap explicit:

$K$ 0

so the overlap recovered by CGP is

$K$ 1

This is the interval during which Slime leaves the trainer idle but CGP can already execute training work on the earliest $K$ 2 materialized groups (Chen et al., 25 Jun 2026).

The correctness argument is invariant-based rather than theorem-based. The paper does not present a named theorem, but it states sufficient conditions for maintaining on-policy correctness: all groups in a round must share the same fixed rollout weights; optimizer-step and weight-publish boundaries follow Slime; weight refresh runs only after all $K$ 3 groups from the round have been consumed; CGP and FGD change dispatch timing and order, not samples, advantages, or gradient computation. From these conditions, the paper concludes that training dynamics and convergence behavior remain consistent with Slime (Chen et al., 25 Jun 2026).

This separation also clarifies the relation between CGP and Frontier-Group Dispatch (FGD). CGP is the training-side handoff and consumption mechanism that makes early training possible at all. FGD is a rollout-side admission policy that prioritizes frontier groups needed to form the next training batch so that trainer-ready groups arrive earlier and more steadily. CGP removes the whole-rollout barrier; FGD improves the supply pattern feeding that pipeline. Confusing the two obscures the architecture: one changes synchronization granularity, the other changes request admission order (Chen et al., 25 Jun 2026).

5. Performance characteristics and empirical evidence

The experiments evaluate RolloutPipe on Qwen3-1.7B using 8 × RTX 4090 24GB for training with TP=4 and DP=2, 2 × A100 40GB PCIe for rollout with TP=2, and four workloads—LSAT-AR, Sci-XW, Sci-JL, OlyPhys—under rollout group counts $K$ 4, with four rounds per configuration (Chen et al., 25 Jun 2026).

Across all 12 settings, CGP+FGD shortens rollout-to-train-end time by 30.7%–42.3% relative to Slime. The gains rise with larger $K$ 5: 30.7%–32.9% at $K$ 6 and 39.8%–42.3% at $K$ 7. The paper attributes most of that improvement specifically to CGP: 71%–96% of the total main-time reduction versus Slime is due to CGP, with CGP’s share increasing from 71%–79% at $K$ 8 to 92%–96% at $K$ 9. On top of CGP, CGP+FGD yields an additional 2.5%–11.4% reduction in main time (Chen et al., 25 Jun 2026).

The dispatch-timing evidence is particularly revealing. At $K$ 0, the first legal dispatch time is 509–543 s for Slime, 46–90 s for CGP, and 52–61 s for CGP+FGD. This indicates that the principal speedup comes from moving training start much earlier in the round rather than from reducing trainer-side work. The paper reinforces that interpretation by reporting that trainer compute time remains nearly identical across Slime, CGP, and CGP+FGD for the same $K$ 1, and that response lengths remain in the same rough range of 3.3k–3.6k tokens (Chen et al., 25 Jun 2026).

Trainer idling decreases correspondingly. Slime’s waiting ratio is reported as 47%–52%, whereas CGP+FGD reduces it to 14%–33% across the twelve settings, with especially low values at $K$ 2: 14.2%, 15.0%, 13.8%, and 13.9%. The abstract summarizes the improvement as a 37%–76% reduction in trainer waiting ratio relative to Slime. Operationally, the experiments describe waiting ratio as trainer-side

$K$ 3

The interpretation offered by the paper is straightforward: CGP wins by moving the same work earlier and overlapping it with rollout, not by shrinking the workload itself (Chen et al., 25 Jun 2026).

6. Scope, limitations, and relation to broader pipelining literature

CGP is most beneficial when rollout and training are disaggregated across separate GPU pools, the baseline system is synchronous on-policy with a whole-rollout barrier, group completion times are staggered, and rollout rounds contain many groups. It provides less leverage when group completion times are already tightly synchronized, $K$ 4 is small, the trainer is not idle under baseline operation, or trainable groups do not arrive early enough to form $K$ 5-group batches quickly (Chen et al., 25 Jun 2026).

Its assumptions are correspondingly narrow. The setting is specifically GRPO / RLVR; the trainable unit is defined by group-level reward normalization; all groups in a round use fixed rollout weights; weight refresh occurs only after the full round is consumed; and the system must expose a clear notion of complete-group formation and materialization. The main tradeoff identified by the paper is that CGP alone does not control which groups finish first, so trainer-ready supply may remain irregular. That is precisely why RolloutPipe pairs CGP with FGD (Chen et al., 25 Jun 2026).

In the broader pipelining literature, the term “CGP” is not used uniformly, and most adjacent papers do not define Complete-Group Pipelining directly. Deep pipelined Conjugate Gradient overlaps delayed global reductions with later iterations through nonblocking collectives and a hierarchy of auxiliary bases (Cools et al., 2019); automatic multi-dimensional pipelining in HLS statically overlaps producer-consumer loop nests across affine schedules (Majumder et al., 2023); PipeOrgan chooses variable-depth inter-operation pipeline segments and spatial organizations for DNN accelerators (Garg et al., 2024); and LayerPipe2 derives shared delay assignments when layers are pipelined in groups, with all layers in a group sharing the same downstream-stage-dependent delay (Unnikrishnan et al., 9 Dec 2025). These works suggest a broader family resemblance—overlap achieved by delaying consumption until a semantically valid boundary—yet they remain distinct from RolloutPipe’s CGP because they do not operate on GRPO group semantics or on fixed-weight on-policy rollout rounds.

A plausible implication is that CGP’s most portable idea is not “pipeline everything early,” but “pipeline at the smallest boundary that is already semantically complete.” In RolloutPipe, that boundary is the trainable complete group. The paper’s contribution is therefore not merely earlier dispatch, but a redefinition of synchronization granularity that matches GRPO’s actual dependence structure (Chen et al., 25 Jun 2026).