Papers
Topics
Authors
Recent
Search
2000 character limit reached

Compression Group Relative Policy Optimization

Updated 14 July 2026
  • C-GRPO is a family of GRPO methods that compress aspects of the training signal via response, update-set, token, or systems-level strategies while retaining group-relative credit assignment.
  • Techniques such as FGO, BPPO, SSPO, TEPO, and TIC-GRPO modify reward construction, select informative updates, or aggregate token signals to reduce redundancy and variance.
  • Empirical findings indicate that these compression methods can yield shorter outputs, improved efficiency, and robust performance, though challenges remain in standardization and binary correctness reliance.

Searching arXiv for recent GRPO variants relevant to compression-oriented formulations and C-GRPO terminology. Compression Group Relative Policy Optimization (C-GRPO) is an umbrella characterization for a family of Group Relative Policy Optimization (GRPO) variants that compress, aggregate, or selectively reduce the optimization payload of group-based reinforcement learning while preserving the defining GRPO principle of within-group relative credit assignment. In the literature, “C-GRPO” is not a standardized method name. Instead, closely related work appears under formulations such as Fine-grained Group Policy Optimization (FGO), Binary Prefix Policy Optimization (BPPO), Soft Sequence Policy Optimization (SSPO), Token-Level Policy Optimization (TEPO), and CoDistill-GRPO, each of which modifies GRPO by compressing reward structure, update sets, token-level corrections, or rollout sources in distinct ways (Han et al., 10 Feb 2026, Zhao et al., 27 May 2026, Glazyrina et al., 22 Feb 2026, Lin et al., 10 Oct 2025, Kwon et al., 9 May 2026). The shared theme is that standard GRPO often propagates coarse sequence-level or group-level signals through token-level updates in a way that is costly, redundant, or statistically brittle; C-GRPO-style methods seek more compact and better-conditioned surrogates without abandoning group-relative optimization.

1. Conceptual scope and naming

The expression “Compression Group Relative Policy Optimization” does not currently denote a single canonical algorithm in the arXiv literature. Several papers are directly relevant to such a label, but they adopt different names and emphasize different forms of compression. Some compress long chain-of-thought (CoT) behavior by reward shaping toward shorter valid trajectories, as in FGO (Han et al., 10 Feb 2026). Others compress the update-bearing subset of completions and tokens, as in BPPO, which keeps only the shortest correct completion and the shortest incorrect completion while preserving full-group advantage normalization (Zhao et al., 27 May 2026). Still others compress token-level importance information into a sequence-level statistic through geometric aggregation, as in SSPO and TEPO (Glazyrina et al., 22 Feb 2026, Lin et al., 10 Oct 2025).

This suggests that “C-GRPO” is best treated as a higher-level editorial category rather than a fixed paper title. Under that interpretation, C-GRPO denotes GRPO-family objectives in which compression occurs along at least one of four axes. First, there is response compression, where the policy is explicitly trained to prefer shorter reasoning traces. Second, there is update-set compression, where only a subset of sampled completions is used for policy updates. Third, there is token-signal compression, where tokenwise policy ratios or gradients are aggregated into a more compact sequence-level form. Fourth, there is systems-level compression, where rollout generation or training structure is shifted toward cheaper models or smaller effective batches (Han et al., 10 Feb 2026, Zhao et al., 27 May 2026, Glazyrina et al., 22 Feb 2026, Kwon et al., 9 May 2026).

A common misconception is that compression-oriented GRPO necessarily means post hoc truncation of reasoning. The available work does not support that reduction. The central design pattern is instead to alter the RL objective or reward construction so that concise, stable, or non-redundant behavior becomes favorable during training itself (Han et al., 10 Feb 2026, Zhao et al., 27 May 2026).

2. GRPO substrate and the compression problem

Standard GRPO samples a group of responses for the same prompt, computes scalar rewards, normalizes these rewards within the group, and then applies a PPO-style clipped policy update using token-level importance ratios. In one representative formulation, the GRPO objective is

JGRPO(θ)=ExD,{yi}i=1Gπθold(x)[1Gi=1G1yit=1yimin(ρi,t(θ)A^i,t,clip(ρi,t(θ),1ε,1+ε)A^i,t)],\mathcal{J}_\text{GRPO}(\theta) = \mathbb{E}_{ x \sim \mathcal{D},\, \{y_i\}_{i=1}^G \sim \pi_{\theta_\text{old}}( \cdot | x) } \left[ \frac{1}{G} \sum_{i=1}^{G} \frac{1}{|y_i|} \sum_{t=1}^{|y_i|} \min \left( \rho_{i,t}(\theta) \widehat{A}_{i,t}, \, \mathrm{clip} \left( \rho_{i,t}(\theta), 1 - {\varepsilon}, 1 + {\varepsilon}\right) \widehat{A}_{i,t} \right) \right],

with token-level importance ratio

ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},

and group-relative normalized reward

A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.

This formulation already contains a kind of compression: every token in a response inherits the same group-normalized scalar advantage (Glazyrina et al., 22 Feb 2026).

Compression-oriented extensions emerge because this baseline induces several mismatches. Rewards in reasoning RL are often sequence-level, yet correction is token-level. Long completions multiply token-level probability ratios across time, increasing variance. PPO-style hard clipping stabilizes updates but can discard useful signal. In binary-verification settings, many groups may also become ineffective when all sampled responses share the same label, causing zero or near-zero relative advantages (Glazyrina et al., 22 Feb 2026, Han et al., 10 Feb 2026).

Another structural issue is redundancy. BPPO reports that, within the same prompt group, same-class completions often induce highly similar update directions, whereas correct-incorrect pairs provide more distinct contrastive signals (Zhao et al., 27 May 2026). This implies that standard GRPO may spend substantial computation updating on near-duplicate information. A related line of analysis argues that standard GRPO’s practical update rule is closer to a stale-policy gradient estimator than to a true current-policy gradient estimator, which further motivates simpler or more aggregated correction schemes (Pang et al., 4 Aug 2025).

The compression problem in C-GRPO therefore concerns both efficiency and statistical alignment: how to preserve the relative-comparison structure that makes GRPO attractive while eliminating unnecessary tokens, completions, variance sources, or model-side computation.

3. Compression by reward shaping: FGO and concise chain-of-thought

The clearest direct instantiation of compression-oriented GRPO is Fine-grained Group Policy Optimization (FGO), proposed for long CoT compression (Han et al., 10 Feb 2026). FGO starts from the observation that RL-trained reasoning models often generate unnecessarily verbose CoT trajectories containing redundant intermediate derivations, repeated self-verification, and overthinking. It defines compression operationally as reducing token length while preserving answer accuracy and reasoning capability (Han et al., 10 Feb 2026).

FGO preserves the group-relative structure of GRPO but modifies reward construction inside the sampled group. In the paper’s formulation, vanilla reward is binary answer verification,

ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}

which in ordinary GRPO yields zero advantage whenever all responses in the group are either correct or incorrect. FGO addresses this by partitioning the group into a correct subgroup G+\mathcal G^+ and an incorrect subgroup G\mathcal G^-, then shaping rewards within each subgroup using response length and entropy (Han et al., 10 Feb 2026).

For the correct subgroup, the weight is

W+=Softmax[(mean(L+)L+)α×(mean(H+)H+)β],\mathcal{W}^+ = \text{Softmax} \left[ \Big( \frac{ \text{mean} (L^+)}{L^+} \Big)^\alpha \times \Big( \frac{ \text{mean}(\mathcal{H}^+)}{\mathcal{H}^+} \Big) ^ \beta \right],

and the shaped reward is

R+=W+×Rˉ+,R^+ = \mathcal{W}^+ \times \bar{R}^+,

with Rˉ+=1\bar R^+=1. Shorter-than-average and lower-entropy-than-average correct responses therefore receive larger positive weights (Han et al., 10 Feb 2026). FGO then replaces the standard GRPO advantage with a Dr.GRPO-style mean-centered version,

Ai,t=rimean({ri}i=1G),riR,A_{i,t} = r_i - \text{mean}(\{r_i\}_{i=1 }^{G}), \qquad r_i \in R,

omitting the standard deviation denominator “for stability” (Han et al., 10 Feb 2026).

The compression mechanism here is not token pruning. It is subgroup-relative reward shaping over whole responses. This distinction matters. FGO does not segment a trajectory into chunks and remove them; it changes the training signal so that shorter, still-correct reasoning paths become more rewarding. The paper reports strong evidence that this works: for Qwen2.5-Math-1.5B on MATH500, FGO improves accuracy from 65.6 under GRPO to 68.6 while reducing average length from 578 to 441 tokens and increasing ACT from 11.3 to 15.6; for DeepSeek-R1-Distill-Qwen-1.5B on MATH500, FGO improves accuracy from 51.0 to 56.4 while reducing length from 828 to 229 (Han et al., 10 Feb 2026).

FGO also addresses a major GRPO degeneracy. Table 3 in the paper reports that invalid sample counts among 3,200 training samples drop from 1584, 1866, 2617, and 2444 under GRPO to 0 under FGO for four different 1.5B models (Han et al., 10 Feb 2026). This makes FGO particularly relevant to C-GRPO as a compression-aware reformulation of reward structure rather than an inference-time compression heuristic.

4. Compression by update selection: BPPO and pairwise prefix optimization

Binary Prefix Policy Optimization (BPPO) is a more aggressive form of C-GRPO in which compression is applied directly to the optimization payload (Zhao et al., 27 May 2026). The method preserves GRPO’s full-group sampling, reward computation, and advantage normalization, but updates only a compact subset of completions and only their prefixes.

The key empirical observation is that within-group same-class completions often induce highly similar update directions, while correct-incorrect pairs provide more informative contrastive signal (Zhao et al., 27 May 2026). Based on this, BPPO defines a binary update unit consisting of the shortest correct completion and the shortest incorrect completion from the same prompt group: ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},0 with selected set

ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},1

The advantage ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},2 is still computed using all ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},3 sampled completions. Only after full-group normalization does BPPO compress the update to this selected pair (Zhao et al., 27 May 2026).

BPPO also applies prefix-focused optimization. Instead of updating over the full response length ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},4, it updates only the first ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},5 response tokens, where ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},6 is set to 50% of the average response length in the reported experiments (Zhao et al., 27 May 2026). The objective becomes

ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},7

This is a direct compression of both the completion axis and the token axis (Zhao et al., 27 May 2026).

The efficiency gains are substantial. The paper reports up to ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},8 speedup over GRPO in the main tables, specifically on MATH with Llama3.2-3B-Instruct at ρi,t(θ)=πθ(yi,tx,yi,<t)πθold(yi,tx,yi,<t),\rho_{i,t}(\theta)=\frac{ \pi_{\theta} (y_{i,t} \mid x, y_{i,<t}) }{ \pi_{\theta_\text{old}} (y_{i,t} \mid x,y_{i,<t})},9, where training time falls from A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.0 seconds to A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.1 seconds (Zhao et al., 27 May 2026). Mean response length is reduced by approximately 30–50% without any explicit length penalty: on MATH at A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.2, average tokens drop from 374.83 to 185.20, a 50.6% reduction; on GSM8K at A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.3, from 262.64 to 147.25; on Geo3K at A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.4, from 3066.45 to 1665.13 (Zhao et al., 27 May 2026).

BPPO therefore exemplifies a strong form of C-GRPO: full-group statistics, pair-only optimization, and prefix-only backpropagation. A plausible implication is that update-set compression can be effective when group redundancy is high, but BPPO also makes clear that this relies on binary correctness structure and on groups containing both classes.

5. Compression by sequence aggregation: SSPO, TEPO, and trajectory-level correction

A different C-GRPO lineage compresses token-level correction signals into a sequence-level statistic. The main motivation is the mismatch between sequence-level rewards and token-level importance weighting.

Soft Sequence Policy Optimization (SSPO) explicitly frames itself as a bridge between GMPO and SAPO within the GRPO family (Glazyrina et al., 22 Feb 2026). Its central idea is to aggregate softly gated token ratios geometrically across the sequence: A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.5 where A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.6 is the SAPO sigmoid gate and A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.7 is the token-level importance ratio (Glazyrina et al., 22 Feb 2026). The gate is

A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.8

SSPO therefore compresses an entire token-ratio trajectory into one softly attenuated sequence-level weight (Glazyrina et al., 22 Feb 2026). This is directly relevant to C-GRPO if compression is understood as robust sequence summarization of off-policy corrections.

TEPO pursues a related objective using a length-normalized sequence importance ratio derived from the Markov factorization of autoregressive sequence likelihood (Lin et al., 10 Oct 2025): A^i,t=A^i=r(x,yi)mean({r(x,yi)}i=1G)std({r(x,yi)}i=1G).\widehat{A}_{i,t} = \widehat{A}_{i} = \frac{r(x, y_i) - \mathrm{mean} \left( \{ r(x, y_i) \}_{i=1}^G \right) }{ \mathrm{std} \left( \{ r(x, y_i) \}_{i=1}^G \right) }.9 Its core loss is

ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}0

which replaces tokenwise importance correction with a single sequence-level geometric-mean ratio (Lin et al., 10 Oct 2025). TEPO’s backward pass then redistributes the sequence-level signal across tokens through the Markov structure.

This suggests a “compressed statistic” view of C-GRPO: rather than store or optimize with all token-level ratios directly, one can compress them into a single trajectory-level or sequence-level factor before backpropagation. That perspective is reinforced by "On the Theory and Practice of GRPO: A Trajectory-Corrected Approach with Fast Convergence" (Pang et al., 4 Aug 2025), which argues that standard practical GRPO’s token-level importance correction estimates the gradient at the old policy rather than the current one. The paper proposes trajectory-level importance-corrected GRPO (TIC-GRPO), replacing token-level ratios with

ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}1

and claims this yields an unbiased estimate of the current-policy gradient while preserving the critic-free structure (Pang et al., 4 Aug 2025). A plausible implication is that trajectory-level correction is a principled compression of token-level importance information, not merely an engineering shortcut.

6. Compression by rollout reduction, co-distillation, and constraints

Several additional papers broaden the meaning of C-GRPO beyond output length and token aggregation.

"It Takes Two: Your GRPO Is Secretly DPO" argues that GRPO can be reframed as a contrastive objective and that a two-rollout version, 2-GRPO, can match 16-GRPO while using only ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}2 of the rollouts and reducing training time by over 70% (Wu et al., 1 Oct 2025). In the pairwise case, if one rollout is positive and one negative, the advantages collapse to

ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}3

and otherwise both are zero. The paper reports 0.15 million generated rollouts for 2-GRPO versus 1.2 million for 16-GRPO, with wall-clock reductions of roughly 74–84% depending on model and dataset (Wu et al., 1 Oct 2025). This is a strong form of rollout compression.

CoDistill-GRPO addresses a different bottleneck: small models often fail under standard GRPO because sparse reward provides too little signal on hard tasks (Kwon et al., 9 May 2026). Its small-model objective adds an on-policy KD reward

ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}4

while the large model is updated off-policy using rollouts generated by the small model (Kwon et al., 9 May 2026). The result is a form of systems-level compression: the expensive model no longer bears the dominant rollout-generation burden. The paper reports an approximate 18% speedup for the larger model and substantial gains for the smaller model, including over 11.6 percentage points over the base model and an additional 6.0 points over GRPO on Minerva for Qwen2.5-Math-1.5B (Kwon et al., 9 May 2026).

Constrained GRPO is relevant when “compression” is expressed as an explicit constraint rather than a soft preference (Girgis et al., 5 Feb 2026). The paper shows that naive scalarization of reward and constraint before group normalization introduces implicit reweighting: ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}5 so the effective coefficient on each component is not ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}6 but ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}7 (Girgis et al., 5 Feb 2026). It proposes scalarizing standardized component advantages instead: ri={1,if a^i=a; 0,if a^ia,r_i = \begin{cases} 1, & \text{if } \hat{a}_i = a; \ 0, & \text{if } \hat{a}_i \neq a, \end{cases}8 This is highly relevant to compression-aware GRPO because token budget, latency, bandwidth, or compute caps can be encoded as indicator costs. The paper’s main lesson is that if compression is treated as a constraint, a C-GRPO implementation should scalarize advantages rather than rewards, or the intended trade-off can be corrupted by variance mismatch (Girgis et al., 5 Feb 2026).

7. Empirical patterns, controversies, and open questions

Across these papers, several empirical patterns recur. Compression-oriented GRPO variants often improve token efficiency, sometimes while preserving or improving accuracy. FGO and BPPO both report shorter outputs with competitive or improved benchmark performance (Han et al., 10 Feb 2026, Zhao et al., 27 May 2026). Pairwise and subset-based methods report large wall-clock savings, especially when rollout generation dominates cost (Wu et al., 1 Oct 2025, Zhao et al., 27 May 2026). Sequence-level aggregation methods are typically motivated by better alignment between reward granularity and correction granularity, though some of their claims remain more conceptual than fully numerically established in the visible text (Glazyrina et al., 22 Feb 2026).

There are also important controversies and caveats. First, several methods rely on binary correctness signals, which may not generalize directly to partial-credit or open-ended tasks (Han et al., 10 Feb 2026, Zhao et al., 27 May 2026, Wu et al., 1 Oct 2025). Second, the distinction between compression of behavior and compression of optimization is often blurred. BPPO, for example, reduces response length without an explicit length penalty by not reinforcing suffixes, whereas FGO explicitly reweights rewards toward shorter trajectories (Zhao et al., 27 May 2026, Han et al., 10 Feb 2026). Third, some formulations contain minor inconsistencies between prose and equations, especially in incorrect-group weighting or clipping notation, which complicates direct transplantation (Han et al., 10 Feb 2026, Pang et al., 4 Aug 2025).

A broader theoretical tension concerns the fidelity of token-level PPO-style correction in GRPO. One line argues that standard GRPO’s token-level correction does not estimate the true current-policy gradient and that simplified stale-policy updates can perform comparably (Pang et al., 4 Aug 2025). Another line emphasizes off-policy GRPO and clipped surrogates with theoretical reward-improvement guarantees, showing that both on-policy and off-policy GRPO can improve reward under appropriate total-variation control (Mroueh et al., 28 May 2025). This suggests that the “right” compression of importance information remains unsettled: pairwise reduction, trajectory-level correction, geometric aggregation, and stale-policy simplification each have distinct justifications.

A final open issue is that “C-GRPO” itself remains non-standardized. Current evidence supports the existence of a coherent design space rather than a canonical named algorithm. This suggests that future work may stabilize around a narrower definition, but at present the term is best understood as a descriptive umbrella for compression-aware GRPO-family methods rather than a single fixed objective.

8. Synthesis

Compression Group Relative Policy Optimization refers most plausibly to GRPO-family methods that retain group-relative credit assignment while compressing some part of the training signal or update process. The literature supports four major instantiations of this idea.

First, reward shaping can compress behavior, as in FGO, by teaching policies to prefer shorter, still-correct reasoning trajectories (Han et al., 10 Feb 2026). Second, update selection can compress the optimization payload, as in BPPO and 2-GRPO, by retaining only informative pairs or prefixes (Zhao et al., 27 May 2026, Wu et al., 1 Oct 2025). Third, sequence aggregation can compress token-level correction into robust trajectory-level statistics, as in SSPO, TEPO, and TIC-GRPO (Glazyrina et al., 22 Feb 2026, Lin et al., 10 Oct 2025, Pang et al., 4 Aug 2025). Fourth, co-distillation and constrained formulations can compress systems cost or impose explicit compression budgets through auxiliary objectives or Lagrangian costs (Kwon et al., 9 May 2026, Girgis et al., 5 Feb 2026).

Taken together, these works suggest that the essential design principle of C-GRPO is not mere shrinking of outputs. It is the replacement of overly fine-grained, redundant, or poorly aligned policy-update signals with compact surrogates that preserve the relative-comparison logic of GRPO. In that sense, C-GRPO is best viewed not as a single algorithm but as a developing research program inside critic-free, group-relative policy optimization.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Compression Group Relative Policy Optimization (C-GRPO).