GC²PO: Group Causal Counterfactual Policy Optimization

• The method decomposes reasoning trajectories into groups and applies counterfactual comparisons to assign non-uniform credit, addressing the limitations of outcome-only rewards.
• GC²PO integrates grouping operations, latent-space perturbations, and span-masking interventions to capture causal importance during policy updates in LLMs.
• Empirical evaluations demonstrate improved performance on reasoning benchmarks with controlled computational overhead compared to uniform credit assignment methods.

Searching arXiv for the cited GC²PO-related papers and adjacent causal policy optimization work. Group Causal Counterfactual Policy Optimization (GC²PO) denotes a causal-policy-optimization paradigm in which grouped units of behavior—most often multiple reasoning trajectories, episodes, or spans—are evaluated through counterfactual comparisons and then used for non-uniform credit assignment during policy updates. In current arXiv usage, the term is most directly associated with post-training frameworks for LLM reasoning that replace outcome-only supervision with causally grounded rewards on intermediate reasoning structure, but closely related papers also use the label descriptively for span-masking-based token credit assignment and for causally informed extensions of groupwise policy optimization (Wang et al., 6 Feb 2026, Khandoga et al., 10 Feb 2026, Gu et al., 7 Aug 2025).

1. Terminology, scope, and recurring structure

The term GC²PO is not yet a single universally fixed algorithmic name. In "Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning" it is the explicit name of a post-training framework for LLM reasoning (Wang et al., 6 Feb 2026). In "Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization," the authors state that their counterfactual importance–weighted policy optimization method can be mapped onto the GC²PO paradigm, with grouping over reasoning spans, counterfactual masking interventions, and policy-gradient reweighting (Khandoga et al., 10 Feb 2026). In "Group Causal Policy Optimization for Post-Training LLMs," the algorithm is named GCPO, and “GC²PO” is described only as a descriptive variant emphasizing the counterfactual or causal projection embedded in GCPO rather than a distinct method (Gu et al., 7 Aug 2025).

Across these uses, three components recur: a grouping operation, a counterfactual estimand, and a policy-optimization rule.

Variant	Grouped object	Counterfactual mechanism
GC²PO for reasoning	Episodes within sampled trajectories	Latent-space perturbations of episode representations
Span-based GC²PO instantiation	Reasoning spans in a completion	Mask a span and measure answer log-probability drop
GCPO / “GC²PO” interpretation	Candidate responses in a group	Causal projection correcting collider-induced dependence

This suggests that GC²PO is best understood as a family resemblance term for methods that reject uniform sequence-level credit and instead attempt to assign credit through causal or counterfactual structure internal to grouped reasoning traces.

2. Causal formulation in LLM reasoning

In the formulation of (Wang et al., 6 Feb 2026), GC²PO is motivated by a failure mode of GRPO-style outcome rewards: correct final answers receive reward $1$, incorrect answers receive $0$, and the reasoning process is not directly evaluated. The paper states that this couples credit to correctness rather than to reasoning quality, so trajectories with sound reasoning but minor mistakes are penalized, while lucky guesses with flawed logic are rewarded. The proposed remedy is to interpret the $K$ sampled trajectories for a fixed question $x$ as parallel counterfactual trials in an SCM equivalent to the underlying MDP.

The method decomposes each candidate trajectory into episodes $E_{k,l}$, maps each episode to a latent vector $u_{k,l}(x) \in \mathbb{R}^d$, and defines an induced answer distribution $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$. A family of latent-space perturbation operators $\{g_m\}_{m=1}^M$ generates perturbed episode representations

$\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$

with corresponding perturbed answer distributions $q^{(m)}(\cdot \mid x)$. Two quantities are then estimated. The robustness or stability term is

$0$0

and the effectiveness or expressiveness term is

$0$1

These are combined into the episodic causal counterfactual reward

$0$2

GC²PO then mixes this episode-level counterfactual reward with outcome reward. For candidate $0$3 and episode $0$4,

$0$5

Token-level rewards are constructed via surprise weights under the old policy,

$0$6

so that

$0$7

After truncated-mean aggregation to $0$8, group-normalized advantages $0$9 are formed, and token-level advantages are defined as

$K$0

The policy objective is a GRPO-style PPO objective with clipping and KL regularization, using $K$1 rather than a uniform completion-level advantage (Wang et al., 6 Feb 2026).

The same paper couples this construction to three theoretical statements. Theorem 1 gives an SCM-equivalence motivation for treating grouped trajectories as counterfactual experiments. Theorem 2 states an identifiability claim for $K$2, asserting that maximizing stability enforces invariance to spurious cues while maximizing expressiveness prevents collapse. Theorem 3 gives an oracle-policy bound of order $K$3 under manifold and perturbation-spanning assumptions (Wang et al., 6 Feb 2026).

3. Span-level counterfactual credit assignment

A second, more concrete instantiation of GC²PO appears in (Khandoga et al., 10 Feb 2026), where the grouped objects are reasoning spans inside a sampled completion. The paper begins from GRPO and DAPO, which assign uniform token credit within a completion. In the DAPO baseline, for prompt $K$4, $K$5 sampled completions $K$6, and rewards $K$7, the completion-level advantage is

$K$8

and the loss is

$K$9

so every token inherits the same scalar advantage.

The GC²PO mapping is explicit: reasoning tokens are partitioned into non-overlapping spans $x$0, where $x$1 and $x$2 in the reported implementation. Span detection uses arithmetic-focused regex patterns, including equations such as “23 + 45 = 68,” multiplication and division, proportion and rate expressions, aggregation steps such as “6 + 3 + 1 = 10,” conclusion cues such as “therefore,” sentence boundaries, and simple headers such as “Step 1:”. Tokens outside detected spans receive baseline weight $x$3; final answer tokens receive a fixed boost $x$4.

For a reasoning trace $x$5 and final answer sequence $x$6, the outcome sink is the teacher-forced answer log-probability

$x$7

For each span $x$8, the method constructs $x$9 by replacing span tokens with a fixed placeholder repeated to match span length, using the model’s pad token or end-of-text token if pad is unavailable. The counterfactual importance signal is

$E_{k,l}$0

The paper also writes this as $E_{k,l}$1 with $E_{k,l}$2. Larger positive $E_{k,l}$3 means masking the span reduces answer probability, so the span is treated as causally important.

Importance is normalized within each completion: $E_{k,l}$4 and converted into token weights

$E_{k,l}$5

with recommended $E_{k,l}$6. The importance-weighted REINFORCE estimator is

$E_{k,l}$7

and the DAPO-compatible loss is

$E_{k,l}$8

This variant is explicitly intervention-based at the token-span level. The paper’s case analysis states that masking scaffolding such as “Let me think…” yields near-zero $E_{k,l}$9, while masking an arithmetic span yields large $u_{k,l}(x) \in \mathbb{R}^d$0, pushing arithmetic tokens toward $u_{k,l}(x) \in \mathbb{R}^d$1 and scaffolding toward $u_{k,l}(x) \in \mathbb{R}^d$2. The authors treat this as evidence that the method captures causal importance rather than merely token salience (Khandoga et al., 10 Feb 2026).

4. Relation to GCPO and to broader causal policy optimization

The GCPO framework of (Gu et al., 7 Aug 2025) occupies a closely related but distinct position. It begins from GRPO with grouped candidate responses $u_{k,l}(x) \in \mathbb{R}^d$3 and argues that candidate responses should not be treated as independent because many pipelines condition on all candidates to form an integrated output $u_{k,l}(x) \in \mathbb{R}^d$4. In the induced SCM, this produces a collider structure $u_{k,l}(x) \in \mathbb{R}^d$5, so candidates that are independent given the query $u_{k,l}(x) \in \mathbb{R}^d$6 become dependent once one conditions on the integrated output. The paper formalizes a causal projection with

$u_{k,l}(x) \in \mathbb{R}^d$7

and defines a causally projected predictor

$u_{k,l}(x) \in \mathbb{R}^d$8

Advantages are then rescaled by a causal factor

$u_{k,l}(x) \in \mathbb{R}^d$9

and optimization includes an additional KL term toward a causally projected reference policy $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$0. The same paper is explicit that GCPO is the algorithmic name and that “GC²PO” is only a descriptive reading emphasizing the causal or counterfactual interpretation (Gu et al., 7 Aug 2025).

Outside LLM post-training, related work supplies a broader causal-policy-optimization substrate. "Towards Causal Model-Based Policy Optimization" introduces C-MBPO, which learns a local SCM over causal state variables $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$1, spurious variables $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$2, and rewards $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$3, then performs intervention-based rollouts using

$q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$4

That paper does not define GC²PO, but it describes a natural extension by adding a group variable $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$5 and optimizing group-conditioned objectives (Caron et al., 12 Mar 2025).

"Counterfactual Influence in Markov Decision Processes" studies a different issue: as a counterfactual MDP path diverges from the observed path, the observed trajectory may cease to influence the counterfactual world, so analysis can collapse from counterfactual to interventional. The paper formalizes influence through support overlap and constructs influence-constrained counterfactual models; the group-level GC²PO connection is presented as a principled extension rather than as the paper’s core algorithm (Kazemi et al., 2024).

"From Plausible to Causal: Counterfactual Semantics for Policy Evaluation in Simulated Online Communities" shifts the emphasis from policy-gradient training to causal policy evaluation. It frames group-aware policy optimization in terms of counterfactual estimands such as the Probability of Necessity $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$6, Probability of Sufficiency $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$7, Probability of Necessary and Sufficient causation $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$8, and $q(\cdot \mid u_{k,l}(x)) \in \Delta(\mathcal{A})$9, and then defines a group-aware objective over simulator-conditional estimates. This is again a GC²PO synthesis rather than a single training rule for neural policies (Goyal et al., 5 Apr 2026).

A plausible implication is that GC²PO has two active meanings in the literature: a narrow meaning centered on LLM reasoning credit assignment, and a broader meaning referring to grouped causal-counterfactual policy optimization more generally.

5. Empirical results and observed effects

The most direct GC²PO evaluation for reasoning appears in (Wang et al., 6 Feb 2026). The reported benchmarks are AIME 2024/2025, AMC 2023, MATH500, MinervaMATH, and GSM8K, with empirical evidence also using HumanEval for process-validity analysis. Base models include DeepScaleR-1.5B-Preview, DeepSeek-R1-Distill-Qwen-1.5B, and DeepSeek-R1-Distill-Qwen-7B. Training uses learning rate $\{g_m\}_{m=1}^M$0, weight decay $\{g_m\}_{m=1}^M$1, batch size $\{g_m\}_{m=1}^M$2, $\{g_m\}_{m=1}^M$3, and $\{g_m\}_{m=1}^M$4. On average Pass@1, GC²PO reaches $\{g_m\}_{m=1}^M$5 on DeepScaleR-1.5B-Preview versus $\{g_m\}_{m=1}^M$6 for $\{g_m\}_{m=1}^M$7L2T-GRPO and $\{g_m\}_{m=1}^M$8 for $\{g_m\}_{m=1}^M$9GRPO; $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$0 on DeepSeek-R1-Distill-Qwen-1.5B versus $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$1 for $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$2L2T-GRPO and $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$3 for $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$4GRPO; and $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$5 on DeepSeek-R1-Distill-Qwen-7B versus $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$6 for $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$7L2T-GRPO and $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$8 for $\tilde u_{k,l}^{(m)}(x) = g_m(u_{k,l}(x)),$9GRPO. The paper also reports computational overhead of approximately $q^{(m)}(\cdot \mid x)$0 versus GRPO, best accuracy under lower token budgets, near-constant gradient norm, and no spikes in sampling ratio (Wang et al., 6 Feb 2026).

The span-based GC²PO instantiation in (Khandoga et al., 10 Feb 2026) reports GSM8K experiments on Qwen3-1.7B, Qwen2.5-3B, and Llama3.2-3B, with LoRA adapters $q^{(m)}(\cdot \mid x)$1, DAPO with $q^{(m)}(\cdot \mid x)$2 completions, binary rewards, temperature $q^{(m)}(\cdot \mid x)$3, top-$q^{(m)}(\cdot \mid x)$4 $q^{(m)}(\cdot \mid x)$5, $q^{(m)}(\cdot \mid x)$6 steps, and $q^{(m)}(\cdot \mid x)$7 random seeds. Test accuracy at step $q^{(m)}(\cdot \mid x)$8 is $q^{(m)}(\cdot \mid x)$9 for Qwen3-1.7B versus $0$00 for DAPO, $0$01 for Qwen2.5-3B versus $0$02 for DAPO with $0$03, and $0$04 for Llama3.2-3B versus $0$05 for DAPO. AUC gains are $0$06 pp, $0$07 pp, and $0$08 pp respectively, with the last reported as non-significant. The method reaches DAPO’s final accuracy several hundred steps earlier, with the paper giving step $0$09 versus $0$10 for Qwen2.5-3B. Inverted importance degrades performance by $0$11 pp on Qwen3-1.7B and $0$12 pp on Llama3.2-3B relative to DAPO, while random weights are near-neutral or slightly worse. The reported ordering is Counterfactual $0$13 Uniform $0$14 Random $0$15 Inverted. Structural analyses report that $0$16 of spans are critical, $0$17 are distractors, calculation chains are $0$18 more prevalent in critical spans, and $0$19 of high-importance tokens receive $0$20 of gradient mass (Khandoga et al., 10 Feb 2026).

The GCPO paper reports additional evidence for causality-aware groupwise optimization (Gu et al., 7 Aug 2025). On DeepScaleR-1.5B-Preview, average Pass@1 is $0$21 for GCPO versus $0$22 for GRPO and $0$23 for GVPO; on DeepSeek-R1-Distill-Qwen-1.5B it is $0$24 versus $0$25 for GRPO and $0$26 for GVPO; on DeepSeek-R1-Distill-Qwen-7B it is $0$27 versus $0$28 for GRPO and $0$29 for GVPO. On HumanEval with Qwen2-7B-Instruct, GCPO reaches Pass@1 $0$30 in 0-shot and $0$31 in 5-shot. Component ablations reduce average performance from $0$32 to $0$33 without advantage weighting and to $0$34 without the causal KL term. The reported training-cost increase is approximately $0$35 relative to GRPO, with the most stable gradient norms across training (Gu et al., 7 Aug 2025).

Taken together, these results establish a repeated empirical pattern: non-uniform, causally structured credit assignment improves over uniform groupwise baselines on mathematical reasoning tasks, while the exact mechanism differs across papers.

6. Assumptions, limitations, and points of dispute

A central misconception is that GC²PO refers to one settled optimizer. The literature does not support that reading. One paper introduces GC²PO as an episodic causal counterfactual reward framework for LLM reasoning; another maps span-masking-based importance weighting onto the GC²PO paradigm; another states that GCPO is the algorithmic name and that “GC²PO” is only a descriptive variant (Wang et al., 6 Feb 2026, Khandoga et al., 10 Feb 2026, Gu et al., 7 Aug 2025).

The assumptions also differ materially across formulations. In the episodic LLM-reasoning version, identifiability depends on latent perturbations predominantly exciting non-causal directions while preserving causal structure, and Monte Carlo variance depends on the number of perturbations $0$36 (Wang et al., 6 Feb 2026). In the span-masking version, answer log-probability must be a faithful proxy for causal contribution, and span masking must approximate the do-operator without inducing pathological distribution shift. The same paper notes mask artifacts from padding in mid-sequence, reports that arithmetic-focused detectors do not capture code structure, states that MBPP+ shows no consistent benefit in code generation, and reports a correlation between longer spans and larger $0$37 of approximately $0$38. It also quantifies compute overhead as $0$39 to $0$40 wall-clock per step depending on model and average $0$41, with CF estimation accounting for approximately $0$42 to $0$43 of total step time (Khandoga et al., 10 Feb 2026).

The GCPO formulation depends on a specific SCM in which grouped candidates become dependent through conditioning on an integrated output, and the causal projection is only approximated in representation space through repeated conditional generations. The paper explicitly notes that exact $0$44 is unknown, that identifiability depends on approximation quality, and that explicit do-operator counterfactuals are not computed (Gu et al., 7 Aug 2025).

Broader causal-policy-optimization work introduces further caution. In the counterfactual-influence analysis of MDPs, counterfactuals can lose observational influence and become interventional as paths diverge from the observed trajectory (Kazemi et al., 2024). In simulator-based policy evaluation, quantities such as $0$45, $0$46, $0$47, and $0$48 are interpreted as simulator-conditional causal estimates whose policy relevance depends on simulator fidelity and transportability to the real deployment domain (Goyal et al., 5 Apr 2026).

The open problems listed across papers are correspondingly heterogeneous: learned segmentation or task-specific rules for span discovery, pairwise or multi-span interventions, actor-critic or preference-model hybrids, proxy-model or asynchronous counterfactual estimation, tighter identification of causal projections, and explicit evaluation of group fairness or group-conditional robustness (Wang et al., 6 Feb 2026, Khandoga et al., 10 Feb 2026, Gu et al., 7 Aug 2025). This suggests that GC²PO is presently a research program rather than a finished standard, with its central commitment being causal credit assignment inside grouped policy optimization rather than any single canonical estimator or loss.