GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning

Abstract: Competitive programming remains one of the last few human strongholds in coding against AI. The best AI system to date still underperforms the best humans competitive programming: the most recent best result, Google's Gemini~3 Deep Think, attained 8th place even not being evaluated under live competition conditions. In this work, we introduce GrandCode, a multi-agent RL system designed for competitive programming. The capability of GrandCode is attributed to two key factors: (1) It orchestrates a variety of agentic modules (hypothesis proposal, solver, test generator, summarization, etc) and jointly improves them through post-training and online test-time RL; (2) We introduce Agentic GRPO specifically designed for multi-stage agent rollouts with delayed rewards and the severe off-policy drift that is prevalent in agentic RL. GrandCode is the first AI system that consistently beats all human participants in live contests of competitive programming: in the most recent three Codeforces live competitions, i.e., Round~1087 (Mar 21, 2026), Round~1088 (Mar 28, 2026), and Round~1089 (Mar 29, 2026), GrandCode placed first in all of them, beating all human participants, including legendary grandmasters. GrandCode shows that AI systems have reached a point where they surpass the strongest human programmers on the most competitive coding tasks.

• The paper introduces Agentic GRPO, a novel reinforcement learning algorithm that decouples immediate and delayed reward assignments to support multi-agent reasoning in competitive programming.
• It employs specialized agentic modules—including main solvers, hypothesis generators, and summarizers—to outperform human grandmasters in live Codeforces contests.
• Empirical evaluations reveal significant gains in test-case generation, long-context handling, and scalable distributed training, marking a milestone in autonomous code synthesis.

GrandCode: Achieving Grandmaster-Level Performance in Competitive Programming via Agentic Reinforcement Learning

Introduction

GrandCode establishes a new state-of-the-art in AI for competitive programming, surpassing human grandmasters on public Codeforces live contests. Distinct from prior approaches, GrandCode leverages a multi-agent RL pipeline with explicit agentic modules and a reinforcement learning algorithm—Agentic GRPO—engineered for severe off-policy drift under delayed and distributed reward feedback. The system is evaluated under stringent, live conditions, consistently obtaining first-place standings in multiple Codeforces rounds and marking a critical milestone in autonomous algorithmic reasoning and generalization capability.

Figure 1: Overview of the full pipeline. In post-training, noisy competitive-programming data is used for continued training, supervised fine-tuning on reference solutions, auxiliary hypothesis generation policy $\pi_{\mathrm{hypothesis}}$ and summarization policy $\pi_{\mathrm{summary}}$ are trained, and multi-component RL is applied. At test/online-contest time, direct generation addresses easy cases, while an online test-time RL loop operates for harder instances.

System Architecture

GrandCode employs an orchestrated set of specialized agentic modules:

• Main Solver ($\pi_{\mathrm{main}}$): Generates reasoning chains and code.
• Hypothesis Generator ($\pi_{\mathrm{hypothesis}}$): Formulates interpretable, verifiable conjectures about problem structure, refined via small-scale empirical validation.
• Summarizer ($\pi_{\mathrm{summary}}$): Compresses and manages extremely long contexts, a necessity for high-difficulty tasks.
• Test Case Generator: Synthesizes adversarial and large-scale stress tests to identify logical errors and inefficiencies.

Post-training integrates continued pretraining on diverse, noisy corpora, SFT with structured (question, thinking, solution) data filtered for fidelity to reference solutions, and joint multi-component RL. At test time, the system routes by operation difficulty, executes direct solution for easy problems, and applies on-the-fly, instance-specific RL adaptation for difficult tasks.

Figure 2: Hypothesis generation and small-scale verification. The agent proposes a compact characterization, validates it against random instances using a brute-force solver, and iterates until hypotheses are consistent with all verification tests.

Figure 3: Examples of contest figures whose visual structure is difficult to capture with text-only descriptions—necessitating native multimodal handling for competitive programming tasks.

Agentic Group Relative Policy Optimization

GrandCode introduces Agentic GRPO, addressing shortcomings of standard RL in distributed, delayed multi-agent rollouts. Unlike vanilla GRPO, which assigns credit only from terminal rewards, Agentic GRPO decouples reward credit assignment:

• Immediate Reward: As soon as an intermediate reward $r_t$ becomes available for stage $s_t$, an update is triggered with normalized group statistics.
• Delayed Correction: When terminal reward $r_N$ is observed, a correction equal to $r_N - r_t$ is applied to previous stages, ensuring accurate long-term credit assignment even with heavy asynchrony and drift.

This mechanism robustly supports multi-stage reasoning and code synthesis processes, where code generation, hypothesis proposal, verification, and adversarial testing unfold asynchronously and with temporally distributed rewards.

Test Case Generation and Robust Verification

Test case generation is not limited to superficial correctness but specifically targets logical, adversarial, and efficiency faults. Core strategies include:

• Difference-Driven Test Synthesis: Differential fuzzing via LLM-simulated input generation and validation, retaining only test cases inducing inconsistent outputs among candidate solutions.
• Solution-Based Attacks: Where gold solutions exist, adversarial cases are generated by probing differences with reference implementations to expose non-obvious flaws.
• Dynamic, Large-Scale Evaluation: By leveraging progressively faster candidate solvers, the system tests the scalability and limits of code robustness on inputs infeasible for brute-force enumeration.

Empirical analysis over 50 real Codeforces tasks demonstrates that composite adversarial test generation, coupled with online adaptation, can close the gap between hidden judge tests and internal verification, achieving perfect alignment in the examined sample set.

Multimodal and Long-Context Problem Handling

Given that modern competitive programming increasingly involves visual, diagrammatic, or multimodal instructions, GrandCode natively processes multimodal inputs rather than relying on lossy text conversion heuristics. Ablations demonstrate clear performance degradation when visual context is textually abstracted, highlighting the necessity of direct visual reasoning capabilities.

Simultaneously, for high-complexity instances, long reasoning traces (potentially 100K+ tokens) are managed via a dedicated summarization policy, which is trained with both RL and conditional SFT to maximize information retention, ensure tractability in rollout computation, and facilitate end-to-end credit assignment.

RL and Inference Infrastructure

GrandCode leverages a modular infrastructure for efficient, stable distributed RL:

• Asynchronous PipelineRL: Enables concurrent sampling and training, overlapping multiple policy versions and updating with token-level staleness weighting.
• Pipelined Context Parallelism: Provides near-linear scalability for long contexts by combining sequential DeltaNet and full attention layers, substantially improving hardware utilization in long-sequence scenarios.

  Figure 4: An illustration of pipelined context parallelism for 3 DeltaNet layers + 1 softmax attention layer, showcasing micro-batch scheduling and synchronization to optimize accelerator utilization.

Batch and context parallelism are both dynamically scheduled based on runtime statistics and instance difficulty, maximizing efficiency across a wide distribution of sequence lengths typical in real-time competition.

Empirical Evaluation: Codeforces Competition Results

GrandCode was evaluated in three consecutive live Codeforces rounds, under three unique participant IDs, achieving first place in every instance—systematically surpassing all human participants, including top-rated grandmasters. The system achieved completion of the full task set with minimal failed submissions (no more than four per task) and aggregate scores (both $S(\mathrm{joint})$ and $\pi_{\mathrm{summary}}$0) that decisively exceeded the closest human followers.

Figure 5: Standings and submission pages for GrandCode (Round 1087, ID averyjones1), showing first-place finish in official contest conditions.

Figure 6: Standings for Round 1088. GrandCode under ID yokeko achieves top placement, validating robustness across divisions and problem sets.

Figure 7: Standings for Round 1089. GrandCode (ID Vortex1) sustains consistent superiority over all human participants.

Task analysis (e.g., Problem G, "Toothless") includes end-to-end illustration of both complex problem statement processing and solution trace generation.

Figure 8: Problem G “Toothless” details with before/after visualizations per move—demonstrating the requirement for reasoning over visually structured input domains.

Benchmark and Ablation Performance

On a curated 100-problem benchmark spanning five difficulty tiers, GrandCode’s pipeline optimization yielded:

• Accept rate: plateauing at 73% post continued, supervised, and summary-augmented training.
• Level 5 (hardest) problems solved: 7 out of 20 with post-training, reaching 15 after full test-time RL.
• Weighted score (0–100): 73.5 after applying all system components, substantially higher than top LLM baselines (Gemini 3.1 Pro, Claude Opus 4.6, GPT-5.4, which remained in the 63–64 range).

Test-time RL (LoRA-based, task-specific adaptation) accounts for the strongest jump in the hardest problem region, verifying the critical role of instance-level adaptation.

Theoretical and Practical Implications

GrandCode’s success underscores the necessity of architectures capable of distributed, asynchronous, and agentic credit assignment in complex, multi-agent reasoning domains. The Agentic GRPO framework represents a tangible advance for RL-based credit propagation in realistic, high-latency evaluation tasks.

Practically, GrandCode’s performance on Codeforces demonstrates that AI systems—when equipped with dedicated agentic loops, thorough adversarial verification, robust RL optimization, and efficient parallel computation—now match or exceed the top echelon of human programmers, not only in offline benchmarks but under adversarial, real-time, multimodal contest conditions.

Theoretically, these results reinforce the importance of modular RL agents, staged credit assignment, and environment coupling across hypothesis, verification, code synthesis, and adversarial testing. Future work will likely probe generalization to broader algorithmic synthesis, dynamic problem domains, and open-ended discovery, as well as deeper integration of symbolic and learned reasoning in agentic frameworks.

Conclusion

GrandCode represents a milestone in agentic AI for code reasoning. By coordinating modular RL agents, robust verification pipelines, and high-throughput distributed infrastructure, it outperforms human grandmasters in real competitive programming contests. These results highlight the effectiveness of deep agentic RL systems with both offline and test-time adaptation, and signal the imminent shift toward fully autonomous, superhuman-level performance on previously human-dominated cognitive domains.

---

For further details and supporting empirical results, see "GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning" (2604.02721).