GLM-5: from Vibe Coding to Agentic Engineering

Abstract: We present GLM-5, a next-generation foundation model designed to transition the paradigm of vibe coding to agentic engineering. Building upon the agentic, reasoning, and coding (ARC) capabilities of its predecessor, GLM-5 adopts DSA to significantly reduce training and inference costs while maintaining long-context fidelity. To advance model alignment and autonomy, we implement a new asynchronous reinforcement learning infrastructure that drastically improves post-training efficiency by decoupling generation from training. Furthermore, we propose novel asynchronous agent RL algorithms that further improve RL quality, enabling the model to learn from complex, long-horizon interactions more effectively. Through these innovations, GLM-5 achieves state-of-the-art performance on major open benchmarks. Most critically, GLM-5 demonstrates unprecedented capability in real-world coding tasks, surpassing previous baselines in handling end-to-end software engineering challenges. Code, models, and more information are available at https://github.com/zai-org/GLM-5.

• The paper introduces a breakthrough in agentic engineering by integrating dynamic sparse attention, massive MoE scaling, and asynchronous RL for enhanced computational efficiency.
• It details novel mechanisms like DeepSeek Sparse Attention and Multi-Latent Attention, enabling long-context reasoning and robust performance on diverse coding and agentic tasks.
• The model achieves significant real-world gains, outperforming predecessors in coding automation benchmarks and optimizing deployment across heterogeneous hardware ecosystems.

GLM-5: From Vibe Coding to Agentic Engineering

Introduction and Motivation

GLM-5, as presented in "GLM-5: from Vibe Coding to Agentic Engineering" (2602.15763), establishes a decisive shift in LLM capabilities—from generating code through direct prompts (“vibe coding”) to achieving autonomous end-to-end agentic engineering. The work targets the dual challenges of extreme computational efficiency and robust agentic autonomy, aiming to reduce the overheads traditionally associated with large-scale MoE architectures while ensuring high performance in complex, real-world codebases and long-horizon, decision-rich scenarios.

Technical Innovations

GLM-5 introduces several architectural and algorithmic advancements over its predecessor, GLM-4.7. The principal technical contributions are as follows:

1. DeepSeek Sparse Attention (DSA): Building on the efficiency theme from DeepSeek-V3.2, DSA deploys a dynamic, content-based sparse attention mechanism, drastically reducing attention computation and memory scaling at long context lengths (up to 200K tokens). DSA replaces full attention with a fine-grained top-$k$ selection strategy, preserving relevant long-range dependencies without quality loss.
2. Massive Scaling with MoE & MLA: The model expands to a 744B total parameter count (40B active), with 256 MoE experts and utilization of Multi-Latent Attention (MLA) for high GPU memory efficiency in long-context training. The MLA implementation is finetuned via Muon Split, resolving performance and head-dimension issues found in earlier implementations.
3. Asynchronous RL Infrastructure: Post-training leverages a new asynchronous RL system that decouples trajectory generation from model weight updates. This supports scalable multi-task RL for agentic workloads, enabling efficient use of diverse agent environments and continuous model improvement with minimized synchronization costs.
4. Novel Agent RL Algorithms: GLM-5 incorporates algorithms for asynchronous, group-wise policy optimization with token-level importance sampling and robust token-in-token-out (TITO) pipelines for lossless action-reward alignment under full asynchrony, effectively controlling off-policy bias and minimizing failure due to long-tailed rollouts.
5. Full-stack Adaptation to Chinese GPU Ecosystems: The engineering stack is co-optimized for a wide array of domestic hardware, using mixed-precision quantization (W4A8), fusion kernels, and distributed inference techniques for maximal deployment versatility and cost reduction.

   Figure 1: The GLM-5 training pipeline integrates massive-scale pre-training, staged mid-training for context extension, and a progressive RL-alignment curriculum.

Training Pipeline and Data Curation

The training regime begins with multi-trillion-token pre-training, with context extension staged through 32K, 128K, and 200K token windows, progressively upsampling agentic, mathematical, and code-centric corpora. Pre-training data includes a 28.5T-token blend covering code, reasoning, and knowledge-intensive long-documents. Notable augmentations include:

• Fine-grained classifier and LLM-based filtration to extract high-quality, long-tail web and code data.
• Systematic upsampling of long-context trajectories, using both natural and synthetic data constructions, with chunk-and-aggregate scoring to defeat lost-in-the-middle phenomena.
• Expanded multilingual code and math data, integrating repository-level issues, PRs, diffs, and multi-file agent environments to enrich agentic context diversity.

Post-Training: Alignment and Agentic RL

Post-training is structured in multi-stage alignment:

• Supervised Fine-tuning (SFT): Data is diversified and scaled, with distinct treatment of general chat, reasoning (mathematical, programming, scientific), and coding/agent actions. SFT supports "interleaved thinking," "preserved thinking," and "turn-level thinking" for controllable reasoning pathways.

  Figure 2: “Interleaved Thinking” triggers explicit reasoning before actions; “Preserved Thinking” maintains reasoning history across agentic turns.

• Reasoning RL: RL training is performed across mathematics, science, code, and tool-integrated reasoning domains. The loss uses an IcePop-style mismatch ratio for robust on-policy optimization, explicitly suppressing samples with high training-inference divergence. DSA’s deterministic top-$k$ operators and frozen indexer weights maintain roll-in/roll-out consistency and RL stability.
• Agentic RL: Asynchronous, multi-task agent RL is enabled by a Multi-Task Rollout Orchestrator and TITO pipelines. Off-policy bias is controlled via direct token-level importance sampling, complemented by robust filtering and cache-locality optimization for high-throughput, long-context RL.
• General RL and On-Policy Cross-Stage Distillation: Final general alignment is achieved with hybrid rule-based, outcome, and generative reward models for correctness, emotional intelligence, and task-specific metrics. On-policy distillation bridges SFT and RL cleanly without catastrophic forgetting.

Evaluation Across Benchmarks

GLM-5 achieves a ~20% margin over GLM-4.7 and matches or surpasses leading proprietary models in rigorous benchmarks, spanning reasoning, agentic tool-use, and real-world coding.

Figure 3: GLM-5 demonstrates competitive or superior performance relative to GPT-5.2 (xhigh), Claude Opus 4.5, Gemini 3 Pro, and DeepSeek-V3.2 across diverse agentic, reasoning, and code benchmarks.

Figure 4: GLM-5 sets a new open-weights record by scoring 50 on the Artificial Analysis Intelligence Index v4.0.

Figure 5: GLM-5 is the leading open model in both the LMArena Text Arena and Code Arena evaluation tracks.

In long-horizon agentic evaluations, such as Vending-Bench 2 and CC-Bench-V2, GLM-5 leads all open models and rivals closed-source leaders in total score and backend, frontend, and long-horizon engineering tasks.

Figure 6: GLM-5 achieves strong outcomes on extended-horizon, simulation-based business and software agent tasks.

Closing the loop on real-world coding automation, GLM-5 demonstrates high build/test pass rates for frontend/backend/long-horizon engineering tasks, enabled by fully automated, agent-as-a-judge pipelines.

Figure 7: The Agent-as-a-Judge pipeline builds, launches, and interactively validates agent-generated projects for functional correctness.

Control of Context and Agentic Robustness

A key advantage of GLM-5 is robust context management, using hierarchical strategies to retain relevant tool-call rounds and discard redundant history. The model’s performance on BrowseComp improves from 55.3% to 62.0% with context folding, and a hybrid "keep-recent+$discard$" policy further increases the final accuracy to 75.9%, leading all open baselines.

Figure 8: Aggressive and hybrid context management improves agentic tool-use on the BrowseComp benchmark.

Slide Generation and Reward Model Red Teaming

Agentic capabilities extend to complex structured outputs, such as slide generation, where multi-level reward functions are used for HTML markup, runtime rendering, and perceptual evaluation. The pipeline successfully thwarts reward hacking, yielding empirical improvements in output quality:

Figure 9: Runtime-verified rewards are robust against model reward hacking in slide layout generation tasks.

Real-World Ability Gains and Deployment

GLM-5 demonstrates consistent improvements across translation, multilingual dialogue, instruction following, world knowledge, and tool calling as measured by internal and external human and automated evaluations.

Figure 10: Across five general ability domains, GLM-5 consistently outperforms its predecessor, GLM-4.7.

The adaptation to major Chinese NPU platforms, with INT4/INT8 quantization and deep kernel fusion, enables deployment parity with dual-GPU international clusters while halving deployment costs for long-sequence inference.

Implications and Future Directions

GLM-5 establishes that open-weights LLMs equipped with efficient architectural designs, scalable RL pipelines, and robust multi-domain agent training can achieve competitive frontier performance in real-world engineering domains. The modular, asynchronous RL infrastructure, full-stack chip optimization, and hierarchical context management strategies pave the way for next-generation, deployable agentic systems that are both cost-effective and autonomously adaptive.

Theoretical implications include the demonstration that nearly lossless attention sparsification (DSA) is viable for token- and task-rich agentic reasoning at scale, without long-range degradation. Practically, the agentic RL pipeline exemplifies how decoupled generation and on-policy optimization unlock both throughput and sample efficiency, inviting further research into dynamic curriculum design, modular reward architectures, and continual context management mechanisms.

GLM-5’s agentic engineering paradigm foreshadows broader use of AI in autonomous tool synthesis, complex software workflows, and adaptive agent frameworks capable of sustained, multi-turn improvement and error correction in open-ended tasks.

Conclusion

GLM-5 advances the state-of-the-art for open-weights foundation models by uniting scalable efficiency, robust real-world alignment, and agentic autonomy. Its architecture, training pipeline, and infrastructure innovations directly address long-standing challenges in efficient agentic engineering, delivering measurable gains in diverse benchmarks and practical deployment scenarios. The methodologies and results outlined in (2602.15763) will define technical baselines for efficient, agent-like large models and inspire further advances in autonomous, real-world AI systems.