---
title: 'Ring-2.6: Scalable Agentic Transformer Models'
url: https://www.emergentmind.com/topics/ring-2-6-model-family
type: topic
---

# Ring-2.6: Scalable Agentic Transformer Models

The Ring-2.6 model family comprises a suite of large-scale transformer-based neural networks specifically engineered for agentic intelligence, combining deep reasoning, efficient serving, and scalable deployment at the trillion-parameter scale. Developed as an upgrade to the Ling-2.0 base, Ring-2.6 introduces a hybrid linear attention mechanism and novel optimization strategies tailored for both token efficiency and complex agent-environment interactions. The model family distinguishes itself by enabling high reasoning capability per output token and robust real-world agentic workflows, establishing new open-weight benchmarks for throughput and efficiency [2606.15079]. 

## 1. Architectural Innovations: Hybrid Linear Attention

The Ring-2.6 models replace standard softmax attention layers with a hybrid linear attention backbone to achieve O($n·d^2$) complexity per layer, crucial at context lengths ≫ 4K where FLOPs and memory for softmax attention become prohibitive. The architecture implements a 7:1 layer ratio between Lightning Attention and Multi-Head Latent Attention (MLA), as determined by scaling-law experiments designed to equate overall FLOPs within deployment constraints.

- **Lightning Attention (FLA):** Implements attention via a causal recurrence,
  $$
  h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad
  o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)
  $$
  where $\phi$, $\kappa$ are element-wise feature maps, accumulating a summary of key-value (K·V) blocks. FLA reduces computational complexity and memory footprint over full softmax.

- **Multi-Head Latent Attention (MLA):** Compresses K,V matrices into a much lower L-dimensional latent space ($L \ll n$), enabling attention over this compact representation. With TransMLA,
  $$
  A_{\rm MLA}(Q,K,V) = (QW_q)\,\mathrm{Softmax}((\tilde K)\tilde K^\top / \sqrt{d})\,(VW_v)
  $$
  Latent attention enables significant memory reductions (by $\sim n/L$) and accelerates both training and inference along long sequences.

Empirical benchmarks report 1.5–4× speedups in decode FLOPs against previous Grouped-Query Attention (GQA) architectures, with direct benefits for scaling to longer contexts [2606.15079].

## 2. Layerwise Migration and Upgraded Training Paradigm

Ring-2.6 retrofits the Ling-2.0-1T base (80 layers, MoE FFNs, GQA attention) through a multistage migration and pre-training regimen:

1. **Lightning Conversion:** Expands QKV to multi-head, inserts gating, preserves QK Norm and rotary position encoding (RoPE).
2. **Linear Warmup:** Freezes all but Lightning parameters; learning rate (LR) warmup restores pre-conversion loss.
3. **MLA Conversion:** Applies QK Norm absorption via layer calibration,
   $$
   \hat W_q = \frac{W_q}{\sqrt{\frac{1}{N d} \sum_{i,j}q_{ij}^2+\epsilon} \odot \gamma_Q},\qquad
   \hat W_k = \frac{W_k}{\sqrt{\frac{1}{N d} \sum_{i,j}k_{ij}^2+\epsilon}\odot \gamma_K}
   $$
   followed by partial-RoPE adaptation and TransMLA integration.
4. **MLA Warmup:** Unfreezes remaining parameters and returns learning rate to baseline.

Continued pre-training utilizes $\sim$8T tokens, with an aggressive data blend (46% math/code, 50% web, 4% multilingual) and learning regime transitions including WSM warmup, constant rate, and checkpoint-merge annealing. Mid-training proceeds in three context-scaled phases (4K→32K, extended to 256K), enabling robust adaptation to ultra-long inputs. Unified co-design spans architecture, optimization (MoE, Evo-CoT, LPO, bidirectional rewards), kernel-fused serving systems, and agentic corpora with executable traces.

## 3. Token-Efficient Reasoning and Optimization Techniques

Ring-2.6 amplifies reasoning capacity per output token through several targeted objectives:

- **Evolutionary Chain-of-Thought (Evo-CoT):** Prunes redundant reasoning steps, with the RL reward
  $$
  R_{\rm Evo} = R_{\rm acc} - \lambda_{\rm red} \sum_{s\in\text{segments}}\mathbb{I}[\text{segment redundant}]
  $$
  fostering concise but correct solutions.
- **Linguistic Unit Policy Optimization (LPO):** Regards semantically coherent spans as policy actions, with the policy gradient objective
  $$
  J_{\rm LPO}(\theta) = \mathbb{E}_{u_{1:T}\sim\pi_\theta}\left[\sum_{t=1}^T A_t\,\log\pi_\theta(u_t|u_{<t})\right]
  $$
  to improve efficiency at the unit level.
- **Bidirectional Preference Alignment:** A single reward model provides both positive and negative signals across multiple qualitative axes, with adaptive focus-reward
  $$
  \nabla J = \sum_t (r_t^+ - r_t^-)\,\nabla\log\pi_\theta(y_t|\cdot)
  $$
  for reinforced learning.
- **Shortest-Correct-Response Distillation:** Minimizes
  $$
  \mathcal{L}_{\rm distill}
  = \mathrm{CE}(\pi_\theta(\cdot|x),\,\mathbf{1}_{y^*}) + \beta\,|y^*|
  $$
  to concentrate model probability on the minimal correct answer length.

The combination of these techniques yields approximately a 4× improvement in reasoning per output token compared to Ling-2.0 models [2606.15079].

## 4. Agentic Workflows and the KPop RL Framework

The KPop (KL-bounded Policy Optimization) framework facilitates stable RL training of agentic Ring-2.6 models using large-scale, environment-grounded datasets. KPop builds upon IcePop, substituting fixed PPO-style clipping with a tokenwise binary KL mask:
$$
D_{\rm KL}^B\bigl(p_{\rm train}(y_t)\|\;p_{\rm infer}(y_t)\bigr) =
p_{\rm train}\ln\frac{p_{\rm train}}{p_{\rm infer}} +(1-p_{\rm train})\ln\frac{1-p_{\rm train}}{1-p_{\rm infer}}
$$
The mask $M_t$ enforces per-token KL constraints:
$$
M_t = \mathbf{1}[D_{\rm KL}^B(p_{\rm train}\|p_{\rm infer}) \le \phi \wedge D_{\rm KL}^B(p_{\rm infer}\|p_{\rm train}) \le \phi]
$$
yielding the final RL objective,
$$
J_{\rm KPop} = \mathbb{E}\left[\sum_t M_t\,\widehat A_t\,\ln\pi_\theta(y_t|\cdot)\right]
$$
Asynchronous rollout architecture coordinates coding (Docker/MCP), search/tool usage, and workflow execution, with partial-rollout per-iteration token-budgets ($\Phi$) and bounded staleness for policy version tags.

Ring-2.6 exhibits stable RL convergence and empirical gains: SWE-bench RL reward rises from 0.54 to 0.68 over training, with verified solve rates on SWE-bench rising from 70.8% to 76.28% [2606.15079].

## 5. Empirical Performance and Efficiency Benchmarks

Ring-2.6 establishes competitive or state-of-the-art open benchmarks in throughput, efficiency, and agentic reasoning.

| Benchmark                     | Ring-2.6-1T Performance | Baseline/Context               |
|-------------------------------|-------------------------|-------------------------------|
| Artificial Analysis Index     | 34 pts on ~16M tokens   | ~4× Ling-2.0-1T token efficiency |
| AIME 2026 Avg@64              | 95.78%                  |                               |
| LiveCodeBench-v6              | 86.95%                  |                               |
| ARC-AGI-2 Pass@2              | 66.18%                  |                               |
| SWE-bench Verified            | 74.00%                  |                               |
| GAIA-2 Search                 | 77.90%                  |                               |
| τ²-bench high (Function call) | Average: 84.26%         | Telecom: 96.71%               |
| OpenClaw PinchBench           | 87.60% (high mode)      | ClawEval: 63.82%              |

Additionally, Ring-2.6 models achieve:

- **Decode throughput:** On H100×4 (batch 32, 64K decode), Ling-2.6-flash achieves 1.3× Nemotron-3-Super, 2.4× Qwen3.5-122B, and 4.3× GLM-4.5-Air. Prefix throughput is up to 4× Ling-2.0.
- **MoE Sparsity:** Only 8/256 experts active at a time ($\sim$3% of weights by FLOPs), achieving trillion-parameter scale at $\sim$1% dense model cost.
- **Mixed-precision and kernel fusions:** BF16/FP8 mixed-precision training/inference accelerates speed by >50% with <2% log-prob drift; linghe kernel fusions boost tokens/s by 60% (BS 1) and 54% (BS 16) [2606.15079].

## 6. Model Family Characteristics and Open-Source Release

The Ring-2.6 family, together with Ling-2.6, represents a unified and open-weight approach to large-scale agentic systems. The co-design encompasses backbone architectures (hybrid attention, MoE), training protocols (migration, token-efficient RL), and optimized systems for long-context inference and rollout-driven training. All checkpoints in the 2.6 family are open-sourced to support further research and reproducibility in practical agentic intelligence, spanning low-latency response (Ling-2.6) and deep agentic reasoning (Ring-2.6).

The design and empirical results suggest convergence between efficient transformer architectures, token economy, and scalable RL for agentic AI, providing a foundation for future research in large-scale, real-world reasoning and workflow automation [2606.15079].

Source: https://www.emergentmind.com/topics/ring-2-6-model-family