Ring-2.6: Scalable Agentic Transformer Models

• Ring-2.6 models are a suite of large-scale transformer networks engineered for agentic intelligence, featuring hybrid linear attention and scalable performance.
• They integrate Lightning Attention and Multi-Head Latent Attention to reduce computational complexity and boost token efficiency for long-context processing.
• Innovative training techniques such as layerwise migration, Evo-CoT, and KPop RL enable 4× improved reasoning per token and robust real-world deployments.

The Ring-2.6 model family comprises a suite of large-scale transformer-based neural networks specifically engineered for agentic intelligence, combining deep reasoning, efficient serving, and scalable deployment at the trillion-parameter scale. Developed as an upgrade to the Ling-2.0 base, Ring-2.6 introduces a hybrid linear attention mechanism and novel optimization strategies tailored for both token efficiency and complex agent-environment interactions. The model family distinguishes itself by enabling high reasoning capability per output token and robust real-world agentic workflows, establishing new open-weight benchmarks for throughput and efficiency (Li et al., 13 Jun 2026).

1. Architectural Innovations: Hybrid Linear Attention

The Ring-2.6 models replace standard softmax attention layers with a hybrid linear attention backbone to achieve O($n·d^2$) complexity per layer, crucial at context lengths ≫ 4K where FLOPs and memory for softmax attention become prohibitive. The architecture implements a 7:1 layer ratio between Lightning Attention and Multi-Head Latent Attention (MLA), as determined by scaling-law experiments designed to equate overall FLOPs within deployment constraints.

• Lightning Attention (FLA): Implements attention via a causal recurrence,

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$

where $\phi$, $\kappa$ are element-wise feature maps, accumulating a summary of key-value (K·V) blocks. FLA reduces computational complexity and memory footprint over full softmax.

• Multi-Head Latent Attention (MLA): Compresses K,V matrices into a much lower L-dimensional latent space ($L \ll n$), enabling attention over this compact representation. With TransMLA,

$$A_{\rm MLA}(Q,K,V) = (QW_q)\,\mathrm{Softmax}((\tilde K)\tilde K^\top / \sqrt{d})\,(VW_v)$$

Latent attention enables significant memory reductions (by $\sim n/L$) and accelerates both training and inference along long sequences.

Empirical benchmarks report 1.5–4× speedups in decode FLOPs against previous Grouped-Query Attention (GQA) architectures, with direct benefits for scaling to longer contexts (Li et al., 13 Jun 2026).

2. Layerwise Migration and Upgraded Training Paradigm

Ring-2.6 retrofits the Ling-2.0-1T base (80 layers, MoE FFNs, GQA attention) through a multistage migration and pre-training regimen:

1. Lightning Conversion: Expands QKV to multi-head, inserts gating, preserves QK Norm and rotary position encoding (RoPE).
2. Linear Warmup: Freezes all but Lightning parameters; learning rate (LR) warmup restores pre-conversion loss.
3. MLA Conversion: Applies QK Norm absorption via layer calibration,

$$\hat W_q = \frac{W_q}{\sqrt{\frac{1}{N d} \sum_{i,j}q_{ij}^2+\epsilon} \odot \gamma_Q},\qquad \hat W_k = \frac{W_k}{\sqrt{\frac{1}{N d} \sum_{i,j}k_{ij}^2+\epsilon}\odot \gamma_K}$$

followed by partial-RoPE adaptation and TransMLA integration.

1. MLA Warmup: Unfreezes remaining parameters and returns learning rate to baseline.

Continued pre-training utilizes $\sim$8T tokens, with an aggressive data blend (46% math/code, 50% web, 4% multilingual) and learning regime transitions including WSM warmup, constant rate, and checkpoint-merge annealing. Mid-training proceeds in three context-scaled phases (4K→32K, extended to 256K), enabling robust adaptation to ultra-long inputs. Unified co-design spans architecture, optimization (MoE, Evo-CoT, LPO, bidirectional rewards), kernel-fused serving systems, and agentic corpora with executable traces.

3. Token-Efficient Reasoning and Optimization Techniques

Ring-2.6 amplifies reasoning capacity per output token through several targeted objectives:

• Evolutionary Chain-of-Thought (Evo-CoT): Prunes redundant reasoning steps, with the RL reward

$$R_{\rm Evo} = R_{\rm acc} - \lambda_{\rm red} \sum_{s\in\text{segments}}\mathbb{I}[\text{segment redundant}]$$

fostering concise but correct solutions.

• Linguistic Unit Policy Optimization (LPO): Regards semantically coherent spans as policy actions, with the policy gradient objective

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$0

to improve efficiency at the unit level.

• Bidirectional Preference Alignment: A single reward model provides both positive and negative signals across multiple qualitative axes, with adaptive focus-reward

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$1

for reinforced learning.

• Shortest-Correct-Response Distillation: Minimizes

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$2

to concentrate model probability on the minimal correct answer length.

The combination of these techniques yields approximately a 4× improvement in reasoning per output token compared to Ling-2.0 models (Li et al., 13 Jun 2026).

4. Agentic Workflows and the KPop RL Framework

The KPop (KL-bounded Policy Optimization) framework facilitates stable RL training of agentic Ring-2.6 models using large-scale, environment-grounded datasets. KPop builds upon IcePop, substituting fixed PPO-style clipping with a tokenwise binary KL mask:

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$3

The mask $$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$4 enforces per-token KL constraints:

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$5

yielding the final RL objective,

$$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$6

Asynchronous rollout architecture coordinates coding (Docker/MCP), search/tool usage, and workflow execution, with partial-rollout per-iteration token-budgets ($$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$7) and bounded staleness for policy version tags.

Ring-2.6 exhibits stable RL convergence and empirical gains: SWE-bench RL reward rises from 0.54 to 0.68 over training, with verified solve rates on SWE-bench rising from 70.8% to 76.28% (Li et al., 13 Jun 2026).

5. Empirical Performance and Efficiency Benchmarks

Ring-2.6 establishes competitive or state-of-the-art open benchmarks in throughput, efficiency, and agentic reasoning.

Benchmark	Ring-2.6-1T Performance	Baseline/Context
Artificial Analysis Index	34 pts on ~16M tokens	~4× Ling-2.0-1T token efficiency
AIME 2026 Avg@64	95.78%
LiveCodeBench-v6	86.95%
ARC-AGI-2 Pass@2	66.18%
SWE-bench Verified	74.00%
GAIA-2 Search	77.90%
τ²-bench high (Function call)	Average: 84.26%	Telecom: 96.71%
OpenClaw PinchBench	87.60% (high mode)	ClawEval: 63.82%

Additionally, Ring-2.6 models achieve:

• Decode throughput: On H100×4 (batch 32, 64K decode), Ling-2.6-flash achieves 1.3× Nemotron-3-Super, 2.4× Qwen3.5-122B, and 4.3× GLM-4.5-Air. Prefix throughput is up to 4× Ling-2.0.
• MoE Sparsity: Only 8/256 experts active at a time ($$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$83% of weights by FLOPs), achieving trillion-parameter scale at $$h_t = h_{t-1} + \phi(q_t)[\kappa(k_t)v_t], \quad o_t = \gamma_{\rm gate} \,\mathrm{RMSNorm}(h_t) \odot \sigma(W_{\rm gate}h_t)$$91% dense model cost.
• Mixed-precision and kernel fusions: BF16/FP8 mixed-precision training/inference accelerates speed by >50% with <2% log-prob drift; linghe kernel fusions boost tokens/s by 60% (BS 1) and 54% (BS 16) (Li et al., 13 Jun 2026).

6. Model Family Characteristics and Open-Source Release

The Ring-2.6 family, together with Ling-2.6, represents a unified and open-weight approach to large-scale agentic systems. The co-design encompasses backbone architectures (hybrid attention, MoE), training protocols (migration, token-efficient RL), and optimized systems for long-context inference and rollout-driven training. All checkpoints in the 2.6 family are open-sourced to support further research and reproducibility in practical agentic intelligence, spanning low-latency response (Ling-2.6) and deep agentic reasoning (Ring-2.6).

The design and empirical results suggest convergence between efficient transformer architectures, token economy, and scalable RL for agentic AI, providing a foundation for future research in large-scale, real-world reasoning and workflow automation (Li et al., 13 Jun 2026).