Ling-2.6: Instant Response Model

• Ling-2.6 is an instant-response model that leverages architectural migration and fine-tuned co-design to achieve low-latency outputs.
• It integrates hybrid linear attention with fine-grained MoE networks to scale processing up to 262K token contexts efficiently.
• Advanced pre-training and post-training methods, including Evolutionary Chain-of-Thought, enhance token efficiency and agentic reasoning capabilities.

Searching arXiv for Ling-2.6 and closely related work to ground the article in the cited literature. Ling-2.6 is the “instant” line within the 2.6 model family, designed for low-latency response generation, high capability per output token, and scalable deployment under ultra-long contexts. In the technical report that introduces the family, Ling-2.6 is positioned against Ring-2.6, the corresponding “thinking” line for deeper, longer-horizon reasoning and agentic workflows. Rather than being trained from scratch, Ling-2.6 upgrades the Ling-2.0 base model through architectural migration pre-training, continued pre-training, mid-training, and large-scale post-training, with a unified co-design spanning model architecture, optimization objectives, serving systems, and agent training environments (Li et al., 13 Jun 2026).

1. Family placement and released configurations

The 2.6 family comprises open-weight checkpoints for both instant and thinking modes. Ling-2.6-flash and Ling-2.6-1T are the instant models, while Ring-2.6-1T is the thinking model. The family scale spans approximately $104\text{B}$ to $1\text{T}$ parameters, and the design goals for Ling-2.6 are stated as instant responses, high token efficiency, and scalable deployment (Li et al., 13 Jun 2026).

The report frames these goals in operational rather than purely architectural terms. Instant responses are pursued by reducing prefill and decode cost, stabilizing long-context latency, and enabling speculative decoding with multi-token prediction. High token efficiency is pursued by maximizing information density per generated token through Evolutionary Chain-of-Thought, Linguistic Unit Policy Optimization, bidirectional preference alignment, and shortest-correct-response distillation. Scalable deployment is pursued through joint optimization of hybrid linear attention, migration and continued training, inference kernels, and agent environments.

Model	Core configuration	Stated role
Ling-2.6-flash	32 layers; 32 heads; hidden 4096; intermediate 9216; 256 experts with 8 active; KV/Q LoRA ranks 512/1536; layer group size 8	Instant model
Ling-2.6-1T	80 layers; 64 heads; hidden 8192; intermediate 18,432; 256 experts with 8 active; KV/Q LoRA ranks 512/1536; layer group size 8	Instant model

Both Ling checkpoints are released with support for contexts up to 262,144 tokens, and both are intended to retain strong general capability while favoring lower latency and higher capability density per emitted token. A plausible implication is that the Ling/Ring split formalizes a deployment distinction between low-latency response serving and longer-horizon deliberative inference, rather than merely separating model sizes.

2. Core architecture

Ling-2.6 inherits a transformer backbone from Ling-2.0 and combines it with fine-grained MoE FFNs and a hybrid linear-attention stack. For Ling-2.6-1T-base, the reported backbone uses 80 layers, hidden size $d=8192$, 64 attention heads, head dimension 128, vocabulary 157,184, and a context window up to 262,144 tokens. Positional encoding uses RoPE with $\theta=6{,}000{,}000$, with Partial RoPE applied to 64 rotary dimensions. RMSNorm with $\epsilon = 10^{-6}$ and SiLU are used for normalization and activation (Li et al., 13 Jun 2026).

The MoE subsystem is explicitly fine-grained. Each MoE layer contains 256 routed experts plus 1 shared expert, with 8 experts active per token. The expert intermediate size is 2048, the routed output is scaled by 2.5 with normalized top-$k$ probabilities, and the router uses FP32 sigmoid scoring with grouped routing, $n_{\text{group}}=8$, top-4 group selection, and auxiliary-loss-free load balancing with expert bias. The first four transformer blocks remain dense FFNs with intermediate size 18,432.

At the attention level, the key design choice is the replacement of quadratic-cost softmax/GQA attention with a hybrid ratio of 7 Lightning Attention layers for every 1 MLA layer. Scaling-law experiments are reported over a group size $M$ in which each group contains 1 MLA layer and $(M-1)$ Lightning layers; $M=8$ gives the best scaling trend under equal FLOPs, while $1\text{T}$0 degrades loss notably. The complexity rationale is direct: Lightning Attention reduces per-token cost from $1\text{T}$1 to $1\text{T}$2, while MLA compresses the KV cache into a low-rank latent space for extreme memory savings during decoding.

The migration from the earlier attention stack requires two structural fixes for MLA conversion. The first is QK Norm absorption:

$1\text{T}$3

The report states that this removes QK Norm and fuses its normalizing effect into the projections, enabling MLA’s KV weight absorption and performance-lossless conversion. The second fix is Partial RoPE adaptation, in which RoPE-affected dimensions are decoupled, rotated by PCA-based weight rotation, and then re-concatenated to preserve Partial RoPE within TransMLA.

A further architectural consequence appears in long-context systems support. For Lightning Attention, the report introduces a context-parallel scheme based on AllGather, in which each rank computes local recurrent hidden state $1\text{T}$4 and local output $1\text{T}$5 on its sequence shard, performs a single AllGather over $1\text{T}$6 states, and folds preceding states to correct outputs. The report attributes linear scalability with $1\text{T}$7 to this design and reports approximately 68% end-to-end speedup at 256K context from Triton-fused correction kernels for variable-length sequences (Li et al., 13 Jun 2026).

3. Migration pre-training and corpus design

A central feature of Ling-2.6 is that it preserves the approximately 20T-token investment of Ling-2.0-1T while replacing the attention bottleneck, rather than reinitializing and retraining a 1T model from scratch. The report describes a three-stage process over approximately 9.6T tokens: Migration Pre-Training, Continue Pre-Training, and Mid-Training (Li et al., 13 Jun 2026).

Migration Pre-Training consumes approximately 400B tokens. It begins with Lightning Attention conversion, using linear warmup with frozen parameters except $1\text{T}$8 and QK Norm until the loss returns to the pre-conversion level. MLA conversion then proceeds through QK Norm removal via absorption, a brief partial-parameter warmup in which $1\text{T}$9, QK Norm, $d=8192$0, and $d=8192$1 are unfrozen, followed by structural conversion through Partial RoPE adaptation and TransMLA. MLA warmup then freezes structurally unmodified parameters and warms the learning rate until the loss again reaches the pre-conversion level.

Continue Pre-Training consumes approximately 8T tokens at 4K context with full-parameter training. The data mix is approximately 46% reasoning-intensive data, including math and code, approximately 50% general web data, and approximately 4% multilingual data. The report emphasizes aggressive data switching, defined as introducing higher-quality data immediately, and states that this yields faster recovery and a higher final ceiling.

Mid-Training consumes approximately 1.2T tokens and extends context while refining capabilities on long-context high-quality data. Phase 1 uses 250B tokens and samples 20% of sequences at 32K context. Phase 2 uses 425B tokens at 32K while increasing the high-quality data ratio. Phase 3 uses 525B tokens and extends to 256K context while maintaining the high-quality mix. Hyperparameters include a WSM learning-rate schedule, router bias-update $d=8192$2 in continue pre-training and $d=8192$3 in mid-training, and MTP loss weight 0.1.

The pre-training corpus is heterogeneous and explicitly agentic. It includes agentic corpora from more than 500 MCP environments and more than 3,000 tools, repository-level coding integration, verified tool traces, long-context synthetic and retrieval data for math, web parsing, summarization, and RAG fusion, general web corpora with textbook-style rewrites and atomic fact construction, math and code augmentations, and multilingual coverage including FineWeb2 and FineWeb2-hq at approximately 1.1T tokens with multilingual share 4%. This suggests that Ling-2.6’s instant-response objective is not pursued by narrowing the training distribution; rather, it is pursued by shifting both architecture and optimization toward more efficient extraction of capability from broad, tool-grounded corpora.

4. Post-training and capability per output token

The post-training stack is explicitly organized around capability density. The report describes a two-stage SFT pipeline followed by specialist fine-tuning and RL, with specialist reasoning, agent, and tool-use models distilled back into a unified Ling-2.6. Continued MTP post-training is retained to support speculative decoding speedup (Li et al., 13 Jun 2026).

For reasoning, the report introduces Evolutionary Chain-of-Thought. Its data refinement rule retains the shortest accurate candidate among expert-model outputs and uses LLM judges to prune “over-reflective” secondary reasoning, reducing average output length by 200–300 tokens. The reward is composite and contains explicit efficiency penalties. Accuracy is modeled as $d=8192$4, formatting incurs $d=8192$5 if explicit reasoning markers such as think tags appear, and a dynamic length term is defined by

$d=8192$6

with correct responses receiving $d=8192$7 and incorrect responses receiving $d=8192$8. A semantic redundancy term from segment-level LLM judgments penalizes circular or repetitive reasoning. The stated objective is to improve the model’s tendency to “think efficiently.”

Linguistic Unit Policy Optimization is cited as part of the token-efficiency suite, shifting optimization from token-level credit assignment to semantically coherent linguistic units. The report does not reproduce detailed LUPO update equations and instead notes that these were introduced in prior Ling-2.0 work. Shortest-Correct-Response Distillation is also referenced conceptually rather than with explicit loss equations; its role is to distill from shortest correct outputs so as to raise capability per token.

For agentic specialists, Group Sequence Policy Optimization is used for tool use. The process reward aligns predicted tool trajectories to ground-truth chains and penalizes unnecessary invocations, while a zlib-compression-based repetition penalty discourages degenerate or highly repetitive outputs. Dynamic Pass Rating provides an adaptive curriculum that treats early-solved and stable problems as easy, and persistently unsolved or unstable problems as hard, thereby focusing updates on samples near the frontier.

Bidirectional Preference Alignment extends the same efficiency logic into preference optimization. The report states that a bidirectional focus reward model integrates positive incentives and negative penalties into a single rubric, broadening dynamic range and improving signal-to-noise ratio. It also shifts training weight away from saturated metrics toward weaker dimensions through focal reweighting. Specialized evaluators include ReportLogic for macro- and discourse-level logic in long-form outputs and a rule-decomposition verifier for instruction following, with the explicit aim of keeping outputs concise and information-dense.

5. Agentic infrastructure, inference kernels, and serving

Although KPop is introduced primarily for Ring-2.6’s long-horizon agentic RL, the report states that the RL infrastructure and environment-grounded data also benefit the Ling line’s agentic specialization and distillation. The shared data mixture includes repository-level coding with approximately 300K PR–Issue pairs and tests across 1,550 repositories in more than 30 languages, mobile-app search in ARE-inspired synthetic universes, web search built from Wikipedia graph expansions designed to resist lexical shortcuts, and general tool use over 197 MCP servers, more than 2,400 tools, and more than 170 synthetic toolkits (Li et al., 13 Jun 2026).

The ASystem RL substrate includes a global rollout scheduler, spillover-based training–inference overlap, inference failover with streaming checkpoints, and support for session isolation, structured trajectory trees, branch detection, reward sharing, and both individual and concatenated training modes. For training and inference numerics, the report describes integrated FP8 support with the LM head retained in FP32, BF16 for attention and shared experts, and FP8 for routed experts, reporting 30% throughput gain while matching BF16 stability at 1T/128K settings.

KPop itself replaces IcePop’s uniform ratio masks with adaptive binary KL masks to control train–infer mismatch heterogeneously across tokens. In the report, this empirically stabilizes RL and scales Ring-2.6-1T agent training, increasing reward from 0.54 to 0.68 and improving SWE-bench Verified from 70.8% to 76.28%. For Ling-2.6, the significance is infrastructural: the same environment-grounded RL stack and asynchronous partial-rollout machinery supply the specialist data and training conditions later distilled into the instant line.

Serving efficiency is treated as a first-class design target. The report attributes high long-context decode throughput to the interaction of hybrid attention, MLA KV compression, Lightning Attention’s $d=8192$9 training and decoding, continued MTP post-training for speculative decoding, prefix caching, and the linghe fused-kernel library. Reported linghe fusions include BF16 kernels for QK Norm plus RoPE, Group RMSNorm plus sigmoid gate, BF16-input and FP32-output GEMMs for the router and LM head, and optimized MLA RoPE and Top-$\theta=6{,}000{,}000$0, alongside FP8 fusions for RMSNorm, SwiGLU with quantization, and Split-$\theta=6{,}000{,}000$1 blockwise FP8 GEMMs.

The measured throughput gains under input length 16,384 and shared prefix 8,192 are substantial. In BF16 at batch size 1, baseline throughput rises from 186 tok/s to 257 tok/s with MTP and to 297 tok/s with MTP plus linghe. In FP8 at batch size 1, the sequence is 155 tok/s, 236 tok/s, and 341 tok/s. For Ling-2.6-flash on 4×H20 with batch 32, tensor parallelism 4, and output length 64K, decode throughput is reported as 1.3× that of Nemotron-3-Super, 2.4× that of Qwen3.5-122B-A10B, and 4.3× that of GLM-4.5-Air. The appendix also reports improved speculative accepted length under 4-step speculation after continuing MTP training with two additional MTP layers, reaching accepted lengths of 2.71 for MTP-1, 3.23 for MTP-3, 3.29 for MTP-3-1, and 3.31 for MTP-3-share (Li et al., 13 Jun 2026).

6. Evaluation, release status, limitations, and terminology

The evaluation results place Ling-2.6 as an instant-response model family that attempts to preserve broad capability while changing its cost profile. Base-model comparisons show that Ling-2.6-1T-base and Ling-2.6-flash-base improve world knowledge on GPQA, SimpleQA, and MMMLU, improve long-context performance on LongBenchv2 and LEval, and preserve or enhance math and code performance on MathBench and LiveCodeBench relative to Ling-2.0 bases (Li et al., 13 Jun 2026).

For Ling-2.6-flash, the report lists reasoning scores of 73.85 on AIME26, 49.29 on HMMT-Feb26, 54.28 on IMO-AnswerBench, and 62.28 on LiveCodeBench-v6. Agentic scores include 61.20 on SWE-bench Verified, 81.30 on PinchBench, 66.81 on BFCL-v4, 76.36 on $\theta=6{,}000{,}000$2-bench, and 64.56 on ClawEval. Long-context and multi-turn results include 75.93 on MRCR across 16K–256K, 74.80 on Multi-IF turn-3, and 39.71 on Multichallenge.

For Ling-2.6-1T, the report lists 87.40 on AIME26, 81.93 on HMMT-Nov25, 65.81 on IMO-AnswerBench, 50.94 on ARCPrize, 52.37 on bbeh, and 65.58 on LiveCodeBench-v6. Agentic scores include 85.24 on PinchBench, 51.00 on ClawEval, 70.64 on BFCL-v4, 72.20 on SWE-bench Verified, 78.36 on $\theta=6{,}000{,}000$3-bench, and 40.45 on terminal-bench 2.0. Long-context results include 48.31 on LongBenchv2 and 80.37 on MRCR across 16K–256K. Knowledge scores include 76.53 on C-SimpleQA, 31.50 on SimpleQA-Verified, 76.17 on GPQA-Diamond, and 58.32 on SuperGPQA. The report also states that Ling-2.6-1T reaches an Artificial Analysis Intelligence Index score of 34 using approximately 16M output tokens, approximately four times better token efficiency than Ling-2.0-1T and comparable to GPT-5.4 in non-reasoning mode.

The release includes Ling-2.6-flash, Ling-2.6-1T, and Ring-2.6-1T checkpoints, with code at https://github.com/inclusionAI/Ling-V2.5, Ling instant-model collections at https://huggingface.co/collections/inclusionAI/ling-26, Ring thinking-model collections at https://huggingface.co/collections/inclusionAI/ring-26, and inference kernels at https://github.com/inclusionAI/linghe. The report describes compatibility with both FP8 and BF16 inference and explicitly supports speculative decoding through continued MTP layers.

The limitations section is unusually direct. For Ling-2.6-flash, the tight deliberation budget constrains reasoning depth, instruction compositionality, and tool-use reliability in high-complexity settings. The token-efficiency objective can conflate low-value repetition with necessary factual elaboration in knowledge-heavy outputs. Long-horizon agent robustness still degrades over extended workflows, shifting tool states, and heterogeneous environments. The report further notes multilingual prompt-stability drift and argues that public benchmarks under-measure persistence, recovery, cost-aware planning, and deployment robustness. Future directions are listed as deeper architecture–systems co-design, advanced attention, low-precision training and inference, KV-cache management, next-generation optimizers such as Muon, native multimodal agents, and improved evaluation for persistence, recovery, and planning under deployment constraints.

The name can invite confusion. In this context, “Ling-2.6” denotes the instant model line described in the 2026 technical report; it is unrelated to Huey Ling-derived binary-adder formulations such as those discussed in “4-Bit High-Speed Binary Ling Adder,” which does not use the term “Ling-2.6” (Gupta, 2019). It is likewise unrelated to the Linux 2.6 kernel-series nomenclature that appears in systems papers such as “Faults in Linux 2.6” and “Design and Implementation of the Connectionless Network Protocol (CLNP) as Loadable Kernel Modules in Linux Kernel 2.6” (Palix et al., 2014, Sugiarto et al., 2012).