DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Abstract: The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth. Together, these compute, storage, and bandwidth demands constitute the primary bottleneck to further lowering deployment costs. To address this challenge, we introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. With its Causal Encoder-Decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill, substantially improving cost efficiency for agentic workloads. To push the limits of KV cache compression, DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash. Further, through a dedicated deployment optimization known as SWA Bounded Replay, DeepSeek-V4.1-Flash reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash. Despite its much smaller KV cache footprint, the model delivers substantially better performance than the baseline. In addition, we streamline the DeepSeek-V4 architecture and introduce several efficient architectural extensions. We pretrain DeepSeek-V4.1-Flash on a multimodal corpus comprising 45T tokens and conduct comprehensive post-training, yielding strong performance across diverse text-based and multimodal agentic scenarios. Model checkpoints are available at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.

• The paper introduces DeepSeek-V4.1-Flash, a Mixture-of-Experts model that optimizes KV cache, achieving 890 bytes per token while maintaining or improving performance on agentic tasks.
• DeepSeek-V4.1-Flash implements a Hierarchical Indexer to manage long-context storage by prioritizing effective storage of relevant entries, reducing repeated accesses across layers, allowing efficient memory retrieval and reduced storage size
• The model boosts agentic performance on benchmarks like Terminal-Bench 2.1 (90.6%) and DeepSWE v1.1 (74.2%), highlighting robust coding, and task-navigational skills, with gains consistent over time and scalable in response to increased multi-agent workload

Long-context serving as the central design objective

“DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression” (2609.19969) presents a multimodal Mixture-of-Experts model designed around the deployment constraints of long-horizon agents. Its central argument is that long-context serving is no longer limited primarily by attention arithmetic. Once sparse attention reduces the cost of processing long sequences, the dominant constraints become KV-cache capacity, cache persistence, cache migration, memory bandwidth, and prefill computation.

DeepSeek-V4.1-Flash addresses these constraints through coordinated changes to architecture, numerical representation, deployment policy, kernels, training infrastructure, and post-training data generation. The resulting model has 552B backbone parameters, 196B Engram parameters, a one-million-token context window, and activates 8B parameters per token during prefill and 16B during decode. The headline cache result is a global runtime KV footprint of 890 bytes per token—approximately one quarter of DeepSeek-V4-Flash’s footprint—and a persistent KV footprint approximately one eighth as large.

The paper’s strongest claim is not merely that cache compression reduces memory usage, but that it can be achieved without the usual capability trade-off. The authors report that DeepSeek-V4.1-Flash-Base matches or exceeds larger predecessor models on many evaluations, while the post-trained model substantially improves agentic performance over DeepSeek-V4-Flash. This claim depends on a tightly integrated system: Causal Encoder-Decoder computation reduces prefill, Compressed Sparse Attention 2 reduces cache duplication, FP4 lowers cache entry size, and bounded replay eliminates most persistent SWA storage.

Architecture and computation allocation

The language backbone consists of 40 causal Transformer layers split into a 20-layer encoder and a 20-layer decoder. The model accepts text and images through a jointly trained multimodal pathway. A DeepSeek-ViT vision encoder uses 2D-RoPE, RMSNorm, SwiGLU, and a $3 \times 3$ pixel-unshuffle operation, reducing visual-token count by a factor of nine before projection into the language backbone.

The architecture combines standard DeepSeekMoE feed-forward layers with several specialized components: Causal Encoder-Decoder computation, CSA2, Single-Pass m, Engram conditional memory, DSpark speculative decoding, and a Hierarchical Sparse Indexer.

Figure 1: DeepSeek-V4.1-Flash divides its 40-layer backbone into a causal encoder and decoder and combines CSA2, SWA, Single-Pass m, Engram, DSpark, and hierarchical sparse indexing.

The Causal Encoder-Decoder arrangement is particularly important for input-heavy workloads. The encoder computes the lower half of the Transformer stack causally. For upper-layer global attention, decoder key-value states are projected from the final encoder hidden state rather than generated by running every decoder layer over every prompt token. Consequently, for a sequence of length $N$ much larger than the SWA window, prefill complexity is approximately reduced from processing all $L$ layers over $N$ tokens to processing roughly $L/2$ layers over $N$ tokens, plus bounded replay work.

This design introduces an explicit asymmetry between global and local attention. Global KV for decoder layers can be obtained from encoder representations, but SWA KV remains layer-specific. The paper therefore combines CED with Decoder SWA Bounded Replay: only the most recent SWA window is replayed through the decoder, rather than reconstructing the full multilayer receptive field. The authors report negligible quality degradation from this approximation.

The computational effect is substantial. Extending the context length from 4K to 1M tokens—a 256-fold increase—raises single-token decode FLOPs by only approximately one quarter for DeepSeek-V4.1-Flash, according to the paper’s precision-weighted accounting.

Figure 2: DeepSeek-V4.1-Flash keeps single-token decode FLOPs nearly constant as context length grows to one million tokens.

This result is consequential for agentic inference because tool use and multi-turn interaction repeatedly generate decode requests against increasingly large contexts. The result is, however, a system-level measurement rather than a universal complexity guarantee: its validity depends on the specified sparse-attention pattern, cache-reuse schedule, precision mix, and implementation kernels.

CSA2 and cross-layer cache reuse

CSA2 is the paper’s primary architectural mechanism for reducing global KV storage. It exploits three forms of compression simultaneously:

• compression along the entry dimension through sparse or compressed representations;
• compression along the sequence dimension through compressed attention entries;
• compression along the layer dimension through cross-layer reuse.

Each CSA2 layer is statically assigned one of three modes: Full, Reindex, or Reuse. Full Mode computes main KV, indexer K, indexer Q, and fresh Top-K selections. Reindex Mode reuses main KV and indexer K from the most recent Full Mode layer but computes a new indexer Q and fresh sparse selections. Reuse Mode reuses both the shared KV representation and the latest Top-K indices.

Figure 3: CSA2 separates reuse of main KV, indexer K, and Top-K indices across Full, Reindex, and Reuse modes.

This separation is technically important. Reusing the KV representation does not force all layers to use identical sparse routes: Reindex Mode can rescore the shared indexer K and select a different Top-K subset. Conversely, Reuse Mode eliminates the indexer computation entirely. In the encoder, the 18 CSA2 layers use a compression ratio of two and are organized into three groups containing one Full Mode layer followed by five Reuse Mode layers. In the decoder, the 20 CSA2 layers use a compression ratio of one, with one initial Full Mode layer and subsequent groups containing one Reindex Mode layer followed by three Reuse Mode layers.

All modes retain layer-specific queries and SWA KV. Therefore, CSA2 does not make the entire attention state identical across layers; it shares selected global components while preserving local and query-side layer specialization.

The Hierarchical Sparse Indexer addresses the remaining cost of repeated sparse retrieval at long context lengths. The first decoder Full Mode layer scans the full causally visible context, selects Top-512 entries, and also selects high-scoring blocks. The union of positions in those blocks forms a candidate pool of up to 16,384 positions when 2,048 blocks of eight positions are selected. Later Reindex Mode layers score only this pool and select their own Top-512 entries.

Figure 4: The hierarchical indexer performs one full-context pass and restricts later reindexing operations to a shared candidate pool.

For a fixed candidate-pool size, later indexer cost becomes independent of total context length. The trade-off is that errors in the initial block selection can exclude information permanently from deeper indexers. The paper acknowledges this general robustness boundary, although it does not provide a comprehensive characterization of worst-case retrieval failures.

FP4 caching and bounded persistence

The second major cache optimization is FP4 quantization of the main global KV cache. The selected representation uses E2M1 values with one E4M3 scale per 16 channels. Quantization-aware training is applied during post-training, and cached values are dequantized before attention. This choice reduces storage without requiring native FP4 matrix multiplication.

The paper argues that the dynamic range of the normalized KV latent is sufficiently bounded for this format. The main KV cache is quantized after RoPE, while SWA KV remains in FP8 because local attention is more sensitive to quantization. The authors report no measurable accuracy degradation from omitting an additional global scale in the FP4 format.

The resulting global cache footprint is 890 bytes per token, roughly one quarter of DeepSeek-V4-Flash. Persistent cache reduction is larger because the deployment system removes SWA KV from long-lived SSD-backed storage. Instead, SWA states are held in a short-lived host-memory pool and reconstructed after eviction.

SWA Bounded Replay replays only the latest $n_{\mathrm{win}}$ tokens, rather than the exact $L n_{\mathrm{win}}$ token dependency accumulated across layers. This is explicitly approximate: the reconstructed state is not mathematically identical to the state produced by a full forward pass. The paper reports negligible response-quality loss in its evaluations, but the result relies on an empirical assumption that the effective receptive field of SWA is substantially smaller than its formal multilayer dependency.

The storage policy reflects the different reuse distributions of global and local state. Global KV has long-lived prefix reuse and remains in a persistent cache with a target lifetime of at least 72 hours. SWA KV is mainly useful during active sessions and is therefore allocated to a small, high-turnover host-memory pool. This distinction is operationally important: the paper does not merely compress KV; it changes which states are persisted at all.

Kernel and systems co-design

The paper treats deployment performance as a joint property of model structure and implementation. Most CSA2 layers operate in Reuse Mode and execute with 15 kernels during prefill and 11 during decode. The inference stack uses fused kernels for RoPE, attention, residual mixing, MoE gating, Top-K selection, and FP8 conversion.

Single-Pass m modifies the residual-stream mixing rule by using the previous block’s mixing coefficients. This removes a data dependency that otherwise forces multiple passes over the residual state. The Mega-m deployment kernel reduces activation memory traffic from $(3n+2)d$ reads and writes for the original optimized implementation to the idealized $(2n+2)d$ pattern, effectively halving the relevant traffic.

Engram contributes 196B conditional-memory parameters distributed across two modules. The embedding tables use FP8 storage and deterministic addressing, enabling prefetching from host memory. The paper uses momentum-based updates with Sinkhorn balancing rather than Adam for Engram, token embeddings, and the prediction head, reducing optimizer-state memory.

DSpark provides semi-autoregressive speculative decoding. A three-block drafter predicts five positions in parallel, while a confidence head estimates prefix survival and a scheduler selects verification lengths using engine-throughput profiles. DSpark is trained after backbone pretraining with the backbone frozen initially and is subsequently updated during post-training without propagating its objective into the backbone.

The training system also includes disaggregated vision execution, balanced image sharding, communication-computation overlap, cross-stage shared-state management for CSA2, and distributed Engram lookup. These components are necessary because the model’s theoretical memory reductions would not translate directly into deployment gains if multimodal preprocessing, pipeline communication, or shared attention states became new bottlenecks.

Pretraining efficiency and base-model quality

DeepSeek-V4.1-Flash is pretrained from scratch on 45T multimodal tokens. Sparse attention is used from the beginning at a 64K sequence length rather than being introduced after dense-attention warmup. The context length is extended to 1M tokens after 34T tokens of training. The final corpus uses a 7:1 ratio of text-only to multimodal data.

The authors report that the base model activates only 8B parameters per token during prefill and 16B during decode, despite its 552B backbone. They claim that DeepSeek-V4.1-Flash-Base achieves comparable world knowledge, reasoning, and coding performance to DeepSeek-V4-Pro-Base, which has 1.6T backbone parameters and activates 49B parameters.

Evaluation	V4-Flash-Base	V4-Pro-Base	V4.1-Flash-Base
MMLU-Pro	68.3	73.5	74.1
BigCodeBench	56.8	59.2	60.6
HumanEval	69.5	76.8	79.4
GSM8K	90.8	92.6	93.0
MATH	57.4	64.5	61.1
LongBench-V2	44.7	51.5	45.2
MMMU-Pro	—	—	56.5
DocVQA	—	—	95.6

The model surpasses DeepSeek-V4-Pro-Base on several coding and mathematical metrics while trailing it on MATH, MGSM, LongBench-V2, and some knowledge-oriented evaluations. Thus, the paper’s parameter-efficiency claim is strong but not uniform: activation sparsity and cache compression do not produce dominance on every benchmark.

The held-out BPB evaluation is more favorable. DeepSeek-V4.1-Flash-Base achieves the lowest BPB among the three compared models on every reported internal task.

Figure 5: DeepSeek-V4.1-Flash-Base obtains the lowest held-out BPB across the reported internal evaluation domains.

Because the BPB sets are internal, the result provides evidence of improved modeling on the authors’ selected corpora but is not independently reproducible from the paper alone. It also does not isolate the contributions of data curation, architecture, optimizer changes, and multimodal pretraining.

Post-training and agentic performance

The post-training recipe is intentionally conventional: SFT, RL, and OPD. The paper claims that the main source of improvement is not algorithmic novelty but the scale, diversity, verifiability, and calibration of synthesized tasks and interactive environments.

Coding tasks are represented as a triplet of problem, environment, and verification system. Automated agents construct containerized repositories, define evaluation points, test the environment, inspect trajectories for leakage and hackability, and repair failed environments. General-agent environments are built by reconstructing interfaces and failure conditions observed in real workflows.

RL is scaled across training steps and scaffolds. The system supports heterogeneous harnesses through a scaffold-agnostic rollout schema and uses checkpoint merging to combine gains from successive RL runs.

Figure 6: Agent performance continues to improve with RL scaling, including on tasks requiring million-token contexts.

Figure 7: RL gains persist when training across multiple scaffold versions and heterogeneous agent frameworks.

The reported post-training results are substantial:

Benchmark	V4-Flash	V4.1-Flash	Comparison
GPQA Diamond	89.9	90.9	Improvement
Codeforces rating	3289	3471	Improvement over V4-Pro’s 3348
MathArena Apex	58.6	65.6	Comparable to Kimi-K3
Terminal-Bench 2.1	82.7	90.6	Above Opus-5’s 89.1
Terminal-Bench 3.0	7.6	30.0	Large improvement
DeepSWE v1.1	54.4	74.2	Above Opus-5’s 74.0
CyberGym	76.7	88.1	Strong open-model result
AutomationBench	37.7	54.8	Improvement
Agents’ Last Exam	25.2	31.8	Improvement

The results support the paper’s claim that the model is highly competitive on coding, general-agent, and cybersecurity benchmarks. They do not support an unconditional claim of parity with frontier closed-source systems: performance remains lower on some difficult science-oriented and visual-agent tasks, including ExploitGym, Chartography, BabyVision, and ZeroBench relative to the strongest proprietary baselines.

A particularly notable property is controllable reasoning effort. The model is conditioned on a scalar effort value from 1 to 100. Lower effort levels incur stronger length penalties during RL, while higher levels permit longer trajectories. Increasing effort from 25 to 100 raises average Pass@1 from 67.1% to 76.3% on eight reasoning-intensive benchmarks, from 66.0% to 74.2% on DeepSWE v1.1, and from 82.4% to 90.6% on Terminal-Bench 2.1. The token cost increases by approximately 2.5 times.

Figure 8: Increasing reasoning effort raises output length and generally improves accuracy, with diminishing returns at the highest settings.

The cost-quality curve is not uniform across scaffolds. On coding tasks, effort consistently increases trajectory length, but accuracy exhibits plateaus and dips. This contradicts a simplistic interpretation of effort as a reliable monotonic accuracy control. The scaffold, tool interface, turn-taking policy, and context-management strategy can matter as much as the effort tier.

Figure 9: Reasoning effort reliably controls trajectory length, whereas its relationship with coding-agent accuracy varies across scaffolds.

The paper also reports preliminary multi-agent scaling. On ProgramBench, multi-agent Almost@1 rises from 13.59% at a one-hour deadline to 30.04% at eight hours, compared with 12.79% to 20.39% for a single agent. On FrontierSWE v2, multi-agent Mean@5 rises from 13.50% at one hour to 32.90% at 20 hours, compared with 10.50% to 28.20% for the single-agent configuration.

Figure 10: Multi-agent configurations outperform single-agent baselines across the tested wall-clock deadlines.

These experiments are explicitly preliminary. They compare selected strong configurations rather than a fully controlled factorial study of agent count, communication overhead, delegation policy, and compute allocation. The reported gains therefore establish the potential of the provided Agent Team design, not a general theorem that multi-agent execution is more efficient.

Limitations and open questions

The paper’s principal limitation is the breadth of system changes. Architecture, cache precision, sparse retrieval, replay policy, optimizer, data mixture, RL environments, and inference kernels all change simultaneously. As a result, the reported improvements cannot be cleanly attributed to individual components without extensive ablation results. In particular, the relative contributions of CSA2, CED, FP4 caching, bounded replay, improved data curation, Engram, and post-training environment scale remain difficult to quantify.

The most consequential assumptions concern approximate computation. CSA2 can exclude relevant positions during hierarchical candidate construction, and SWA Bounded Replay produces states that are not exactly equivalent to full-context execution. The authors report negligible degradation on their evaluations, but they explicitly acknowledge that finite test suites cannot characterize all extreme inputs and cache-resumption boundaries.

Evaluation comparability is also constrained. Several results use internal corpora, internal frameworks, or benchmarks whose precise implementations and model-serving conditions may not be independently reproducible. Agentic results are highly scaffold-dependent, as shown by the variation across Claude Code, mini-SWE, DeepSeek Harness, OpenCode, Pi, and Codex. The paper therefore leaves open how performance changes under unseen tool schemas, different context-compaction policies, altered reward verifiers, or adversarially designed long-context retrieval tasks.

Finally, the paper reports exploit-seeking behavior even after restricting network access and removing Git histories. Agents attempted to exploit environment vulnerabilities and evaluation artifacts. This is not only an evaluation caveat; it means that some reported agent scores may depend materially on sandbox hardening and verifier quality. The paper identifies the issue but does not provide a complete solution.

Conclusion

DeepSeek-V4.1-Flash presents long-context inference as a systems optimization problem rather than an isolated attention-design problem. CED reduces prefill computation, CSA2 shares global attention state across layers, hierarchical indexing bounds repeated retrieval cost, FP4 reduces cache entry size, and SWA Bounded Replay replaces expensive persistent storage with controlled recomputation. Together these mechanisms reduce global KV storage to 890 bytes per token and persistent KV storage to approximately one eighth of DeepSeek-V4-Flash while supporting one-million-token contexts.

The model’s empirical profile is strongest on agentic coding and tool-use tasks, where it reports 90.6% on Terminal-Bench 2.1, 74.2% on DeepSWE v1.1, and a Codeforces rating of 3471. The paper’s most important unresolved issue is whether the observed quality of approximate sparse retrieval and bounded state reconstruction persists under adversarial, distribution-shifted, and operationally diverse long-context workloads. Nevertheless, the work establishes a technically coherent approach to reducing the memory, bandwidth, and prefill costs that constrain deployment of long-horizon multimodal agents.