Optimize AsyncLLM for Multi-head Latent Attention

Develop an optimized AsyncLLM implementation supporting Multi-head Latent Attention (MLA), including the model-specific attention kernel and cache representation required for cache-view operations.

Background

AsyncLLM represents concurrent computation through CacheBlocks and cache views, with attention kernels adapted to account for logical positions when blocks are rearranged. The paper explains that Multi-head Latent Attention (MLA) stores compressed latent representations and, in decoupled-RoPE implementations, a separate positional component. The latent state can be reused across cache views, while the positional component requires logical-position correction. However, the reported implementation does not provide the model-specific MLA kernel and cache representation needed to realize this compatibility efficiently.

References

We leave an optimized MLA implementation to future work.

— LLMs are General Asynchronous Agents  (2609.35427 - Yakushev et al., 28 Sep 2026) in Appendix, Section "Full Attention variations", paragraph "Grouped-query, multi-query, and latent attention"

We leave the implementation and empirical evaluation of this to future work.

— LLMs are General Asynchronous Agents  (2609.35427 - Yakushev et al., 28 Sep 2026) in Appendix, Section "Full Attention variations", paragraph "Learned sparse attention"

The same CacheBlock abstraction can in principle expose these memories, but we leave this implementation to future work.

— LLMs are General Asynchronous Agents  (2609.35427 - Yakushev et al., 28 Sep 2026) in Appendix, Section "Non-early-fusion multi-modal LLMs"