Multi-context coordination for mzCache

Develop coordination policies and data structures that enable mzCache to manage multiple coexisting LLM applications, each maintaining its own model weights and KV cache.

Background

The current mzCache system supports only a single LLM context. In realistic mobile environments, multiple LLM applications may coexist, and each application may maintain separate model weights and KV caches. Supporting this setting would require coordination mechanisms and data structures to manage these multiple memory objects while preserving mzCache’s memory-eviction and restoration functionality. The paper identifies this extension as outside the scope of the current work and leaves it unresolved.

References

mzCache currently supports a single LLM context scenario. However, as LLM applications become more prevalent, multiple LLM applications may coexist, each maintaining its own weights and KV caches. mzCache can be extended to handle such scenarios by adding coordination policies and data structures for managing multiple weights and KV caches. Such extensions are beyond the scope of this work and we leave them for future work.

— mzCache: On-Device LLM Memory Management under Multitasking  (2609.01338 - Yu et al., 1 Sep 2026) in Section Discussion, subsection “Multi-context support”