Optimized Heterogeneous Chiplet Composition for Hybrid LLM Workloads

Determine the optimized composition of heterogeneous compute, memory, and communication chiplets for serving hybrid Transformer–Mamba large language model workloads under architectural and runtime constraints.

Background

Hybrid Transformer–Mamba LLMs combine Attention and Mamba or state-space-model components with substantially different computational, memory, and communication behaviors. Efficient serving therefore requires selecting an appropriate mixture of specialized chiplets, including resources tailored to Transformer and Mamba operations as well as prefill and decode phases.

The paper identifies the composition of heterogeneous chiplets as unresolved despite prior design-space-exploration frameworks addressing portions of the chiplet architecture problem. Existing approaches generally assume static workloads and do not fully capture the dynamic, multi-tenant behavior of LLM serving, including varying prefill–decode lengths and asynchronous request arrivals. Resolving the composition problem is intended to support efficient system-level throughput and latency under these heterogeneous and dynamic conditions.