--- title: Heterogeneous Compute-in-Memory Accelerators url: https://www.emergentmind.com/topics/heterogeneous-compute-in-memory-cim-accelerators type: topic --- # Heterogeneous Compute-in-Memory Accelerators Heterogeneous Compute-in-Memory (CIM) accelerators constitute a class of architectures that exploit multiple memory and compute modalities—typically analog and digital, and often spanning diverse memory device types, numerical precisions, and functional blocks—to overcome the data-movement and scalability limits of traditional von Neumann systems in advanced ML workloads. A heterogeneous CIM system combines crossbar-based analog compute (e.g., ReRAM, PCM, FeFET, SRAM) with digital-side compute (ALUs, SIMD units, on-chip or chiplet-level functional units), sometimes coordinating different array technologies spatially or hierarchically. This approach enables fine-grained tradeoffs between energy efficiency, throughput, precision, endurance, and scalability, and presents unique challenges in system integration, compilation, and workload mapping. ## 1. Architectural Principles of Heterogeneous CIM Accelerators A heterogeneous CIM accelerator integrates multiple memory technologies or compute paradigms (analog, digital, mixed-signal) at varying architectural levels—array, core, chip, or system. Notable examples include architectures that mix analog crossbar arrays (e.g., eNVM, SRAM, FeFET) for in-situ multiply–accumulate with digital logic for control and non-linear/post-processing steps; those interleaving analog and digital CIM (DCIM) chips in a chiplet mesh; and hybrids offering both memory mode (as storage) and compute mode (as in-place MAC engine) with dynamic switching. Key generalizations: - **Device Diversity**: Systems employ SRAM, RRAM, PCM, FeFET, eDRAM, and MRAM—each optimized for specific tasks: e.g., static VMM (analog), dynamic computation (SRAM), or associative search (CAM), reflecting their different density, precision, and endurance characteristics [2401.14428][2511.15397][2401.12428][2308.15040]. - **Functional Heterogeneity**: Analog CIM blocks perform dense MVM for convolution/projection layers; digital units handle index-sensitive or control-heavy operations, non-linearities, normalization, and accumulate crossbar outputs with high precision. - **Hierarchy and Chiplets**: Chiplet-style platforms (e.g., Hemlet) tile arrays of ACIM and DCIM chiplets with global buffer and intermediate processing chiplets interconnected by network-on-package (NoP), for scalable deployment, especially in large workloads like vision transformers [2511.15397]. - **Dynamic Reconfiguration**: Dual-mode arrays (compute/memory) allow adaptive allocation and programmable dataflow, reallocating resources as workload demands shift [2502.17006]. ## 2. Device and Modality Integration Multiple CIM implementations exploit heterogeneity at the device and circuit level to harmonize compute and storage. - **Analog-Digital Integration**: Core compute steps—such as MVM—use crossbar-based analog in-memory operations, followed by digital accumulation, quantization, or logic. For example, RRAM and PCM-based designs utilize conductance-based dot product with digital ADC/readout, while SRAM/FeFET analog macros employ charge-domain or capacitive multiplication, digitized by ADC/flash ADC [2401.14428][2511.15397]. - **Dynamic Precision and Hybrid Arrays**: Hybrid CIM arrays (as in OSA-HCIM) blend ACIM and DCIM modes in a split-port SRAM cell, allowing application-driven, per-operation dynamic selection of digital or analog MAC, controlled through saliency-aware precision mapping [2308.15040]. - **Technology-aware Partitioning**: Frameworks like DNN+NeuroSim assign different layers or processing elements to different device types: SRAM for rapid weight updates/gradients in training, eNVM devices (e.g., FeFET, PCM, RRAM) for large, high-fan-in layers in inference [2003.06471]. Table 1 summarizes representative heterogeneous CIM designs: | Architecture | Memory Technology | Heterogeneity Mode | |---------------------------------|---------------------------------------------|-----------------------------------| | Hemlet [2511.15397] | RRAM (ACIM), SRAM (DCIM), SRAM (IDP) | Chiplet partitioned + analog/digital | | OSA-HCIM [2308.15040] | Hybrid 6T SRAM | Dynamic analog/digital boundary | | DNN+NeuroSim V2.0 [2003.06471] | FeFET, RRAM, PCM, SRAM | Layer-wise device assignment | | IBM 64-core PCM [2401.14428] | PCM, on-chip digital | Analog in-array compute + digital post-processing | ## 3. Compilation, Abstraction, and Programming Methodologies Emergence of heterogeneous CIM imposes nontrivial challenges for mapping, scheduling, and code generation. - **Hierarchical and Multi-level Compilation**: Networks are mapped using multi-level abstractions—chip, core, crossbar/array—combined with iterative, tiered compilation passes (e.g., CIM-MLC) that partition workloads, tile matrices, pipeline operations, and remap data granularity, accounting for each device’s regime and resource [2401.12428]. - **Device- and Mode-aware Abstractions**: Compilers such as CMSwitch extend standard architectures with dynamic hardware annotations (e.g., array dual-mode switching, mode-switch latency) and employ DP+MIP passes to jointly segment graphs and allocate modes across arrays for optimal throughput and resource utilization [2502.17006]. - **Meta-operator and Multi-target IR**: Advanced compilers (CINM, CIM-MLC) emit meta-operators at each architectural tier (e.g., core, crossbar, wordline), which a backend lowers to native commands, supporting portability across backends [2401.12428][2301.07486]. CINM, built on MLIR, uses parallel dialects (e.g., “cim”, “cnm”, “memristor-dialect”) and progressive lowering for device-agnostic and device-aware optimizations [2301.07486]. - **Hardware-software Co-design**: In HASTILY, ISA extensions (e.g., EXP_VEC, MAX_REDUCE_VECTOR) and compiler-aware scheduling eliminate memory bottlenecks in transformer softmax by co-optimizing dataflow, reduction, and in-memory element-wise operations [2502.12344]. ## 4. Performance Metrics, Energy Efficiency, and Trade-offs Heterogeneous CIM systems achieve significant gains in throughput and energy efficiency compared to both monolithic architectures and von Neumann baselines, but highlight nuanced trade-offs between area, latency, and architectural parameters. - **Throughput and Efficiency**: Chiplet-based designs (Hemlet) deliver up to 8.68 TOPS per system, with system-level speedups ranging from 1.44× to 4.07× versus monolithic baselines, and up to 3.86 TOPS/W energy efficiency. OSA-HCIM achieves 5.79 TOPS/W, a 1.95× gain over pure DCIM for ≤0.1% accuracy drop [2511.15397][2308.15040]. - **Latency and Mode-switching**: Dynamic mode allocation (CMSwitch) incurs minimal (3–5%) runtime overhead, but offers up to 2.03× speedup versus single-mode mapping [2502.17006]. - **Area/performance/energy:** SiTe CiM I in 8T-SRAM incurs 18% area overhead for nearly 7.7× latency improvement; SiTe CiM II reduces area cost (6%) at some expense in speed. HASTILY's UCLM modules add <3% area and <2.5% dynamic power [2408.13617][2502.12344]. - **Device Variation and Endurance:** Analog NVMs remain limited by retention, endurance, and fabrication variability, while digital SRAM/DCIM features larger area per bit and higher energy but superior endurance [2401.14428][2511.15397]. ## 5. Scalability, System Integration, and Workload Mapping Scaling heterogeneous CIM accelerators beyond single-die constraints leverages chiplets, network-on-package, and hierarchical resource allocation. - **Chiplet Modularization**: Hemlet's mesh of ACIM, DCIM, and IDP chiplets interconnected by NoP enables expansion in compute and memory capacity. However, NoP bandwidth and hop latency dominate when system is scaled, capping throughput scaling [2511.15397]. - **Group-Level Parallelism**: GLP (as in Hemlet) interleaves columns from different layers within an ACIM group, enabling all columns in a group to be digitized in parallel, optimizing ADC utilization and throughput scaling within analog CIM tiles [2511.15397]. - **Dynamic Mapping and Reconfiguration**: By exposing per-array, per-chip, and per-segment constraints, compilers (CMSwitch, CIM-MLC) automatically adapt mapping and resource allocation to changing model structure, arithmetic intensity, and array availability [2502.17006][2401.12428]. - **Heterogeneous Task Partitioning**: Certain frameworks (e.g., DNN+NeuroSim) partition networks layer-wise, mapping high-fan-in layers to eNVM arrays and computation-heavy, update-intense stages (e.g., weight gradients) to SRAM or PCM. Offloading is performed for GEMM-heavy kernels (TDO-CIM), with less compute-intensive tasks retained in host CPU [2003.06471][2007.00060]. ## 6. Challenges, Limitations, and Future Outlook Although heterogeneous CIM accelerators present substantial promise, several technical challenges and open questions persist. - **Device/Integration Barriers**: Yield, endurance, analog drift/noise, and device variation (both D2D and C2C) continue to limit broad deployment and reliability in large-scale systems [2401.14428][2003.06471]. - **Software and Programming Model Complexity**: Extracting the full benefit of heterogeneity mandates robust, hierarchical compilation stacks, device- and workload-aware scheduling, and generalized IRs, which increases programming complexity and hinders adoption [2301.07486][2401.12428]. - **On-chip Communications and Bottlenecks**: As in Hemlet, NoP link bandwidth and communication latency cap practical scaling, and tradeoffs must be negotiated between compute density and communication-intensive interchip or interarray dataflows [2511.15397]. - **Cross-layer/Function Adaptivity**: Runtime or co-design mechanisms for allocating modes, splitting workloads, or tuning analog/digital boundaries (e.g., saliency-aware in OSA-HCIM) are still emerging and typically lack OS/runtime coordination [2308.15040][2502.17006]. - **Precision and Generalizability**: Most efficiency gains are in low- to moderate-precision settings (1–8 bit), with extensions to high-precision or non-dense/sparse workloads requiring additional research and, in many cases, new analog primitives [2408.13617][2003.06471]. A plausible implication is that future generations of heterogeneous CIM accelerators will increasingly integrate hardware–compiler codesign, programmable dual-mode or multi-modal arrays, advanced communication fabrics (e.g., photonic NoP), and mixed-technology stacks within and across chiplets or system tiles. Systematic performance–energy–precision trade-off exploration, enabled by co-simulation and design frameworks (e.g., DNN+NeuroSim), remains critical for balanced designs across diverse ML benchmarks [2003.06471][2401.14428]. Source: https://www.emergentmind.com/topics/heterogeneous-compute-in-memory-cim-accelerators