---
title: Chiplet-based Memory Module Overview
url: https://www.emergentmind.com/topics/chiplet-based-memory-module
type: topic
---

# Chiplet-based Memory Module Overview

A chiplet-based memory module is a memory subsystem constructed by partitioning memory and related logic into multiple optimized chiplets, typically connected via an advanced interposer or high-speed in-package interconnect. This architectural paradigm leverages advances in heterogeneous integration, network-on-package topologies, and protocol standardization (UCIe, CXL, CHI), and is designed to overcome scalability, area, yield, bandwidth, and energy constraints inherent to monolithic memory hierarchies. Chiplet-based memory modules implement memory technologies such as DRAM, SRAM, RRAM (ReRAM), or even processing-in-memory (PIM) logic as disaggregated, modular chiplets, which enables tailoring process nodes, integrating compute/logic functions, and providing scalable bandwidth for AI/ML and HPC workloads [2511.15397, 2511.12286, 2510.06513, 2108.08903, 2406.15068, 2501.07330, 2307.02666].

## 1. Architectural Organization

Chiplet-based memory modules employ several integration approaches:

- **2.5D and MCoOI Integration:** Multiple chiplets (memory, logic, compute) are mounted side-by-side on a silicon interposer, which facilitates high-density, low-latency die-to-die communication. For example, Hemlet integrates analog CIM (ACIM, RRAM) and digital CIM (DCIM, SRAM) chiplets, plus a digital IDP (SRAM-based processing) chiplet via a 2D mesh NoP [2511.15397]. Sangam physically separates DRAM "bank" chiplets and logic chiplets on a passive interposer, providing direct logic-bank point-to-point links [2511.12286].

- **On-Package Memory via UCIe:** Multiple memory chiplets (DRAM, SRAM) or logic-die mediators are serially interfaced with compute chiplets using UCIe (Universal Chiplet Interconnect express), enabling asymmetric, scalable, and protocol-flexible on-package memory modules [2510.06513].

- **Multi-Channel HBM/DRAM Assemblies:** Accelerator systems (e.g., Occamy, SIAM) deploy multiple HBM or DRAM stacks as distinct chiplets, each with dedicated controllers on the compute chiplets, with traffic distributed via wide PHYs and hierarchical routers/NoCs transmitted through a silicon interposer [2406.15068, 2501.07330, 2108.08903].

- **On-Chip SRAM Memory Fabrics:** Modular AI supercomputer designs (Chiplet Cloud) utilize distributed SRAM-dominated CC-MEM chiplets for in-package storage, directly interfaced to compute clusters [2307.02666].

### Typical Memory Module Chiplet Subtypes

| Chiplet Type    | Example Memory Tech   | Function/Role                      |
|-----------------|----------------------|-------------------------------------|
| ACIM            | RRAM                 | Static parameter VMM/MLP weights    |
| DCIM            | SRAM                 | Dynamic VMM (attention, dynamic wts)|
| DRAM banklet    | DRAM                 | Capacity-optimized storage array    |
| Logic chiplet   | 7 nm logic/SRAM/SIMD | PIM compute, controller, reduction  |
| IDP/Buffer      | SRAM                 | Global scratchpad, SIMD processing  |

These modules are further interconnected by package-level 2D mesh or bus networks, high-speed SerDes, or standards-based PHYs (CXL, UCIe) [2511.15397, 2511.12286, 2510.06513].

## 2. Memory Hierarchy and Subarray Structure

Chiplet-based modules expose explicit, multi-tiered memory organization:

- **Subarray Granularity:** Each memory chiplet (e.g., ACIM or DRAM banklet) is partitioned into arrays or subarrays (e.g., 256×256 RRAM or multiple DRAM banks), with shared local peripheral logic (e.g., MUX, ADC for ACIM) [2511.15397, 2108.08903].
- **Tile and PE Grouping:** Subarrays aggregate into tiles, then further into processing elements (PEs). For example, Hemlet’s ACIM features a Tile (16 PEs, each with 6 subarrays), collectively forming a three-level buffer hierarchy [2511.15397].
- **Bank/Channel Partitioning:** HBM and DRAM chips expose extensive parallelism via multi-bank, multi-channel partitioning. For instance, Occamy utilizes 8 HBM2E channels per stack, with each channel subdivided into banks and pseudo-channels, offering >1 TB/s bandwidth per module [2406.15068, 2501.07330].

## 3. Inter-Chiplet Interconnects and Protocols

High-bandwidth, low-latency, and scalable chiplet interconnects are essential:

- **Network-on-Package (NoP):** For Hemlet, a 2D mesh NoP connects chiplets via 8 SerDes links/node, with up to 256 GB/s per chiplet [2511.15397]. SIAM also employs a mesh of passive interposer links (32 lanes at 250 MHz) [2108.08903].
- **Serial Protocols:** UCIe links in modern designs employ flit-based serialization and asymmetrical data lane allocation, supporting JEDEC-compliant DRAM protocols (LPDDR6/HBM4), CXL.Mem, or CHI with configurable bandwidth, latency, and protocol adaptation [2510.06513, 2511.12286].
- **Bandwidth and Latency:** CXL-attached modules (Sangam) operate at x8 PCIe 6.0/CXL Gen 4 (102.4 GB/s per module) [2511.12286]. UCIe can deliver per-lane 32 GT/s, achieving >256 GB/s per group with sub-10 ns end-to-end latency [2510.06513]. Hybrid chiplets exploit aggregate channelization, e.g., Occamy’s dual HBM2E modules deliver up to 3.3 TB/s [2406.15068].

## 4. Compute-in-Memory, PIM, and Buffering

Chiplet-based memory modules increasingly incorporate CIM/PIM for near-data compute:

- **Analog and Digital CIM:** Heterogeneous integration enables high-density RRAM-based analog MACs for static layers (ACIM) and flexible, SRAM-based digital MACs for dynamic layers (DCIM), as in Hemlet’s architecture [2511.15397]. Analog CIM supports up to 7 bit precision, with energy per MAC ~0.5 pJ; DCIM is area-flexible but incurs higher read/write energy.
- **Processing-in-Memory (PIM) Logic:** Sangam separates dense DRAM banks (in DRAM chiplets) from PIM-capable logic (in center-stripe chiplet), supporting local systolic arrays, SIMD units, bank-local SRAM scratchpads, and reduction trees [2511.12286].
- **Intermediate Data Processing:** The IDP chiplet, as in Hemlet, organizes SRAM banks for cross-chiplet SIMD, LayerNorm, GELU, and residual additions [2511.15397].
- **SRAM-Based Fast Memory Modules:** Chiplet Cloud’s CC-MEM implements high-bandwidth, low-latency SRAM arrays tightly co-located with compression/decompression hardware, matching SIMD cores’ access patterns [2307.02666].

## 5. Performance, Energy, and Scaling Models

Closed-form models and empirically validated results characterize fundamental performance and scaling traits:

- **Throughput, Latency, and Energy:**
  - Throughput $\Phi$ in chiplet-based CIM: $\Phi = \frac{N_{MAC}}{T_{comp} + T_{comm}}$ [2511.15397].
  - Memory access latency: e.g., $L_{read}^{AC} = L_{WL\,dr} + L_{ADC} + L_{adder},\;L_{mem} = B(L_{read} + L_{write})/\mathrm{pipeline\,depth}$ [2511.15397].
  - Energy per op (ACIM/DCIM): $E_{op} = E_{cell}^{read} + E_{ADC} + E_{buffer} + E_{comm}$ [2511.15397]. Detailed cost models separate DRAM, logic-die, package/test [2510.06513].
  - HBM2E peak bandwidth: $B_{\rm chiplet} = 2 \times 819.2 = 1638.4\,\mathrm{GB/s}$ [2406.15068].
  - CC-MEM per-chiplet bandwidth: 1.5–4.2 TB/s, SRAM latency $\sim$1 ns [2307.02666].

- **Bandwidth Density and Latency Improvement:** UCIe-based modules reach linear bandwidth densities of 658 GB/s/mm, with 7.5 ns end-to-end latency (versus 18–20 ns for HBM/LPPDR) and 0.25 pJ/bit energy (versus 0.9–2.8 pJ/bit for DRAM/HBM) [2510.06513].

- **Empirical Outcomes:**
  - Hemlet: 3×–8× subarray speedup, 1.5×–2× overall chiplet speedup via GLP; 8.68 TOPS throughput; 3.86 TOPS/W; NoP bandwidth utilization up to 70% (vs. 40% baseline) [2511.15397].
  - Sangam: 2.8–4.2× speedup over H100 GPU in LLM inference latency, $>$10× decode throughput, and 12× energy savings [2511.12286]. 
  - SIAM: up to 130× (versus V100) energy efficiency improvement on DNNs [2108.08903].
  - Occamy: up to 3.3 TB/s sustained bandwidth, area/power cost reductions, FPU utilization >80% for stencil/dense and >40% for sparse codes [2406.15068, 2501.07330].
  - Chiplet Cloud: 97× TCO/token improvement over GPU cloud; 1.7× larger sparse models at same memory capacity [2307.02666].

## 6. Design Trade-Offs, Generalization, and Future Directions

Chiplet-based memory modules admit several key trade-offs and extensibility paths:

- **Analog vs. Digital CIM:** ACIM enables >5–10× area density at the cost of write endurance and high reprogramming energy; DCIM offers flexibility but has higher area/energy per operation [2511.15397]. Sangam and SIAM demonstrate that decoupling dense logic from DRAM array avoids severe density penalties and enables advanced PEs [2511.12286, 2108.08903].
- **Cost/Area/Yield Optimization:** Spreading memory and compute across multiple smaller chiplets enables higher manufacturing yield ($\eta = e^{-D_0 A}$), with up to 60% cost reduction vs. monolithic dies for large AI models [2108.08903, 2307.02666].
- **Protocol and Integration Standardization:** Adoption of UCIe, CXL.Mem, and CHI enables vendor-agnostic, scalable interfaces across logic, DRAM, and NVM chiplets [2510.06513].
- **Applicability Beyond DNNs:** Group-level parallelism and cross-chiplet mapping schemes generalize to CNNs, GNNs, graph and diffusion models [2511.15397]. 
- **3D/Heterogeneous Stacking:** Emerging directions include 3D stacking for further area and wire-length compression, hybrid photonic/electrical links, and integration of nonvolatile memory (ReRAM, MRAM) with native chiplet interfaces [2510.06513, 2511.12286].
- **Programmable In-Package Memory Networks:** Dynamic compression, decode, and all-reduce protocols (as in CC-MEM), and flexible mapping via analytic models, support robust scaling to exascale AI deployments [2307.02666].

## 7. Representative Comparative Metrics

| Module/Design        | Peak BW (GB/s) | Latency (ns) | Energy/bit (pJ) | Capacity/Chiplet | Area Overhead | Notable Features      |
|----------------------|----------------|--------------|-----------------|------------------|---------------|----------------------|
| Hemlet (ACIM/DCIM)   | 256 (NoP)      | 10 hops: 10/H+0.015·S | 0.5–1.5/MAC     | 24 Mb RRAM      | –             | 2.5D, RRAM/SRAM      |
| Sangam (CXL-PIM)     | 102.4 (CXL x8) | –            | DRAM read power dominates (>80%) | 16 DDR5 chips (128–512 GB total) | – | 17 chiplets/module   |
| UCIe-DRAM (CXL.Mem)  | 256+           | 7.5          | 0.25            | Flexible         | –             | UCIe+logic-die/native|
| Occamy (HBM2E)       | 1638           | 140          | ~16 (PHY+ctrl)  | 16 GiB/stack     | 19% die       | 2.5D dual-chiplet    |
| CC-MEM (SRAM)        | 1500–4200      | ~1           | 0.03–0.1 (SRAM/xbar) | 80–230 MB      | ~75%+         | On-chip crossbar+CDR |

Data reflect design-point specifics from [2511.15397, 2511.12286, 2510.06513, 2108.08903, 2406.15068, 2501.07330, 2307.02666].

---

Chiplet-based memory modules therefore represent a convergence of advanced packaging, scalable memory-subsystem design, and in-/near-memory computing, enabling AI and HPC systems to transcend classical bottlenecks in capacity, bandwidth density, power efficiency, and cost. They provide a flexible substrate not only for DNN acceleration, but also for general-purpose memory-bound workloads. Future work will extend these modules via 3D stacking, NVM integration, photonic links, and open protocol ecosystems [2510.06513, 2511.12286].

Source: https://www.emergentmind.com/topics/chiplet-based-memory-module