---
title: Heterogeneous-Hybrid PIM (HH-PIM)
url: https://www.emergentmind.com/topics/heterogeneous-hybrid-pim-hh-pim
type: topic
---

# Heterogeneous-Hybrid PIM (HH-PIM)

A heterogeneous-hybrid processing-in-memory (HH-PIM) architecture integrates multiple, distinct PIM device types or compute modalities within a unified accelerator platform, orchestrating the complementary strengths of each type for a target workload’s diverse memory, compute, latency, or power requirements. In an HH-PIM system, various memory and compute elements—such as analog/memristive PIM arrays, digital systolic arrays, SRAM-PIM, DRAM-PIM, or MRAM/FeFET PIM modules—operate as specialized compute islands or tiers under coordinated scheduling. This approach overcomes the fixed granularity, resource rigidity, or device-specific bottlenecks of homogeneous or simple hybrid PIM by dynamically partitioning and mapping sub-tasks to the optimal device type. Recent HH-PIM architectures have demonstrated order-of-magnitude improvements in throughput, energy efficiency, and resilience to workload skew or device non-idealities across applications ranging from large language models (LLMs) to GNN training, AI inference, and transactional-analytical database integration.

## 1. Foundational Design Principles of HH-PIM

HH-PIM design mandates modularity at the device, array, and interconnect levels. Representative implementations combine:

- Multiple compute substrates (e.g., analog RRAM crossbars, digital accumulators, embedded SRAM/FeFET, MRAM) on planar or 3D-integrated stacks [2508.16011, 2508.10691, 2504.01468].
- Discrete digital accelerator blocks (e.g., systolic arrays, TPUs/NPUs) coupled to PIM blocks for high-precision or nonlinear tasks [2504.01994, 2511.06838, 2509.12993, 2403.00579].
- Clusters or chiplets grouped by PIM type, linked via an on-chip or chiplet/interposer-level network [2508.10691, 2504.01468].
- Centralized or distributed schedulers, responsibility for graph partitioning, data movement, and kernel-to-device mapping [2504.01994, 2509.12993, 2211.10516].

A key tenet is that each memory technology is tasked according to its performance envelope; for example, analog RRAM PIM performs energy-dense low-precision projections, while high-endurance SRAM or FeFET PIM handles frequent writes or nonlinear reductions. This separation is fundamental when faced with workload dynamism, non-uniform memory access patterns, and hardware constraints typical of real AI and data workloads.

## 2. Architectural and Device-Level Heterogeneity

Device heterogeneity spans NVM (ReRAM, MRAM, FeFET), SRAM, analog/digital modalities, and voltage domains. Notable architectural examples include:

- **PIM-LLM**: Combines analog RRAM crossbar PIM (256×256, 1b×8b, for projection layers) with an 8b digital systolic array (32×32, for attention heads), orchestrated by a CPU layer scheduler. The on-chip network routes activations/weights, and LPDDR serves as the backing storage medium [2504.01994].
- **HePGA**: 3D-stacked “tiles” with tiered device types: bottom two tiers employ high-density ReRAM (static data), mid-tier FeFET (fast, moderate write endurance), top-tier SRAM (write-dominated flows, e.g. gradient updates) [2508.16011].
- **THERMOS**: Chiplet-level heterogeneity by mixing ReRAM, SRAM (digital/analog), FeFET-based PIM, in various crossbar sizes and memory bit-widths, grouped into clusters per technology [2508.10691].
- **HH-PIM for edge AI**: Implements “big.LITTLE” cluster structure of high-performance MRAM+SRAM and low-power MRAM+SRAM PIM modules, both managed by dynamic allocation and power gating [2504.01468].

This configuration enables dynamic mapping of compute or storage sub-tasks to device types with the best matching latency, endurance, or energy profile.

## 3. Scheduling, Data Mapping, and Dynamic Resource Allocation

Scheduling in HH-PIM is central to achieving Pareto-optimal deployment. Key strategies observed:

- **Centralized dispatch**: CPU or top-level controller statically and/or dynamically partitions computation, delegating high-precision kernels to digital arrays or SRAM-PIM, low-precision or bandwidth-bound operations to analog/NVM PIM.
    - e.g., PIM-LLM’s master scheduler partitions Transformer layers into projections (PIM) and attention heads (TPU), controlling weight and activation movement [2504.01994].
    - HPIM leverages a hardware-aware compiler to label and distribute operators (GEMM/GEMV) to SRAM-PIM or HBM-PIM, with intra-token pipelining [2509.12993].
- **Dynamic, workload-adaptive resource allocation**: THERMOS achieves multi-objective optimization (latency, energy, thermal) by training a multi-objective RL policy to select the device cluster per neural layer, factoring device temperature, available memory, and inter-chiplet distance [2508.10691].
- **Data placement optimization**: HH-PIM for edge AI applies DP-based (knapsack) allocation to minimize energy for a given latency budget by splitting neural weights across HP-SRAM/MRAM and LP-SRAM/MRAM, with runtime migration and power-gating [2504.01468].
- **Hardware/software co-design for concurrency**: NeuPIMs unlocks lock-free concurrency by introducing dual row-buffers in each DRAM bank, enabling NPU and PIM to access separate banks simultaneously and batch interleaving to maximize utilization [2403.00579].

Such scheduling and mapping approaches are essential to exploit the full heterogeneity and avoid underutilization or thermal violations across complex hardware planes.

## 4. Application Domains and Workload-Specific Mapping

HH-PIM has delivered empirical and modeled performance advances in distinct workload domains:

- **Transformer LLM Inference**: 
    - **PIM-LLM** shows up to 80× throughput and 70% higher tokens/J versus traditional accelerators; analog PIM is assigned low-precision projection matmuls, systolic digital arrays handle attention heads. Peak analog MatMul throughput is modeled as $N_{xbar} \times N_{row} \times f_{CLK}$; digital OS systolic array achieves 102.4 GOP/s at $f_{TPU}=100$ MHz [2504.01994].
    - **HPIM** partitions LLM kernels such that latency-critical attention is on SRAM-PIM (ultra-low latency, full ISA), while HBM-PIM handles bandwidth-bound GEMV, with tightly coupled inter-domain pipelining. Speedups up to 22.8× against NVIDIA A100; parse-phase (GEMM) on TCU, decode-phase (GEMV) on PIM units [2509.12993].
    - **IANUS** integrates NPU and DRAM-PIM as unified memory, offloading memory-bound small mat-vecs to PIM, compute-bound mat-mats to NPU. Achieves 6.2× speedup and 3.7–4.4× energy efficiency versus NPU-memory baselines [2410.15008].
    - **P³-LLM** and NeuPIMs extend this paradigm with mixed-precision quantization and quantized PIM compute-units or concurrent NPU/PIM execution, further boosting decode throughput and energy efficiency [2511.06838, 2403.00579].
- **GNN Training (HePGA)**: Maps large, static, sparse graph adjacency to ReRAM, fast moderate-size combine kernels to FeFET, and write-intensive gradient flows to SRAM, optimizing for energy, area, and temperature-induced device variation. Up to 3.8× energy and 6.8× area efficiency improvements [2508.16011].
- **Edge AI**: HH-PIM for edge devices dynamically distributes weight/data file across performance/power-optimized clusters in response to real-time latency constraints, providing average 60.4% energy savings vs. prior PIMs [2504.01468].
- **HTAP Databases (PUSHtap)**: Fuses conventional CPU and PIM access via a unified row/column-aligned block-circulant format and in-place MVCC; achieves 3.4×/4.4× OLAP/OLTP throughput increase over multi-instance PIM [2508.02309].
- **Skew-resistant Ordered Indexing**: PIM-tree partitions B+-/skip-list hybrids between shared-memory CPU (L3/top) and distributed PIM (L1/L2/leaves), dynamically “pushes” or “pulls” requests based on contention, guaranteeing $O(\log_B \log_B P)$ communication per query even under power-law skew [2211.10516].

A common motif is per-operator or per-layer assignment to the “best-fit” PIM device, either via static analysis or runtime adaptation.

## 5. Quantitative Results, Device Characterization, and Performance Models

Reported empirical and modeled performance advances are achieved via device specialization and fine-grained orchestration. Systems provide:

- Substantial throughput and energy improvements:
    - PIM-LLM: ≥ 2× and ≥ 5× improvement in GOPS and GOPS/W vs. previous PIM LLM accelerators [2504.01994].
    - HPIM: Average 6.2× speedup, up to 22.8× versus A100 GPU (OPT family, up to 30B parameters) [2509.12993].
    - IANUS: 6.2× faster than A100 GPU for single-token GPT-2, reducing system energy by factors of 3.7–13× [2410.15008].
    - HH-PIM Edge: 60.4% average energy savings, up to 86.2% under light loads [2504.01468].
    - HePGA: 3.8× energy, 6.8× area efficiency vs. homogeneous PIMs [2508.16011].
    - PUSHtap: 3.4× OLTP/4.4× OLAP throughput vs. multi-instance PIM HTAP [2508.02309].
    - NeuPIMs: Up to ×3 throughput, ×2–3× utilization over NPU-only/NPU+PIM [2403.00579].
- Device parameters (selected, from [2508.16011, 2508.10691]):
    - ReRAM: 32 nm, 2 bits/cell, read ≈ 30 ns/50 pJ, non-ideality: conductance window shrinks with temperature.
    - FeFET: 28 nm, 1 bit/cell, read ≈ 5 ns/30 pJ, non-ideality: Vt window reduces linearly with temperature.
    - SRAM: 14 nm, 6T cell, ≈ 1 ns/10 pJ read, very high endurance.
    - Analog PIM (RRAM): 0.8 pJ/MAC (DAC ≈ 0.1 pJ, ADC ≈ 0.5 pJ, conduction ≈ 0.2 pJ).
- Scheduling and power models are tailored to each architecture, e.g., THERMOS uses a multi-objective RL framework, while HH-PIM edge applies DP-based allocation and LUT power-gating [2508.10691, 2504.01468].

The HH-PIMs are systematically evaluated on standard benchmarks (Transformer LLMs, GNNs, CNNs, HTAP workloads), often employing cycle-accurate simulation or FPGA prototyping.

## 6. Implementation Challenges, Limitations, and Generalization

Several practical challenges are identified:

- **Resource constraints**: PIM program size, host-PIM bandwidth, modest cache sizes in today's hardware [2211.10516].
- **Thermal and non-ideality management**: Crossbar device parameters and variability require aware mapping and thermal modeling [2508.16011, 2508.10691].
- **Complexity and scheduling**: Coordinated allocation policies, RL-based schedulers, dynamic power/latency trade-off mechanisms introduce additional design and runtime complexity.
- **Extension to new domains**: Push-pull scheduling and shadow replication in PIM-tree, or block-circulant data mapping in PUSHtap, can be generalized to B-trees, radix indexes, graphs, and new PIM device types [2211.10516, 2508.02309].

A plausible implication is that as both memory-embedded PIM and specialized compute islands (e.g., digital NPUs, analog crossbars) scale, HH-PIM architectures and their associated scheduling software will become the de facto baseline for AI and data-intensive workload acceleration on heterogeneous platforms.

## 7. Comparative Summary of HH-PIM Architectures

| Architecture      | Main Device Types         | Application Domain         | Performance Summary/Advantage                        |
|-------------------|--------------------------|---------------------------|-----------------------------------------------------|
| PIM-LLM [2504.01994] | Analog RRAM PIM + digital systolic | 1-bit LLM inference        | Up to 80× tokens/s, 2–5× GOPS(W) vs. prior         |
| HPIM [2509.12993]     | SRAM-PIM + HBM-PIM           | LLM single-batch inference | Up to 22.8× speedup vs. A100, 6.2× avg.            |
| HePGA [2508.16011]    | 3D ReRAM, FeFET, SRAM        | GNN training/inference     | ≤3.8× energy, 6.8× area efficiency                  |
| THERMOS [2508.10691]  | ReRAM, SRAM/FeFET chiplets   | Multi-model AI inference   | Up to 89% faster, 57% less energy vs. baselines     |
| IANUS [2410.15008]    | NPU + DRAM-PIM               | LLM end-to-end inference   | 6.2× speedup, 3.7–4.4× energy efficiency            |
| HH-PIM edge [2504.01468]| HP/LP MRAM+SRAM             | Edge AI adaptation         | Avg. 60.4% energy savings vs. conventional PIM      |
| PIM-tree [2211.10516] | CPU + 2048 DRAM-PIM nodes     | Ordered index/DB           | 59.1–69.7× speedup vs. prior skip lists             |
| PUSHtap [2508.02309]  | CPU + DRAM-PIM, unified data | OLTP/OLAP (HTAP)           | 3.4× OLTP/4.4× OLAP throughput frontier shift       |

These results collectively demonstrate that the HH-PIM paradigm achieves a new Pareto frontier in bandwidth, energy, throughput, and adaptivity for AI and data workloads by leveraging heterogeneous device specializations, dynamic mapping, and workload-aware orchestration.

Source: https://www.emergentmind.com/topics/heterogeneous-hybrid-pim-hh-pim