---
title: Low-Batch Interleaving Mode (LBIM)
url: https://www.emergentmind.com/topics/low-batch-interleaving-mode-lbim
type: topic
---

# Low-Batch Interleaving Mode (LBIM)

CD-PIM refers to digital Processing-In-Memory (PIM) architectures that implement either cross-division bank-level computation for bandwidth and utilization optimization or confidential computation through memory-resident security enclaves. Recent CD-PIM designs have emerged to address the challenges of memory-bound large language model (LLM) inference on edge devices and securely offloading data-intensive workloads within trusted memory banks. Architectures such as CD-PIM for low-batch LLM acceleration [2601.12298] and PIM-Enclave for confidential computation [2111.03307] implement distinct hardware and computational strategies to extend the PIM paradigm.

## 1. Architectural Foundations of CD-PIM

Bank-level CD-PIM leverages digital computation units placed within DRAM banks to accelerate general matrix-vector multiplication (GEMV) and general matrix-matrix multiplication (GEMM) operations central to transformer-based LLM inference. The architecture utilizes commodity LPDDR5 DRAM equipped with two pipelined computing units (CUs) per bank, partitioning each physical LPDDR5 bank into four pseudo-banks ("Pbanks") to maximize internal bandwidth and parallelism. High-level features include:

- Per-bank compute: Two identical CUs, each capable of 8-bit MACs at double the DRAM internal clock frequency ($f_{CU} = 2 \times f_{int}$).
- Bank segmentation: Each bank is divided into upper/lower and left/right segments via isolation transistors and split global bitlines, enabling independent activation of four Pbanks.
- Overlapped workload support: Specialized modes for simultaneous GEMM (CPU) and GEMV (PIM).

In contrast, confidential CD-PIM as realized by PIM-Enclave [2111.03307] organizes each memory bank (or vault) to include a lightweight RISC core, AES-GCM-capable DMA engine, and access-control logic tied to a root-of-trust key and ROM. The memory module enables remote attestation, secure session and data key provisioning, and hardware-based protection against unauthorized access and side-channel leakage.

## 2. Modes of Operation: HBCEM and LBIM

CD-PIM for LLM acceleration introduces two principal operating modes on LPDDR5 [2601.12298]:

### High-Bandwidth Compute-Efficient Mode (HBCEM)
- Each LPDDR5 bank is subdivided into four Pbanks, each with independent sense amplifiers.
- All Pbanks are activated in parallel via the PIM_MAC_FM instruction, multiplying the bandwidth by 4. For example, with $f_{int}=200$ MHz and $W=16$ B, one bank’s bandwidth increases from $3.2$ GB/s to $12.8$ GB/s.
- Across 16 banks, the aggregate internal bandwidth reaches $204.8$ GB/s.

### Low-Batch Interleaving Mode (LBIM)
- The four Pbanks per bank are time-multiplexed: two process GEMV operations for LLM decoding, while the remaining two handle regular DRAM requests for CPU-driven GEMM.
- Enables concurrent execution where GEMV (decode) is processed on PIM and GEMM (prefill) on the CPU, reducing total inference latency:
  - $T_{HBCEM} = T_{prefill} + T_{decode}$
  - $T_{LBIM} = \max(T_{prefill}, T_{decode\_PIM}/2)$
- LBIM leverages bandwidth/compute tradeoff for improved latency in compute-bound low-batch settings, empirically yielding up to $1.46\times$ speedup over HBCEM.

## 3. Compute Unit Microarchitecture and Data-Mapping

Each Pbank integrates two deeply pipelined CUs, designed for maximal multiply-accumulate throughput:
- Each CU contains a 64 B input buffer and a 128 B output partial-sum buffer.
- The CU operates on 32 B wide weight vectors, processing two input-by-weight outer products per cycle.
- At twice the DRAM internal clock ($400$ MHz for $f_{int}=200$ MHz), each CU delivers $128$ MACs per cycle; two CUs yield $256$ MACs per bank per cycle, achieving $\sim819.2$ GMAC/s on 16 banks.

Optimized data-mapping strategies maximize CU utilization for both transformer K-cache (outer-product flow) and V-cache (inner-product flow):
- Column-wise (K-cache): $2 \times 64$ tile per bank, broadcasted $Q$ vector, bankwise outer product accumulation.
- Row-wise (V-cache): $64 \times 2$ tile per bank, broadcasted attention weight vector, bankwise inner product calculation.

## 4. Confidential Processing-In-Memory: PIM-Enclave CD-PIM

PIM-Enclave [2111.03307] supports confidential CD-PIM by constructing a memory-resident secure enclave leveraging:
- AES-GCM-capable DMA engine for zero-copy encrypted data transfer.
- Access-control logic per bank to enforce "in-enclave" memory regions, inaccessible to unauthorized host reads/writes via efficient on-die range checks.
- EK (endorsement key) and ROM for attestation, key establishment, and root-of-trust.

Operational workflow includes:
- Remote attestation where host verifies enclave integrity and provisions session/data keys using asymmetric encryption.
- Protected regions are locked via base and mask registers.
- All DMA transfers are authenticated and encrypted via hardware, ensuring IND-CPA confidentiality and tamper detection (integrity).

Programming interface resembles GPU offload paradigms, with explicit enclaving, encrypted code/data transfer, execution, and result retrieval.

## 5. Evaluation Metrics and System Impact

Evaluation of CD-PIM for LLMs on edge devices via Ramulator 2.0 simulation and real device benchmarks [2601.12298] demonstrates:

| System      | Batch      | Mode                 | Speedup vs. GPU-only | Speedup vs. AttAcc | Area Overhead |
|-------------|------------|----------------------|----------------------|--------------------|---------------|
| Jetson Orin | 1 (single) | HBCEM                | 11.42×               | 4.25×              | ~0.8%         |
| iPhone 15   | 1 (single) | HBCEM                | Range 4.5×–18.6×     | -                  | -             |
| Jetson Orin | 4 (low)    | LBIM vs. HBCEM       | 1.01–1.46×           | -                  | -             |

Each CU incurs $\sim14,941\,\mu\text{m}^2$ area and $4.5$ mW; cumulative addition for two CUs per bank remains modest relative to die size and power ($\approx144$ mW on $32$ Gb die).

Confidential CD-PIM via PIM-Enclave [2111.03307] achieves:
- Linear speedup with >6 banks.
- Encrypted DMA overhead of only $3.7$–$22\%$.
- Robust mitigation of bus side-channel leakages, cold-boot, replay/tampering.

## 6. Security Properties and Threat Model

Confidential CD-PIM designs define adversaries capable of system software compromise, physical bus analysis, DMA, and cold-boot attacks. Security goals are:

- Confidentiality: IND-CPA assurance for enclave memory contents.
- Integrity: AES-GCM tag-based tampering/replay protection.
- Side-channel resilience: No runtime leakage of memory access patterns on external buses.

Hardware-based access controls ensure in-enclave memory exclusivity during execution; all host read attempts to protected regions return $\perp$. Remote attestation allows cryptographic verification of enclave state prior to execution.

## 7. Context and Adaptability

CD-PIM architectures, through either high-bandwidth multi-bank compute or secure bank-resident enclaving, offer scalable, efficient solutions for both edge intelligence workloads (LLM inference) and confidential data processing. CD-PIM/LLM implementations optimize for low-latency, low-batch scenarios, achieving significant throughput and latency improvements over GPU-only and previous PIM baselines. Confidential CD-PIM strategies eliminate practical bus-level side-channels while maintaining minimal performance overhead. Adoption in memory-centric acceleration and cloud security primitives suggests ongoing relevance as DRAM and PIM paradigms evolve [2601.12298][2111.03307].

Source: https://www.emergentmind.com/topics/low-batch-interleaving-mode-lbim