--- title: PIM Execution Primitives (PEPs) url: https://www.emergentmind.com/topics/pim-execution-primitives-peps type: topic --- # PIM Execution Primitives (PEPs) PIM Execution Primitives (PEPs) are the fundamental hardware- and microcode-level actions exposing computational capabilities within modern digital processing-in-memory (PIM) architectures. They form the lowest-level compute and data movement units directly executable inside memory arrays such as DRAM or memristive crossbars, providing the enabling substrate for hardware acceleration of data-intensive workloads across scientific computing, machine learning, tensor algebra, and graph analytics. Recent architectural and compiler frameworks—such as inclusive-PIM designs for DRAM-based systems, abstractPIM’s technology-independent ISA for crossbar logic, AME-PIM’s matrix micro-kernels, and PyPIM’s programmatic Python integration—systematically formalize the PEP notion, codifying them as a compositional ISA layer with well-defined performance, parallelism, and technological mapping [2309.07984][2208.14472][2604.27808][2206.04218][2308.14007]. ## 1. Formal Definition and Taxonomy of PEPs PEPs are defined as the smallest indivisible computation/memory actions supported by the PIM hardware, exposed either as instruction-mnemonics (e.g., ADD, NOR₂, FILL), microprogrammed micro-ops, or fixed-function micro-kernels. PEP sets vary depending on PIM substrate (HBM, DRAM, memristive crossbars) and system design. **Taxonomy of PEPs:** | PEP Category | Representative PEPs | Typical Platforms | |------------------------------------------|-------------------------|----------------------------------| | Bitwise digital logic | NOT, NOR₂, AND₂ | Crossbar PIM: abstractPIM, PyPIM | | Element-wise arithmetic | ADD-PEP, MUL-PEP | HBM-PIM: AME-PIM | | Data movement/activation | FILL, MOV, MASK | HBM-PIM, PyPIM | | Aggregate/outer-product kernel | MAC-PEP, GEMM microkern.| AME-PIM, Inclusive-PIM | | High-level tensor operations (R-type ISA)| ADD, SUB, CMP, MOVE | PyPIM | - In abstractPIM, the canonical ISA of PEPs includes all 1–4 input Boolean functions, MUX, half-adders, and their symmetric extensions, with a microcode mapping for each PIM technology [2208.14472]. - PyPIM exposes both micro-ops (MASK, READ, WRITE, LOGIC_H, LOGIC_V, MOVE) and a macro R-type ISA for standard arithmetic, comparison, and logical operations [2308.14007]. - DRAM/HBM-PIM architectures (AME-PIM) implement PEPs as fixed-function micro-kernels matched to matrix, vector, and tile arithmetic, e.g., MAC-PEP for outer-product accumulation and SIMD ADD/MUL microkernels [2604.27808]. - AritPIM details bit-serial and bit-parallel arithmetic PEPs, including full carry-lookahead and carry-save primitives for addition, multiplication, and division in both fixed- and floating-point [2206.04218]. - Inclusive-PIM analyzes five "acceleratable" PEPs: vector-sum, wavesim-volume, wavesim-flux, ss-GEMM, and push-based graph kernels, grounding the definition in mathematical, memory, and hardware constraints [2309.07984]. ## 2. Instruction Set Design and Encoding The implementation of PEPs at the ISA and microarchitectural layer is critical for portability, backward compatibility, and hardware abstraction. - In abstractPIM, each PEP is associated with a fixed-width instruction; for example, a 16-bit encoding with distinct fields for OPCODE, destination row, and source rows. Complex multi-output primitives (e.g., half-adder) extend fields as needed [2208.14472]. - PyPIM’s 64-bit micro-ops encode the operation type, data width, partition or crossbar selectors, gate types, and operand selectors. The R-type ISA overlays the micro-ops as 32-bit macro-instructions, akin to RISC integer ALUs [2308.14007]. - DRAM/HBM-PIM’s PEPs are microprogrammed into per-channel command registers, triggered via standard DRAM commands (column, row) that synchronize wide SIMD datapaths across banks [2604.27808]. This hierarchical encoding enables decoupling high-level compute graph scheduling from hardware-specific (e.g., MAGIC, IMPLY) instruction synthesis, permitting code generation, reuse, and microcode library replacement as technologies evolve [2208.14472]. ## 3. Algorithmic Realization and Technological Mapping Each PEP must be realized efficiently for the native in-memory logic or DRAM compute unit. - Memristive crossbar PEPs (abstractPIM, PyPIM) map to sequences of (possibly parallel) stateful logic cycles: single-row NOR, AND, NOT using row/column biasing and partition isolation. Primitives such as addition, multiplication, and division employ canonical arithmetic algorithms (bit-serial ripple-carry, parallel-prefix, carry-save, Karatsuba, Brent–Kung, non-restoring division) decomposed into sequences of base logic PEPs [2208.14472][2206.04218]. - HBM-PIM (AME-PIM) designs map AME RISC-V tile instructions to combinations of SIMD PEPs such as ADD-PEP, MUL-PEP, SUB-PEP, and MAC-PEP. MAC-PEP implements an outer-product update with fully in-memory accumulation to obviate host-PIM data shuttling and maximize bank-locality [2604.27808]. - Data-movement PEPs (FILL, MOV in AME-PIM; MASK, MOVE in PyPIM) are designed to leverage DRAM/getter bus widths or H-tree on-chip interconnects, supporting register refill, accumulator flush, and cross-array communication in a bank/partition-parallel manner [2308.14007][2604.27808]. The crossbar mapping for logic PEPs emphasizes parallel gate execution per partition; DRAM-PIM focuses on leveraging bank-parallelism and minimizing row activation penalties via bank-group pipelining [2309.07984][2604.27808]. ## 4. Performance Modeling and Amenability PEPs must align with the memory and compute bandwidth regimes of the PIM platform to realize speedup versus traditional compute. - Inclusive-PIM introduces a PIM-amenability test based on four quantitative criteria: memory-bandwidth limit (operational intensity $R_{\mathrm{alg}}$ against PIM BW), memory residency/on-chip reuse, operand locality (bank alignment), and aligned data parallelism (SIMD group/grid compliance) [2309.07984]. - For crossbar PIM, aggregate performance is analyzed as a function of micro-op latency and macro-op sequencing, with bit-parallel primitives collapsing arithmetic latency from $O(N)$–$O(N^2)$ (serial) to $O(\log N)$ (parallel), yielding up to $10^{12}$ ops/sec and >$10^9$ ops/W efficiency [2206.04218][2308.14007]. - DRAM/HBM-PIM can deliver up to 14.9 GFLOP/s (59.4 FLOP/cycle) per pseudo-channel for matrix multiplication using MAC-PEPs in outer-product mode, populating ridgeline points on roofline models limited by row activation, register bandwidth, and command issue rates [2604.27808][2309.07984]. PEP set choice and algorithmic mapping are thus directly informed by memory and computation bottlenecks; co-design of data placement, PEP scheduling, and microkernel design is required to achieve effective BW utilization [2309.07984][2604.27808]. ## 5. Co-Design Optimizations, Trade-Offs, and ISA Portability Co-design of hardware PEP sets and software/hardware interfaces is a central theme in state-of-the-art PIM platforms. - Inclusive-PIM demonstrates that extending PEP execution by overlapping row activations, skipping PEPs for dynamic zeros (sparse matrix multiply), or partially offloading updates to on-chip caches (for graph algorithms) increases realized speedup from $<1.2\times$ to $2$–$5\times$ relative to high-end GPU baselines [2309.07984]. - abstractPIM emphasizes backward compatibility: by separating a target-independent PEP ISA stream from family-specific microcode, the same PEP program can execute on different memristive logic families (MAGIC, IMPLY, etc.)—with device-specific microcode libraries installed in the PIM controller. This approach reduces code size, supports technology upgrades, and encapsulates device/sequence-level scheduling detail [2208.14472]. - PyPIM demonstrates programmatic mapping from high-level tensor operations into R-type ISA instructions, each expanded into optimized micro-op sequences (LOGIC_H/V, MOVE, etc.). The abstraction enables full utilization of crossbar parallelism, minimizes control bottlenecks, and facilitates flexible data movement across horizontal, vertical, and inter-array axes [2308.14007]. - In AME-PIM, mapping AME instructions to PEP microkernels and eliminating host-side reduction is key for achieving near-peak sustained throughput and minimal off-chip transfers, though inner-loop limitations (e.g., counter range, accumulator width) still bound some use cases [2604.27808]. PEP granularity, ISA richness, and technology-specific trade-offs—latency, area, energy, code length, flexibility—are thus central architectural decisions, with richer ISA sets reducing code size but increasing per-PEP control complexity [2208.14472][2206.04218]. ## 6. Application Domains and Representative Use Cases PEP-defined architectures and their associated ISAs enable a broad spectrum of data-centric applications. - Scientific computing: PEPs form the substrate for high-order PDE solvers (wavesim-volume/flux), sparse linear algebra (ss-GEMM), and block-structured matrix multiply/accumulate microkernels (MAC-PEP, GEMM) [2309.07984][2604.27808]. - Machine learning: Elementwise primitives (ADD, MUL), outer-product accumulations, and tensor-format data movement are fundamental to DNN forward and backward computation; PEPs permit tiling, pipelined accumulation, and all-in-memory execution [2309.07984][2604.27808][2308.14007]. - Graph workloads: PEPs implement push-based update kernels, exploiting locality and selective host/offload decisions depending on cache-hit rates [2309.07984]. - Arithmetic logic: Crossbar platforms realize full IEEE754 floating-point add, multiply, divide, modulo, sign/compare—and their vectorized variants—via cascades of basic logic PEPs using serial or parallel-pipeline techniques [2206.04218][2308.14007]. - Tensor libraries: PyPIM translates high-level Python/NumPy operations (.add, .sum, .matmul) directly into PEP-based instruction streams executable in massively parallel fabric [2308.14007]. Performance results demonstrate up to two orders-of-magnitude improvement in throughput and energy efficiency relative to high-end GPUs for core arithmetic, as well as 2–4× near-peak end-to-end acceleration for full matrix/tensor workloads [2206.04218][2309.07984][2604.27808][2308.14007]. ## 7. Limitations and Outlook Although PEP-based architectures demonstrate significant performance and energy scaling, several constraints are inherent to current realizations. - Some DRAM/HBM-PIMs lack native cross-bank reduction or non-arithmetic instructions, requiring emulation via data movement and broader PEP sequences; e.g., SUB-PEP is implemented by MUL–1 + ADD, and reductions often need nontrivial data relayout [2604.27808]. - micro-op/PEP loop depth, row/column activation overhead, register file width, and fixed tile geometry can restrict attainable parallelism or resource utilization for irregular workloads [2309.07984][2604.27808]. - Floating-point compliance (full IEEE754, NaN, subnormals), rare control-flow, and support for large-scale dynamic data structures require additional control FSM complexity or microcode extensions [2206.04218][2308.14007]. - Device technology variations (memristor drift, DRAM retention, process variation) can impact latency, error rate, or per-PEP sense margin; design must trade off gate count, periphery overhead, and microcontroller complexity for flexibility and sustained throughput [2208.14472][2206.04218]. Nevertheless, PIM Execution Primitives provide a formalized, extensible foundation for the architectural and algorithmic co-design of modern in-memory accelerators, decoupling instruction-level parallelism from technology-specific implementation and enabling both portability and performance across disparate compute-in-memory fabrics. The systematic formulation of PEPs—via ISAs, micro-op templates, and kernel fusion—will likely continue to shape future generations of application-specific memory-centric computing [2208.14472][2308.14007][2604.27808][2309.07984][2206.04218]. Source: https://www.emergentmind.com/topics/pim-execution-primitives-peps