--- title: 'CIM: Compute-in-Memory Architectures' url: https://www.emergentmind.com/topics/cim type: topic --- # CIM: Compute-in-Memory Architectures CIM is an overloaded acronym. In the hardware-systems literature cited here, it most often denotes **compute-in-memory**, a class of architectures that performs arithmetic in memory arrays or their immediate periphery in order to reduce data movement across the processor–memory boundary. In other arXiv literatures represented in the same corpus, CIM also denotes **coherent Ising machines** and **code index modulation**. The compute-in-memory usage dominates current architectural work because modern DNNs, transformers, and related workloads are governed by matrix operations whose cost is often set more by movement of operands than by arithmetic itself [2401.14428][2604.15944]. ## 1. Scope and terminology Compute-in-memory is distinct from compute-near-memory. In the terminology of the survey literature, CIM performs operations in the array or directly on array bitlines and peripherals, whereas compute-near-memory places logic close to memory banks but still fetches operands into nearby execution units [2401.14428]. The same survey further distinguishes **CIM-periphery** and **CIM-array**, and separates **digital CIM** from **analog or mixed-signal CIM**. This taxonomy remains central because precision, energy, array organization, and peripheral overhead all depend on where the arithmetic is realized [2401.14428]. The motivation is the memory wall. The survey reports that transferring data between logic and commodity memories typically costs **10–100× more energy per bit** than the logic operation itself, and Eva-CiM cites that moving **256 bits** from main memory to a processor can cost about **200×** the energy of a single floating-point operation [2401.14428][1901.09348]. This is why recent work treats CIM not as an isolated circuit technique but as a systems problem spanning devices, arrays, compilers, schedulers, memory hierarchies, and workload mapping. ## 2. Foundations and mathematical formulation The canonical CIM kernel is matrix–vector multiplication. In abstract form, the workload is $$y = W x.$$ In analog crossbar realizations, weights are mapped to conductances and inputs to voltages, producing column currents such as $$I_j = \sum_i V_i G_{ij},$$ after which local ADCs, accumulators, or digital post-processing recover the numerical result [2401.14428]. Digital SRAM-based CIM instead decomposes multiplication into bit-wise logic and reduction, avoiding ADC/DAC overhead and retaining deterministic, bit-accurate behavior [2604.15944][2505.01107]. This basic formulation now intersects directly with transformer workloads. Self-attention is expressed as $$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,$$ and the cited transformer literature emphasizes that the large dynamic activation matrices $Q$, $K$, and $V$, together with softmax’s global dependency, amplify memory traffic and create pipeline bubbles in conventional accelerators [2604.15944]. That observation explains why contemporary CIM work is no longer confined to static MAC arrays: current designs increasingly integrate quantization, nonlinear approximation, buffering, and scheduling in order to keep attention-local dataflows inside or adjacent to memory macros [2604.15944][2604.27384]. ## 3. Implementation styles and operation sets The implementation space is now broad. The survey literature reports prototyped and commercialized CIM systems spanning SRAM, DRAM, PCM, RRAM, MRAM, FeFET, and flash. Representative figures include IBM’s **64-core PCM CIM** prototype with **63.1 TOPS peak at 9.76 TOPS/W**, a **TSMC 7 nm 8T SRAM** macro at **351 TOPS/W**, and an **Intel 22 nm LP FinFET** SRAM macro at **32.2 TOPS/W** [2401.14428]. These numbers are not directly comparable across workloads and precision modes, but they establish that CIM has progressed from device-level demonstrations to processor-scale integration. Recent designs also broaden the operation set beyond fixed dot products. FlexSpIM, a fabricated **40-nm** digital CIM-SRAM macro for spiking neural networks, supports arbitrary operand resolution and shape with bitwise granularity, demonstrates a **2× improvement in bit-normalized energy efficiency** over prior fixed-precision digital CIM-SNNs, and reaches **95.8%** classification accuracy on IBM DVS Gesture [2410.23082]. OSA-HCIM introduces on-the-fly saliency-aware hybrid SRAM CIM with a dynamic digital-to-analog boundary and reports a **1.95×** enhancement in energy efficiency while maintaining minimal accuracy loss on CIFAR100 [2308.15040]. Acore-CIM combines SRAM-based weight storage with linear-resistor mixed-signal compute and an RV32IMFC control plane; its RISC-V–controlled self-calibration improves compute SNR by **25 to 45%** to **18–24 dB** and raises ENOB from **2.3 to 3.3 bits** [2506.15440]. GEM3D-CIM pushes further by targeting general-purpose matrix operations directly in memory, reporting transpose in **$N+1$ cycles** and, for a **32×32** instance, **15.51 GOPS** and **12.77 GOPS/W** for transpose, **27.86 GOPS** and **432.25 GOPS/W** for addition, and **13.93 GOPS** and **436.61 GOPS/W** for element-wise multiplication [2604.13969]. ## 4. Transformer and LLM acceleration Transformer acceleration is currently one of the most active CIM subfields. CIMple is a fully digital, standard-cell SRAM-based self-attention accelerator that keeps the entire attention pipeline in INT8, including a LUT-based split softmax adjacent to the CIM core. The design is implemented as a **32 kb** CIM-based accelerator in **ST 28 nm FD-SOI** and reports **26.1 TOPS/W at 0.85 V** and **2.31 TOPS/mm$^2$ at 1.2 V**. Its split-and-pipeline softmax reduces activation-to-activation latency by **33%** for an encoder-only mapping at **400 MHz**, $d_k=64$, and **1024 tokens**, while task-level deltas on TinyLlama INT8 remain within **$\pm 0.6\%$** relative to PyTorch LogSoftmax [2604.15944]. RCW-CIM addresses a different transformer bottleneck: weight-update latency when LLM weights exceed a single CIM macro. It introduces read-compute/write overlap and a WS-OCS dataflow, reducing external DRAM access by **51.6%** and internal CIM weight updates by **87.6%** during prefill for **1024 tokens**. The same work reports a **21.59%** decode compute-latency reduction from RCW, a **69.17%** decode-latency reduction from nonlinear operator fusion, **3.28 TOPS**, **42.3 TOPS/W**, **4.2 ms** prefill latency, and **26.87** decoded tokens per second for INT4-weight Llama2 on dual DDR5-6400 memory [2604.27384]. UniCAIM attacks long-context KV-cache growth through a FeFET-based unified CAM/CIM architecture that combines static and dynamic pruning; it performs approximate dynamic top-$k$ selection in **$O(1)$** time and reports circuit-level AEDP reductions ranging from **8.2×** to **831×** over prior CIM-based LLM accelerators, while maintaining application-level accuracy comparable with dense attention [2504.07479]. At a higher architectural level, replacing TPU systolic MXUs with digital SRAM-based CIM-MXUs yields up to **44.2%** performance improvement for LLM inference and up to **27.3×** reduction in MXU energy consumption relative to the TPUv4i baseline [2503.00461]. ## 5. Reliability, precision, sparsity, and thermal constraints As CIM matures, reliability and precision have become first-order concerns. Unicorn-CIM studies high-precision SRAM-based floating-point CIM and finds that exponent errors are the critical failure mode: inference accuracy begins to fluctuate at BER around **$1\times 10^{-8}$** and collapses to near zero by BER around **$1\times 10^{-6}$**, while mantissa flips are largely benign up to BER around **$1\times 10^{-3}$**. Its algorithm–hardware co-design combines exponent alignment with a row-based One4N ECC, reducing redundancy to **512 bits** on a **256×256** SRAM array, adding **8.98%** minimal logic overhead on the exponent processing path, and only about **1.48%** overall accelerator power overhead [2506.02311]. A complementary accuracy-oriented result appears in column-wise quantization for analog CIM: aligning weight and partial-sum quantization at the column level improves top-1 accuracy by **0.99%** on ResNet-20/CIFAR-10, **2.69%** on ResNet-20/CIFAR-100, and **1.01%** on ResNet-18/ImageNet relative to the best-performing related works, while improving robustness to memory cell variations [2502.07842]. Workload structure matters equally. CIMinus provides a sparsity-aware framework for SRAM-based CIM and validates its estimates within **5.27%** error against contemporary sparse CIM architectures; it reports that input sparsity alone yields **1.2× to 1.4×** speedups and energy savings, while the effectiveness of coarse or fine-grained sparsity patterns depends strongly on array alignment, routing overhead, and macro organization [2511.16368]. At the opposite end of the operating environment, Space-CIM studies thermally constrained orbital platforms and links allowable TOPS directly to radiator capacity. Under tight radiator budgets, the work reports that CIM can achieve up to about **40×** higher effective TOPS than GPU+HBM systems, largely because GPU/HBM packages develop severe hotspots and require throttling, whereas CIM heat distribution remains substantially more uniform [2606.05741]. ## 6. Toolchains and system-level co-design The contemporary CIM literature treats evaluation and deployment as compiler and systems problems rather than as circuit characterization alone. Eva-CiM is an early system-level framework that combines GEM5, McPAT, and DESTINY with dependency- and locality-aware trace analysis, and reports whole-system energy improvements of **1.3–6.0×** for SRAM-based CiM and **2.0–7.9×** for FeFET-RAM [1901.09348]. WWW extends this architectural perspective to the cache hierarchy and asks what type of CiM to use, when to use it, and where to place it; its analytical study reports up to **3.4×** energy-efficiency improvement and up to **15.6×** throughput improvement relative to an INT8 baseline [2312.15896]. More recent tools make the design flow programmable and workload-aware. CIMFlow provides an ISA-based compilation and cycle-accurate simulation framework for digital CIM, including capacity-aware partitioning and MLIR-based operator mapping; it reports up to **2.8×** speedup and **61.7%** energy reduction versus baseline mapping strategies [2505.01107]. ChatNeuroSim overlays a large-language-model agent on top of NeuroSim-based CIM exploration, automating request parsing, parameter validation, script generation, and optimization; in a Swin Transformer Tiny case study at **22 nm**, its pruning-based optimizer reduces average runtime to **0.42×–0.79×** of the no-pruning baseline [2603.08745]. Taken together, these frameworks indicate that “CIM” now denotes an ecosystem of device models, compiler passes, array mappings, workload transformations, and system-level cost models, not only a memory-macro circuit style [2505.01107][2603.08745]. ## 7. Other meanings of CIM Outside the hardware-architecture meaning, CIM also denotes **coherent Ising machines** in optical optimization. In that literature, a time-multiplexed network of degenerate optical parametric oscillator pulses searches for low-energy states of the Ising Hamiltonian $$H(s) = -\sum_{i