CIM: Compute-in-Memory Architectures

• CIM is an overloaded acronym that primarily denotes compute-in-memory, where arithmetic operations occur within memory arrays to overcome energy and latency issues of data movement.
• It differentiates compute-in-memory from compute-near-memory by leveraging digital, analog, and mixed-signal implementations across technologies like SRAM, DRAM, and PCM.
• Recent CIM advances integrate quantization, scheduling, and sparsity-aware strategies to boost transformer and LLM performance while optimizing energy efficiency.

CIM is an overloaded acronym. In the hardware-systems literature cited here, it most often denotes compute-in-memory, a class of architectures that performs arithmetic in memory arrays or their immediate periphery in order to reduce data movement across the processor–memory boundary. In other arXiv literatures represented in the same corpus, CIM also denotes coherent Ising machines and code index modulation. The compute-in-memory usage dominates current architectural work because modern DNNs, transformers, and related workloads are governed by matrix operations whose cost is often set more by movement of operands than by arithmetic itself (Khan et al., 2024, Ahn et al., 17 Apr 2026).

1. Scope and terminology

Compute-in-memory is distinct from compute-near-memory. In the terminology of the survey literature, CIM performs operations in the array or directly on array bitlines and peripherals, whereas compute-near-memory places logic close to memory banks but still fetches operands into nearby execution units (Khan et al., 2024). The same survey further distinguishes CIM-periphery and CIM-array, and separates digital CIM from analog or mixed-signal CIM. This taxonomy remains central because precision, energy, array organization, and peripheral overhead all depend on where the arithmetic is realized (Khan et al., 2024).

The motivation is the memory wall. The survey reports that transferring data between logic and commodity memories typically costs 10–100× more energy per bit than the logic operation itself, and Eva-CiM cites that moving 256 bits from main memory to a processor can cost about 200× the energy of a single floating-point operation (Khan et al., 2024, Gao et al., 2019). This is why recent work treats CIM not as an isolated circuit technique but as a systems problem spanning devices, arrays, compilers, schedulers, memory hierarchies, and workload mapping.

2. Foundations and mathematical formulation

The canonical CIM kernel is matrix–vector multiplication. In abstract form, the workload is

$y = W x.$

In analog crossbar realizations, weights are mapped to conductances and inputs to voltages, producing column currents such as

$I_j = \sum_i V_i G_{ij},$

after which local ADCs, accumulators, or digital post-processing recover the numerical result (Khan et al., 2024). Digital SRAM-based CIM instead decomposes multiplication into bit-wise logic and reduction, avoiding ADC/DAC overhead and retaining deterministic, bit-accurate behavior (Ahn et al., 17 Apr 2026, Qi et al., 2 May 2025).

This basic formulation now intersects directly with transformer workloads. Self-attention is expressed as

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,$$

and the cited transformer literature emphasizes that the large dynamic activation matrices $Q$, $K$, and $V$, together with softmax’s global dependency, amplify memory traffic and create pipeline bubbles in conventional accelerators (Ahn et al., 17 Apr 2026). That observation explains why contemporary CIM work is no longer confined to static MAC arrays: current designs increasingly integrate quantization, nonlinear approximation, buffering, and scheduling in order to keep attention-local dataflows inside or adjacent to memory macros (Ahn et al., 17 Apr 2026, Guo et al., 30 Apr 2026).

3. Implementation styles and operation sets

The implementation space is now broad. The survey literature reports prototyped and commercialized CIM systems spanning SRAM, DRAM, PCM, RRAM, MRAM, FeFET, and flash. Representative figures include IBM’s 64-core PCM CIM prototype with 63.1 TOPS peak at 9.76 TOPS/W, a TSMC 7 nm 8T SRAM macro at 351 TOPS/W, and an Intel 22 nm LP FinFET SRAM macro at 32.2 TOPS/W (Khan et al., 2024). These numbers are not directly comparable across workloads and precision modes, but they establish that CIM has progressed from device-level demonstrations to processor-scale integration.

Recent designs also broaden the operation set beyond fixed dot products. FlexSpIM, a fabricated 40-nm digital CIM-SRAM macro for spiking neural networks, supports arbitrary operand resolution and shape with bitwise granularity, demonstrates a 2× improvement in bit-normalized energy efficiency over prior fixed-precision digital CIM-SNNs, and reaches 95.8% classification accuracy on IBM DVS Gesture (Chauvaux et al., 2024). OSA-HCIM introduces on-the-fly saliency-aware hybrid SRAM CIM with a dynamic digital-to-analog boundary and reports a 1.95× enhancement in energy efficiency while maintaining minimal accuracy loss on CIFAR100 (Chen et al., 2023). Acore-CIM combines SRAM-based weight storage with linear-resistor mixed-signal compute and an RV32IMFC control plane; its RISC-V–controlled self-calibration improves compute SNR by 25 to 45% to 18–24 dB and raises ENOB from 2.3 to 3.3 bits (Numan et al., 18 Jun 2025). GEM3D-CIM pushes further by targeting general-purpose matrix operations directly in memory, reporting transpose in $N+1$ cycles and, for a 32×32 instance, 15.51 GOPS and 12.77 GOPS/W for transpose, 27.86 GOPS and 432.25 GOPS/W for addition, and 13.93 GOPS and 436.61 GOPS/W for element-wise multiplication (Chakraborty et al., 15 Apr 2026).

4. Transformer and LLM acceleration

Transformer acceleration is currently one of the most active CIM subfields. CIMple is a fully digital, standard-cell SRAM-based self-attention accelerator that keeps the entire attention pipeline in INT8, including a LUT-based split softmax adjacent to the CIM core. The design is implemented as a 32 kb CIM-based accelerator in ST 28 nm FD-SOI and reports 26.1 TOPS/W at 0.85 V and 2.31 TOPS/mm$^2$ at 1.2 V. Its split-and-pipeline softmax reduces activation-to-activation latency by 33% for an encoder-only mapping at 400 MHz, $d_k=64$, and 1024 tokens, while task-level deltas on TinyLlama INT8 remain within $\pm 0.6\%$ relative to PyTorch LogSoftmax (Ahn et al., 17 Apr 2026).

RCW-CIM addresses a different transformer bottleneck: weight-update latency when LLM weights exceed a single CIM macro. It introduces read-compute/write overlap and a WS-OCS dataflow, reducing external DRAM access by 51.6% and internal CIM weight updates by 87.6% during prefill for 1024 tokens. The same work reports a 21.59% decode compute-latency reduction from RCW, a 69.17% decode-latency reduction from nonlinear operator fusion, 3.28 TOPS, 42.3 TOPS/W, 4.2 ms prefill latency, and 26.87 decoded tokens per second for INT4-weight Llama2 on dual DDR5-6400 memory (Guo et al., 30 Apr 2026). UniCAIM attacks long-context KV-cache growth through a FeFET-based unified CAM/CIM architecture that combines static and dynamic pruning; it performs approximate dynamic top-$I_j = \sum_i V_i G_{ij},$0 selection in $I_j = \sum_i V_i G_{ij},$1 time and reports circuit-level AEDP reductions ranging from 8.2× to 831× over prior CIM-based LLM accelerators, while maintaining application-level accuracy comparable with dense attention (Xu et al., 10 Apr 2025). At a higher architectural level, replacing TPU systolic MXUs with digital SRAM-based CIM-MXUs yields up to 44.2% performance improvement for LLM inference and up to 27.3× reduction in MXU energy consumption relative to the TPUv4i baseline (Zhu et al., 1 Mar 2025).

5. Reliability, precision, sparsity, and thermal constraints

As CIM matures, reliability and precision have become first-order concerns. Unicorn-CIM studies high-precision SRAM-based floating-point CIM and finds that exponent errors are the critical failure mode: inference accuracy begins to fluctuate at BER around $I_j = \sum_i V_i G_{ij},$2 and collapses to near zero by BER around $I_j = \sum_i V_i G_{ij},$3, while mantissa flips are largely benign up to BER around $I_j = \sum_i V_i G_{ij},$4. Its algorithm–hardware co-design combines exponent alignment with a row-based One4N ECC, reducing redundancy to 512 bits on a 256×256 SRAM array, adding 8.98% minimal logic overhead on the exponent processing path, and only about 1.48% overall accelerator power overhead (Li et al., 2 Jun 2025). A complementary accuracy-oriented result appears in column-wise quantization for analog CIM: aligning weight and partial-sum quantization at the column level improves top-1 accuracy by 0.99% on ResNet-20/CIFAR-10, 2.69% on ResNet-20/CIFAR-100, and 1.01% on ResNet-18/ImageNet relative to the best-performing related works, while improving robustness to memory cell variations (Kim et al., 11 Feb 2025).

Workload structure matters equally. CIMinus provides a sparsity-aware framework for SRAM-based CIM and validates its estimates within 5.27% error against contemporary sparse CIM architectures; it reports that input sparsity alone yields 1.2× to 1.4× speedups and energy savings, while the effectiveness of coarse or fine-grained sparsity patterns depends strongly on array alignment, routing overhead, and macro organization (Qi et al., 20 Nov 2025). At the opposite end of the operating environment, Space-CIM studies thermally constrained orbital platforms and links allowable TOPS directly to radiator capacity. Under tight radiator budgets, the work reports that CIM can achieve up to about 40× higher effective TOPS than GPU+HBM systems, largely because GPU/HBM packages develop severe hotspots and require throttling, whereas CIM heat distribution remains substantially more uniform (Mugdho et al., 4 Jun 2026).

6. Toolchains and system-level co-design

The contemporary CIM literature treats evaluation and deployment as compiler and systems problems rather than as circuit characterization alone. Eva-CiM is an early system-level framework that combines GEM5, McPAT, and DESTINY with dependency- and locality-aware trace analysis, and reports whole-system energy improvements of 1.3–6.0× for SRAM-based CiM and 2.0–7.9× for FeFET-RAM (Gao et al., 2019). WWW extends this architectural perspective to the cache hierarchy and asks what type of CiM to use, when to use it, and where to place it; its analytical study reports up to 3.4× energy-efficiency improvement and up to 15.6× throughput improvement relative to an INT8 baseline (Sharma et al., 2023).

More recent tools make the design flow programmable and workload-aware. CIMFlow provides an ISA-based compilation and cycle-accurate simulation framework for digital CIM, including capacity-aware partitioning and MLIR-based operator mapping; it reports up to 2.8× speedup and 61.7% energy reduction versus baseline mapping strategies (Qi et al., 2 May 2025). ChatNeuroSim overlays a large-language-model agent on top of NeuroSim-based CIM exploration, automating request parsing, parameter validation, script generation, and optimization; in a Swin Transformer Tiny case study at 22 nm, its pruning-based optimizer reduces average runtime to 0.42×–0.79× of the no-pruning baseline (Lee et al., 4 Mar 2026). Taken together, these frameworks indicate that “CIM” now denotes an ecosystem of device models, compiler passes, array mappings, workload transformations, and system-level cost models, not only a memory-macro circuit style (Qi et al., 2 May 2025, Lee et al., 4 Mar 2026).

7. Other meanings of CIM

Outside the hardware-architecture meaning, CIM also denotes coherent Ising machines in optical optimization. In that literature, a time-multiplexed network of degenerate optical parametric oscillator pulses searches for low-energy states of the Ising Hamiltonian

$I_j = \sum_i V_i G_{ij},$5

The cited femtosecond-pumped platform uses a 1555 nm mode-locked fiber laser at 100 MHz, frequency doubles it to 777.5 nm, and demonstrates an average success rate of 55% on a 100-vertex Möbius ladder graph while maintaining stable operation over 8 hours; the same work reports applications to molecular docking and credit scoring (Wei et al., 8 Dec 2025).

In wireless communications, CIM also denotes code index modulation. There the information is carried jointly by a QAM symbol and the indices of active Walsh–Hadamard spreading codes, with the paper defining the transmitted bits per symbol period as

$I_j = \sum_i V_i G_{ij},$6

The cited SIMO Rayleigh-fading image-transmission system reports lower BER than QAM and QPSK at matched $I_j = \sum_i V_i G_{ij},$7, higher throughput at lower SNR, and energy-efficiency improvements of 50%, 67%, and 67% over SM, QAM, and QPSK for one configuration, and 65%, 80%, and 80% for another (Ozden et al., 13 Jul 2025). These meanings are technically unrelated to compute-in-memory, but the acronym overlap is now established across multiple research communities.