---
title: 'BitROM CiROM: Scalable AI Inference'
url: https://www.emergentmind.com/topics/bitrom-cirom-architecture
type: topic
---

# BitROM CiROM: Scalable AI Inference

BitROM CiROM (Compute-in-Read-Only-Memory) architectures refer to a category of specialized computing-in-memory (CiM) designs that exploit high-density, read-only memory arrays for matrix-vector multiplication (MVM), with the principal aim of reducing energy and area costs associated with the storage and movement of neural network weights. BitROM and its successors, including YOLoC and the advanced LLM-oriented BitROM, leverage both ROM device physics and circuit/algorithmic co-design to achieve scalable, energy-efficient, and high-capacity AI inference engines that address the core limitations of conventional SRAM-CiM and weight reload bottlenecks [2206.00379], [2304.02908], [2509.08542].

## 1. Foundational BitROM Cell and Crossbar Macro

At the circuit level, the prototypical BitROM cell is a single NMOS transistor whose gate is irreversibly tied—at fabrication—to either the wordline (WL) or ground, representing a binary “1” or “0” weight respectively. No write capability is available post-fabrication; thus, data (weights) are immutable, a property that distinguishes BitROM from SRAM- or eDRAM-based CiM. In operation, computation is performed in situ: an activation bit $a_i \in \{0, 1\}$ is applied to the WL, and multiplication ($w_i a_i$) reduces to a logical AND; enabled cells discharge the bitline (BL), and the total BL voltage drop accumulates the columnwise dot-product. This analog sum is digitized by a column ADC, yielding multi-bit partial sums for further processing [2206.00379].

Macros are arranged in crossbar architectures (e.g., $128 \times 256$ arrays), with groupwise ADC sharing (e.g., 5-bit ADC per 16 BLs). Weight mapping for multi-bit quantization is performed on a bit-plane basis; for $b$-bit weights, $b$ physical ROM planes are required. Peripheral circuits manage pre-charging, input serialization, and accumulation, while post-ADC logic reconstructs the full-precision MAC result.

## 2. Advanced ROM Array Designs and Ternary Quantization

The BitROM concept was further extended to address the demands of billion-parameter LLMs by storing ternary weights ($w \in \{-1, 0, +1\}$) with maximized density. The Bidirectional ROM Array (BiROMA) implements two ternary weights per transistor via bidirectional access: each nMOS source/drain terminal is prewired to one of three voltage rails (0, $0.25V_{DD}$, $0.5V_{DD}$). Depending on read direction, each transistor emits a separate ternary value, and two independent weights are extracted per cell. Comparator thresholds are set at $1/8 V_{DD}$ and $3/8 V_{DD}$, discriminating between ternary states with $1.58$ effective bits per weight consistent with BitNet quantization. This results in raw bit densities up to $4,967$ kb/mm$^2$ and area reductions of $10\times$ versus previous digital CiROMs [2509.08542].

## 3. Peripheral Circuits: Accumulation, Sensing, and Dataflow

ROM-CiM systems require dedicated periphery for analog-to-digital conversion, serialization, partial summation, and global accumulation:

- Sense amplifiers and ADCs convert BL voltage to digital partial sums, with one ADC typically shared across multiple columns for area optimization.
- Local shift-and-add units, as in Tri-Mode Local Accumulators (TriMLA), implement ternary accumulation: zero-skip gating ($w=0$), addition ($w=+1$), or subtraction ($w=-1$) on a per-channel basis. Hardware comparators separate weights and select arithmetic operations in real time.
- For high-throughput scalability, local accumulation is further aggregated in multiway adder trees before final stage pooling or nonlinear activation.

Latency and energy models are explicitly characterized as functions of precharge, wordline drive, comparator operation, and accumulation time. Example per-MAC figures: $E_\text{MAC} \approx 117\ \text{fJ}$, $t_\text{MAC} \approx 0.8\ \text{ns}$ (TriMLA, 65 nm) [2509.08542].

## 4. Architectural Extensions: ReBranch, On-Die Caching, and LoRA Adapters

The immutability of ROM-based storage is fundamentally offset by three key mechanisms:

- **Residual Branch (ReBranch)**: For transfer learning or adaptation, a lightweight SRAM-CiM "branch" operates in parallel with the ROM-CiM "trunk." The branch employs a compressed–residual–decompressed sequence using additional pointwise and small conv layers, requiring only $1/(D \cdot U)$ the parameters of a full convolution. Incoming activations are broadcast to both engines, outputs are merged via on-chip adders, and a controller synchronizes sequencing and post-processing [2206.00379].
- **Decode-Refresh eDRAM (KV-Cache Management)**: For LLM inference, BitROM integrates a small on-die eDRAM—organized as a cache for the key/value (KV) projections generated during decoding. Caching the first $M$ tokens on-die yields a $43.6\%$ reduction in external DRAM traffic, substantially lowering energy and DRAM bandwidth requirements [2509.08542].
- **LoRA Adapters**: Hardware LoRA integration is achieved by augmenting the TriMLA infrastructure with small multiplier–adder units for low-rank projection matrices $U$ and $V$. LoRA updates, quantized to 6 bits, are co-processed with core inference, resulting in only $0.7\%$ area/power overhead and negligible latency increase ($<2\%$), while enabling rapid domain transfer and state-of-the-art accuracy benchmarks.

## 5. Dual-Context ROM-Augmented SRAM Architectures

BitROM/CiROM architectures can also be realized as ROM-augmented SRAM (notably 8T-SRAM). By VT-programming the isolated read port transistors (LVT↔ROM "1", HVT↔ROM "0"), a single bank can act as pure ROM, pure RAM, or support true dual-context (simultaneous RAM/ROM read). Sense margin, delay, and energy are analytically modeled; area penalties are minimal, with 1.3× density gain over separate 8T-SRAM + ROM, and only $5$–$7\%$ extra metal routing. Monte-Carlo simulations demonstrate $>99.99\%$ yield and robust sense separation. Typical read delays are $1.25$–$1.95\times$ baseline 8T SRAM delays, with $3$–$8\%$ dynamic energy overhead [2304.02908].

Potential use cases include on-chip LUTs for mathematical functions, in-memory neural inference (weights in ROM, activations in RAM), and context-switching in security/IoT accelerators without reconfiguration overhead.

## 6. Area, Energy, and System-Level Trade-Offs

The primary system-level advantage of BitROM/CiROM lies in memory density and weight reload elimination:

- **Cell/array density**: BitROM achieves $\sim16\times$ higher density than 6T-SRAM-CiM ($0.014\ \mu\text{m}^2$/bit vs. $0.22\ \mu\text{m}^2$/bit at 28 nm); BiROMA at 65 nm achieves $4,967$ kb/mm$^2$, a $10\times$ improvement over prior architectures [2206.00379], [2509.08542].
- **Throughput/energy**: YOLoC demonstrates $11.5$ TOPS/W ($119$ GOPS/mm$^2$); BitROM achieves $20.8$ TOPS/W with system energy savings enhanced by local cache and no runtime weight reloading.
- **Application-level performance**: On object detection (YOLO/ResNet-18), YOLoC attains $4.8$–$14.8\times$ energy savings versus SRAM-CiM with $<8\%$ latency overhead and $<0.5\%$ mean average precision (mAP) difference [2206.00379].
- **LLM scalability**: BitROM enables practical LLM inference in advanced nodes; e.g., LLaMA-7B fits in $\sim54$ cm$^2$ (vs. $>1,000$ cm$^2$ without density enhancement).

Process variation and ADC sharing present ongoing design challenges, with mitigation schemes including calibration, ECC, and controlled ADC column multiplexing.

## 7. Applications and Future Directions

BitROM/CiROM architectures support a broad range of workloads:

- Large-scale CNNs and LLMs where weight reload elimination and density are critical.
- Embedded and edge AI where area and power budgets preclude DRAM-intensive architectures.
- Multi-context compute engines (e.g., shared RAM/ROM compute for on-chip functions, security, and IoT).

Emerging work in BiROMA and TriMLA demonstrates further integration of algorithmic–circuit co-design (e.g., BitNet quantization, LoRA hardware), suggesting that BitROM/CiROM frameworks will remain key to scaling efficient, flexible inference processors for contemporary and future machine learning workloads [2206.00379], [2304.02908], [2509.08542].

Source: https://www.emergentmind.com/topics/bitrom-cirom-architecture