BiROMA: Bidirectional ROM Array for Edge LLMs

• Bidirectional ROM Array (BiROMA) is a compute-in-ROM design that exploits NMOS symmetry to store two independently retrievable ternary weights per transistor for efficient LLM inference.
• It employs a bidirectional read protocol with dual-threshold comparators and specialized peripheral circuitry to achieve high storage density and reliable ternary encoding.
• Quantitative metrics indicate up to 10× density improvement and ultra-low energy per operation, though this requires tight analog tolerances and precise voltage control.

Searching arXiv for the cited papers and closely related accelerator/ROM work to ground the article. arxiv_search(query="(Zhang et al., 10 Sep 2025) OR BitROM Bidirectional ROM Array ternary weights", max_results=5) arxiv_search(query="(Wang et al., 17 Mar 2025) OR ROMA Read-Only-Memory-based Accelerator QLoRA on-device LLM", max_results=5) arxiv_search(query="id:(Zhang et al., 10 Sep 2025)", max_results=10) Bidirectional ROM Array, abbreviated BiROMA, is a compute-in-read-only-memory array organization introduced in BitROM for edge LLM inference, in which a single NMOS transistor stores two independently retrievable ternary weights by exploiting the symmetry of its source/drain terminals and directional readout (Zhang et al., 10 Sep 2025). In the BitROM formulation, one read direction yields one ternary value and the reverse direction yields a second ternary value, so that one FET encodes two weights in $\{-1,0,+1\}$. The structure is presented as part of a co-design with BitNet’s 1.58-bit quantization model, a Tri-Mode Local Accumulator, and an integrated Decode-Refresh eDRAM for KV-cache management, with the stated objective of making CiROM practical for billion-parameter LLM inference at the edge (Zhang et al., 10 Sep 2025).

1. Architectural role within compute-in-ROM LLM inference

BiROMA appears in BitROM as the storage substrate for ternary model weights. The motivating problem is that conventional CiROM accelerators, while energy-efficient for CNNs, do not scale naturally to LLMs because of parameter volume. The BitROM description states that LLaMA-7B, identified there as the smallest model in the LLaMA series, would require more than 1,000 cm2 of silicon area even in advanced CMOS nodes if approached naively, and it positions BiROMA as one of three architectural innovations intended to address that constraint (Zhang et al., 10 Sep 2025).

The decisive property of BiROMA is not mutability but packing density. The array remains ROM: it is described as update-free storage, and the gain comes from storing two ternary weights per transistor rather than from runtime weight modification. This suggests that the term “bidirectional” refers to directional read semantics over a symmetric device rather than to a conventional bidirectional bus or dual-port memory abstraction.

At the system level, BitROM couples BiROMA with a Tri-Mode Local Accumulator optimized for ternary-weight computations and with Decode-Refresh eDRAM that supports on-die KV-cache management. The same BitROM description also states that LoRA-based adapters are integrated to enable efficient transfer learning across downstream tasks. In that setting, BiROMA functions as the immutable base-weight store, while the rest of the architecture manages accumulation, decoding-state retention, and task adaptation (Zhang et al., 10 Sep 2025).

2. Cell structure and ternary-weight encoding

Each BiROMA cell consists of a single NMOS transistor whose two source/drain terminals are each connected, through 1-to-3 multiplexers, to three metal-rail voltages. One terminal is designated the source line (SL) and the other the bitline (BL) during a given read. The rails correspond to three analog levels: $\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$ (Zhang et al., 10 Sep 2025).

The ternary-weight mapping is defined explicitly as

$$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$

This encoding assigns a distinct SL voltage to each ternary state. The state $w=0$ is labeled “skip,” and the later sensing logic uses that state to gate a local enable. Because the FET is symmetric, the roles of SL and BL can be swapped and the same physical device can be reread in the opposite direction to recover the second stored ternary value (Zhang et al., 10 Sep 2025).

The encoding density is summarized in the BitROM quantitative section as “1 transistor $\Rightarrow 2\times 3$-level weights $\Rightarrow 3.17$ bits per cell.” The important technical point is that the gain comes from multiplicity of stored ternary states per device, not from reducing each weight to a binary code. A common simplification is to describe the array as storing “two values per transistor”; the more precise statement is that it stores two independently addressable ternary weights per transistor, one per direction of read.

3. Read-out mechanism and bidirectional access protocol

A BiROMA read begins with bitline precharge. The selected BL is pre-charged to $V_{DD}$ and equalized across the column. One side of the cell is driven to the SL voltage corresponding to the requested ternary code, while the other side is left floating as BL. When the wordline is asserted, the NMOS conducts and the bitline relaxes toward the source-line voltage according to

$$V_{BL}(t)=V_{SL} +\bigl(V_{pre}-V_{SL}\bigr)\exp\!\bigl(-t/\tau\bigr), \qquad \tau=R_{on}C_{BL}.$$

After a fixed evaluation interval $t_{eval}$, the bitline voltage settles close to $V_{SL}$ and is digitized by two comparators:

$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$0

with

$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$1

The MSB comparator gates a local “enable” for the skip case $\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$2, while the LSB determines add versus subtract for $\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$3 (Zhang et al., 10 Sep 2025).

Bidirectional access is implemented as a two-phase protocol. In the forward read, the even-side rails are driven as SL and the odd-side rails are precharged as BL; the selected wordline is then activated and one weight is decoded. In the reverse read, the roles are swapped: odd-side rails become SL, even-side rails become BL, and the same wordline is activated again to fetch the second stored weight (Zhang et al., 10 Sep 2025).

This protocol directly addresses a common misconception. “Bidirectional” does not denote simultaneous extraction of both values, nor does it denote arbitrary reversibility of state. The BitROM description states that bidirectional reads “double the number of sensing steps per cycle if both weights are needed,” and therefore compute scheduling must interleave forward and reverse reads to maintain throughput. The array remains a read-only structure with directional sensing phases rather than a rewritable or simultaneously dual-read storage primitive (Zhang et al., 10 Sep 2025).

4. Array organization and peripheral circuitry

BiROMA is organized into 2,048 wordlines and 1,024 column pairs. Each column pair comprises three even-side rails and three odd-side rails, each labeled $\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$4–$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$5, together with a chain of NMOS cells sharing one wordline per row, a column-select switch (CS), and a digit equalizer (DEQ) used to pre-equalize bitlines before each read (Zhang et al., 10 Sep 2025).

The directional protocol is embedded into the physical organization. The even and odd rail sets are not merely duplicated wiring resources; they form the two directional interfaces through which the same transistor can be interrogated in opposite orientations. Forward read uses Even$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$6Odd, and reverse read uses Odd$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$7Even. This suggests that the array’s “bidirectionality” is fundamentally an architectural commitment at the rail and column-pair level, not just a local cell property.

Peripheral support is correspondingly specialized. The DEQ mitigates residual mismatch before sensing, and the CS isolates one column at a time. The array therefore combines analog-level settling, local equalization, and directional rail drive with digital decode by dual-threshold comparators. That combination is characteristic of CiROM designs in which storage density and in-situ computation are coupled at the bitcell and peripheral levels rather than separated into a conventional memory macro plus arithmetic datapath (Zhang et al., 10 Sep 2025).

5. Quantitative characteristics, margins, and limitations

The minimum noise margin is given as

$\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$8

Because the three SL levels are relatively close, comparator offsets and transistor threshold variation must be controlled tightly. The BitROM description states that they must be held below approximately $\{ \tfrac12 V_{DD}, \tfrac14 V_{DD}, 0 \}$9 to avoid misreads. In the design-tradeoff summary, that requirement is restated more concretely as comparator offset below $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$0 mV in $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$1 V systems and tight $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$2 control of $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$3 mV across process/voltage/temperature guardbands (Zhang et al., 10 Sep 2025).

Read disturbance is addressed through careful precharge-equalization and by keeping unselected wordlines at $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$4 V to isolate them. At the same time, the description notes that residual leakage on long bitlines can limit maximum row length, reported here as 2048 rows in the implemented design. The multiple metal-rail buses also increase routing complexity and impose precise IR-drop control requirements (Zhang et al., 10 Sep 2025).

The post-layout 65 nm metrics reported for the full BitROM macro are:

• effective bit density of $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$5 Kb/mm$$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$6;
• a stated 10$$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$7 improvement over prior digital CiROM, with DCiROM listed at $$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$8 Kb/mm$$w\in\{-1,0,+1\} \;\longleftrightarrow\; V_{SL}= \begin{cases} 0 & w=-1,\ \frac14V_{DD} & w=+1,\ \frac12V_{DD} & w=0~(\text{“skip”}). \end{cases}$$9;
• $w=0$0 TOPS/W;
• $w=0$1 pJ/op;
• approximately $w=0$2 pJ/read;
• access latency per ternary-weight fetch of $w=0$3 ns, inclusive of precharge, wordline activation, and sense (Zhang et al., 10 Sep 2025).

These numbers define the central trade-off of BiROMA. The array exchanges reduced noise margin and tighter analog tolerances for a substantial increase in storage density. A plausible implication is that its practical viability depends as much on comparator design, equalization discipline, and PVT control as on the nominal one-transistor encoding concept.

6. Relation to forward-only ROM arrays and ROMA’s B-ROM

BiROMA is usefully contrasted with the B-ROM structure used in ROMA, another ROM-based LLM accelerator. ROMA stores quantized base models in ROM and uses SRAM for LoRA weights and KV cache. Its B-ROM groups every four consecutive rows into one block, pre-generates all $w=0$4 possible 4-bit patterns in that block, and selects the correct one with a small 4-to-1 multiplexer. The stated effect is a roughly $w=0$5 reduction in ROM cell transistor count relative to an ungrouped conventional ROM array, corresponding to approximately $w=0$6 area saving once mux overhead is included (Wang et al., 17 Mar 2025).

The read model in ROMA is explicitly different. The ROMA description states that the “Forward read sequence” is the only mode supported and that “No bidirectional readback is exploited.” In ROMA, the key optimization is row grouping and pattern sharing across four addresses, followed by physical fusion of the B-ROM with adjacent low-precision compute logic in a “Fused-Cell,” which yields an additional approximately $w=0$7 area saving on top of the $w=0$8 from B-ROM alone (Wang et al., 17 Mar 2025).

The contrast clarifies what is distinctive about a bidirectional ROM array. In ROMA, density comes from replacing four per-bit pass-transistor instances with one shared structure plus CGen/MUX logic while preserving ordinary one-direction read semantics. In BiROMA, density comes from using device symmetry and directional rail assignment so that one transistor stores two ternary weights. Both approaches are ROM-centric and edge-LLM-oriented, but they optimize different dimensions of the design space: B-ROM emphasizes grouped pattern generation for quantized base models, whereas BiROMA emphasizes bidirectional ternary packing for CiROM inference (Wang et al., 17 Mar 2025).

At the system level, the two works also frame their storage hierarchies differently. ROMA reports on-chip storage of 4-bit 3B and 2-bit 8B LLaMA models without external memory and decoding throughput exceeding 20,000 tokens/s, up to 31.8 k tokens/s. BitROM, by contrast, frames BiROMA as part of a BitNet-oriented CiROM stack with integrated DR eDRAM that reduces external DRAM access by 43.6%. This suggests complementary research directions in ROM-based LLM acceleration: one based on stable quantized base-model storage with LoRA adaptation, and one based on extreme-density ternary CiROM with directional read encoding (Wang et al., 17 Mar 2025).