Hardware-Software Co-Design of Scalable, Energy-Efficient Analog Recurrent Computations

Abstract: Always-on AI applications, from environmental sensors to biomedical implants, require ultra-low power consumption. Analog circuits offer a path to sub-microwatt inference, yet existing analog implementations are limited to feedforward architectures: extending them to recurrent dynamics has been considered impractical due to noise accumulation through temporal feedback. We demonstrate that this barrier can be overcome through hardware-software co-design. Specifically, we identify that Bistable Memory Recurrent Units (BMRUs), a class of Recurrent Neural Networks (RNNs) with discrete-valued outputs and hysteretic dynamics, admit an ultra-low power current-mode analog implementation which we design from first principles. The resulting circuit establishes a one-to-one correspondence between each learned parameter and a circuit element. The discrete outputs suppress analog noise by at least 20-fold at each cell boundary, breaking the noise accumulation that prevents analog recurrence. We reformulate BMRUs for first-quadrant operation with fixed thresholds, enabling the direct correspondence while preserving expressivity and trainability. Transistor-level simulations in 180 nm Complementary Metal-Oxide-Semiconductor (CMOS) show near-perfect agreement between software predictions and circuit-level behavior, with the software model thereby serving as a high-fidelity simulator of the physical hardware at low computational cost. We leverage this fidelity to conduct large-scale noise immunity and power scaling analyses: the power cost of adding recurrence scales linearly with state dimension, while the feedforward layers dominating total power scale quadratically, meaning recurrence is added at linear marginal cost relative to the feedforward backbone. End-to-end keyword spotting achieves sub-microwatt inference at the RNN core.

• The paper presents the BMRU architecture as a breakthrough for enabling noise-immune analog recurrent computations through discrete state dynamics.
• It details a fully current-mode, transistor-only implementation validated by Cadence Spectre simulations, ensuring high-fidelity hardware-software congruence.
• Benchmark results demonstrate near state-of-the-art accuracy on sequential tasks with ultra-low power consumption, supporting scalable edge AI applications.

Hardware-Software Co-Design of Scalable, Energy-Efficient Analog Recurrent Computations

Motivation and Problem Statement

Modern always-on AI edge applications, such as environmental monitoring and biomedical implants, demand ultra-low power sequence processing beyond the feasible range of digital microcontrollers and existing neuromorphic chips. While analog circuits offer sub-microwatt inference and inherent energy advantages, fully analog implementations capable of recurrent computations have been hampered by degradation from temporal noise accumulation. Feedforward architectures dominate available analog solutions; genuinely recurrent dynamics remained impractical due to sensitivity to analog noise in the feedback loop. This paper addresses the outstanding challenge: enabling robust, scalable analog recurrent neural networks (RNNs) through principled hardware-software co-design, specifically by leveraging the Bistable Memory Recurrent Unit (BMRU) architecture.

Co-Design Methodology and BMRU Properties

The authors identify BMRUs—a class of RNNs with discrete-valued outputs and hysteretic dynamics—as uniquely compatible with monolithic analog implementation. BMRUs possess three critical properties for this domain: (1) parallelizable training via instantaneous convergence (associative scan), (2) persistent, multistable memory without capacitive settling, and (3) discrete outputs that enforce structurally noise-immune cell boundaries. This discrete cell state blocks propagation of analog noise, directly addressing the core practical obstacle to recurrent analog computation.

The resultant analog architecture is fully current-mode, capacitor-free, and transistor-only. Each learned parameter in the BMRU variant maps directly to a physical circuit element: thresholds are implemented as bias currents, weights are encoded as transistor width ratios in current mirrors, and discrete output amplitudes are set via dedicated analog bias.

First-Quadrant BMRU Reformulation

To align with unipolar current-mode circuit constraints, the authors introduce the First-Quadrant BMRU (FQ BMRU). All internal signals and outputs remain non-negative, fixed at the circuit design stage to DC bias values. FQ BMRUs maintain the expressive capacity of their original counterparts through a window-comparator update rule—when the candidate input crosses learned thresholds ($h_t < B_{\text{lo}}$, $h_t > B_{\text{hi}}$), cell states are instantaneously set or reset; otherwise, previous state persists. This windowed hysteresis is critical for robust bistability and noise suppression.

Theoretical results demonstrate universal approximation equivalence: FQ BMRUs, augmented with ReLU MLP layers for pre-processing, can approximate any function computed by original BMRUs with input-dependent thresholds and bipolar outputs, validating the reconfigured architecture’s expressiveness.

Analog CMOS Implementation and Validation

Transistor-level implementation of the FQ BMRU in X-FAB 180 nm CMOS is presented, with all cells and layers operating in ultralow-power subthreshold regimes. The circuit leverages current mirrors for weighted summation, diode-connected transistors for ReLU, and a dual-Heaviside feedback structure for hysteresis. Each BMRU cell consists of 9 transistors, yielding nanowatt-scale per-cell power consumption and aggregate inference below 100 nW for a minimal keyword-spotting network.

Cadence Spectre simulation validates the analog implementation against software, achieving hardware-software prediction congruence at every signal stage—including input projection, BMRU layer candidates, recurrent cell states, skip connections, and class output logits. The software model thus serves as a high-fidelity proxy for hardware with negligible computational overhead.

Benchmark Results and Numerical Analysis

On benchmark tasks spanning sequential MNIST, permuted MNIST, spoken-word keyword spotting, ListOps, and character-level language modeling, the FQ BMRU matches or closely approaches state-of-the-art parallelizable RNN baselines (LRU, minGRU). For the keyword spotting task (Google Speech Commands), inference accuracy exceeds 97% at state dimension $d \ge 16$, saturating at 98% near $d=16$. The analog RNN core operates continuously at sub-100 nW power for $d=4$, and remains within the sub-microwatt envelope up to $d=16$, all validated in hardware simulation.

Quantization analysis demonstrates negligible (<4%) accuracy degradation under 4-bit parameter discretization, confirming hardware robustness for programmable implementations using binary-weighted current mirrors.

Structural Noise Immunity

A central technical result is the quantitative demonstration of noise immunity. Injecting analog noise levels (derived from transistor-level mismatch and PVT corner analysis) into software simulations, FQ BMRU networks maintain full accuracy up to twice the measured analog noise. Discrete-valued BMRU outputs suppress accumulated noise by at least 20-fold at cell boundaries, preventing propagation through the recurrent feedback loop—a property not shared by purely linear SSMs and only weakly present in minGRUs.

Monte Carlo mismatch and PVT analyses corroborate hardware robustness, with all misclassifications attributable to input ambiguity rather than systematic circuit errors. Multi-class extension (11-digit keyword classification) maintains competitive accuracy and increased output margin separation, further improving robustness.

Power and Scalability Analysis

Component-level power breakdown reveals a fundamental architectural asymmetry: BMRU cell power scales linearly in state dimension ($\mathcal{O}(d)$) while feedforward FC layers scale quadratically ($\mathcal{O}(d^2)$). Consequently, adding recurrence incurs linear marginal power overhead relative to dominant feedforward costs. This composability implies that any analog feedforward platform can acquire recurring sequence processing at minimal energy overhead merely by integrating BMRU cells.

Programmable circuit overhead (area and power) for 4-bit resolution is linear in parameter size; estimated maximum system power remains within sub-microwatt bounds for all practical network dimensions.

Implications and Future Directions

This work establishes the first fully analog recurrent architecture with structural noise immunity, parallelizable training, and practical power scalability. Practically, the approach enables always-on temporal intelligence for edge AI devices previously constrained by battery and energy budgets—ranging from distributed sensors to biomedical implants.

Theoretically, the hardware-software co-design paradigm exemplified here can be extended to other recurrent architectures and hybrid analog-digital systems. With analog feedforward inference already well-established, the demonstration that recurrence can be added at modest linear cost unlocks new architectural horizons. Hybrid designs combining bistable memory and transient dynamics could enable more sophisticated temporal processing on ultra-low power substrates. Further exploration of post-fabrication programmability, integration of on-chip feature extraction, and physical silicon validation are open avenues.

Conclusion

The paper introduces a principled co-design methodology for scalable, energy-efficient analog recurrent neural networks, overcoming the temporal noise barrier via BMRU’s discrete-state dynamics. Transistor-level simulation validates high-fidelity software-hardware congruence, sub-microwatt inference feasibility, and robust structural noise immunity. The results support the integration of temporal processing into analog inference platforms, fundamentally broadening the scope of edge AI in power-constrained environments (2605.15216).