Binarized Sensing Layer

• Binarized sensing layer is a system architecture that converts activations, weights, and sensing operations to binary values, facilitating efficient in-sensor processing.
• It leverages techniques such as XNOR logic, bit-plane encoding, and in-memory compute to achieve significant reductions in energy consumption and latency on edge devices.
• Despite a modest drop in accuracy compared to full-precision models, its design supports scalable applications in compressive sensing and low-power neural network deployment.

A binarized sensing layer is a system or architectural front end, implementable in both hardware and software, that constrains all elements—input activations, weights, and often the core sensing/transduction operation itself—to binary (typically ±1 or {0,1}) precision. These layers are integral to contemporary binary neural networks, in-sensor processing, and binarized compressive sensing; they combine minimal compute/memory requirements with high speed and low energy consumption. Implementations span from pixel processor arrays with in-pixel ALUs, to MRAM/PCM-based compute pixels and fully digital in-memory XNOR arrays, and even learned binary input binning for pre-processing sensor data. Though intrinsically lossy compared to full-precision alternatives, they are engineered and optimized to realize near-state-of-the-art accuracy with orders-of-magnitude improvements in latency and energy, enabling edge deployment at scale.

1. Fundamental Principles and Binarization Mechanisms

At the algorithmic core, binarized sensing layers reduce both network weights and activations to representations constrained to ±1 or {0,1}. This is accomplished using sign-thresholding for activations, and weight clipping/projecting for filters or synaptic weights. Training is performed using a straight-through estimator (STE) for the non-differentiable sign function, where gradients are estimated as if the quantization is identity within the (–1,+1) interval (using hard tanh or similar) (Liu et al., 2022).

For post-processing of activations, batch normalization (BN) is typically algebraically fused into the binarization threshold. For example, rearranging $$y = \gamma \cdot (x - \mu)/\sqrt{\sigma^2 + \epsilon} + \beta$$ leads to replacing the sequence BN$\rightarrow$sign by a direct sign$(x-\tau^*)$ comparison, where $$\tau^* = \mu - (\beta/\gamma) \cdot \sqrt{\sigma^2+\epsilon}$$ can be stored as a single scalar per channel (Liu et al., 2022).

In hardware, multiplication by ±1 is realized as an XNOR logic gate: $+1 \cdot +1 \rightarrow +1$, $-1 \cdot -1 \rightarrow +1$, $+1 \cdot -1 \rightarrow -1$, $-1 \cdot +1 \rightarrow -1$. Accumulations become simple bitwise popcount or current summation, depending on the substrate (digital or analog).

2. Architectural Implementations: Sensor and Compute-Hardware

Pixel Processor Array (PPA) Architectures

The SCAMP-5 PPA demonstrates in-pixel ALUs capable of XNOR, bit-shift, and limited integer arithmetic. In-sensor binarized FCNs are mapped via group convolution to minimize in-pixel storage and computation. Each layer reads input and filter bitmaps from registers, computes 3x3 XNOR-accumulate dot products, compares to the fused threshold, and writes results back, all without external RAM or floating-point operations (Liu et al., 2022).

Network structure example (SCAMP-5):

Layer	Kernel	Groups	Out Channels	Local Mem (per pixel)
Conv1	3×3	4	16	16b (in)+16b (wts)
Conv2	3×3	4	32	32b+32b
Conv3	3×3	8	64	64b+64b
Conv4	3×3	8	64	64b+64b
Conv5	3×3	16	128	128b+128b

Processing-In-Sensor Accelerator (PISA)

PISA uses an array of compute pixels with STT-MRAM that stores 1-bit weights directly at the sensor site. Each path multiplies the photocurrent by its kernel weight via current steering (T4/T5 branches, no explicit digital XNOR), with the output sign detected analogically. Subsequent BWNN layers are performed in an adjacent near-sensor processing-in-memory (PIM) compute unit in DRAM, with the architecture supporting a hybrid mode-switching for coarse/fine-grained operation (Angizi et al., 2022).

In-Memory XNOR Arrays

Fully digital in-memory XNOR realization leverages 2T–2R differential OxRAM cells to represent ±1 weights. The precharge sense-amplifier directly implements the XNOR operation between weight (stored as the LRS/HRS state of paired RRAMs) and input activation by steering precharge discharge based on the activation bit. Popcount and thresholding at the periphery finalize each binary neuron update. This architecture is robust to bit errors and enables sub-100 nJ/sample inference energy (Hirtzlin et al., 2019).

3. Techniques for Input and First-Layer Binarization

Conventional wisdom excluded the first (input) layer from binarization due to severe accuracy losses, but several schemes now enable full-stack binary pipelines:

• Bit-plane Encoding decomposes the 8-bit input into $P$ bit-planes $x_b$, each convolved with its own binary filter, reweighted (e.g., by learned $\alpha_b$), fused by summation, then passed through a sign (binarization) activation (Vorabbi et al., 2023).
• Binary Input Layer (BIL) expands the M-bit input into $C \times M$ binary channels. A 1×1 binary convolutional layer (with learned weights $w^b_{p,k}$) fuses these into K binary features. BIL enables per-bit significance learning and chip area/energy drops to ~2% of baseline designs while retaining competitive accuracy, especially for multimodal inputs (Dürichen et al., 2018).
• Data-Driven Binning/Masking learns a binary (or real-valued) mask $B$ mapping inputs $x \in \mathbb{R}^N$ to $y = Bx$, where $B \in \{0,1\}^{M \times N}$. End-to-end training is performed using surrogate gradients, and binarization enforced by straight-through estimators or row normalization. The method is directly hardware-realizable as fill-factor masks and provides rapid recognition and SNR gains via spatial binning (Mennel et al., 2021).

4. Binarized Sensing in Compressive Sensing and Signal Acquisition

In 1-bit compressive sensing, both the measurement process and the reconstruction operator (sensing matrix) are binarized. The acquisition model is $y = \text{sign}(\Phi x + \xi^y)$, with reconstruction via a binarized sensing matrix $\Psi = \text{sign}(\Phi + \xi^{\Phi})$. Estimation leverages a quantized projected back-projection (QPBP): $x̂ = (1/m) \mathsf{H}_s(\Psi^T z)$, with $\mathsf{H}_s(\cdot)$ denoting hard thresholding to sparsity $s$. Memory and compute are reduced 32$\times$/64$\times$ relative to floating-point, and reconstruction error decays as $O(m^{-1/2})$ (Feuillen et al., 2020).

5. Performance, Energy, and Robustness Characteristics

Throughput and Efficiency

• PPA-based binarized FCNs achieve >280 fps at sub-5 ms/ frame on SCAMP-5 (172×182 resolution).
• PISA achieves 1000 fps for layer-1 convolution, >80% reduction in ADC/I/O energy, and 1.7 TOPS/W compute-level efficiency (Angizi et al., 2022).
• In-memory XNOR arrays enable full-image inference at 25 nJ/sample, exceeding conventional CPUs/GPUs by 10⁴–10⁶× in energy efficiency (Hirtzlin et al., 2019).

Accuracy

• PPA FCN: IoU = 82.3%, pixel-wise accuracy = 93.7% on road-scene segmentation; typical decrease in accuracy versus floating-point: 5–7% (Liu et al., 2022).
• PISA (coarse→PIM): MNIST 95.1%, SVHN 90.4%, CIFAR-10 79.8%—trading <5% accuracy for 10× energy savings (Angizi et al., 2022).
• Bit-plane input and BIL methods: For CIFAR-10 (VGG-Small), bit-plane: 85.9% vs. baseline 89.2%; BIL (K=256): 86.35% (Vorabbi et al., 2023, Dürichen et al., 2018).
• 1-bit compressive sensing: $O(m^{-1/2})$ error decay, ≈2 dB loss versus full-precision with highly reduced hardware (Feuillen et al., 2020).

Robustness

Binarized networks show inherent tolerance to raw bit errors up to BER $10^{-4}$–$10^{-3}$ with negligible accuracy degradation, due to averaging and popcount operations (Hirtzlin et al., 2019).

6. Limitations, Scalability, and Deployment Considerations

Despite marked advantages in energy and latency, binarized sensing layers present limitations:

• ~5–7% accuracy drop may be prohibitive for some tasks; performance is more favorable for applications where this loss is acceptable.
• On-sensor binary compute is often limited by in-pixel memory/register capacity, constraining kernel size (≤5×5), channels (≤128), and network depth.
• In analog or mixed-signal implementations (e.g., PISA), analog variability demands careful threshold calibration and noise-aware training (Angizi et al., 2022).
• In compressive sensing, reconstruction performance degrades if quantization parameters are not chosen carefully in concert with the underlying RIP properties (Feuillen et al., 2020).

Integration is straightforward for edge hardware with binary ALUs or existing DRAM/flash architectures, but may require nontrivial hardware design for analog compute pixels or memory arrays.

7. Comparative Overview and Research Directions

Binarized sensing layers unify several research threads, from in-memory digital computing and analog in-sensor inference, to learned binary input convolutions and 1-bit compressive acquisition. They consistently exhibit:

• Extreme energy/area efficiency: up to two orders of magnitude gain versus full-precision, allowing inference on resource-constrained devices (Liu et al., 2022, Angizi et al., 2022, Hirtzlin et al., 2019).
• Robustness to hardware non-idealities, making them suitable for emerging memory and mixed-signal substrates.
• Scalability is presently upper-bounded by hardware resource constraints and algorithmic accuracy trade-offs.

Active research focuses on improving accuracy through hybrid precision, noise-adaptive learning, integration with PIM architectures for deeper layers, and expanding applicability to more complex modalities. These advances are positioning binarized sensing as a primary candidate for pervasive low-power intelligent sensing at the edge.