Balanced Binary Neural Networks with Gated Residual

• The paper introduces a dual-path quantization framework that balances binary weights with entropy-maximizing normalization and restores activations using a gated residual pathway.
• It replaces standard Conv-BN-ReLU modules with BBG blocks, seamlessly integrating into architectures like ResNet, VGG, and SSD for classification and detection tasks.
• Experimental results show BBG achieves notable accuracy gains—up to a cumulative improvement of 1.21% over conventional binary networks—with minimal computational overhead.

Balanced Binary Neural Networks with Gated Residual (BBG) are a network quantization framework that simultaneously maximizes information retention in both binary weights and binary activations. The BBG approach addresses a major limitation of conventional 1-bit neural networks—namely, severe accuracy degradation due to information loss and biased quantization. BBG integrates two mathematically motivated modules into generic deep learning backbones: a weight-balanced binarization scheme designed to maximize entropy by enforcing unbiased and high-variance binary filters, and a lightweight per-channel gated residual pathway that restores lost information by merging real-valued activations into the main computation stream. This design enables BBG to match or outperform the accuracy of prior binary networks with minimal computational and memory overhead, and is compatible with diverse architectures such as VGG, ResNet, and SSD for both classification and detection tasks (Shen et al., 2019).

1. Architectural Principles and Network Integration

The BBG module replaces each conventional full-precision convolution–activation pair in a standard network backbone with a dual-path construct. In the main (binary) path, real-valued "proxy" weights $v \in \mathbb{R}^{d \times d}$ are adjusted via differentiable centering, followed by symmetric binarization and mean scaling. Binary activations $x^b$ are convolved with binarized weights $w^b$, with the output further normalized via BatchNorm. In parallel, a gated-residual path computes a per-channel elementwise product between the real activation and a learnable scaling vector, which is then summed with the main path output.

When configuring a BBG block, the original macro-architecture (e.g., ResNet, VGG, SSD) remains unchanged, with all "Conv–BN–ReLU" modules replaced by BBG blocks. In detection architectures such as SSD, only backbone convolutions are binarized, while feature heads remain in full precision (Shen et al., 2019).

2. Weight-Balanced Binarization Formulation

To mitigate information loss in binarized weights, BBG introduces "balanced normalization" to maximize binary filter entropy by forcing their mean to zero:

$$w = v - \frac{1}{d} \mathbf{1} (\mathbf{1}^{\!\top} v) \quad\in\mathbb{R}^{d\times d}$$

where $v$ is the learnable proxy, $d$ is the filter size, and $\mathbf{1}$ an all-ones vector. The centered weights are binarized via:

$$\begin{aligned} \text{Forward:} &\quad w^b = \mathrm{sign}(w)\, E(\left| w \right|), \quad E(\left| w \right| )= \frac{1}{d} \sum_{ij} \left| w_{ij} \right| \ \text{Backward:} &\quad \frac{\partial \mathcal{L}}{\partial w} = \frac{\partial \mathcal{L}}{\partial w^b} \quad \text{(STE)} \end{aligned}$$

This normalization ensures $\mathbb{E}[w^b] = 0$, yielding high-entropy, unbiased quantized filters that preserve more activation information. Only the proxy weights $v$ are updated in training; only the binary weights $x^b$0 are stored at inference (Shen et al., 2019).

3. Gated Residual Compensation for Binary Activations

Activation quantization follows a hard clipping and rounding:

$x^b$1

This 1-bit discretization induces significant information loss. BBG compensates by adding a "gated residual" per output channel:

$x^b$2

The main output merges this with the binary path:

$x^b$3

where $x^b$4 denotes the path with binary conv and BN. The residual’s initial scaling ($x^b$5) acts as an identity shortcut. During training, the gated path also receives gradients:

$x^b$6

This mechanism provides a low-mismatch path for both forward information flow and backward gradient propagation, with only $x^b$7 FLOPs overhead per block (Shen et al., 2019).

4. Module Instantiation and Training Workflow

The end-to-end BBG block is formalized as follows:

$x^b$8

All non-smooth steps (sign, round) use STE (straight-through estimator) in backpropagation. BBG blocks are used in place of traditional conv–BN–ReLU sequences in major architectures. For classification (e.g., CIFAR-10/100, ImageNet), all main 3×3 convolutions are replaced, with first and last layers kept at full precision. For detection (e.g., Pascal VOC/SSD300), only backbone layers are binarized (Shen et al., 2019).

5. Benchmark Performance and Quantitative Analysis

BBG outperforms previous binary neural network methods across image classification and detection tasks, with marginal computational and memory overhead.

Dataset/Task	Architecture	Competing Method	Top-1 Acc / mAP	FLOPs (G)
CIFAR-10 (cls)	ResNet-20	Vanilla binary	84.13%	-
		+ Balanced weights	84.71%	-
		+ Gated residual	84.89%	-
		BBG (both)	85.34%	-
ImageNet (cls)	ResNet-18	XNOR-Net	51.2%	-
		DoReFa-Net	53.0%	-
		BiReal-Net	56.4%	-
		PCNN	56.9%	-
		ResNetE	58.2%	-
		BBG-Net	59.5%	-
Pascal VOC (detect)	SSD300-VGG16	FP	74.3%	30
		BDN (DenseNet-45)	68.2%	1.96
		BBG-VGG16	68.5%	1.06
	SSD300-Res34	BBG-Res34	62.8%	0.36

On ARM Cortex-A53 (daBNN runtime), BBG ResNet-18 requires 251 ms per image, indicating a 5.8× speed-up over the full-precision baseline (1457 ms). BBG networks require only 1 bit per stored weight (∼1/32 of FP32), with negligible additional cost for the gated scalars (Shen et al., 2019).

6. Ablation Studies and Component Analysis

Ablation experiments confirm the contribution of each BBG component. On CIFAR-10 with ResNet-20, weight-balancing alone yields +0.58% accuracy, gated-residual alone yields +0.76%, and their combination achieves a cumulative +1.21% gain over vanilla binary networks. Further comparison across four module designs under various channel widths shows BBG consistently matches or outperforms all prior binary network modules, reaching near full-precision accuracy when width is modestly increased.

A plausible implication is that the entropy-maximizing binarization synergizes with the low-mismatch residual path, jointly bridging the accuracy gap imposed by 1-bit quantization (Shen et al., 2019).

7. Context, Applicability, and Generalization

BBG modules are architecturally agnostic and can be used in VGG, ResNet, SSD, and potentially other topologies, as a generic quantization mechanism. In classification, all 3×3 convolutions are binarized except for the first and last layers; in detection networks (e.g., SSD300), only the backbone convolutions are binarized, with detection heads left in full precision. The weight-balanced binarization method and gated residual compensation can be generally adopted as modular enhancements for 1-bit quantization schemes in convolutional neural networks.

The design addresses both the entropy bottleneck in binary weights and the information loss in binary activations, empirically demonstrating improved trade-offs in accuracy, inference speed, and energy efficiency over previous binary networks (Shen et al., 2019).