Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BARS: Joint Search of Cell Topology and Layout for Accurate and Efficient Binary ARchitectures (2011.10804v3)

Published 21 Nov 2020 in cs.AI

Abstract: Binary Neural Networks (BNNs) have received significant attention due to their promising efficiency. Currently, most BNN studies directly adopt widely-used CNN architectures, which can be suboptimal for BNNs. This paper proposes a novel Binary ARchitecture Search (BARS) flow to discover superior binary architecture in a large design space. Specifically, we analyze the information bottlenecks that are related to both the topology and layout architecture design choices. And we propose to automatically search for the optimal information flow. To achieve that, we design a two-level (Macro & Micro) search space tailored for BNNs and apply a differentiable neural architecture search (NAS) to explore this search space efficiently. The macro-level search space includes width and depth decisions, which is required for better balancing the model performance and complexity. We also design the micro-level search space to strengthen the information flow for BNN. %A notable challenge of BNN architecture search lies in that binary operations exacerbate the "collapse" problem of differentiable NAS, for which we incorporate various search and derive strategies to stabilize the search process. On CIFAR-10, BARS achieves 1.5% higher accuracy with 2/3 binary operations and 1/10 floating-point operations comparing with existing BNN NAS studies. On ImageNet, with similar resource consumption, BARS-discovered architecture achieves a 6% accuracy gain than hand-crafted binary ResNet-18 architectures and outperforms other binary architectures while fully binarizing the architecture backbone.

User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (9)
  1. Tianchen Zhao (27 papers)
  2. Xuefei Ning (52 papers)
  3. Xiangsheng Shi (2 papers)
  4. Songyi Yang (2 papers)
  5. Shuang Liang (84 papers)
  6. Peng Lei (12 papers)
  7. Jianfei Chen (63 papers)
  8. Huazhong Yang (80 papers)
  9. Yu Wang (940 papers)
Citations (7)

Summary

We haven't generated a summary for this paper yet.