Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

NeuSpin: Design of a Reliable Edge Neuromorphic System Based on Spintronics for Green AI (2401.06195v1)

Published 11 Jan 2024 in cs.ET, cs.AI, cs.LG, and cs.NE

Abstract: Internet of Things (IoT) and smart wearable devices for personalized healthcare will require storing and computing ever-increasing amounts of data. The key requirements for these devices are ultra-low-power, high-processing capabilities, autonomy at low cost, as well as reliability and accuracy to enable Green AI at the edge. AI models, especially Bayesian Neural Networks (BayNNs) are resource-intensive and face challenges with traditional computing architectures due to the memory wall problem. Computing-in-Memory (CIM) with emerging resistive memories offers a solution by combining memory blocks and computing units for higher efficiency and lower power consumption. However, implementing BayNNs on CIM hardware, particularly with spintronic technologies, presents technical challenges due to variability and manufacturing defects. The NeuSPIN project aims to address these challenges through full-stack hardware and software co-design, developing novel algorithmic and circuit design approaches to enhance the performance, energy-efficiency and robustness of BayNNs on sprintronic-based CIM platforms.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. U. Gupta et al., “Chasing carbon: The elusive environmental footprint of computing,” in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA).   IEEE, 2021, pp. 854–867.
  2. C. Magazzino et al., “Investigating the link among ict, electricity consumption, air pollution, and economic growth in eu countries,” Energy Sources, Part B: Economics, Planning, and Policy, vol. 16, no. 11-12, pp. 976–998, 2021.
  3. T. Lee et al., “World-most energy-efficient mram technology for non-volatile ram applications,” in 2022 IEDM.   IEEE, 2022, pp. 10–7.
  4. A. Nguyen et al., “Deep neural networks are easily fooled: High confidence predictions for unrecognizable images,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 427–436.
  5. Y. Gal et al., “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning.   PMLR, 2016, pp. 1050–1059.
  6. R. Cai et al., “Vibnn: Hardware acceleration of bayesian neural networks,” ACM SIGPLAN Notices, vol. 53, no. 2, pp. 476–488, 2018.
  7. S. T. Ahmed et al., “SpinBayes: Algorithm-Hardware Co-Design for Uncertainty Estimation Using Bayesian In-Memory Approximation on Spintronic-Based Architectures,” ACM Transactions on Embedded Computing Systems, vol. 22, no. 5s, pp. 131:1–131:25, Sep. 2023. [Online]. Available: https://doi.org/10.1145/3609116
  8. H. Awano et al., “Bynqnet: Bayesian neural network with quadratic activations for sampling-free uncertainty estimation on fpga,” in 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2020, pp. 1402–1407.
  9. H. Fan et al., “Fpga-based acceleration for bayesian convolutional neural networks,” IEEE TCAD, vol. 41, no. 12, pp. 5343–5356, 2022.
  10. A. Malhotra et al., “Exploiting oxide based resistive ram variability for bayesian neural network hardware design,” IEEE Transactions on Nanotechnology, vol. 19, pp. 328–331, 2020.
  11. T. Dalgaty et al., “In situ learning using intrinsic memristor variability via markov chain monte carlo sampling,” Nature Electronics, vol. 4, no. 2, pp. 151–161, 2021.
  12. S. Dutta et al., “Neural sampling machine with stochastic synapse allows brain-like learning and inference,” Nature Communications, vol. 13, no. 1, p. 2571, Dec. 2022. [Online]. Available: https://www.nature.com/articles/s41467-022-30305-8
  13. D. Bonnet et al., “Bringing uncertainty quantification to the extreme-edge with memristor-based bayesian neural networks,” Researchsquare, 2023.
  14. K. Yang et al., “All-spin bayesian neural networks,” IEEE Transactions on Electron Devices, vol. 67, no. 3, pp. 1340–1347, 2020.
  15. A. Lu et al., “An Algorithm-Hardware Co-Design for Bayesian Neural Network Utilizing SOT-MRAM’s Inherent Stochasticity,” IEEE-JXCDC, 2022.
  16. M. Horowitz, “1.1 computing’s energy problem (and what we can do about it),” in 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2014, pp. 10–14.
  17. A. Mobiny et al., “Dropconnect is effective in modeling uncertainty of bayesian deep networks,” Scientific reports, 2021.
  18. S. T. Ahmed et al., “Binary bayesian neural networks for efficient uncertainty estimation leveraging inherent stochasticity of spintronic devices,” in NANOARCH’22: 17th ACM International Symposium on Nanoscale Architectures.   ACM, 2022, pp. 1–6.
  19. ——, “Spindrop: Dropout-based bayesian binary neural networks with spintronic implementation,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 13, no. 1, pp. 150–164, 2023.
  20. ——, “Spatial-spindrop: Spatial dropout-based binary bayesian neural network with spintronics implementation,” arXiv preprint arXiv:2306.10185, 2023.
  21. T. Gokmen et al., “Training deep convolutional neural networks with resistive cross-point devices,” Frontiers in neuroscience, vol. 11, p. 538, 2017.
  22. X. Peng et al., “Optimizing weight mapping and data flow for convolutional neural networks on rram based processing-in-memory architecture,” in IEEE ISCAS.   IEEE, 2019, pp. 1–5.
  23. S. T. Ahmed et al., “Scale-dropout: Estimating uncertainty in deep neural networks using stochastic scale,” arXiv preprint arXiv:2311.15816, 2023.
  24. ——, “Enhancing reliability of neural networks at the edge: Inverted normalization with stochastic affine transformations,” in 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2024, pp. 1–6.
  25. ——, “Scalable spintronics-based bayesian neural network for uncertainty estimation,” in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2023, pp. 1–6.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Soyed Tuhin Ahmed (10 papers)
  2. Kamal Danouchi (4 papers)
  3. Guillaume Prenat (8 papers)
  4. Lorena Anghel (9 papers)
  5. Mehdi B. Tahoori (25 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets