Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VECOM: Variation Resilient Encoding and Offset Compensation Schemes for Reliable ReRAM Based DNN Accelerator (2312.11042v1)

Published 18 Dec 2023 in cs.ET

Abstract: Resistive Random Access Memory (ReRAM) based Processing In Memory (PIM) Accelerator has emerged as a promising computing architecture for memory intensive applications, such as Deep Neural Networks (DNNs). However, due to its immaturity, ReRAM devices often suffer from various reliability issues, which hinder the practicality of the PIM architecture and lead to a severe degradation in DNN accuracy. Among various reliability issues, device variation and offset current from High Resistance State (HRS) cell have been considered as major problems in a ReRAM based PIM architecture. Due to these problems, the throughput of the ReRAM based PIM is reduced as fewer wordlines are activated. In this paper, we propose VECOM, a novel approach that includes a variation resilient encoding technique and an offset compensation scheme for a robust ReRAM based PIM architecture. The first technique (i.e., VECOM encoding) is built based on the analysis of the weight pattern distribution of DNN models, along with the insight into the ReRAM's variation property. The second technique, VECOM offset compensation, tolerates offset current in PIM by mapping the conductance of each Multi level Cell (MLC) level added with a specific offset conductance. Experimental results in various DNN models and datasets show that the proposed techniques can increase the throughput of the PIM architecture by up to 9.1 times while saving 50% of energy consumption without any software overhead. Additionally, VECOM is also found to endure low R ratio ReRAM cell (up to 7) with a negligible accuracy drop.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. A. Shafiee, A. Nag, N. Muralimanohar, R. Balasubramonian, J. P. Strachan, M. Hu, R. S. Williams, and V. Srikumar, “ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars,” in 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).   IEEE, jun 2016.
  2. P. Chi, S. Li et al., “Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory,” SIGARCH Comput. Archit. News, vol. 44, no. 3, p. 2739, jun 2016.
  3. V. Joshi, M. Le Gallo, S. Haefeli, I. Boybat, S. R. Nandakumar, C. Piveteau, M. Dazzi, B. Rajendran, A. Sebastian, and E. Eleftheriou, “Accurate deep neural network inference using computational phase-change memory,” Nature communications, vol. 11, no. 1, pp. 1–13, 2020.
  4. J. Lin, C.-D. Wen, X. Hu, T. Tang, I.-C. Lin, Y. Wang, and Y. Xie, “Rescuing RRAM-based computing from static and dynamic faults,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 10, pp. 2049–2062, oct 2021.
  5. Y. Park, S. Y. Lee, H. Shin, J. Heo, T. J. Ham, and J. W. Lee, “Unlocking wordline-level parallelism for fast inference on RRAM-based DNN accelerator,” in Proceedings of the 39th International Conference on Computer-Aided Design.   ACM, nov 2020.
  6. L. Chen, J. Li, Y. Chen, Q. Deng, J. Shen, X. Liang, and L. Jiang, “Accelerator-friendly neural-network training: Learning variations and defects in rram crossbar,” Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017, pp. 19–24, 3 2017.
  7. T. Liu, W. Wen, L. Jiang, Y. Wang, C. Yang, and G. Quan, “A fault-tolerant neural network architecture,” in 2019 56th ACM/IEEE Design Automation Conference (DAC).   IEEE, 2019, pp. 1–6.
  8. M. A. Lastras-Montaño, D. Pozo-Zamudio, L. Glebsky, M. Zhao, H. Wu, K.-T. Cheng et al., “Ratio-based multi-level resistive memory cells,” Scientific reports, vol. 11, no. 1, pp. 1–12, 2021.
  9. W.-H. Chen, C. Dou, K.-X. Li, W.-Y. Lin, P.-Y. Li, J.-H. Huang, J.-H. Wang, W.-C. Wei, C.-X. Xue, Y.-C. Chiu et al., “Cmos-integrated memristive non-volatile computing-in-memory for ai edge processors,” Nature Electronics, vol. 2, no. 9, pp. 420–428, 2019.
  10. Y. Long, X. She, and S. Mukhopadhyay, “Design of reliable dnn accelerator with un-reliable reram,” in 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2019, pp. 1769–1774.
  11. S. Jeong, J. Kim, M. Jeong, and Y. Lee, “Variation-tolerant and low r-ratio compute-in-memory reram macro with capacitive ternary mac operation,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no. 7, pp. 2845–2856, 2022.
  12. W.-H. Chen, K.-X. Li, W.-Y. Lin, K.-H. Hsu, P.-Y. Li, C.-H. Yang, C.-X. Xue, E.-Y. Yang, Y.-K. Chen, Y.-S. Chang et al., “A 65nm 1mb nonvolatile computing-in-memory reram macro with sub-16ns multiply-and-accumulate for binary dnn ai edge processors,” in 2018 IEEE International Solid-State Circuits Conference-(ISSCC).   IEEE, 2018, pp. 494–496.
  13. H. Shin, M. Kang, and L.-S. Kim, “Fault-free: A fault-resilient deep neural network accelerator based on realistic reram devices,” in 2021 58th ACM/IEEE Design Automation Conference (DAC).   IEEE, 2021, pp. 1039–1044.
  14. T.-H. Yang, H.-Y. Cheng, C.-L. Yang, I.-C. Tseng, H.-W. Hu, H.-S. Chang, and H.-P. Li, “Sparse reram engine: Joint exploration of activation and weight sparsity in compressed neural networks,” in Proceedings of the 46th International Symposium on Computer Architecture, ser. ISCA ’19.   New York, NY, USA: Association for Computing Machinery, 2019, p. 236–249.
  15. B. Crafton, Z. Wan, S. Spetalnick, J.-H. Yoon, W. Wu, C. Tokunaga, V. De, and A. Raychowdhury, “Improving compute in-memory ecc reliability with successive correction,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, ser. DAC ’22.   New York, NY, USA: Association for Computing Machinery, 2022, p. 745–750.
  16. B. Feinberg, S. Wang, and E. Ipek, “Making memristive neural network accelerators reliable,” in 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2018, pp. 52–65.
  17. Q. Lou, T. Gao, P. Faley, M. Niemier, X. S. Hu, and S. Joshi, “Embedding error correction into crossbars for reliable matrix vector multiplication using emerging devices,” in Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design, ser. ISLPED ’20.   New York, NY, USA: Association for Computing Machinery, 2020, p. 139–144.
  18. C. Ma, Y. Sun, W. Qian, Z. Meng, R. Yang, and L. Jiang, “Go unary: A novel synapse coding and mapping scheme for reliable reram-based neuromorphic computing,” in 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020, pp. 1432–1437.
  19. S.-S. Lee and J.-S. Yang, “Value-aware parity insertion ecc for fault-tolerant deep neural network,” in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2022, pp. 724–729.
  20. J.-S. Kim and J.-S. Yang, “Dris-3: Deep neural network reliability improvement scheme in 3d die-stacked memory based on fault analysis,” in 2019 56th ACM/IEEE Design Automation Conference (DAC), 2019, pp. 1–6.
  21. T.-H. Nguyen, M. Imran, and J.-S. Yang, “Dynapat: A dynamic pattern-aware encoding technique for robust mlc pcm-based deep neural networks,” in Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, 2022, pp. 1–9.
  22. B. K. Joardar, J. R. Doppa, H. Li, K. Chakrabarty, and P. P. Pande, “Learning to train cnns on faulty reram-based manycore accelerators,” ACM Transactions on Embedded Computing Systems (TECS), vol. 20, no. 5s, pp. 1–23, 2021.
  23. J. J. Ryu, B. K. Park, T.-M. Chung, Y. K. Lee, and G. H. Kim, “Optimized method for low-energy and highly reliable multibit operation in a hfo2-based resistive switching device,” Advanced Electronic Materials, vol. 4, no. 12, p. 1800261, 2018.
  24. H. Li, Z. Jiang, P. Huang, Y. Wu, H.-Y. Chen, B. Gao, X. Liu, J. Kang, and H.-S. Wong, “Variation-aware, reliability-emphasized design and optimization of rram using spice model,” in 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2015, pp. 1425–1430.
  25. M. Saberi, R. Lotfi, K. Mafinezhad, and W. A. Serdijn, “Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation adcs,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 58, no. 8, pp. 1736–1748, 2011.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now