Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

H3DFact: Heterogeneous 3D Integrated CIM for Factorization with Holographic Perceptual Representations (2404.04173v1)

Published 5 Apr 2024 in cs.AR and cs.LG

Abstract: Disentangling attributes of various sensory signals is central to human-like perception and reasoning and a critical task for higher-order cognitive and neuro-symbolic AI systems. An elegant approach to represent this intricate factorization is via high-dimensional holographic vectors drawing on brain-inspired vector symbolic architectures. However, holographic factorization involves iterative computation with high-dimensional matrix-vector multiplications and suffers from non-convergence problems. In this paper, we present H3DFact, a heterogeneous 3D integrated in-memory compute engine capable of efficiently factorizing high-dimensional holographic representations. H3DFact exploits the computation-in-superposition capability of holographic vectors and the intrinsic stochasticity associated with memristive-based 3D compute-in-memory. Evaluated on large-scale factorization and perceptual problems, H3DFact demonstrates superior capability in factorization accuracy and operational capacity by up to five orders of magnitude, with 5.5x compute density, 1.2x energy efficiency improvements, and 5.9x less silicon footprint compared to iso-capacity 2D designs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Y. Burak, U. Rokni, M. Meister, and H. Sompolinsky, “Bayesian model of dynamic image stabilization in the visual system,” Proceedings of the National Academy of Sciences, vol. 107, no. 45, pp. 19 525–19 530, 2010.
  2. Z. Wan, C.-K. Liu, H. Yang, R. Raj, C. Li, H. You, Y. Fu, C. Wan, A. Samajdar, Y. Lin, T. Krishna, and A. Raychowdhury, “Towards cognitive ai systems: Workload and characterization of neuro-symbolic ai,” in 2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).   IEEE, 2024.
  3. M. Hersche, M. Zeqiri, L. Benini, A. Sebastian, and A. Rahimi, “A neuro-vector-symbolic architecture for solving raven’s progressive matrices,” Nature Machine Intelligence, vol. 5, no. 4, pp. 363–375, 2023.
  4. X. Yang, Z. Wang, X. S. Hu, C. H. Kim, S. Yu, M. Pajic, R. Manohar, Y. Chen, and H. H. Li, “Neuro-symbolic computing: Advancements and challenges in hardware-software co-design,” IEEE Transactions on Circuits and Systems II: Express Briefs, 2023.
  5. Z. Wan, C.-K. Liu, H. Yang, C. Li, H. You, Y. Fu, C. Wan, T. Krishna, Y. Lin, and A. Raychowdhury, “Towards cognitive ai systems: a survey and prospective on neuro-symbolic ai,” arXiv preprint arXiv:2401.01040, 2024.
  6. D. Kleyko, D. Rachkovskij, E. Osipov, and A. Rahimi, “A survey on hyperdimensional computing aka vector symbolic architectures,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–52, 2023.
  7. M. Hersche, F. Di Stefano, T. Hofmann, A. Sebastian, and A. Rahimi, “Probabilistic abduction for visual abstract reasoning via learning rules in vector-symbolic architectures,” arXiv preprint arXiv:2401.16024, 2024.
  8. D. Kleyko, M. Davies, E. P. Frady, P. Kanerva, S. J. Kent, B. A. Olshausen, E. Osipov, J. M. Rabaey, D. A. Rachkovskij, A. Rahimi et al., “Vector symbolic architectures as a computing framework for emerging hardware,” Proceedings of the IEEE, vol. 110, no. 10, pp. 1538–1571, 2022.
  9. E. P. Frady, S. J. Kent, B. A. Olshausen, and F. T. Sommer, “Resonator networks, 1: An efficient solution for factoring high-dimensional, distributed representations of data structures,” Neural Computation, vol. 32, no. 12, pp. 2311–2331, 2020.
  10. A. Renner, L. Supic, A. Danielescu, G. Indiveri, E. P. Frady, F. T. Sommer, and Y. Sandamirskaya, “Neuromorphic visual odometry with resonator networks,” arXiv preprint arXiv:2209.02000, 2022.
  11. S. Yu, W. Shim, J. Hur, Y.-c. Luo, G. Choe, W. Li, A. Lu, and X. Peng, “Compute-in-memory: from device innovation to 3d system integration,” in ESSDERC 2021-IEEE 51st European Solid-State Device Research Conference (ESSDERC).   IEEE, 2021, pp. 21–28.
  12. B. Crafton, Z. Wan, S. Spetalnick, J.-H. Yoon, W. Wu, C. Tokunaga, V. De, and A. Raychowdhury, “Improving compute in-memory ecc reliability with successive correction,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022, pp. 745–750.
  13. C.-K. Liu, H. Chen, M. Imani, K. Ni, A. Kazemi, A. F. Laguna, M. Niemier, X. S. Hu, L. Zhao, C. Zhuo et al., “Cosime: Fefet based associative memory for in-memory cosine similarity search,” in Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, 2022, pp. 1–9.
  14. M. Chang, A. S. Lele, S. D. Spetalnick, B. Crafton, S. Konno, Z. Wan, A. Bhat, W.-S. Khwa, Y.-D. Chih, M.-F. Chang et al., “A 73.53 tops/w 14.74 tops heterogeneous rram in-memory and sram near-memory soc for hybrid frame and event-based target tracking,” in 2023 IEEE International Solid-State Circuits Conference (ISSCC).   IEEE, 2023, pp. 426–428.
  15. J. Langenegger, G. Karunaratne, M. Hersche, L. Benini, A. Sebastian, and A. Rahimi, “In-memory factorization of holographic perceptual representations,” Nature Nanotechnology, vol. 18, no. 5, pp. 479–485, 2023.
  16. W. Li, M. Manley, J. Read, A. Kaul, M. S. Bakir, and S. Yu, “H3datten: Heterogeneous 3-d integrated hybrid analog and digital compute-in-memory accelerator for vision transformer self-attention,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023.
  17. H. Li, “Emerging hardware technologies and 3d system integration for ubiquitous machine intelligence,” in 2023 60th ACM/IEEE Design Automation Conference (DAC).   IEEE, 2023, pp. 1–2.
  18. Y. Luo and S. Yu, “H3d-transformer: A heterogeneous 3d (h3d) computing platform for transformer model acceleration on edge devices,” ACM Transactions on Design Automation of Electronic Systems, 2024.
  19. P. Kanerva, “Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors,” Cognitive Computation, vol. 1, pp. 139–159, 2009.
  20. G. Murali, A. Iyer, L. Zhu, J. Tong, F. M. Martínez, S. R. Srinivasa, T. Karnik, T. Krishna, and S. K. Lim, “On continuing dnn accelerator architecture scaling using tightly coupled compute-on-memory 3-d ics,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023.
  21. S. Dutta, H. Ye, W. Chakraborty, Y.-C. Luo, M. San Jose, B. Grisafe, A. Khanna, I. Lightcap, S. Shinde, S. Yu et al., “Monolithic 3d integration of high endurance multi-bit ferroelectric fet for accelerating compute-in-memory,” in 2020 IEEE International Electron Devices Meeting (IEDM).   IEEE, 2020, pp. 36–4.
  22. S. D. Spetalnick, M. Chang, S. Konno, B. Crafton, A. S. Lele, W.-S. Khwa, Y.-D. Chih, M.-F. Chang, and A. Raychowdhury, “A 2.38 mcells/mm22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT 9.81-350 tops/w rram compute-in-memory macro in 40nm cmos with hybrid offset/IOFFsubscriptIOFF\text{I}_{\text{OFF}}I start_POSTSUBSCRIPT OFF end_POSTSUBSCRIPT cancellation and ICELLsubscriptICELL\text{I}_{\text{CELL}}I start_POSTSUBSCRIPT CELL end_POSTSUBSCRIPT RBLSLsubscriptRBLSL\text{R}_{\text{BLSL}}R start_POSTSUBSCRIPT BLSL end_POSTSUBSCRIPT drop mitigation,” in 2023 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits).   IEEE, 2023, pp. 1–2.
  23. G. Karunaratne, M. Schmuck, M. Le Gallo, G. Cherubini, L. Benini, A. Sebastian, and A. Rahimi, “Robust high-dimensional memory-augmented neural networks,” Nature communications, vol. 12, no. 1, p. 2468, 2021.
  24. H. E. Barkam, S. Yun, P. R. Genssler, C.-K. Liu, Z. Zou, H. Amrouch, and M. Imani, “In-memory acceleration of hyperdimensional genome matching on unreliable emerging technologies,” IEEE Transactions on Circuits and Systems I: Regular Papers, 2024.
  25. S. D. Spetalnick, M. Chang, B. Crafton, W.-S. Khwa, Y.-D. Chih, M.-F. Chang, and A. Raychowdhury, “A 40nm 64kb 26.56 tops/w 2.37 mb/mm 2 rram binary/compute-in-memory macro with 4.23 x improvement in density and >>> 75% use of sensing dynamic range,” in 2022 IEEE International Solid-State Circuits Conference (ISSCC), vol. 65.   IEEE, 2022, pp. 1–3.
  26. F. Zhou and Y. Chai, “Near-sensor and in-sensor computing,” Nature Electronics, vol. 3, no. 11, pp. 664–671, 2020.
  27. S. Yu, X. Guan, and H.-S. P. Wong, “On the switching parameter variation of metal oxide rram—part ii: Model corroboration and device design strategy,” IEEE Transactions on Electron Devices, vol. 59, no. 4, pp. 1183–1188, 2012.
  28. D. Bankman, L. Yang, B. Moons, M. Verhelst, and B. Murmann, “An always-on 3.8μj/\mu j/italic_μ italic_j /86% cifar-10 mixed-signal binary cnn processor with all memory on chip in 28-nm cmos,” IEEE Journal of Solid-State Circuits, vol. 54, no. 1, pp. 158–172, 2018.
  29. R. Swaminathan, “Advanced packaging: Enabling moore’s law’s next frontier through heterogeneous integration,” in IEEE Hot Chip Conference, 2021, pp. 22–24.
  30. X. Peng, W. Chakraborty, A. Kaul, W. Shim, M. S. Bakir, S. Datta, and S. Yu, “Benchmarking monolithic 3d integration for compute-in-memory accelerators: overcoming adc bottlenecks and maintaining scalability to 7nm or beyond,” in 2020 IEEE International Electron Devices Meeting (IEDM).   IEEE, 2020, pp. 30–4.
  31. X. Peng, S. Huang, H. Jiang, A. Lu, and S. Yu, “Dnn+ neurosim v2. 0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 11, pp. 2306–2319, 2020.
  32. UVA HotSpot, “Hotspot 6.0,” https://lava.cs.virginia.edu/HotSpot/, 2019.
  33. Z. Fang, H. Yu, W. Liu, Z. Wang, X. Tran, B. Gao, and J. Kang, “Temperature instability of resistive switching on hfox-based rram devices,” IEEE Electron Device Letters, vol. 31, no. 5, pp. 476–478, 2010.
  34. C. Zhang, F. Gao, B. Jia, Y. Zhu, and S.-C. Zhu, “Raven: A dataset for relational and analogical visual reasoning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 5317–5327.
  35. Z. Wan, K. Swaminathan, P.-Y. Chen, N. Chandramoorthy, and A. Raychowdhury, “Analyzing and improving resilience and robustness of autonomous systems,” in Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, 2022, pp. 1–9.
  36. Y.-S. Hsiao, Z. Wan, T. Jia, R. Ghosal, A. Mahmoud, A. Raychowdhury, D. Brooks, G.-Y. Wei, and V. J. Reddi, “Silent data corruption in robot operating system: A case for end-to-end system-level fault analysis using autonomous uavs,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023.
  37. Z. Wan, N. Chandramoorthy, K. Swaminathan, P.-Y. Chen, V. J. Reddi, and A. Raychowdhury, “Berry: Bit error robustness for energy-efficient reinforcement learning-based autonomous systems,” in 2023 60th ACM/IEEE Design Automation Conference (DAC).   IEEE, 2023, pp. 1–6.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now