Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Cognitive AI Systems: a Survey and Prospective on Neuro-Symbolic AI (2401.01040v1)

Published 2 Jan 2024 in cs.AI and cs.AR

Abstract: The remarkable advancements in AI, primarily driven by deep neural networks, have significantly impacted various aspects of our lives. However, the current challenges surrounding unsustainable computational trajectories, limited robustness, and a lack of explainability call for the development of next-generation AI systems. Neuro-symbolic AI (NSAI) emerges as a promising paradigm, fusing neural, symbolic, and probabilistic approaches to enhance interpretability, robustness, and trustworthiness while facilitating learning from much less data. Recent NSAI systems have demonstrated great potential in collaborative human-AI scenarios with reasoning and cognitive capabilities. In this paper, we provide a systematic review of recent progress in NSAI and analyze the performance characteristics and computational operators of NSAI models. Furthermore, we discuss the challenges and potential future directions of NSAI from both system and architectural perspectives.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Pyreason: Software for open world temporal logic. arXiv preprint arXiv:2302.13482, 2023.
  2. Logic tensor networks. Artificial Intelligence, 303:103649, 2022.
  3. Thinking fast and slow in ai. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  15042–15046, 2021.
  4. Improving compute in-memory ecc reliability with successive correction. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pp.  745–750, 2022.
  5. Bridging machine learning and logical reasoning by abductive learning. Advances in Neural Information Processing Systems, 32, 2019.
  6. Neural logic machines. In International Conference on Learning Representations, 2019.
  7. Explainable ai (xai): Core ideas, techniques, and solutions. ACM Computing Surveys, 55(9):1–33, 2023.
  8. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1–64, 2018.
  9. Neurosymbolic ai: The 3 rd wave. Artificial Intelligence Review, pp.  1–20, 2023.
  10. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv preprint arXiv:1905.06088, 2019.
  11. Neural-symbolic learning and reasoning: a survey and interpretation. Neuro-Symbolic Artificial Intelligence: The State of the Art, 342(1), 2022.
  12. A neuro-vector-symbolic architecture for solving raven’s progressive matrices. Nature Machine Intelligence, pp.  1–13, 2023.
  13. Ontology reasoning with deep neural networks. Journal of Artificial Intelligence Research, 68:503–540, 2020.
  14. Stratified rule-aware network for abstract visual reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  1567–1574, 2021.
  15. Kaut, H. Robert s. engelmore memorial lecture at aaai 2020. https://roc-hci.com/announcements/the-third-ai-summer/, 2020.
  16. Roofline model for uavs: A bottleneck analysis tool for onboard compute characterization of autonomous unmanned aerial vehicles. In 2022 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp.  162–174. IEEE, 2022a.
  17. Automatic domain-specific soc design for autonomous unmanned aerial vehicles. In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), pp.  300–317. IEEE, 2022b.
  18. Graph neural networks meet neural-symbolic computing: A survey and perspective. In IJCAI-PRICAI 2020-29th International Joint Conference on Artificial Intelligence-Pacific Rim International Conference on Artificial Intelligence, 2020.
  19. Deep learning for symbolic mathematics. In International Conference on Learning Representations, 2019.
  20. Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 298:103504, 2021.
  21. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations, 2019.
  22. Introducing PyTorch Profiler - the new and improved performance tool, 2021. https://pytorch.org/blog/introducing-pytorch-profiler-the-new-and-improved-performance-tool/, accessed 2021-05-21.
  23. Neupsl: Neural probabilistic soft logic. arXiv preprint arXiv:2205.14268, 2022.
  24. Logical neural networks. arXiv preprint arXiv:2006.13155, 2020.
  25. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017.
  26. Neuro-symbolic ai: An emerging class of ai workloads and their characterization. arXiv preprint arXiv:2109.06133, 2021.
  27. Analyzing and improving fault tolerance of learning-based navigation systems. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pp.  841–846. IEEE, 2021.
  28. Robotic computing on fpgas: Current progress, research challenges, and opportunities. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp.  291–295. IEEE, 2022.
  29. Towards data-and knowledge-driven artificial intelligence: A survey on neuro-symbolic computing. arXiv preprint arXiv:2210.15889, 2022.
  30. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795–813, 2022.
  31. Neurasp: Embracing neural networks into answer set programming. In 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), 2020.
  32. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems, 31, 2018.
  33. Clevrer: Collision events for video representation and reasoning. In International Conference on Learning Representations, 2020.
  34. Raven: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5317–5327, 2019.
  35. Alphazero. Deep Reinforcement Learning: Fundamentals, Research and Applications, pp.  391–415, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Zishen Wan (33 papers)
  2. Che-Kai Liu (10 papers)
  3. Hanchen Yang (9 papers)
  4. Chaojian Li (34 papers)
  5. Haoran You (33 papers)
  6. Yonggan Fu (49 papers)
  7. Cheng Wan (48 papers)
  8. Tushar Krishna (87 papers)
  9. Yingyan Lin (67 papers)
  10. Arijit Raychowdhury (51 papers)
Citations (11)
View on Semantic Scholar

Summary

Towards Cognitive AI Systems: A Survey and Prospective on Neuro-Symbolic AI

This paper provides a comprehensive review and prospective analysis of the advancements and challenges within the domain of Neuro-symbolic AI (NSAI). As the landscape of AI continues to expand, the integration of neural networks and symbolic reasoning promises to address critical issues related to explainability, robustness, data efficiency, and energy consumption.

Overview of Neuro-Symbolic AI

The authors begin by framing NSAI as an interdisciplinary paradigm that combines neural, symbolic, and probabilistic techniques to further the capabilities of current AI systems. While deep neural networks have shown significant promise in tasks like natural language understanding and perception, they often demand vast computational resources and offer limited interpretability. On the other hand, symbolic methods, which provide logical reasoning capabilities, can potentially enhance AI's transparency and efficiency by leveraging models grounded in the physical world. Probabilistic methods supplement these approaches, improving systems' ability to manage uncertainty and thus enhancing robustness.

Analysis of NSAI Algorithms

The paper categorically reviews recent NSAI algorithms, organizing them according to their integration strategies. Five distinct paradigms emerge from this categorization: Symbolic[Neuro], Neuro∣|Symbolic, Neuro:Symbolic→\rightarrowNeuro, $\mbox{Neuro_{\mbox{Symbolic}$, and Neuro[Symbolic]. These paradigms represent various integrations of neural and symbolic components with specific examples like DeepMind's AlphaGo, IBM's neuro-vector-symbolic architecture, and logical neural networks.

Each paradigm demonstrates unique strengths in leveraging symbolic reasoning and neural learning. Neuro∣|Symbolic systems, for instance, show promise by effectively combining neural preprocessing with symbolic reasoning in a sequential pipeline, while $\mbox{Neuro_{\mbox{Symbolic}$ models map symbolic logic onto embeddings, blending logical rigor with the flexibility of neural approaches.

System Profiling and Computational Insights

The paper proceeds by profiling the performance of several NSAI models, highlighting how symbolic workloads can become a bottleneck due to their computational intensity and lack of parallelism. This profiling emphasizes the importance of understanding workload characteristics, such as compute operators and memory access patterns, to enhance system efficiency. Key findings show that symbolic processes, when dominated by sequential rule detection, demand significant computation time, underpinning the need for optimized dataflows and parallelization strategies.

Challenges and Opportunities

Future research directions highlighted by the authors focus on developing unified models that cohesively incorporate neural, symbolic, and probabilistic components. The creation of standardized, challenging datasets similar to ImageNet is essential for benchmarking and advancing NSAI capabilities in cognitive tasks.

Moreover, the paper emphasizes the need for efficient software frameworks that support diverse reasoning logics and provide modularity and extensibility. Novel cognitive hardware architectures are also necessary to address the diverse and complex computational requirements of NSAI systems. Such architectures would need to incorporate specialized processing units, adaptable interconnects, and comprehensive memory strategies.

Conclusion

The synthesis of neural, symbolic, and probabilistic approaches presents a promising avenue for developing the next generation of AI systems capable of cognitive functionalities akin to human reasoning. This paper sets a valuable foundation for further exploration within NSAI, encouraging continued innovation and interdisciplinary collaboration to overcome existing challenges and leverage the full potential of these systems.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube