Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc (2308.04445v1)

Abstract: Generative AI, the most popular current approach to AI, consists of LLMs that are trained to produce outputs that are plausible, but not necessarily correct. Although their abilities are often uncanny, they are lacking in aspects of reasoning, leading LLMs to be less than completely trustworthy. Furthermore, their results tend to be both unpredictable and uninterpretable. We lay out 16 desiderata for future AI, and discuss an alternative approach to AI which could theoretically address many of the limitations associated with current approaches: AI educated with curated pieces of explicit knowledge and rules of thumb, enabling an inference engine to automatically deduce the logical entailments of all that knowledge. Even long arguments produced this way can be both trustworthy and interpretable, since the full step-by-step line of reasoning is always available, and for each step the provenance of the knowledge used can be documented and audited. There is however a catch: if the logical language is expressive enough to fully represent the meaning of anything we can say in English, then the inference engine runs much too slowly. That's why symbolic AI systems typically settle for some fast but much less expressive logic, such as knowledge graphs. We describe how one AI system, Cyc, has developed ways to overcome that tradeoff and is able to reason in higher order logic in real time. We suggest that any trustworthy general AI will need to hybridize the approaches, the LLM approach and more formal approach, and lay out a path to realizing that dream.

Bridging the Gap Between Generative and Trustworthy AI: Insights from Cyc

The paper "Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc" by Doug Lenat and Gary Marcus presents a critical examination of the limitations inherent in current Generative AI approaches and proposes a hybridized methodology combining the strengths of LLMs with systems like Cyc that depend on symbolic reasoning. The paper sets forth 16 desiderata crucial for developing a trustworthy AI, moving beyond the superficially fluent generative models toward a more reliable form of artificial reasoning.

Traditional LLMs, such as OpenAI's ChatGPT and Google's BARD, leverage massive data-driven learning to create outputs reflective of the patterns observed in the training data. However, these systems are often marred by issues of trustworthiness, unpredictability, and interpretability. The authors illustrate these deficiencies by pointing to several examples where LLMs provide results that are either unsubstantiated or imply incorrect logic. The primary critique lies in the LLMs' inability to perform rigorous reasoning and handle detailed world models effectively. In contrast, humans possess innate abilities for knowledge integration and deliberative reasoning akin to Kahneman's System 2, which LLMs lack.

The authors explore an alternative represented by Cyc, an AI system that has pursued explicit representation of knowledge and reasoning via higher-order logic since the 1980s. Cyc's methodology involves curating axioms of common sense and employing a logical inference engine to ensure each conclusion is meticulously documented. This enables Cyc to verify assertions, trace the origin of its reasoning, and operate under a structured logic. However, this expressiveness comes with a computational trade-off, as higher-order logic can pose scalability challenges due to its computational intensity.

One of the paper's primary contributions is articulating a vision for unifying the intuitive data-processing strengths of LLMs with the deductive reasoning capabilities of systems like Cyc. By harnessing LLMs' breadth and speed and Cyc's scrutinized and auditable reasoning processes, the authors propose a robust path forward to creating AI that is simultaneously knowledgeable, fast, and trustworthy.

Key Discussions and Implications:

1. Sixteen Desiderata for Trustworthy AI: The authors introduce 16 capabilities indicative of trustworthy AI, including explainability, deduction, induction, theory of mind, and defeasibility. These form a comprehensive framework against which current AI systems fall short. The iterative breakdown of these requirements highlights both the complexity and the necessity of ethical AI systems capable of handling nuanced reasoning tasks.
2. Symbolic vs. Generative Approaches: Cyc exemplifies a symbolic framework featuring higher-order logic and a richly populated knowledge base. This contrasts with LLMs' probabilistic methods, which enable rapid responses but often lack depth in understanding. The Cyc system overcomes expressiveness/efficiency dilemmas through methodological advancements such as heuristic-level reasoning modules to supplement general theorem proving.
3. Potential Synergies: The paper emphasizes opportunities for synergy between LLMs and Cyc. Symbolic systems could mitigate inaccuracies in LLM outputs by critically evaluating and confirming claims. Conversely, LLMs might help augment symbolic systems through natural language processing capabilities, bringing in expressive flexibility and generating real-time semantic inputs.

Conclusion and Speculation on Future AI Developments:

The proposed convergence of generative and symbolic AI could provide substantive advancements in the quest for a general, trustworthy AI. A hybrid approach, integrating the heuristic advantages and scalability of LLMs with Cyc's rigorous inferential logic, may be pivotal in crafting AI systems that are both adept and auditable. This synthesis could pave the way for AI applications in sensitive domains such as healthcare, where decision-making necessitates reliability and accountability.

This paper lays down a substantial foundational context and intellectual roadmap, challenging researchers to explore innovative integrations of disparate AI methodologies. The synthesis it describes could contribute meaningfully to the trajectory towards trusted, universally applicable AI, a cornerstone for future AI endeavors. The research community is thus called upon to pursue such hybridization initiatives as a pragmatic strategy for addressing trust, reliability, and operational efficiency in next-generation AI systems.