Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety (2312.06798v1)

Abstract: Explainability and Safety engender Trust. These require a model to exhibit consistency and reliability. To achieve these, it is necessary to use and analyze data and knowledge with statistical and symbolic AI methods relevant to the AI application - neither alone will do. Consequently, we argue and seek to demonstrate that the NeuroSymbolic AI approach is better suited for making AI a trusted AI system. We present the CREST framework that shows how Consistency, Reliability, user-level Explainability, and Safety are built on NeuroSymbolic methods that use data and knowledge to support requirements for critical applications such as health and well-being. This article focuses on LLMs as the chosen AI system within the CREST framework. LLMs have garnered substantial attention from researchers due to their versatility in handling a broad array of NLP scenarios. For example, ChatGPT and Google's MedPaLM have emerged as highly promising platforms for providing information in general and health-related queries, respectively. Nevertheless, these models remain black boxes despite incorporating human feedback and instruction-guided tuning. For instance, ChatGPT can generate unsafe responses despite instituting safety guardrails. CREST presents a plausible approach harnessing procedural and graph-based knowledge within a NeuroSymbolic framework to shed light on the challenges associated with LLMs.

• The paper presents the CREST framework that integrates consistency, reliability, explainability, and safety to boost AI trustworthiness.
• It outlines NeuroSymbolic methods using deep knowledge-enriched ensembles to stabilize outputs and improve LLM reliability.
• Empirical tests on the PRIMATE dataset demonstrate enhanced answerability and ethical alignment, especially in healthcare applications.

Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety

This academic paper introduces the CREST framework, a rigorous methodology centered on the development of trustworthy NeuroSymbolic AI systems. The framework emphasizes the four pillars of Consistency, Reliability, user-level Explainability, and Safety (CRES) in AI, demonstrating their critical role in enabling trust within AI systems, particularly LLMs. The authors adopt a NeuroSymbolic AI approach, leveraging both neuro-based statistical methods and symbolic knowledge integration to address inherent challenges in LLMs, such as those evidenced in tools like ChatGPT and Google's MedPaLM.

Key Contributions

The paper delineates the CREST framework, offering a comprehensive approach to integrating Consistency, Reliability, Explainability, and Safety into AI models. By concentrating on LLMs within the NeuroSymbolic AI paradigm, the authors aim to enhance trust, specifically for applications in health and well-being. The framework synthesizes several strategies for optimizing AI behavior, notably:

• Consistency: The authors stress the necessity for AI systems to generate stable outputs across different but semantically identical inputs. This is particularly pertinent to avoid adverse responses in sensitive domains, emphasizing the need for enhanced paraphrasing techniques and adversarial perturbation methods.
• Reliability: Achieving high confidence in AI outputs is paramount for sensitive applications. The authors advocate for deep knowledge-infused ensemble methods that surpass traditional statistical ensembling in capturing the nuances across various knowledge domains. This is instrumental in transitioning from monolithic LLMs to more robust ensembles.
• Explainability: The paper introduces the concept of User-level Explainability using evaluator modules alongside generative AI to offer domain-specific, actionable insights. This facilitates transparency in AI decision-making, which is especially critical when AI systems interact with healthcare professionals.
• Safety: Grounding, instructability, and alignment emerge as fundamental attributes of safe AI as per the CREST framework. The paper highlights the integration of domain-specific guidelines (e.g., clinical practice guidelines) to ensure outputs align with ethical and safe practices.

Practical Implications

The authors provide empirical validation of the framework through experiments on the PRIMATE dataset, highlighting notable improvements in metrics related to answerability and BLEURT over traditional LLMs like GPT 3.5. By emphasizing domain knowledge integration, particularly in mental health scenarios, the CREST framework demonstrates a path forward for extending AI applications into critical areas with higher stakes.

Theoretical Implications

The CREST framework suggests that the fusion of symbolic reasoning with statistical learning advances the frontier in AI trustworthiness. This amalgamation allows systems to grasp abstract concepts and maintain user trust by generating reliable and contextually relevant outputs. The authors contend that this synthesis allows AI systems to engage in anticipatory thinking and adapt more precisely to diverse scenarios.

Future Directions

The paper advocates for the continued exploration of knowledge-infused learning and more effective training methods for ensemble LLMs. Potential future research includes refining paraphrasing and adversarial generation techniques, further automating user-level explainability processes, and developing quantitative metrics to assess the CREST framework's suitability and effectiveness across broader domains.

In conclusion, the presented framework underscores a significant stride towards constructing AI systems that are not only effective but also inherently trustworthy, aligning closely with the ongoing demands for ethical and safe AI applications in critical fields such as healthcare.