A theory of understanding for artificial intelligence: composability, catalysts, and learning (2408.08463v1)

Abstract: Understanding is a crucial yet elusive concept in AI. This work proposes a framework for analyzing understanding based on the notion of composability. Given any subject (e.g., a person or an AI), we suggest characterizing its understanding of an object in terms of its ability to process (compose) relevant inputs into satisfactory outputs from the perspective of a verifier. This highly universal framework can readily apply to non-human subjects, such as AIs, non-human animals, and institutions. Further, we propose methods for analyzing the inputs that enhance output quality in compositions, which we call catalysts. We show how the structure of a subject can be revealed by analyzing its components that act as catalysts and argue that a subject's learning ability can be regarded as its ability to compose inputs into its inner catalysts. Finally we examine the importance of learning ability for AIs to attain general intelligence. Our analysis indicates that models capable of generating outputs that can function as their own catalysts, such as LLMs, establish a foundation for potentially overcoming existing limitations in AI understanding.

• The paper introduces a framework based on composability for evaluating AI's ability to process diverse inputs and generate verified outputs.
• The paper demonstrates that catalysts, acting as internal or external enhancers, significantly boost AI understanding and enable autocatalytic learning.
• The paper evaluates existing AI systems, revealing limitations in universality and scale while outlining a path towards achieving general intelligence.

A Theory of Understanding for Artificial Intelligence: Composability, Catalysts, and Learning

The paper "A theory of understanding for artificial intelligence: composability, catalysts, and learning" by Zijian Zhang, Sara Aronowitz, and Alán Aspuru-Guzik presents a novel framework to assess and analyze understanding in AI systems. This framework is rooted in the idea of "composability" and introduces "catalysts" as instrumental components in the enhancement of understanding. The paper also emphasizes the significance of learning ability as a critical aspect of AI's path towards achieving general intelligence.

Core Concepts

Composability

The authors propose composability as the central mechanism for defining understanding. Composability pertains to a subject's ability to process inputs and generate outputs that a verifier deems appropriate. This framework allows for a versatile application across various entities, both human and non-human.

Catalyst

Catalysts are conceptualized as elements that enhance the subject's compositional processes, thereby improving the outputs. These catalysts can be internal (within the subject) or external. The paper suggests that crucial intellectual constructs such as explanations and theories function as catalysts.

Characterization of Understanding

The framework characterizes understanding through sets of input-output pairs assessed by a verifier. This characterization highlights two essential dimensions:

1. Universality: The capability of processing various types of inputs and producing diverse outputs.
2. Scale: The ability to handle inputs and outputs of varying magnitudes and complexities.

These dimensions are crucial in assessing the extent and depth of understanding in AI systems.

Subject Decomposition and Learning

The framework leverages subject decomposition to uncover the internal structure of understanding. By identifying inner catalysts and primitive subjects within a composite subject, one can gain insights into the improvement of understanding. The ability to produce and utilize inner catalysts is linked to the learning ability of the subject.

Analysis of Current AI Systems

The authors dissect current AI systems, particularly LLMs, using their proposed framework. They point out that, despite significant advancements, these systems showcase limitations in universality and scale.

Universality and Tool Use

A significant discussion revolves around the universality of LLMs, particularly in their ability to use tools, which positions them favorably compared to narrow AI models. This tool use is crucial in enabling LLMs to bridge various tasks and input types, enhancing their overall utility and perceived understanding.

Internal Learning and Autocatalysis

A key observation in the paper is the autocatalytic property of LLMs, wherein they can use their outputs as catalysts for further processing. This self-reinforcing feature is pivotal in advancing AI learning capabilities, moving closer to human-like learning processes.

Implications for General Intelligence

The learning ability of AI systems is emphasized as a vital component in achieving general intelligence akin to human cognition. The paper posits that this learning ability hinges on the AI's capacity to produce and apply new inner catalysts, driven by composability.

Future Directions

The paper suggests that the proposed framework itself acts as a catalyst for further research and development in AI understanding. It calls for deeper investigation into characteristics such as creativity and the formation of composite subjects, as well as the practical implementation of AI systems that leverage these concepts.

Conclusion

This work provides a comprehensive and practical framework for analyzing and enhancing understanding in AI systems. By focusing on composability, catalysts, and learning, the authors offer a robust methodology for future advancements towards AI with general intelligence. The insights derived from this framework can guide the development of more sophisticated and adaptable AI systems.