Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity & Creativity (0709.0674v1)

Abstract: I postulate that human or other intelligent agents function or should function as follows. They store all sensory observations as they come - the data is holy. At any time, given some agent's current coding capabilities, part of the data is compressible by a short and hopefully fast program / description / explanation / world model. In the agent's subjective eyes, such data is more regular and more "beautiful" than other data. It is well-known that knowledge of regularity and repeatability may improve the agent's ability to plan actions leading to external rewards. In absence of such rewards, however, known beauty is boring. Then "interestingness" becomes the first derivative of subjective beauty: as the learning agent improves its compression algorithm, formerly apparently random data parts become subjectively more regular and beautiful. Such progress in compressibility is measured and maximized by the curiosity drive: create action sequences that extend the observation history and yield previously unknown / unpredictable but quickly learnable algorithmic regularity. We discuss how all of the above can be naturally implemented on computers, through an extension of passive unsupervised learning to the case of active data selection: we reward a general reinforcement learner (with access to the adaptive compressor) for actions that improve the subjective compressibility of the growing data. An unusually large breakthrough in compressibility deserves the name "discovery". The "creativity" of artists, dancers, musicians, pure mathematicians can be viewed as a by-product of this principle. Several qualitative examples support this hypothesis.

• The paper proposes that subjective beauty, curiosity, and creativity in intelligent agents arise from the drive to improve the compressibility of sensory data based on algorithmic information theory.
• It introduces a curiosity drive where agents seek data that maximizes the improvement in their ability to compress new observations, linking discovery to compression breakthroughs.
• The research outlines a computational framework based on reinforcement learning, rewarding agents internally for progress made in data compression, with implications for developing more autonomous and creative AI systems.

Overview of Simple Algorithmic Principles of Discovery

The paper, authored by Jürgen Schmidhuber, explores how intelligent agents, including humans, should approach the management of sensory data and learning. It introduces a framework where all sensory data is treated as valuable and subject to compression based on algorithmic regularity and simplicity. The fundamental hypothesis suggests that data segments that display repetitive patterns or regularity are considered "beautiful," following principles of algorithmic information theory. These ideas are extended to cover the concepts of curiosity and creativity, positing that agents should actively seek data that improves the compressibility of their sensory histories.

Key Concepts and Theoretical Contributions

The paper is structured around the fundamental idea of optimizing the compressibility of observed data, which is linked to the intrinsic motivation of curiosity. It argues that curiosity-driven learning arises from the desire to transform seemingly random data into structured and compressible patterns.

1. Data Compression as Regularity Detection: At any time, an agent encounters data that can be compressed into shorter, more efficient representations using algorithmic techniques. This compression not only reveals the order within the data but also contributes to its perceived "beauty."
2. Curiosity as a Driving Force: The paper introduces the concept of a curiosity drive which encourages agents to execute actions that explore environments rich in unknown and learnable regularities. Curiosity is quantified in terms of how much an agent improves its ability to compress new data, essentially maximizing the "first derivative" of subjective beauty.
3. Discovery through Compression Breakthroughs: Schmidhuber characterizes significant improvements in data compressibility as discoveries. The act of discovering involves the creation of sequences that extend historical observations into new regularities, providing a deeper understanding of the observed environment.
4. Implications for Creativity: Creativity in artists, musicians, and scientists is identified as a by-product of the quest for compressible data patterns. The paper argues that artistic and scientific processes are motivated by the potential to uncover new patterns that yield compressibility.

Computational Framework

The research proposes a computational architecture where agents optimize across their own limitations. Here, agents actively select actions to increase overall utility by gaining external rewards or improving data compressibility. This framework, built on reinforcement learning principles, leverages improvements in data compression to guide agents.

1. Reinforcement Using Compression Progress: Internal rewards are associated with improvements in data compression capability. Agents are reinforced using these intrinsic rewards to adapt behavior towards discovering compressible patterns.
2. Optimal Action Selection: The paper discusses theoretically optimal reinforcement learning methods capable of dealing with infinite or asymptotic action spaces. Methods like Solomonoff induction and G\"{o}del Machine implementations are presented as theoretically sound approaches for optimal exploration.

Practical and Theoretical Implications

The work has profound implications for future developments in AI, particularly in designing systems that emulate human curiosity and creativity. By equipping AI with a framework that rewards discovery through compressibility, it hints at developing more autonomous systems capable of intricate pattern recognition and creativity beyond externally defined objectives.

Speculation on Future Developments: The research prescribes a direction where AI systems not only optimize externally provided tasks but also showcase intrinsic motivation to explore novel, unexplored data landscapes. As computational resources grow, the feasibility of implementing these architectures in real-world applications becomes more tangible, potentially revolutionizing fields such as autonomous reasoning and creative problem-solving.

In conclusion, Schmidhuber's work emphasizes the importance of subjective beauty and discovery in intelligent systems. By tightly coupling curiosity with data compression efficiencies, the paper provides a compelling framework to understand and harness creativity in artificial agents.