Intelligence from Learnable Novelty

Abstract: Intelligence appears under different names in different fields: as data compression in statistics and machine learning, as universal computation in dynamical systems, and as adaptive behavior in agents. Each field carries its own objective, and the two most influential drives often fail in mirror image: novelty search, which seeks surprise, is transfixed by a noisy television screen, while the free-energy principle, which avoids surprise, is most content in a dark room. Both failures have a single cause: each objective treats as one quantity the surprise a learner can convert into knowledge and the surprise it never can. Here we show that the learnable part of that information, which we call learnable novelty, yields the seemingly disparate projections of intelligence, and we give a closed-form estimator of it built on a cheap and differentiable reservoir computer. Used as a measure, with no supervision of any kind, the estimator recovers decades of complexity classification, ranking the Turing-complete rule~110 highest among the elementary cellular automata. Used as an objective, its gradient carries a neural cellular automaton from simple dynamics into a regime of solitons, the traveling, colliding structures by which rule~110 computes, as well as organizes the representation of an image encoder around the ten digit classes of MNIST, fully unsupervised: no label ever enters training. Handed to a reinforcement-learning agent as an intrinsic reward, it supplies the exploration that task rewards lack, improving on the task baseline in nine of ten environments and collapsing in none. Complexity generation, abstraction, and exploration, ordinarily pursued with unrelated objectives in separate fields, thus emerge from ascent on one differentiable quantity, and the projections of intelligence gain a common quantitative footing.

• The paper introduces learnable novelty as a novel measure, epiplexity, that distinguishes learnable from unlearnable surprise in data.
• It applies the concept to cellular automata, re-ranking rule complexity and evolving simple rules into complex, soliton-rich patterns using gradient ascent.
• The approach boosts representation and reinforcement learning by promoting self-organized clustering and enhanced exploration without external supervision.

Intelligence from Learnable Novelty

Introduction

The concept of intelligence traverses several disciplines, manifesting through diverse interpretations such as data compression in statistics and machine learning, universal computation in dynamical systems, and adaptive behavior in agent-environment interactions. These interpretations often carry unique objectives and reveal limitations when pursued independently, including fixation with unpredictability or overly predictable scenarios. This paper introduces the concept of learnable novelty, aiming to unify these manifestations by focusing on the aspect of information that can genuinely be learned.

Learnable Novelty and Epiplexity

The exploration of novelty in data presents challenges where existing strategies like novelty search and the free-energy principle fail. The former seeks unpredictability but often settles on noise, while the latter avoids surprise, leading to stagnation. Learnable novelty provides a refined objective by distinguishing between the learnable and unlearnable components of surprise in data. This results in a measure called epiplexity, calculated via a differentiable estimator leveraging reservoir computing.

Figure 1: (a) Learnable novelty and (b) its application as an objective, reshaping various systems.

Applications and Experiments

Dynamical Systems: The evaluation of elementary cellular automata (ECA) using epiplexity as a measure accurately ranks the complexity of the rules, with the Turing-complete rule 110 obtaining the highest score among the set due to its capacity for universal computation.

Figure 2: $S^\phi$ scores for different ECA rules, highlighting rule 110.

Inverse Design in Cellular Automata: Gradient ascent maximization of learnable novelty demonstrated the evolution of cellular automaton rules from straightforward dynamics to displaying complex patterns like solitons.

Figure 3: Evolution of neural cellular automata showing development of solitons.

Representation Learning: Applying learnable novelty on an encoder transforming MNIST data exhibited self-organized clustering according to digit labels without any supervised labels, enhancing representation utility naturally.

Figure 4: Progression of representation clustering for MNIST data over training.

Reinforcement Learning: When integrated as an intrinsic reward, learnable novelty significantly enhanced agent performance across multiple environments by improving exploration tendencies beyond those achievable through task-focused rewards alone.

Discussion and Conclusion

The introduced framework of learnable novelty unifies seemingly disparate aspects of intelligence under a single measurable objective, providing a consistent foundation for assessing and inducing intelligent behavior. This approach not only yields predictive accuracy and efficiency in learning tasks but also fosters innovative patterns and useful representations without necessitating external supervision. The implications are broad, suggesting further exploration into observer-centric models of intelligence where systems evolve in conjunction with the bounded learning capacity of their observers.

In conclusion, learnable novelty offers a robust metric to drive complexity generation, abstraction, and adaptive exploration. Future research may focus on incorporating evolving observers into this framework and exploring the mutual development of observer-system dynamics in more open-ended environments.