Intelligence from Predicting Complexity

Establish whether intelligent behavior in artificial systems arises from the ability to predict complexity and whether creating intelligence requires only exposure to complex, hard-to-predict dynamics, as suggested by models trained on data generated by elementary cellular automata.

Background

The paper trains modified GPT-2 models on datasets generated by individual elementary cellular automata (ECAs) and evaluates their performance on downstream reasoning and chess move prediction tasks. The authors observe that models pretrained on rules producing higher complexity exhibit greater downstream performance, with a sweet spot near the “edge of chaos.”

Motivated by these empirical findings, the authors explicitly conjecture a broader principle: that intelligence emerges from the capacity to predict complexity and that creating intelligence may require only exposure to complex behaviors, even when the data-generating process itself is not intelligent.