Amortized Planning with Large-Scale Transformers: A Case Study on Chess (2402.04494v2)

Abstract: This paper uses chess, a landmark planning problem in AI, to assess transformers' performance on a planning task where memorization is futile $\unicode{x2013}$ even at a large scale. To this end, we release ChessBench, a large-scale benchmark dataset of 10 million chess games with legal move and value annotations (15 billion data points) provided by Stockfish 16, the state-of-the-art chess engine. We train transformers with up to 270 million parameters on ChessBench via supervised learning and perform extensive ablations to assess the impact of dataset size, model size, architecture type, and different prediction targets (state-values, action-values, and behavioral cloning). Our largest models learn to predict action-values for novel boards quite accurately, implying highly non-trivial generalization. Despite performing no explicit search, our resulting chess policy solves challenging chess puzzles and achieves a surprisingly strong Lichess blitz Elo of 2895 against humans (grandmaster level). We also compare to Leela Chess Zero and AlphaZero (trained without supervision via self-play) with and without search. We show that, although a remarkably good approximation of Stockfish's search-based algorithm can be distilled into large-scale transformers via supervised learning, perfect distillation is still beyond reach, thus making ChessBench well-suited for future research.

• The paper demonstrates that a transformer trained on millions of annotated chess positions can achieve grandmaster-level play without traditional search algorithms.
• It systematically analyzes the effects of model size and dataset scale on generalization, offering detailed ablation studies and scaling insights.
• It reveals that learned action-values suffice for complex chess strategy, showcasing the potential of neural approximators in replacing explicit search methods.

Introduction

The breakthroughs in AI over the past few years have been driven by the application of large-scale models and massive datasets. This paper investigates the application of such paradigms to the game of chess, traditionally dominated by engines using deep search algorithms and large databases of heuristics. Through the lens of supervised learning, the authors of the paper hypothesize that it is possible to achieve strong chess performance purely from learned action-values, without the traditional explicit search algorithms.

Methodology

The team constructed a dataset by extracting and annotating 10 million games from Lichess with action-values sourced from the elite chess engine Stockfish 16. A transformer model with 270 million parameters was then trained to predict win probabilities for given chess board positions. The researchers systematically studied the effects of varying model sizes and dataset scales to understand their impact on generalization and chess performance.

Results and Contributions

The model's prowess is commendable, with a Lichess blitz Elo of 2895, which situates it in the grandmaster echelon. It also outperforms AlphaZero's policy and value networks—sans the Monte Carlo Tree Search—and GPT-3.5-turbo-instruct over a select set of metrics, solidifying the notion of neural predictors as strong generalizers. Key contributions include:

• A neural predictor trained on millions of annotated board states capturing Stockfish 16's expertise.
• A grandmaster-level policy derived from this predictor, capable of tackling challenging chess puzzles, without any domain-specific enhancements or search algorithms.
• Demonstrated significance of scale for achieving robust generalization and performance in chess, accompanied by a detailed analysis through rigorous ablations and scaling studies.

Discussion and Implications

While achieving overwhelming success, the paper acknowledges the limitations of lacking historical context which traditional engines use for strategic planning, exposing minor weaknesses. These were pragmatically mitigated through workarounds that, however, do not imply a search algorithm. Furthermore, the paper clarifies the scale needed for similar neural predictors to bridge the gap between current model performance and oracle-powered engines like Stockfish 16.

The work underlines a shift in AI research, suggesting that highly complex algorithmic reasoning—once deemed exclusive to sophisticated search-based systems—can be distilled into transformer models using supervised learning. As such, transformers are not simply pattern recognition systems, but powerful approximators capable of substituting intricate algorithmic processes. The implications for future research are profound, promising further shifts in how AI is applied across varied cognitive domains.