Do sequence-trained game models encode formal rules?

Determine whether superhuman-level game-playing foundation models trained purely on move sequences (for example, transformer-based models trained on chess or Othello game records) encode the formal rules of their respective games in their internal representations, rather than achieving high performance solely through statistical pattern-matching that does not reflect the underlying rule structure.

Background

The paper discusses cases where models achieve impressive performance without necessarily capturing underlying mechanisms, highlighting shortcut learning and failures to generalize. In the context of game-playing, models trained only on sequences of moves have been argued to exhibit superhuman play, prompting debate about whether such performance implies an internalization of the game's rules or reliance on heuristics and correlations.

This uncertainty is framed as part of a broader question about whether strong predictive capabilities indicate true mechanistic understanding. References to studies on Othello and chess-trained models suggest mixed evidence, with performance degrading under shifts outside training conditions, reinforcing the need to test for rule-encoding versus pattern-matching.