MetaOthello: A Controlled Study of Multiple World Models in Transformers

Abstract: Foundation models must handle multiple generative processes, yet mechanistic interpretability largely studies capabilities in isolation; it remains unclear how a single transformer organizes multiple, potentially conflicting "world models". Previous experiments on Othello playing neural-networks test world-model learning but focus on a single game with a single set of rules. We introduce MetaOthello, a controlled suite of Othello variants with shared syntax but different rules or tokenizations, and train small GPTs on mixed-variant data to study how multiple world models are organized in a shared representation space. We find that transformers trained on mixed-game data do not partition their capacity into isolated sub-models; instead, they converge on a mostly shared board-state representation that transfers causally across variants. Linear probes trained on one variant can intervene on another's internal state with effectiveness approaching that of matched probes. For isomorphic games with token remapping, representations are equivalent up to a single orthogonal rotation that generalizes across layers. When rules partially overlap, early layers maintain game-agnostic representations while a middle layer identifies game identity, and later layers specialize. MetaOthello offers a path toward understanding not just whether transformers learn world models, but how they organize many at once.

• The paper introduces MetaOthello, a controlled suite of Othello variants, and finds that pure and mixed models achieve α > 0.98, with mixed training reducing performance by only about 0.5 percentage points.
• The paper shows that board-state representations remain causally transferable across rule systems, align up to an orthogonal rotation under token scrambling, and are not cleanly separated into isolated game-specific subspaces.
• The paper finds a layered arbitration mechanism in which early layers encode shared state, middle layers infer game identity, and later layers apply rules, with representational divergence increasing as the games’ predicted states conflict.

Motivation and scope

Mechanistic interpretability has produced strong evidence that sequence models trained on next-token prediction learn causally complete internal state representations ("world models"), most famously in Othello-playing transformers (Li et al., 2022), with linear probes sufficient for decoding and intervening on board states when tiles are classified as mine/yours/empty (Nanda et al., 2023). However, this line of work studies a single generative process in isolation. Foundation models must instead arbitrate among multiple, potentially conflicting rule systems within one set of weights, and it is not known how such systems organize multiple world models in a shared representation space.

The paper introduces MetaOthello, a controlled suite of Othello variants sharing an $8\times 8$ board but differing in validation rules ($V$), update rules ($U$), or tokenization (a syntax map $\phi_g$ from tokens to moves). Three variants anchor the study:

• NoMidFlip: identical to Classic except the update flips only the outermost two flanked tiles, producing high game-tree overlap.
• DelFlank: spread initialization, neighbor-based validity, and deletion of flanked pieces; its game tree is exponentially larger at every depth.
• Iago: computationally isomorphic to Classic but with an orthogonal token-to-move permutation, isolating surface syntax from latent structure.

Small GPT models (8 layers, 8 heads, $d_{\text{model}}=512$, context 59) are trained on pure datasets (20M sequences per game) or mixed pairwise datasets (40M total). Because standard loss metrics are incomparable across games with different branching factors, the paper proposes an $\alpha$ score—a normalized KL divergence measuring resolved excess entropy relative to uniform guessing—which reaches $\alpha > 0.98$ for all pure and mixed models, with mixed models losing roughly half a percentage point relative to their single-game counterparts. This establishes that multi-game training does not measurably degrade predictive competence before any representational analysis.

Cross-variant alignment of board-state representations

The central question is whether mixed-game transformers partition the residual stream into game-specific subspaces or converge on a shared format. Comparing linear probe weight matrices ($192 \times 512$ per layer, one row per tile-state feature) across variants yields three regimes. Classic versus NoMidFlip probes show raw cosine similarity near 0.95 in early and late layers, where Procrustes alignment adds little improvement. Classic versus DelFlank shows raw similarity around 0.67, rising above 0.90 across most layers after alignment—well beyond the Procrustes-aligned random baseline of roughly 0.68, confirming the correspondence reflects learned structure rather than high-dimensional coincidence.

Crucially, geometric similarity is supplemented by causal testing. Using the intervention protocol of prior Othello work, the authors add scaled probe vectors $h_l(s) \rightarrow h_l(s) + \gamma w^l_{i,c}$ across all eight layers to flip or erase tile states. Cross-probe interventions—using a probe trained on one variant to steer activations during the other's sequences—are nearly as effective as matched probes, even for Classic–DelFlank where raw similarity is only moderate. This dissociation between representational similarity and causal efficacy supports the claim that board states are encoded in a largely shared linear format reused across rule systems, consistent with the "Platonic representation" conjecture (Huh et al., 2024).

A caveat applies here: interventions use a fixed scaling factor $\gamma = 5$ applied at all layers simultaneously, which the authors note is unoptimized and may over-intervene; potential compensation effects of the kind documented in self-repairing LLM circuits (McGrath et al., 2023) remain unexamined.

Syntax invariance under the Iago experiment

For the Iago pair, raw cosine similarity between Classic and scrambled-token probes is approximately 0.03—indistinguishable from random—yet after per-layer orthogonal Procrustes alignment similarity reaches 0.98 against a random-alignment baseline of 0.68. The transformer thus learns identical latent structure up to a rotation induced by input vocabulary permutation.

The result extends beyond probes to the residual stream itself. Estimating a single global orthogonal transformation $V$0 on pooled activations from all layers and positions, then applying it at an intervention layer while feeding Classic sequences, recovers near-baseline Iago performance at every layer except the last; without the rotation, Iago move prediction on Classic inputs scores $V$1, i.e., confidently wrong. The implication is that the abstract state geometry is syntax-invariant and recoverable by an approximately orthogonal map. The authors are careful about interpretation: this demonstrates representational isomorphism but does not establish whether the model maintains one shared world model or two distinct instantiations with similar structure—an ambiguity that recurs in the ambiguity analysis below.

Handling ambiguous sequences

Mixed training creates genuinely conflicting supervision: many sequences $V$2 are valid under both games yet induce different board states and legal-move sets, so optimal cross-entropy prediction requires a posterior-weighted mixture over games. The paper dissects how the Classic–NoMidFlip model manages this.

Representational divergence tracks conflict probability. For ambiguous sequences, the probability that a given tile's value differs between the two games predicts the cosine dissimilarity between that tile's Classic and NoMidFlip representations, with correlation strengthening sharply across layers—from $V$3 at layer 5 to $V$4 at layer 6, reaching $V$5 at layer 8. Principal-angle analysis of each position's (mine, yours) subspaces gives qualitatively similar results ($V$6) across all layers. This is direct evidence of representational economization: the model allocates divergent capacity only where the worlds actually disagree, keeping "Empty" representations essentially identical since occupancy is game-invariant.

Layer 5 encodes game identity. Linear probes recover the Bayesian ground-truth posterior $V$7 with roughly 90% accuracy in layers 5–8, while layers 1–4 perform no better than a probe trained on raw one-hot move-and-position inputs. Injecting a steering vector $V$8 derived from these game-ID probes shifts behavior toward NoMidFlip-valid moves only at layer 5, with negligible effect at layers 1–4 and 6–8—and notably changes output behavior without degrading downstream NoMidFlip board representations.

Out-of-distribution arbitration in DelFlank. Because DelFlank's game tree dwarfs Classic's, ambiguous sequences beyond roughly move 12 are effectively absent from training data (of 20M sampled DelFlank sequences, fewer than one remains ambiguous past move 12), so maintaining a DelFlank board state there would be almost useless. On these sequences the DelFlank probe at layer 5 drops to about 67% accuracy versus 99% for Classic probes—the model has effectively selected the Classic world model. Early-layer steering (layers 1–3) with the DelFlank-minus-Classic identity vector produces a large shift toward DelFlank-consistent predictions ($V$9 early, declining to about 0.3 by move 30) and causally improves downstream DelFlank board-probe accuracy from 67.1% to 77.4% after layer-1 steering. Layer-5 steering, effective for NoMidFlip, has no effect here.

Together, these results identify a layered routing architecture: early layers maintain game-agnostic state, a middle layer performs game identification, and later layers apply game-specific rules—with the mechanism's locus depending on how sharply the two games' likelihoods diverge.

Limitations and open questions

The authors are explicit that findings are established in a synthetic setting far simpler than naturalistic multi-task corpora where rules are implicit and numerous. Only 50/50 mixtures, pairwise combinations, and a single 8-layer, $U$0 architecture are tested; imbalanced mixtures, scale, and many-way arbitration are untested. Each configuration is trained once with a fixed seed due to computational constraints. The analysis relies entirely on linear probes and additive interventions, so nonlinear mechanisms and compensation effects could underlie results that linear analysis attributes to shared representations. The central unresolved puzzle, stated plainly by the authors, is that board representations appear causally interchangeable across games yet still carry game-specific information sufficient for correct arbitration on ambiguous sequences—suggesting linear probing alone cannot fully explain the computation.

Conclusion

MetaOthello provides a controlled testbed in which ground-truth latent states are known, enabling rigorous validation of probing and causal-intervention methodology. Its substantive findings are that transformers trained on heterogeneous generative processes do not fragment into isolated sub-models: they converge on mostly shared board-state representations that transfer causally across rule variants, become exactly aligned up to a single orthogonal rotation under vocabulary permutation, and route through an identifiable middle-layer game-identity mechanism that specializes only where rules conflict. Whether these localized routing mechanisms persist at scale, under imbalanced task mixtures, and under nonlinear circuit-level scrutiny remains the concrete question the paper leaves open.