Persistence of geometric grokking

Establish whether geometric grokking persists as task complexity increases and whether the phenomenon is partly an artifact of the particular arithmetic tasks and interface choices used in the experiments.

Background

The paper uses “geometric grokking” for the emergence of regular, interpretable space-time mechanisms in EM43 that remain valid beyond the training range. The authors explicitly identify as unresolved whether this behavior survives more complex tasks or depends primarily on the selected tasks and interface.

References

Does the $D{-2}$ exponent generalize to other group-structured tasks, or is it specific to modular arithmetic?

— Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking  (2609.10657 - Kataria, 9 Sep 2026) in Paragraph "Open questions," Section 8 (Discussion)

Whether geometric grokking persists as task complexity increases, and whether it is partly an artifact of the particular tasks and interface choices employed, remains to be established.

— Emergent Models: Intelligence from Tiny Substrates  (2608.14019 - Bocchese et al., 14 Aug 2026) in Conclusion, paragraph beginning “Among the individual results”