Role of grokking in mitigating FER

Investigate whether and how grokking transitions in deep networks lead to holistic unification of learned representations and reduction of fractured entangled representations, including the conditions under which such mitigation occurs and its stability across tasks and architectures.

Background

The paper discusses the possibility that exposure to extensive data or late-stage training dynamics (grokking) might unify fractured representations into UFR. Prior work suggests grokking can involve cleanup of memorized circuits, but its reliability depends on factors like dataset size and weight decay.

This open question targets the mechanisms and preconditions by which grokking could systematically convert FER into UFR, and whether such effects generalize beyond synthetic settings to complex, multi-capability models.