Conditions and mechanisms for transitions from FER to UFR

Determine the timing and mechanisms by which training induces transitions from fractured entangled representations to unified factored representations in neural networks, identifying specific processes (e.g., grokking) and the circumstances under which such unification occurs.

Background

The authors note that improved behavior at scale may reflect better coverage without representational unification, or true consolidation into UFR. They suggest phenomena like grokking could mediate such transitions but emphasize that the onset and pathways of unification are not understood.

Clarifying when and how models move from FER to UFR is crucial for designing training regimes that proactively promote unified, modular representations rather than relying on incidental or data-heavy cleanup effects.