Component attribution in Mixture-of-Experts looped models

Determine the separate contributions of the iteration envelope, input injection, and recurrent-state initialization in Mixture-of-Experts looped language models through matched ablations, including combinations of shared high/low recurrent states with expert routing and repeated route-freezing interventions across checkpoints and generation tasks.

Background

The MoE experiments compare complete Ouro-style and Huginn-style configurations, which simultaneously differ in recurrence placement, input injection, and state initialization. Consequently, the reported Huginn-versus-Ouro advantage does not fully identify which architectural components produce the observed effects in sparse models.

The authors explicitly leave separate MoE ablations, shared high/low-state combinations, and broader route-freezing tests unresolved.

References

The MoE study also leaves a component-attribution question. It compares complete Ouro- and Huginn-style configurations, which change the iteration envelope, input injection, and state initialization together. Matched MoE ablations should vary these axes separately. Further experiments could combine the shared H/L state with expert routing and repeat the route-freezing intervention across checkpoints and generation tasks.

— Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design  (2610.23036 - Huang et al., 16 Oct 2026) in Section 5, Future Work (Section 1? label sec:future-work) and Appendix D, subsection “Our MoE comparison tests recurrent-family ordering”