Dynamic expert management to prevent modular explosion

Develop algorithms to dynamically merge or compress redundant expert subnetworks within the sparsely activated, expert-gated components of the Tri-Memory architecture to prevent long-term modular explosion during continual learning.

Background

The architecture leverages sparse activation and expert gating to compartmentalize knowledge, improving efficiency and reducing interference. Over extended operation, the number of experts may proliferate, increasing management complexity and memory footprint.

The authors identify an explicit open need for methods that detect redundancy across experts and perform merging or compression to maintain scalability without degrading performance.

References

While the framework offers a promising foundation for Personalized AGI on the edge, several open challenges and research opportunities remain: Can redundant experts be dynamically merged or compressed to prevent modular explosion over long-term learning?

— Personalized Artificial General Intelligence (AGI) via Neuroscience-Inspired Continuous Learning Systems  (2504.20109 - Gupta et al., 27 Apr 2025) in Section 6.4 Open Questions and Future Research Directions

However, modularity does not push the reuse axis further, as forward transfer only matches what the monolithic model already achieved. Moreover, PWM's advantage holds only under the privileged frozen encoder, without which its performance is on par with TD-MPC2. Therefore, a design that reuses components more effectively, and an encoder that supplies a stable representation without privileged access to the full task distribution, both remain open.

— Benchmarking World Models for Continual Learning on Compositional Tasks  (2609.22055 - Zhou et al., 18 Sep 2026) in Section 4, paragraph “Explicit modularity pushes the frontier further”; Section 5, Conclusion