Justify the use of exactly two latent features in HRM

Determine and rigorously justify the design choice of using exactly two latent feature vectors (z_L and z_H) in the Hierarchical Reasoning Model (HRM), rather than one, three, or more latent features, by establishing theoretical criteria or empirical evidence that specify when two features are optimal and how alternative numbers of features affect performance and learning dynamics.

Background

The Hierarchical Reasoning Model (HRM) employs two latent features, z_L and z_H, each produced and consumed by distinct recurrent modules operating at different frequencies. The paper critiques the biological motivation for this design and notes the absence of clear ablations that isolate why two features are chosen over other configurations. This uncertainty is highlighted as a specific unresolved point in the model’s rationale.

The authors later propose Tiny Recursion Models (TRM) with a simplified interpretation (y as the embedded solution and z as latent reasoning) and present experiments on Sudoku-Extreme suggesting two features outperform one or many in that setting. However, a general and principled justification for the “two-feature” design across tasks and model families remains unestablished.