Comparative benefit of HRM hierarchical H/L modules

Determine the performance contribution of the high-level (H) and low-level (L) hierarchical modules in the Hierarchical Reasoning Model relative to a plain transformer architecture, assessing whether the dual-loop design provides measurable advantages over a non-hierarchical transformer of similar capacity on reasoning tasks.

Background

The Hierarchical Reasoning Model (HRM) proposes a dual-loop architecture with a fast-updating low-level (L) module and a slower high-level (H) module. The paper critically evaluates HRM and presents variants, including an 8-layer L-only configuration, that perform similarly to the original 4-layer L plus 4-layer H model on tasks such as Sudoku-Extreme.

These findings, and external analyses cited by the authors, raise doubts about the necessity of the hierarchical design. The authors explicitly state uncertainty regarding the added value of the H/L modules compared to a plain transformer, motivating a focused comparative assessment.