Convergence of HRM recursions to fixed points
Determine conditions under which HRM’s recursive updates converge to fixed points that justify the use of the Implicit Function Theorem and one-step gradient approximation, or else rigorously characterize regimes where such fixed points are not attained.
References
Most importantly, there is no guarantee that a fixed-point is reached. Thus, while the application of the IFT theorem and 1-step gradient approximation to HRM has some basis since the residuals do generally reduce over time, a fixed point is unlikely to be reached when the theorem is actually applied.
— Less is More: Recursive Reasoning with Tiny Networks
(Jolicoeur-Martineau, 6 Oct 2025) in Section “Implicit Function Theorem (IFT) with 1-step gradient approximation”