Close the linear-to-exponential head gap for multi-hop induction

Determine the exact single-layer head complexity of the m-hop induction-head task by closing the gap between the linear lower bound of at least m heads and the known exponential-in-m upper bound.

Background

The paper proves that a one-layer k-head attention-only transformer cannot solve the (k+1)-hop induction-head task, yielding a lower bound of at least m heads for an m-hop task. It notes that existing constructive methods use exponentially many heads in the number of hops. The precise complexity between these bounds remains unresolved.

References

This complements the constructive side, where such tasks are solved with a number of heads exponential in the number of hops; closing the gap between the linear lower bound and the exponential upper bound is left open.

— The Head Complexity of Boolean Functions in Single-Layer Attention  (2609.04046 - Rajaraman et al., 3 Sep 2026) in Remark following Theorem \ref{thm:khop-lb}, Section 8