Close the polynomial gap in worst-case head complexity

Determine the exact worst-case head complexity of Boolean functions computed by a single-layer attention-only transformer by closing the gap between the known lower bound d4aa(2^n/n^2) and the upper bound 2^n heads.

Background

The paper proves that every n-bit Boolean function can be computed with at most 2n heads, while a counting argument shows that almost all such functions require d4aa(2n/n2) heads, even with unrestricted embedding dimension and numerical precision. The authors identify the remaining polynomial-factor gap as a genuine unresolved problem for the worst-case head complexity of a single attention layer.

References

It remains open to exhibit an explicit, natural family with exponential head complexity, thereby complementing the counting lower bound with a concrete witness.

— The Head Complexity of Boolean Functions in Single-Layer Attention  (2609.04046 - Rajaraman et al., 3 Sep 2026) in Paragraph 'An explicit hard function for one layer,' Section 6

Is there an explicit family $f_n$ computable by an attention network of $O(1)$ (or $O(\log n)$) layers with $\operatorname{poly}(n)$ heads per layer, yet requiring $2{\Omega(n)}$ heads in any single layer?

— The Head Complexity of Boolean Functions in Single-Layer Attention  (2609.04046 - Rajaraman et al., 3 Sep 2026) in Paragraph 'An explicit function that forces depth,' Section 6

The one $\operatorname{poly}(n)$ gap that remains --- closing $\Omega(2{n}/n{2})$ up to the trivial $2{n}$ from below --- is a genuine open problem: a single layer has no composition, so heads cannot reuse sub-computations the way a Lupanov-type circuit does.

— The Head Complexity of Boolean Functions in Single-Layer Attention  (2609.04046 - Rajaraman et al., 3 Sep 2026) in Remark 'The bounds meet; head count is the sole resource,' Section 5.2