Do AHATs recognize all regular languages?

Ascertain whether Average Hard Attention Transformers with position embeddings can recognize all regular languages over finite alphabets, thereby determining if the class of languages definable by AHAT with position embeddings coincides with the full class of regular languages.

Background

The paper establishes that Unique Hard Attention Transformers (UHAT) with position embeddings contain all regular languages that lie in AC, and that AHAT languages are contained in TC. It further shows that Counting LTL with unary numerical predicates compiles into AHAT with position embeddings.

However, a complete characterization of AHAT’s expressivity is unavailable, and the authors explicitly state uncertainty about whether AHAT can capture all regular languages. Resolving this would bridge the gap between current logical characterizations (e.g., Counting LTL) and the full scope of regular languages.