Theoretical Limitations of Self-Attention in Neural Sequence Models (1906.06755v2)

Abstract: Transformers are emerging as the new workhorse of NLP, showing great success across tasks. Unlike LSTMs, transformers process input sequences entirely through self-attention. Previous work has suggested that the computational capabilities of self-attention to process hierarchical structures are limited. In this work, we mathematically investigate the computational power of self-attention to model formal languages. Across both soft and hard attention, we show strong theoretical limitations of the computational abilities of self-attention, finding that it cannot model periodic finite-state languages, nor hierarchical structure, unless the number of layers or heads increases with input length. These limitations seem surprising given the practical success of self-attention and the prominent role assigned to hierarchical structure in linguistics, suggesting that natural language can be approximated well with models that are too weak for the formal languages typically assumed in theoretical linguistics.

• The paper demonstrates that self-attention lacks the theoretical capacity to model hierarchical and recursive structures without scaling complexity.
• The analysis uses both hard and soft attention frameworks to show transformers struggle with formal language tasks like bracket closure and parity evaluation.
• The findings suggest that integrating recurrence or hybrid architectures may be necessary to overcome the inherent expressiveness limitations of self-attention.

Theoretical Limitations of Self-Attention in Neural Sequence Models

The paper "Theoretical Limitations of Self-Attention in Neural Sequence Models" by Michael Hahn explores the computational boundaries of self-attention mechanisms in neural sequence models, particularly transformers, for processing formal languages. Despite the vast success of transformers in NLP, the paper analytically investigates whether self-attention can model hierarchical structures and periodic finite-state languages, which are critical in theories of formal languages and linguistics.

Key Contributions

1. Theoretical Analysis of Self-Attention: The paper explores the computational power limitations of self-attention, investigating whether it can process hierarchical and recursive structures without the need for recurrent computations. Self-attention is more parallelizable than LSTMs but is believed to have limited expressiveness because it processes input sequences non-sequentially.
2. Formal Languages and Hierarchical Structures: Two fundamental computational tasks representing hierarchical structure are considered: (i) correctly closing brackets (Dyck language) and (ii) evaluating iterated negation (Parity). The paper demonstrates that neither of these tasks can be robustly solved by transformers relying solely on self-attention, unless the number of layers or heads increase with input length.
3. Hard Attention Analysis: In the setting of hard attention, where each head attends to the input position with the highest attention score, the paper shows significant limitations. The computational power of hard attention transformers is proved to be insufficient for Parity and Dyck languages through combinatorial arguments. These results hold without assumptions on activation functions or parameter norms.
4. Soft Attention Analysis: For soft attention, the paper adapts a probabilistic approach to analyze transformers. While it does not achieve the strength of hard attention bounds, it still establishes that self-attention cannot achieve minimal cross-entropy on distributions over sequences generated by regular or context-free languages, indicating a limited predictive capability on these tasks.

Implications

Theoretical Implications:

The inability of transformers to model hierarchical structures challenges the belief that current models are sufficient for understanding complex linguistic phenomena. The paper showcases mathematical techniques necessary for analyzing self-attention, contrasting its capacity with that of recurrent neural networks. The theoretical position here confirms limitations in the expressiveness of non-recurrent models for tasks involving hierarchy and recursion extensively studied under formal language theories.

Practical Impact:

Practically, while the analysis is asymptotic, it suggests that transformers might still perform well on constrained input lengths, leveraging an increasing number of layers and heads to approach tasks empirically. This insight may guide researchers in constructing more effective neural architectures tailored for specific lengths of natural language sequences, potentially balancing computational efficiency and performance.

Future Research Directions

The paper invites several directions for future research. Firstly, combining empirical studies and theoretical techniques can provide finer-grained understanding and more comprehensive bounds on the performance of transformers. Secondly, exploring hybrid models that integrate both self-attention and recurrence might leverage the strengths of both mechanisms, potentially overcoming the expressiveness limitations highlighted here.

Moreover, examining the connection between limited hierarchical representation in self-attention and cognitive constraints in human language processing opens an intriguing interdisciplinary research avenue. This line of inquiry might yield novel architectures inspired by human sentence processing models that operationalize recursive structures effectively.

Conclusion

This paper offers a crucial perspective on the theoretical capabilities and limitations of transformers, particularly self-attention mechanisms, in modeling formal languages and hierarchical structures. The results underline significant constraints of self-attention, urging the development of more expressive or hybrid neural architectures for linguistic modeling. This work stands as a foundational reference point for researchers studying the theoretical expressiveness and practical performance of advanced neural sequence models in natural language processing.