Researchers compare five different positional encoding approaches in Transformer-based language models to analyze their impact on length generalization.
The study shows that explicit position embeddings are not essential for decoder-only Transformers to generalize well to longer sequences.