---
title: 'AS2DRoPE: 2D Rotary Positional Encodings'
url: https://www.emergentmind.com/topics/two-dimensional-rotary-positional-encodings-as2drope
type: topic
---

# AS2DRoPE: 2D Rotary Positional Encodings

Two-dimensional rotary positional encodings (AS2DRoPE) generalize 1D Rotary Position Embeddings (RoPE) to structured tensors with spatial coordinates, primarily for use in Vision Transformers and agent-based modeling frameworks. AS2DRoPE extends the translation-invariant, relative-position properties of RoPE to 2D, enabling transformers to encode spatial displacements explicitly and efficiently in multi-head self-attention. The term AS2DRoPE is used for canonical “axial-separable” 2D rotary schemes in several works, and forms the reference implementation for practical 2D relative position encoding [2602.03227, 2512.04963, 2503.15029, 2502.02562].

## 1. Mathematical Formulation

Let $d$ be the embedding dimension of a token. In the standard axis-separable 2D RoPE (“AS2DRoPE”), the $d$-dimensional token embedding is divided into $d/2$ contiguous 2D subspaces. For token position $(x, y)$, each block is rotated in the complex plane by an angle that is a linear function of $x$ and $y$:

$$
\mathrm{Rot}(x, y) = \bigoplus_{n=1}^{d/2} \rho(x\,\theta_n^x + y\,\theta_n^y),
$$

where $\rho(\phi)=\begin{bmatrix}\cos \phi & -\sin \phi \\ \sin \phi & \cos \phi\end{bmatrix}$. The frequencies $\theta_n^x, \theta_n^y$ are set per block (e.g., as geometric sequences or learned parameters).

This construction can also be seen as the composition of two commuting rotation blocks:

$$
\mathrm{Rot}(x, y) = \mathrm{Rot}_x(x) \cdot \mathrm{Rot}_y(y),
$$

which ensures that the embedding is translation-invariant in 2D:
$$
\mathrm{Rot}(x_i, y_i)^\top \mathrm{Rot}(x_j, y_j) = \mathrm{Rot}(x_j - x_i, y_j - y_i).
$$

In practical variants [2502.02562, 2503.15029], the $d$-dimensional embedding is split evenly into $x$ and $y$ blocks, each block rotated independently by the respective spatial coordinate multiplied by a frequency or a global scaling parameter. The attention score between two tokens is then a function only of their relative displacement, preserving the essential property of RoPE.

## 2. Practical Implementation

AS2DRoPE inserts the 2D rotary operation directly into the attention mechanism of Multi-Head Self-Attention (MHSA) layers. For each query/key vector:
1. Project to embedding dimension $d$.
2. Partition into $d/2$ blocks (size 2).
3. For block $n$, apply the SO(2) rotation defined by the token’s 2D coordinates.

Efficient implementation leverages inplace block-wise rotations, with $O

Source: https://www.emergentmind.com/topics/two-dimensional-rotary-positional-encodings-as2drope