Polar Attention Mechanisms

• Polar Attention is a family of mechanisms that transform inputs into polar or log-polar representations to align with natural geometric structures such as radial or wedge-shaped layouts.
• It reparameterizes Cartesian inputs into polar coordinates, enabling methods like PolarFormer or DocPolarBERT to improve multi-camera 3D detection, document layout analysis, and ultrasound imaging.
• Empirical results demonstrate that polar representations yield improved accuracy and scale invariance in tasks, though benefits vary with application-specific data geometries.

Polar Attention denotes a family of attention mechanisms in which the organizing variables are expressed in polar or log-polar form—typically radius and angle, sometimes relative distance and direction, and, in one recent language-modeling formulation, a decoupling of retrieved direction from retrieved magnitude rather than an explicit geometric transform. In the geometric sense, Polar Attention replaces Cartesian grids or absolute 2D layouts with representations such as $(r,\theta)$ or $(\rho,\theta)$ that better match wedge-shaped camera frusta, fixation-centered vision, document layout relations, or radial sensing geometry (Jiang et al., 2022, Kiritani et al., 2020, Uthayasooriyar et al., 11 Jul 2025). The term is also used more broadly in the literature for mechanisms that are “polarized” or “polarity-aware” rather than coordinate-polar, so the phrase is not semantically uniform across subfields (Liu et al., 2021, Meng et al., 25 Jan 2025).

1. Coordinate foundations

In multi-camera 3D detection, PolarFormer parameterizes bird’s-eye-view space by

$r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$

with inverse mapping

$x=r\cos\theta,\qquad y=r\sin\theta.$

The stated motivation is that the conventional Cartesian coordinate system with perpendicular axis does not fit the ego car’s perspective as well as a curved $(r,\theta)$ grid aligned with the six-camera wedge view (Jiang et al., 2022).

In recurrent visual attention, RAM-LPM centers a log-polar coordinate system at the current fixation $(x_t,y_t)$ and maps image coordinates by

$\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$

A field of view is defined by restricting $\rho_{\min}<\rho<\rho_{\max}$ and $-\pi\le \theta<\pi$, then tessellating that rectangular $(\rho,\theta)$ region uniformly into an $(\rho,\theta)$0 tensor. The result is a fixed-size glimpse irrespective of the original image size (Kiritani et al., 2020).

In document understanding, DocPolarBERT defines token-to-token relative polar coordinates from OCR bounding-box centers:

$(\rho,\theta)$1

with $(\rho,\theta)$2. Radial distances are clipped and bucketized into $(\rho,\theta)$3 bins, while angles are divided into $(\rho,\theta)$4 uniform buckets (Uthayasooriyar et al., 11 Jul 2025).

In lung ultrasound, Kerdegari et al. use a polar resampling defined by

$(\rho,\theta)$5

so that radial B-lines become approximately vertical structures in the polar image and the large black corner regions present in Cartesian scans are removed (Kerdegari et al., 2021).

Across these formulations, the common design move is geometric reparameterization before or inside attention. This suggests that “polar” is primarily an inductive bias: it makes the attention mechanism operate in coordinates that are closer to the acquisition process or the relational structure of the data.

2. Attention operators on polar domains

PolarFormer implements the most explicit geometric Polar Attention operator. For each camera $(\rho,\theta)$6, BEV scale $(\rho,\theta)$7, and azimuth bin $(\rho,\theta)$8, it defines learned “polar-ray” queries

$(\rho,\theta)$9

with $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$0 radial bins and $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$1 azimuth bins. These queries attend only to the corresponding image-column slice $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$2. The per-ray projections are

$r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$3

and the multi-head attention is

$r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$4

Here $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$5 is a small additive $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$6–$r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$7 positional bias, and the best results are obtained with fixed sine/cosine positional encoding over $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$8 (Jiang et al., 2022).

After computing per-camera ray features, PolarFormer stacks them in azimuth order to form $r=\sqrt{x^2+y^2}, \qquad \theta=\arctan2(y,x),$9, then applies a lightweight alignment that projects each $x=r\cos\theta,\qquad y=r\sin\theta.$0 bin through camera extrinsics and intrinsics, tests visibility, and averages contributions across cameras into a shared BEV polar grid $x=r\cos\theta,\qquad y=r\sin\theta.$1. A decoder then refines 3D cylindrical object queries $x=r\cos\theta,\qquad y=r\sin\theta.$2 by deformable attention over the multi-scale maps $x=r\cos\theta,\qquad y=r\sin\theta.$3, regressing $x=r\cos\theta,\qquad y=r\sin\theta.$4, box sizes, orientation offsets relative to $x=r\cos\theta,\qquad y=r\sin\theta.$5, and velocity in polar components (Jiang et al., 2022).

DocPolarBERT integrates relative polar information directly into self-attention logits. Let $x=r\cos\theta,\qquad y=r\sin\theta.$6 and $x=r\cos\theta,\qquad y=r\sin\theta.$7 denote learned embeddings for bucketized $x=r\cos\theta,\qquad y=r\sin\theta.$8 and $x=r\cos\theta,\qquad y=r\sin\theta.$9. The attention bias is

$(r,\theta)$0

and the head uses

$(r,\theta)$1

The model removes all absolute 2D positional embeddings used in prior layout models and replaces vanilla attention in every layer with this bias-augmented form (Uthayasooriyar et al., 11 Jul 2025).

ATMA uses the name Polar Attention for a different factorization. Instead of a geometric $(r,\theta)$2 transform, it replaces softmax-normalized mixing with a three-channel mechanism: a count-blind unit-vector direction channel, a bounded magnitude channel, and a long-term recurrent compression memory. For query index $(r,\theta)$3, the direction vector is

$(r,\theta)$4

the effective participation is derived from the inverse Simpson index of real-key weights gated by the null weight,

$(r,\theta)$5

and the bounded magnitude is

$(r,\theta)$6

The output recombines these streams as

$(r,\theta)$7

In this usage, “polar” refers to a decomposition into what and how much, not to a coordinate transform (Akbar, 23 Jun 2026).

3. Architectural patterns and invariances

A recurrent form of Polar Attention appears in RAM-LPM. At each time step, a location network proposes a fixation, a log-polar patch is sampled, and two parallel streams process it: a “what” pathway for classification and a “where” pathway for recurrent fixation control. The “what” pathway uses a CNN followed by an LSTM whose hidden state feeds a classification head and a baseline head. The “where” pathway uses a second CNN plus LSTM and predicts Gaussian fixation shifts $(r,\theta)$8, with the next fixation sampled stochastically. The “where” parameters are trained with REINFORCE and the “what” pathway with cross-entropy backpropagation (Kiritani et al., 2020).

RAM-LPM also includes a reflex-like pathway: after the very first glimpse, the classification head already produces logits, so one can read out a “reflex” decision before the full recurrent roll-out is finished. The paper reports a quick but coarse accuracy after 1 or 2 glimpses, sharpening over 5–10 glimpses. Wrap-around padding is used along $(r,\theta)$9 so that features spanning $(x_t,y_t)$0 connect seamlessly (Kiritani et al., 2020).

The stated invariances follow directly from the log-polar representation. Zooming the entire image about the fixation shifts $(x_t,y_t)$1 by a constant, rotating the image about the fixation circularly shifts the tensor along $(x_t,y_t)$2, and each glimpse costs $(x_t,y_t)$3 compute independent of original image resolution (Kiritani et al., 2020).

PolarFormer addresses a different structural issue: irregular polar grids in BEV have no rectangular topology in $(x_t,y_t)$4–$(x_t,y_t)$5. The paper therefore performs all reasoning by pure sequence-to-sequence attention, imposes no fixed 2D or 3D convolution kernels, and handles visibility with a binary mask $(x_t,y_t)$6 that zeros out rays behind the vehicle or outside a camera frustum. During alignment it bilinearly samples only those $(x_t,y_t)$7 bins that backproject into valid image columns (Jiang et al., 2022). To manage unconstrained object scale variations along the distance dimension, it uses $(x_t,y_t)$8 scales:

• scale 1: $(x_t,y_t)$9, $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$0
• scale 2: $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$1, $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$2
• scale 3: $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$3, $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$4

In lung ultrasound, the attention equations themselves are unchanged by the coordinate transform:

$\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$5

The difference is representational: after polar resampling, B-lines become nearly vertical, columnar features, and the authors report that the learned attention weights $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$6 peak more sharply on true B-line frames and are suppressed on non-B-line frames (Kerdegari et al., 2021). This is an example of attention benefiting from a polar input domain rather than implementing a new polar-specific attention rule.

4. Empirical behavior across application domains

On nuScenes, PolarFormer reports under a prototype setting with a ResNet-101 backbone initialized from FCOS3D that a Cartesian baseline in CenterPoint style yields 38.1 mAP / 45.5 NDS, whereas full PolarFormer obtains 39.6 mAP / 45.8 NDS, a +1.5 mAP gain. With a stronger VoVNet-99 backbone and DD3D pre-training, BEVFormer achieves 48.1 mAP / 56.9 NDS, while PolarFormer-T with simple temporal fusion reaches 49.3 mAP / 57.2 NDS. The ablations state that Polar representation benefits near and medium-range objects most, with the multi-scale strategy counteracting radial scale shrinkage (Jiang et al., 2022).

On ImageNet at $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$7, RAM-LPM with glimpseNum=10 and mcSample=20 at test time reaches 64.8% top-1 accuracy, while a single-glimpse reflex yields approximately 50% accuracy. Under $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$8, $\rho=\ln\sqrt{(x-x_t)^2+(y-y_t)^2},\qquad \theta=\atan2(y-y_t,x-x_t).$9 attacks, the paper reports that attacks on full-resolution images had far lower success rates than attacks on $\rho_{\min}<\rho<\rho_{\max}$0 crops, and describes this as underlining the fixed-size glimpse defense. On SIM2MNIST, RAM-LPM achieves 5.00% error (Kiritani et al., 2020).

DocPolarBERT reports micro-F1 results averaged over 10 seeds, with 80.95 for DocPolarBERT and 81.30 for DocPolarBERT + 1-LOP. The ablation identified as “polar-only” is reported to jump from 79.89 → 81.29 when absolute 2D information is removed. On PAYSLIPS it scores 73.5% versus 62.3% for LayoutLM-base; on SROIE 96.96% versus 94.38%; on CORD 95.97% versus 95.66% (Uthayasooriyar et al., 11 Jul 2025).

In lung ultrasound video classification, the polar representation improves video-level F1 at $\rho_{\min}<\rho<\rho_{\max}$1 from 81.0% in Cartesian coordinates to 83.5% in polar coordinates, with paired t-test $\rho_{\min}<\rho<\rho_{\max}$2, $\rho_{\min}<\rho<\rho_{\max}$3. Temporal-localization IoU at the same resolution improves from 67.1% to 73.2% (Kerdegari et al., 2021).

ATMA reports that the Polar Attention core alone is not sufficient for long-context language modeling: in its memoryless configuration, “Polar” NIAH retrieval drops from 96% at 2,048 tokens to 0% at 65,536 tokens. With memory enabled, the reported Table 2 gives Polar+Mem NIAH of 91% at 2,048, 98% at 16,384, 96% at 32,768, and 93% at 65,536 tokens, while the abstract states that the combined Polar + memory model maintains induction needle-in-a-haystack retrieval accuracy above 90% out to 64K tokens (Akbar, 23 Jun 2026).

Taken together, these results indicate that polar parameterizations are most effective when the underlying signal already has a radial, wedge-shaped, fixation-centered, or relative-layout structure. They do not by themselves guarantee gains in unrelated operating regimes.

5. Terminological variants and adjacent mechanisms

One recurring source of confusion is that “polar” in attention papers does not always denote polar coordinates.

“PolaFormer: Polarity-aware Linear Attention for Vision Transformers” uses polarity-aware linear attention rather than geometric polar coordinates. Its key idea is to model both same-signed and opposite-signed query-key interactions through

$\rho_{\min}<\rho<\rho_{\max}$4

together with a learnable power function that restores the spiky properties of attention maps. The paper frames its contribution as recovering negative interactions lost by non-negative kernel feature maps and retaining $\rho_{\min}<\rho<\rho_{\max}$5 complexity (Meng et al., 25 Jan 2025).

“Polarized Self-Attention” likewise does not use a polar coordinate system. Its “polarized filtering” means collapsing one axis while preserving maximal resolution on the other: a channel-only branch computes $\rho_{\min}<\rho<\rho_{\max}$6 and a spatial-only branch computes $\rho_{\min}<\rho<\rho_{\max}$7, both with Softmax followed by Sigmoid enhancement. The paper targets high-quality pixel-wise regression and reports consistent gains on COCO pose and semantic segmentation (Liu et al., 2021).

In coding theory, “polar” often refers to polar codes rather than geometry. The LAT decoder is a latent-attention based transformer for polar decoding in which $\rho_{\min}<\rho<\rho_{\max}$8 and $\rho_{\min}<\rho<\rho_{\max}$9 are learned independently of $-\pi\le \theta<\pi$0 through layer-specific positional encoders, and a code-aware mask enforces algebraic constraints across varying code lengths and rates (Zhu et al., 20 Jul 2025). DeepPolar+ injects standard multi-head self-attention into neural successive-cancellation decoding kernels for DeepPolar codes, with a structured loss that combines bit-level and block-level terms (Srivastava et al., 11 Jun 2025).

A nearby but distinct development is PoPE, “Polar Coordinate Positional Embeddings.” PoPE decouples “what” and “where” in positional encoding by using content magnitudes and purely positional phases, but it is a positional encoding scheme integrated into standard attention, not itself a standalone attention operator (Gopalakrishnan et al., 5 Sep 2025).

The literature therefore supports a strict distinction between coordinate-polar attention, polarity-aware attention, polarized attention, and attention for polar codes. Treating them as a single mechanism would be inaccurate.

6. Limitations, misconceptions, and open questions

A common misconception is that polar parameterization is universally superior to Cartesian parameterization. The reported evidence is narrower. PolarFormer attributes its gains to the ego-vehicle camera geometry and explicitly introduces a multi-scale strategy to offset scale variations along the radial dimension (Jiang et al., 2022). The lung-ultrasound study finds statistically significant gains at higher resolutions but not at $-\pi\le \theta<\pi$1, where the difference is reported as not significant (Kerdegari et al., 2021). RAM-LPM derives scale and rotation invariance around a fixation point, not globally for arbitrary scenes (Kiritani et al., 2020).

Another misconception is that the attention mechanism itself must always be reformulated in polar coordinates. The literature includes at least three patterns. In PolarFormer, the attention operator is intrinsically polar and ray-aligned (Jiang et al., 2022). In DocPolarBERT, attention remains dot-product attention with additive polar biases (Uthayasooriyar et al., 11 Jul 2025). In lung ultrasound, the attention equations are unchanged and only the input representation is polar (Kerdegari et al., 2021).

The design granularity of polar information also varies substantially. DocPolarBERT uses only 4 radial bins and 8 angular bins (Uthayasooriyar et al., 11 Jul 2025). A plausible implication is that coarse relative layout can be sufficient for document NER even when the model abandons absolute 2D embeddings. By contrast, PolarFormer builds full multi-scale $-\pi\le \theta<\pi$2 BEV maps and performs visibility-aware fusion across cameras (Jiang et al., 2022).

Recent long-context work adds a further caution: ATMA states explicitly that neither the Polar Attention core nor the recurrent memory is sufficient alone (Akbar, 23 Jun 2026). This suggests that “polar” factorization may solve only one part of a broader stability problem, especially when context length, retrieval fidelity, and bounded activations must be addressed simultaneously.

Open questions follow directly from these divergences. The current literature does not present a single canonical Polar Attention formalism; instead, it presents a set of task-specific constructions linked by radial geometry, relative directionality, or factorized retrieval semantics. Whether these constructions can be unified into a common theory of polar inductive bias remains unresolved by the cited works.