Row-Column Decoupled Attention (RCDA)

• RCDA is an attention mechanism that decouples 2D self-attention into separate 1D row and column operations to efficiently capture long-range spatial dependencies.
• It reduces computational complexity from O((hw)^2) to O(h^2 + w^2), making it well-suited for processing elongated objects like lane markings.
• Integration in the Laneformer encoder and ablation studies show that RCDA improves F₁ scores in lane detection while minimizing overhead.

Row-Column Decouple Attention (RCDA) is an attention mechanism introduced to efficiently model long-range dependencies in spatial feature maps, specifically targeting the challenges posed by lane detection in visual perception for autonomous driving. RCDA operates by decoupling full two-dimensional self-attention into two complementary one-dimensional attentions along rows and columns, achieving substantial computational and memory efficiency while maintaining global context propagation suited for objects with elongated geometries such as lane markings (Han et al., 2022).

1. Mathematical Formulation of Row-Column Decoupled Attention

Given a spatial feature map $H \in \mathbb{R}^{h \times w \times d}$ output by a ResNet backbone, RCDA proceeds by projecting the map into two sets of 1-D tokens—row tokens and column tokens—prior to separate self-attention operations.

Row Tokens:

• $H_r \in \mathbb{R}^{h \times (w\,d)}$, where $H_r[i]$ is the flattened $i$-th row.
• Projected as $$H'_r = H_r W^{r, \text{proj}} \in \mathbb{R}^{h \times d'}$$.
• Add sine-cosine positional embedding $$E^r \in \mathbb{R}^{h \times d'}: \;\; \widetilde{H}_r = H'_r + E^r$$.
• Compute $Q^r = \widetilde{H}_r W_Q^r$, $K^r = \widetilde{H}_r W_K^r$, $V^r = H'_r W_V^r \in \mathbb{R}^{h \times d'}$.

Column Tokens:

• $H_c \in \mathbb{R}^{w \times (h\,d)}$, each $H_c[j]$ is the flattened $j$-th column.
• Projected as $$H'_c = H_c W^{c, \text{proj}} \in \mathbb{R}^{w \times d'}$$.
• Add $$E^c \in \mathbb{R}^{w \times d'}: \;\; \widetilde{H}_c = H'_c + E^c$$.
• Compute $Q^c = \widetilde{H}_c W_Q^c$, $K^c = \widetilde{H}_c W_K^c$, $V^c = H'_c W_V^c \in \mathbb{R}^{w \times d'}$.

The learned matrices $$W^{r, \text{proj}}, W_Q^r, W_K^r, W_V^r \in \mathbb{R}^{(w d) \times d'}$$ (and analogous for the column path) define the projection and attention parameterizations.

2. Attention Mechanism and 1-D Decoupling

RCDA replaces the standard full 2D spatial self-attention (complexity $O((hw)^2 d')$) with two 1D attentions along each axis.

• Row Self-Attention: For $i, j = 1, \dots, h$,

$$A^r_{i, j} = \mathrm{softmax}_j \left( \frac{Q^r_i \cdot (K^r_j)^T}{\sqrt{d'}} \right), \quad O^r_i = \sum_{j=1}^h A^r_{i, j} V^r_j \in \mathbb{R}^{d'}$$

Each $O^r_i$ is broadcast across $w$ columns to form a $h \times w \times d'$ tensor.

• Column Self-Attention: For $p, q = 1, \dots, w$,

$$A^c_{p, q} = \mathrm{softmax}_q \left( \frac{Q^c_p \cdot (K^c_q)^T}{\sqrt{d'}} \right), \quad O^c_p = \sum_{q=1}^w A^c_{p, q} V^c_q \in \mathbb{R}^{d'}$$

Each $O^c_p$ is broadcast across $h$ rows.

• Aggregation: For each spatial position $(i,p)$, outputs are summed:

$O_{i,p} = O^r_{i,p} + O^c_{i,p}$

This mechanism specifically addresses the topological priors of lane-like objects, which are typically long and thin, by enabling efficient exchange of information along spatial axes (Han et al., 2022).

3. Computational Efficiency and Scaling Properties

The decoupling approach leads to significant reductions in computational cost and memory usage relative to full spatial self-attention:

• For a full $h \times w$ map, standard self-attention scales as $O((hw)^2 d')$ time and $O((hw)^2)$ memory.
• RCDA row and column phases:
  • Projections: $O(h w d d')$ (row) and $O(h w d d')$ (column).
  • Attention and weighted sum: $O(h^2 d')$ (row), $O(w^2 d')$ (column).
  • Total: $O(h w d d' + h^2 d' + w^2 d')$ time and $O(h^2 + w^2)$ memory.

In practice, for spatial resolutions with $h, w \sim 10$–$100$, the total memory and computational footprint for RCDA is 10–100× smaller than full 2D attention at negligible accuracy loss for elongated objects such as lanes (Han et al., 2022).

4. Integration within the Laneformer Transformer Encoder

RCDA is a core component of Laneformer’s encoder architecture (Han et al., 2022), combined with deformable pixel-wise self-attention and object-aware detection attention:

1. The backbone (ResNet) generates feature maps $H_f \in \mathbb{R}^{h \times w \times d}$.
2. Deformable self-attention (as in DETR) captures sparse, global context.
3. RCDA modules perform row and column self-attention in parallel.
4. Pixel-to-BBox detection attention incorporates features of detected object instances via bounding box location information (added in the Key module) and ROI-aligned features (added in the Value module). For each pixel in $H_f'$, queries interact with $M$ detected-object embeddings $Z_b, Z_r \in \mathbb{R}^{M \times d'}$ via:

$$O_{p2b} = \mathrm{softmax}\left( \frac{H_f' Z_b^T}{\sqrt{d'}} \right) Z_r$$

1. The outputs from deformable self-attention, RCDA, and pixel-to-BBox attention are algebraically aggregated into a memory tensor of size $h \times w \times d'$, passed to the decoder stage.
2. In the decoder, queries perform standard self-attention, cross-attention to the encoder memory, and query-to-BBox detection attention in parallel; results are summed and processed via an MLP head to yield lane point predictions (x-coordinates, start/end).

5. Empirical Impact and Quantitative Ablation

Ablation experiments on the CULane benchmark quantify the impact of RCDA in isolation and in conjunction with detection attention. The following table summarizes key results [(Han et al., 2022), Table 6]:

Model	F₁	Prec	Rec
Baseline (no RCDA, no detection attn)	75.45	81.65	70.11
+ row–column attention only	76.04	82.92	70.22
+ detection-attention (bbox only)	76.08	85.30	68.66
+ detection + score information	76.25	83.56	70.12
+ detection + score + category (full model)	77.06	84.05	71.14

The inclusion of RCDA alone yields a +0.59 point improvement in F₁ score over the deformable DETR backbone; the complete architecture combining RCDA and object-aware attention shows a total +1.61 point gain in F₁. This demonstrates that RCDA can materially enhance lane detection accuracy at marginal computational overhead (Han et al., 2022).

6. Context and Applicability within Long-Range Spatial Modeling

RCDA was motivated by the limitations of convolutional approaches in capturing both long-range dependencies and the global context essential for structured object parsing in autonomous driving scenarios. By structuring attention along two principal spatial axes, RCDA provides a low-latency alternative to dense self-attention that retains global feature propagation conducive to structured, geometrically-constrained objects such as lanes. While tailored to lane detection, the underlying principle of decoupled 1-D attention is extensible to other domains where spatial anisotropy or object geometry suggests similar topological priors. A plausible implication is that RCDA or analogous decomposed attention structures may offer substantial efficiency and accuracy benefits in other computer vision tasks featuring long-thin or similarly oriented objects (Han et al., 2022).