Center Attention Head (CenterAtt) Overview

• CenterAtt is a neural module employing Transformer-style attention to associate object centers with candidate features, enhancing prediction and grouping.
• It integrates into two-stage detection and pose estimation pipelines by replacing conventional MLP heads with attention-based Transformer encoder blocks.
• Empirical results demonstrate modest gains in mAPH and AP with reduced inference time, confirming its efficiency in real-time 3D detection and pose grouping.

A Center Attention Head (CenterAtt) is a neural architecture module that employs attention mechanisms to associate object centers with candidate features or detections, facilitating refined prediction and grouping. Introduced in multiple contexts, CenterAtt is a versatile, fully differentiable replacement for traditional multi-layer perceptron (MLP) heads in object detection pipelines and grouping stages for pose estimation, offering both architectural and empirical advantages through Transformer-style attention (Xu et al., 2021, Brasó et al., 2021).

1. Architectural Formulation and Workflow

CenterAtt is incorporated at the second stage of two-stage detection or grouping frameworks, replacing conventional MLPs with an attention-based Transformer encoder block. In 3D object detection pipelines (e.g., CenterPoint), the main workflow proceeds through LiDAR point cloud voxelization and 3D convolutions to yield BEV feature maps. Proposals (coarse 3D boxes) are extracted via a keypoint head and ROI-pooled using bilinear interpolation at five canonical locations: box center and face centers. For each proposal, a feature vector is constructed as the concatenation of five BEV features (one per location) and a fixed 128-dimensional sine positional embedding, yielding the input $x_i \in \mathbb{R}^{D_{\rm in}}$.

The CenterAtt head processes an input set $$X = [x_1;\ldots;x_N] \in \mathbb{R}^{N \times D_{\rm in}}$$ via a single-layer, multi-head self-attention module ($h=8$ heads) followed by a 2048-dimensional feed-forward sublayer and LayerNorm, both with residuals. The refined output $X'$ is then consumed by two lightweight, independent linear heads for class logits and 3D box regression offsets. The pipeline concludes with Hungarian-matched loss over proposals and ground-truth via a cost combining classification loss and 3D IoU (Xu et al., 2021).

In multi-person pose estimation with CenterGroup, CenterAtt is realized as a bank of $J$ separate dot-product attention heads—one per joint type—where each detected center embedding queries the set of keypoint embeddings (with learned joint-type vectors) to directly group joints into their corresponding person instance. All embeddings are contextually enhanced by a sequence of Transformer encoder blocks using positional summation and LayerNorm (Brasó et al., 2021).

2. Mathematical Formulation

For 3D detection (Xu et al., 2021), CenterAtt follows the canonical Transformer formulation. Given $X \in \mathbb{R}^{N \times D_{\rm in}}$, queries, keys, and values are computed as: $Q = XW^Q, \quad K = XW^K, \quad V = XW^V,$ with $$W^Q, W^K, W^V \in \mathbb{R}^{D_{\rm in} \times D_{\rm in}}$$. Each is split into $h$ heads of dimension $d_k = D_{\rm in}/h$. Attention for head $j$ is computed as: $$\mathrm{head}_j = \mathsf{softmax}\left(\frac{Q_j K_j^T}{\sqrt{d_k}}\right)V_j.$$ Head outputs are concatenated and projected, followed by a two-layer feed-forward ReLU network: $$\mathrm{FFN}(Z) = \mathrm{ReLU}(Z W_1 + b_1) W_2 + b_2.$$ Residual and layer normalization yield: $$Z_1 = \mathsf{LayerNorm}(X + \mathrm{MHA}(X)),\quad X' = \mathsf{LayerNorm}(Z_1 + \mathrm{FFN}(Z_1)).$$ Final outputs are obtained via independent linear heads for class and regression.

In CenterGroup (Brasó et al., 2021), for each center $c$ and joint type $i$, the attention is formulated as: $$q_{c, i} = W_i^Q h_c, \quad k_{k, i} = W_i^K(h_k + \phi_{\mathrm{type}(k)}),$$

$$\alpha_{c, k}^{(i)} = \mathrm{softmax}_{k \in \mathcal{K}} \left(\frac{q_{c, i}^\top k_{k, i}}{\sqrt{d_H}}\right), \quad \widehat{\mathrm{loc}^i_c} = \sum_{k \in \mathcal{K}} \alpha_{c, k}^{(i)} \mathrm{loc}_k.$$

Here, $d_H$ is the per-head dimensionality and $\phi_i$ is a learned type-embedding.

3. Implementation Details and Training

For 3D detection (Xu et al., 2021), ROI pooling is realized via five bilinear interpolations per proposal from a BEV feature map. The sine positional encoding is fixed and 128-dimensional. All attention projections and heads use $D_{\rm in}$ as the dimensionality. The feed-forward block hidden size is 2048. Real-time optimizations include GPU-accelerated voxelization via TorchSparse, merged BatchNorm weights, and FP16 inference. Training is staged: initial pre-training of a one-stage detector (36 epochs), followed by CenterAtt-augmented finetuning (3–6 epochs) with AdamW, OneCycleLR, and data augmentations.

For CenterGroup (Brasó et al., 2021), embeddings (dimension $d=128$) are processed by three Transformer encoder blocks (four heads per block, hidden dimension 512, LayerNorm and residuals throughout). Type-specific $W_i^Q, W_i^K, W_i^V$ are learned for each joint type. Positional encodings (sine-cosine, length $d$) are added initially. All grouping, detection, and classification is trained end-to-end, with focal loss for center classification and $L_1$ loss for joint location, using only differentiable operations.

4. Empirical Findings and Ablation Results

In the Waymo Open Dataset real-time 3D detection challenge, addition of CenterAtt to the CenterPoint pipeline yields a performance increase from 67.5% to 67.7% mAPH at a latency of 65.4 ms (vs. 64.9 ms baseline), with further optimizations reaching 67.3% mAPH at 54.1 ms end-to-end (Xu et al., 2021). Visual analysis indicates improved suppression of overlapping low-score proposals, leading to more accurate box selection and reduced need for aggressive NMS. Ablations attribute the 0.2% mAPH gain strictly to the CenterAtt block.

For multi-person pose, the Center Attention Head delivers 67.5 AP for type-specific attention sans transformer (vs. 67.1 AP for AE-based grouping), 67.9 AP with type-encoding, and up to 68.6 AP with full transformer and positional encoding. The HRNet-W48 backbone achieves 69.6 AP single-scale and 71.4 AP multi-scale with CenterAtt, and grouping time is 52 ms (vs. 327 ms for associative embedding methods). Total single-image inference is 178 ms (vs. 453 ms) (Brasó et al., 2021).

5. Algorithmic and Conceptual Significance

CenterAtt replaces clustering, thresholding, or non-learnable grouping mechanisms with a fully differentiable attention-based module. In 3D detection, this introduces context-aware reasoning between ROI proposals at minimal computational cost, enabling proposal interaction and efficient box refinement without custom NMS tricks. In pose estimation, CenterAtt achieves accurate, end-to-end grouping of joints by type via specialized attention heads, eliminating the need for posthoc clustering or integer-programming, resulting in both accuracy and computational efficiency. The design is modular and integrates seamlessly with existing transformer-based encoders, taking advantage of positional encodings and learnable type representations to encode spatial and categorical information.

6. Comparative Analysis and Broader Impact

Ablation studies demonstrate clear, albeit modest, empirical improvements over MLP-based and associative grouping approaches, particularly in speed and ease of integration with modern, highly optimized pipelines. The generality of dot-product attention (both type-specific and general) suggests CenterAtt is applicable beyond its original contexts for center-keypoint grouping, ROI feature aggregation, or structured object-part association. The fully differentiable and trainable design opens avenues for integrating CenterAtt in tasks requiring assignment, grouping, or selection conditioned on set-level context and structured outputs (Xu et al., 2021, Brasó et al., 2021).