DenseBEV: BEV-Based NMS in 3D Detection

• DenseBEV is a method that applies differentiable BEV-based non-maximum suppression to reduce redundant anchor proposals in dense grids.
• It integrates pre-decoder and in-decoder suppression within transformer-based architectures to optimize gradient flow and detection precision.
• Empirical results on datasets like nuScenes and Waymo Open show significant improvements in small-object recall and overall detection accuracy.

Bird's-Eye-View (BEV)-based Non-Maximum Suppression (NMS) is a differentiable suppression mechanism tailored for transformer-based multi-camera 3D object detection architectures in automotive and robotics domains. Unlike traditional approaches, BEV-based NMS operates directly on the 2D BEV projection of predicted objects, offering a computationally efficient alternative to full 3D NMS. Its integration within the training and inference loop enables effective duplicate removal among densely proposed anchors while propagating gradients only through non-suppressed detections. The DenseBEV system exemplifies this approach, leveraging BEV-based NMS to select and refine object proposals on a dense BEV grid before and during transformer decoding, thus improving efficiency and detection quality, particularly for small objects (Dähling et al., 18 Dec 2025).

1. Motivation and Design Principles

BEV-based NMS arises from the need to manage the significant redundancy inherent in dense BEV-based anchor proposals. In architectures such as DenseBEV, each cell in a typical $200 \times 200$ BEV grid is treated as a potential detection anchor, resulting in up to 40,000 proposals per frame. Naïvely passing all proposals into a DETR-style transformer decoder is computationally prohibitive and results in massive duplicate hypotheses.

Standard NMS, often performed on 3D rotated bounding boxes as a post-processing step, is computationally expensive and non-differentiable, preventing the network from learning effective de-duplication strategies throughout training. By contrast, BEV-based NMS exploits the sparsity of vertical object overlap in automotive scenes, reducing NMS complexity by suppressing proposals only on the 2D BEV plane (where $z$-overlap is rare). Importantly, incorporating class-agnostic BEV-based NMS within the computational graph both reduces the query set entering the decoder and restricts gradient flow to viable, non-suppressed anchors, thus avoiding the need for post-hoc filtering (Dähling et al., 18 Dec 2025).

2. Mathematical Formulation

BEV-based NMS relies on the calculation of Intersection over Union (IoU) between rotated 2D bounding boxes projected onto the BEV plane. Each candidate box $b_k$ is parameterized by its center $(x, y)$, orientation $\theta$, width $w$, and length $\ell$. The BEV IoU between two boxes $b_i$ and $b_j$ is defined as

$$\mathrm{IoU}_{\mathrm{BEV}}(b_i, b_j) = \frac{\mathrm{area}(b_i \cap b_j)}{\mathrm{area}(b_i) + \mathrm{area}(b_j) - \mathrm{area}(b_i \cap b_j)}$$

where intersection and area computations account for the box rotation in the BEV plane. CUDA-accelerated routines, as implemented in OpenPCDet, facilitate efficient computation of rotated box overlaps (Dähling et al., 18 Dec 2025).

3. Stepwise BEV-Based NMS Algorithm

Given an input grid of $Q=N \cdot M$ candidate queries, each cell predicts a BEV box $b_k$ and an associated confidence score $s_k$ using an auxiliary detection head. The BEV-based NMS selection process proceeds as follows:

1. Sort all candidate indices by descending confidence score.
2. Iteratively consider each candidate in order. For candidate $i$, suppress it if its BEV IoU with any already-selected (kept) candidate $j$ exceeds the suppression threshold $\tau$.
3. Retain candidates that are not suppressed, producing a list of "kept" indices.

This process is applied both before the decoder (pre-decoder suppression) and within each decoder layer (using the same mask). After initial BEV-NMS, the highest-scoring $K_\text{in}$ (e.g., 900) queries are selected for decoding. During each decoder layer, an auxiliary head produces new boxes and scores; BEV-NMS is recomputed to generate an updated attention mask, blocking attention from and to suppressed queries. Gradients are propagated only through non-suppressed queries.

The following table summarizes the principal algorithmic steps as implemented in DenseBEV:

Step	Description	Purpose
BEV IoU Computation	Compute pairwise IoU in BEV between all candidate boxes	Identifies duplicate spatial anchors
Pre-Decoder Suppression	Run BEV_NMS to select a subset of queries and build initial attention mask	Reduces input queries to decoder
Top-$K_\text{in}$ Selection	Choose $K_\text{in}$ highest-scoring queries from kept set	Enforces limited, high-quality input
In-Decoder Suppression	Per decoder layer: update boxes/scores, re-run BEV_NMS, update mask	Ensures ongoing de-duplication
Gradient Propagation	Only "kept" queries receive gradients; suppressed queries' gradients are zeroed out	Focuses learning on non-duplicates

4. Hyper-Parameterization and Ablation

The suppression threshold $\tau$ governs the aggressiveness of NMS. For sparser grids, $\tau=0.2$ is typical; for denser, "base" grids, $\tau=0.1$ is preferred. Lower thresholds eliminate more duplicates but risk suppressing true positives; empirical ablation indicates optimal performance for $\tau$ in the $0.1$–$0.2$ range, while accuracy degrades for $\tau > 0.3$.

$K_\text{in}$, the number of decoder input queries, is fixed at 900 in DenseBEV experiments, balancing computational tractability with detection performance. For hybrid temporal modeling (which incorporates temporal BEV information), the memory queue length of stored prior queries (300) is treated as another top-$K$ hyper-parameter (Dähling et al., 18 Dec 2025).

5. Integration within Transformer-Based Detection Architectures

In transformer-based pipelines such as DenseBEV, BEV-based NMS operates at the interface between the BEV encoder and the DETR-style transformer decoder. The pipeline proceeds as:

• BEV encoder (e.g., BEVFormer) produces feature grid $G^t \in \mathbb{R}^{n \times m \times C}$.
• Each cell is decoded to a BEV box and confidence score.
• BEV-based NMS is applied to select the initial sparse set of non-overlapping proposals (keep_pre).
• The top-$K_\text{in}$ queries are provided as initial queries for decoding.
• Each decoder layer recomputes BEV-NMS and updates the attention mask, aligning cross- and self-attention to only operate among non-suppressed queries.
• Suppressed queries are blocked (masked to $-\infty$ in attention, resulting in zeroed gradients).

Pre-decoder BEV-NMS incurs $O(Q^2)$ complexity in theory ($Q = N \cdot M$), but in practice, aggressive suppression and CUDA-optimized routines keep run-time overhead tractable (approximately +18.5% on a 2080Ti for a $200 \times 200$ grid). Decoder complexity remains fixed since the number of queries is controlled. Memory overhead is negligible (Dähling et al., 18 Dec 2025).

6. Comparison with Standard Post-Processing NMS

Standard 3D detector pipelines apply NMS to final bounding boxes after network inference as a non-differentiable post-processing step. This prevents the model from learning to propose separated, high-confidence anchors and does not optimize end-to-end for reduced duplication. BEV-based NMS, applied "in-loop" during training and inference, enables the model to learn confident, well-separated hypotheses immediately at the anchor generation stage.

BEV-based suppression is computationally favorable, leveraging 2D overlap computations that avoid the complexities of z-axis overlap in 3D IoU. By directing learning and refinement capacity to the non-duplicated subset of proposals, BEV-NMS facilitates both improved localization and duplicate mitigation at every decoder layer. Empirically, the approach recovers small-object recall (e.g., for pedestrians and traffic cones) by dense anchor consideration followed by aggressive duplicate filtering—capabilities not achievable using only random or post-hoc NMS (Dähling et al., 18 Dec 2025).

7. Empirical Impact and Observed Benefits

DenseBEV, employing BEV-based NMS, demonstrates significant empirical advances on standard benchmarks. On the nuScenes dataset, stability in NDS and mAP is observed even with sparser BEV grids, and pedestrian mAP improves by 3.8%. On the Waymo Open dataset, the approach achieves a LET-mAP of 60.7%, surpassing prior methods by 5.4%. Notably, these gains are consistent across configurations and especially pronounced for small-object classes, underscoring the practical effectiveness of BEV-based NMS in transformer-based detection pipelines (Dähling et al., 18 Dec 2025).