KeypointBEV: A BEV Keypoints Framework

• KeypointBEV is a design pattern that uses sparse keypoints in a BEV coordinate system rather than dense grids for object, lane, and hand keypoint prediction.
• It adapts task-specific decoding strategies, supporting LiDAR object detection, monocular 3D lane estimation, graph-based lane topology recovery, and stereo event-based hand pose refinement.
• By eliminating dense voxelization, KeypointBEV achieves efficiency and flexibility for real-time applications on embedded hardware and various sensor modalities.

KeypointBEV denotes a set of bird’s-eye-view formulations in which prediction is organized around keypoints in a canonical spatial frame rather than around dense 3D convolutions or fixed top-down lane templates. In the cited literature, the label covers BEVDetNet’s single unified model for object center detection, box prediction, and orientation prediction from LiDAR BEV (Mohapatra et al., 2021), BEV-LaneDet’s key-points representation for monocular 3D lane detection (Wang et al., 2022), GLane3D’s graph of 3D lane keypoints with PointNMS and connection prediction (Öztürk et al., 31 Mar 2025), and EgoEV-HandPose’s geometry-anchored stereo fusion module that lifts event-based features into a canonical BEV space for 3D hand pose estimation (Wang et al., 12 May 2026). The common structure is a BEV-referenced keypoint layer followed by task-specific decoding: boxes for objects, clustered curves for lanes, graphs for lane topology, and iterative reprojection refinement for 3D joints.

1. Terminological scope and task variants

The term is not tied to a single benchmark or sensor stack. In the available literature it appears in LiDAR 3D object detection, monocular and graph-based 3D lane detection, and stereo event-based hand pose estimation. What remains stable is the use of BEV as the organizing coordinate system and the use of keypoints as the atomic prediction units.

System	Input and BEV construction	Keypoint role
BEVDetNet	LiDAR point cloud projected to a 512 × 256 BEV image	Object-center detection
BEV-LaneDet	Monocular image warped to a virtual camera and transformed to a BEV grid	Lane key-points per cell
GLane3D	Front-view image features inverse-perspective-mapped to non-uniform BEV anchors	Lane keypoint proposals and graph nodes
EgoEV-HandPose	Synchronized left/right event streams fused under calibrated stereo geometry	42 hand keypoints in a canonical 3D frame

A recurrent misconception is that BEV keypoint methods are necessarily dense raster predictors. That is true for BEVDetNet and BEV-LaneDet, but GLane3D uses sparse top-$N$ anchor proposals in BEV, and EgoEV-HandPose explicitly states that it does not build a dense voxel grid and does not explicitly rasterize a dense BEV grid (Öztürk et al., 31 Mar 2025, Wang et al., 12 May 2026). Another misconception is that “keypoint” always means an anatomical landmark. In these works, the term spans object centers, lane-support points, graph nodes, and hand joints.

2. BEV parameterization and geometric lifting

In BEVDetNet, the input is a Bird’s-Eye-View image of size $512 \times 256$ with three channels per cell: max-height, occupancy, and reflectance. The geographic extents are $x \in [0, 60]$ m, $y \in [-30, 30]$ m, and $z \in [-2.73, 1.27]$ m, with cell resolution $\delta = 0.1$ m. Max-height is defined as the maximum $z$ coordinate of all points falling into a $0.1$ m $\times 0.1$ m cell, clipped to $[-2.73$ m, $512 \times 256$0 m and normalized; occupancy is binary; reflectance is the maximum return intensity in the cell, normalized (Mohapatra et al., 2021).

BEV-LaneDet begins from the road-ground coordinate frame $512 \times 256$1 with plane $512 \times 256$2. A rectangular region

$512 \times 256$3

is split into an $512 \times 256$4 grid, typically with $512 \times 256$5 m. The method first warps every image into a single virtual camera view $512 \times 256$6 using a homography on the road plane and then applies the Spatial Transformation Pyramid to map multiscale front-view features into BEV features (Wang et al., 2022).

GLane3D also starts from front-view image features, but the BEV grid is defined as a fixed set of anchor locations $512 \times 256$7. Using known camera intrinsics and extrinsics, each BEV cell center is inverse-perspective-mapped to the front view, and features are bilinearly sampled to form $512 \times 256$8. To avoid the usual sparsity near the vehicle, the BEV grid spacing is non-uniform: longitudinal spacing $512 \times 256$9 increases linearly from $x \in [0, 60]$0 m near to $x \in [0, 60]$1 m far, while the lateral span narrows near the ego vehicle and widens farther out (Öztürk et al., 31 Mar 2025).

EgoEV-HandPose makes the BEV interpretation explicit as a coordinate-frame construction rather than a raster. Given calibrated intrinsics $x \in [0, 60]$2 and extrinsics $x \in [0, 60]$3, a homogeneous 3D point $x \in [0, 60]$4 projects to view $x \in [0, 60]$5 as

$x \in [0, 60]$6

The module first obtains soft 2D joint positions from stereo heatmap peaks and forms an initial 3D estimate $x \in [0, 60]$7 by weighted linear-least-squares triangulation. It then reprojects the current 3D estimate into both views, samples local features, and refines the coordinates iteratively. The paper states that the module “implicitly anchor[s] each keypoint’s features in a consistent 3D coordinate frame (‘bird’s-eye’ relative to the head-mounted cameras)” (Wang et al., 12 May 2026).

3. Prediction heads and keypoint semantics

BEVDetNet is a single “segmentation-style” encoder-decoder with three parallel heads: a keypoint heatmap head for object-center classification, a box regression head for $x \in [0, 60]$8, and an orientation classification head for binned yaw. The encoder is “Hybrid-DLA,” with five downsampling blocks, dilated convolutions, and large-kernel Context-Aggregation Modules in the first three blocks; channel width doubles from $x \in [0, 60]$9. The decoder restores full $y \in [-30, 30]$0 resolution and feeds three small heads: $y \in [-30, 30]$1 conv to $y \in [-30, 30]$2 channels for the keypoint head, $y \in [-30, 30]$3 conv to three channels for $y \in [-30, 30]$4, and $y \in [-30, 30]$5 conv to $y \in [-30, 30]$6 channels for rotation $y \in [-30, 30]$7 angle bins $y \in [-30, 30]$8 background$y \in [-30, 30]$9 (Mohapatra et al., 2021).

BEV-LaneDet defines each BEV grid cell $z \in [-2.73, 1.27]$0 as a lane key-point candidate at its center $z \in [-2.73, 1.27]$1. For every cell, the network predicts four quantities: a confidence score $z \in [-2.73, 1.27]$2, an embedding vector $z \in [-2.73, 1.27]$3, a lateral offset $z \in [-2.73, 1.27]$4, and a height $z \in [-2.73, 1.27]$5. This representation decouples existence, geometry, and instance identity, which is why clustering can be deferred to inference rather than imposed by anchors or a fixed lane template (Wang et al., 2022).

GLane3D uses a single-channel segmentation head $z \in [-2.73, 1.27]$6 to predict lane-foreground scores over BEV anchors. Top-$z \in [-2.73, 1.27]$7 scoring anchors are taken as proposals $z \in [-2.73, 1.27]$8, their associated BEV features are gathered as

$z \in [-2.73, 1.27]$9

and a lightweight Transformer block with self-attention, cross-attention, and deformable attention predicts, for each proposal, a foreground score $\delta = 0.1$0, a lateral offset $\delta = 0.1$1, a height $\delta = 0.1$2, and a connection feature vector $\delta = 0.1$3. A distinctive design choice is that each ground-truth keypoint may be covered by multiple anchor proposals within a lateral distance $\delta = 0.1$4, after which the model regresses a small offset $\delta = 0.1$5 (Öztürk et al., 31 Mar 2025).

EgoEV-HandPose uses stereo event streams converted by a shared backbone into a 2D heatmap tensor $\delta = 0.1$6, a hand-segmentation mask, and a local feature map $\delta = 0.1$7, with $\delta = 0.1$8. KeypointBEV then operates in three stages: coarse lifting by weighted triangulation, a transformer-based refinement loop with $\delta = 0.1$9 iterations, and final 3D output together with reprojections for refined 2D heatmaps. Here the keypoints are anatomical joints rather than detection centers, but the logic remains BEV-referenced, geometry-constrained, and sparse (Wang et al., 12 May 2026).

4. Suppression, association, and output decoding

In BEVDetNet, inference starts from the object heatmap. The $z$0 object channel is scanned for local peaks, followed by $z$1 non-maximum suppression and an additional distance-based suppression step that rejects peaks closer than a BEV cell diagonal. Each surviving peak at pixel $z$2 is decoded to a 3D center by

$z$3

with $z$4 m. Box dimensions are recovered by exponentiation,

$z$5

and orientation is decoded from the predicted yaw bin among $z$6 bins of width $z$7 plus background (Mohapatra et al., 2021).

BEV-LaneDet performs thresholding and clustering rather than NMS. The set $z$8 is clustered by mean-shift or greedy clustering on $z$9 with gap $0.1$0 to assign lane IDs. For each cluster, the method constructs 3D key-points $0.1$1 and then fits a spline or low-order curve to recover the final 3D lane. The embedding head is therefore central to instance separation, not merely auxiliary (Wang et al., 2022).

GLane3D introduces PointNMS to suppress redundant proposals before graph construction. Each proposal is treated as the center of a small box of size $0.1$2, standard box-NMS is applied with IoU threshold $0.1$3, and only the top $0.1$4 non-overlapping keypoints are kept. After that, each retained keypoint feature is concatenated with positional encoding, passed through separate origin and destination MLPs, and converted into an adjacency matrix

$0.1$5

where $0.1$6 is the probability that keypoint $0.1$7 connects to $0.1$8. Lane extraction defines the directed edge set $0.1$9, identifies start and end nodes by incoming and outgoing degree conditions, and applies Dijkstra’s shortest-path with edge cost $\times 0.1$0 to recover complete lane polylines (Öztürk et al., 31 Mar 2025).

EgoEV-HandPose replaces clustering or graph extraction with iterative reprojection-guided refinement. At iteration $\times 0.1$1, the current 3D estimate $\times 0.1$2 is projected into both views, coordinates are normalized to $\times 0.1$3, features are bilinearly sampled by PyTorch-style grid_sample, and left-right visual features are fused with a spatial encoding derived from the current 3D estimate. A small Transformer Decoder $\times 0.1$4 predicts $\times 0.1$5, and the update is

$\times 0.1$6

After $\times 0.1$7 iterations, $\times 0.1$8 is output as the 3D keypoint set and reprojected for 2D supervision (Wang et al., 12 May 2026).

5. Objective functions and training regimes

BEVDetNet uses a three-term loss,

$\times 0.1$9

with $[-2.73$0 set to $[-2.73$1. Keypoint classification is weighted cross-entropy with class weights $[-2.73$2; box regression is Smooth $[-2.73$3 on $[-2.73$4 at positive keypoints; rotation is weighted cross-entropy over $[-2.73$5 angle bins plus background, using the same frequency-based weighting scheme. The yaw normalization maps $[-2.73$6 into $[-2.73$7 before binning (Mohapatra et al., 2021).

BEV-LaneDet separates supervision into confidence, offset, height, and embedding losses. Confidence uses binary cross-entropy,

$[-2.73$8

offset and height are MSE losses over positive cells, and the embedding term is

$[-2.73$9

with pull-push behavior for same-lane and different-lane embeddings. The method is therefore jointly supervised for occupancy, geometry, and instance grouping (Wang et al., 2022).

GLane3D employs Hungarian matching twice: proposals $512 \times 256$00 are matched to repeated ground-truth keypoints so that each true point can match multiple proposals, and post-NMS keypoints $512 \times 256$01 are matched once. The per-head losses are $512 \times 256$02 for segmentation/classification, $512 \times 256$03 for $512 \times 256$04 regression, $512 \times 256$05 for lane type if used, and $512 \times 256$06 as focal loss on adjacency entries for matched keypoint pairs. The total objective is

$512 \times 256$07

Implementation details include Adam with learning rate $512 \times 256$08, linear warm-up and cosine annealing, $512 \times 256$09 epochs on OpenLane v1.2 with batch size $512 \times 256$10, and $512 \times 256$11 epochs on Apollo with batch size $512 \times 256$12 (Öztürk et al., 31 Mar 2025).

EgoEV-HandPose adopts a staged curriculum. First, the 2D backbone is pretrained with

$512 \times 256$13

Then KeypointBEV is trained with frozen backbone using

$512 \times 256$14

Action classification is pretrained separately with cross-entropy, and end-to-end fine-tuning combines heatmap, segmentation, action, and iterative 2D/3D losses. Typical weights are $512 \times 256$15, $512 \times 256$16, $512 \times 256$17, and $512 \times 256$18. The paper explicitly states that the stage-wise schedule gradually shifts emphasis from early 2D losses to later 3D refinement losses (Wang et al., 12 May 2026).

6. Empirical characteristics, efficiency, and scope

BEVDetNet emphasizes deterministic, low-latency deployment on embedded hardware. On KITTI Car, it reports AP @ IoU $512 \times 256$19 of $512 \times 256$20 for moderate, with easy $512 \times 256$21, hard $512 \times 256$22, and average $512 \times 256$23; at IoU $512 \times 256$24, moderate AP is $512 \times 256$25, with easy $512 \times 256$26, hard $512 \times 256$27, and average $512 \times 256$28. Reported latency is approximately $512 \times 256$29 ms/frame on a desktop RTX-class GPU and approximately $512 \times 256$30 ms/frame on embedded NVIDIA Xavier AGX with TensorRT optimization. It is reported as approximately $512 \times 256$31 faster than PointPillars on Xavier, with only about $512 \times 256$32 AP drop at IoU $512 \times 256$33, and more than $512 \times 256$34 faster than many two-stage detectors such as PointRCNN at $512 \times 256$35 ms. Quantized variants report FP16 at $512 \times 256$36 ms/frame, $512 \times 256$37 MB, AP approximately $512 \times 256$38, and INT8 at $512 \times 256$39 ms/frame, $512 \times 256$40 MB, AP approximately $512 \times 256$41 (Mohapatra et al., 2021).

BEV-LaneDet emphasizes a light-weight and chip-friendly formulation. On OpenLane, it reports $512 \times 256$42 versus PersFormer’s $512 \times 256$43, $512 \times 256$44-error near/far of $512 \times 256$45 m, $512 \times 256$46-error near/far of $512 \times 256$47 m, and speed on V100 of $512 \times 256$48 FPS in PyTorch and $512 \times 256$49 FPS in TensorRT. On Apollo 3D synthetic, it reports balanced-scene $512 \times 256$50, rarely-observed $512 \times 256$51, and visual-variant $512 \times 256$52. Its OpenLane ablation attributes gains to Virtual Camera, Spatial Transformation Pyramid, and Key-Points Representation, with the full system reaching $512 \times 256$53 at $512 \times 256$54 FPS and a ResNet18 variant reaching $512 \times 256$55 at $512 \times 256$56 FPS (Wang et al., 2022).

GLane3D centers its empirical claim on generalization. On OpenLane, GLane3D-Base with ResNet-50 reports $512 \times 256$57 at the $512 \times 256$58 m threshold, compared with PersFormer $512 \times 256$59, BEV-LaneDet $512 \times 256$60, LATR $512 \times 256$61, and PVALane $512 \times 256$62. At the $512 \times 256$63 m threshold, it reaches $512 \times 256$64 versus LATR $512 \times 256$65 and DV-3DLane $512 \times 256$66. Category-wise gains are reported as $512 \times 256$67 on curve lanes, $512 \times 256$68 on merge/split, and $512 \times 256$69 on intersection. On Apollo Synthetic, balanced scenes achieve $512 \times 256$70 and rare scenes $512 \times 256$71. In cross-dataset evaluation from OpenLane to Apollo, it reports $512 \times 256$72 at $512 \times 256$73 m and $512 \times 256$74 at $512 \times 256$75 m, ahead of PersFormer and LATR (Öztürk et al., 31 Mar 2025).

EgoEV-HandPose extends the paradigm beyond automotive perception. It introduces EgoEVHands, a stereo event-camera dataset containing $512 \times 256$76 annotated sequences with dense 3D/2D keypoints across $512 \times 256$77 gesture classes. The full system reports an MPJPE of $512 \times 256$78 mm and $512 \times 256$79 Top-1 gesture recognition accuracy. KeypointBEV itself is described as sparse and efficient because no dense voxel or cost-volume is constructed; all sampling is sparse at the $512 \times 256$80 keypoint locations, yielding approximately $512 \times 256$81 GFLOPs and $512 \times 256$82 M parameters (Wang et al., 12 May 2026).

Taken together, these results indicate that KeypointBEV is best understood not as one fixed architecture but as a reusable design pattern: establish a BEV-referenced coordinate system, predict or lift sparse keypoints, and recover structure through decoding mechanisms appropriate to the task. The cited works show that this pattern supports real-time LiDAR detection, fast monocular lane estimation, graph-based lane topology recovery, and stereo event-based 3D hand pose refinement under explicit camera geometry.