---
title: 'EV-Pose: Event-Based 3D Pose Estimation'
url: https://www.emergentmind.com/topics/ev-pose
type: topic
---

# EV-Pose: Event-Based 3D Pose Estimation

EV-Pose denotes an event-based human pose estimation framework that models the asynchronous output of event cameras as a 3D spatiotemporal signal rather than collapsing it into conventional accumulated event frames. In its initial formulation, EV-Pose estimates 2D human pose directly from a rasterized 3D event point cloud and then recovers 3D pose by triangulation on DHP19 [2206.04511]. A subsequent extension generalized this line of work through two explicit 3D event representations—Rasterized Event Point Cloud (RasEPC) and Decoupled Event Voxel (DEV)—and evaluated them on DHP19, EV-3DPW, EV-JAAD, and outdoor collections [2311.04591]. The central thesis across these works is that event-frame accumulation is too lossy for exploiting the temporal order, sparsity, and low-latency characteristics that make event cameras useful under high dynamic range, fast motion, and motion blur.

## 1. Problem formulation and motivation

EV-Pose addresses human pose estimation from event streams, where a raw event is represented as
\[
e=(x,y,t,p),
\]
with \((x,y)\) denoting pixel location, \(t\) the timestamp, and \(p\) the polarity of brightness change [2206.04511]. The initial EV-Pose formulation uses an event stream from a Dynamic Vision Sensor as input, predicts 2D human joint locations in each camera view, and then triangulates the 2D predictions to recover 3D joint positions [2206.04511].

The methodological motivation is a critique of the dominant event-frame paradigm. Prior event-based pose methods commonly accumulate asynchronous events into 2D frames or voxel grids and then apply CNNs. EV-Pose argues that this weakens or discards the temporal order that is crucial for distinguishing actions. The later EV-Pose paper states explicitly that actions such as moving a hand up versus down may generate similar accumulated event images even though their temporal order differs, and therefore reframes the event stream as a 3D spatiotemporal signal rather than as a 2D image sequence [2311.04591].

The initial benchmark setting is DHP19, described as a real-world event dataset for 3D human pose estimation with four synchronized cameras and Vicon motion-capture ground truth. In the 2022 protocol, training uses subjects S1–S12, testing uses S13–S17, and only the two front camera views are used, matching the original DHP19 protocol [2206.04511]. This setup places EV-Pose in a multi-view event-based HPE regime rather than a monocular 3D lifting regime.

## 2. Rasterized Event Point Cloud

The first core representation in EV-Pose is the Rasterized Event Point Cloud, designed to preserve the discrete point-cloud nature of events while reducing redundancy. Events within a short window are divided into \(K\) temporal slices, and events at the same pixel position within a slice are aggregated. For events in slice \(k\) at pixel \((x,y)\),
\[
E_k(x,y)=(x,y,t_i,p_i),\quad i=1,\dots,M,
\]
the rasterized event is
\[
E'_k(x,y)=(x,y,t_{avg},p_{acc},e_{cnt}),
\]
where
\[
t_{avg}=\frac{1}{M}\sum_i^M t_i,\quad p_{acc}=\sum_i^M p_i,\quad e_{cnt}=M.
\]
The timestamp channel \(t_{avg}\) is normalized to \([0,1]\), and \(K=4\) is used as a practical trade-off between temporal resolution and compactness [2206.04511; 2311.04591].

This representation preserves spatial location \((x,y)\), average timing information \(t_{avg}\), accumulated polarity \(p_{acc}\), and local event density \(e_{cnt}\). The papers characterize these as multiple statistical cues that retain 3D event structure while significantly reducing memory consumption and computation complexity. Inference then proceeds by sampling the rasterized point set to a fixed size, with random point sampling preferred over furthest point sampling for latency reasons; 2048 points are used as the default compromise in the 2022 study [2206.04511].

RasEPC is paired with point-cloud backbones including PointNet, DGCNN, and Point Transformer. In the original EV-Pose pipeline, the backbone output is decoded by two linear layers into two 1D coordinate distributions, one for \(x\) and one for \(y\), using a SimDR-style target:
\[
\boldsymbol{p}_v=[v_0,v_1,\ldots,v_S]\in\mathbb{R}^S,\quad v\in\{x,y\},
\]
with Gaussian-smoothed entries
\[
v_i=\frac{1}{\sqrt{2\pi}\sigma}\exp\left(-\frac{(i-v')^2}{2\sigma^2}\right).
\]
The predicted coordinate is obtained by
\[
\mathrm{pred}_v=\arg\max_j \hat{\boldsymbol{p}}_v(j),
\]
and the 2022 paper reports \(\sigma=8\) as the best-performing setting on DHP19 [2206.04511].

The practical implication is that RasEPC treats the event stream as sparse structured geometry rather than as a dense image tensor. This suggests a closer affinity to point-set learning than to conventional image-based HPE.

## 3. Decoupled Event Voxel and Decoupled Event Attention

The second major EV-Pose representation is the Decoupled Event Voxel, introduced to retain richer spatiotemporal structure while avoiding the cost of full 3D convolutions. A dense voxel representation is written as
\[
V \in \mathbb{R}^{C \times H \times W \times T},
\]
which incurs \(O(HWT)\) storage and computation. DEV reduces this by projecting the event volume onto three orthogonal planes,
\[
D=[D^{HW},D^{TH},D^{WT}],
\]
with
\[
D^{HW}\in\mathbb{R}^{C\times H\times W},\quad
D^{TH}\in\mathbb{R}^{C\times T\times H},\quad
D^{WT}\in\mathbb{R}^{C\times W\times T},
\]
corresponding to the \(xy\), \(xt\), and \(yt\) planes. The resulting complexity is
\[
O(HW+TH+WT),
\]
which is explicitly presented as much lower than full voxelization [2311.04591].

For a point \((x,y,\tau)\), features are sampled from the three planes and aggregated by Decoupled Event Attention (DEA). DEA summarizes the temporal planes along their temporal axes, expands them to the \(xy\)-plane size, computes correlations with the spatial plane, and concatenates the resulting features:
\[
f_{DEV}=f_{hw}\oplus(C_h\cdot f_{th})\oplus(C_w\cdot f_{wt}).
\]
The paper reports that average pooling is better than max pooling in DEA because event data can be noisy and average pooling is more robust [2311.04591].

DEV uses standard 2D CNN backbones on the projected planes, including the DHP19 backbone, ResNet, and MobileHP. Its labels are standard 2D Gaussian heatmaps,
\[
\boldsymbol{p}=\exp\left(-\frac{(\boldsymbol{v}-\boldsymbol{v}')^2}{2\sigma^2}\right),
\]
with heatmap size \(64\times64\) and \(\sigma=2\) [2311.04591].

| Representation | Construction | Key properties |
|---|---|---|
| RasEPC | Aggregate same-pixel events within \(K\) slices into \((x,y,t_{avg},p_{acc},e_{cnt})\) | Sparse point-cloud input; used with PointNet, DGCNN, Point Transformer |
| DEV | Project voxelized events onto \(xy\), \(xt\), and \(yt\) planes and fuse with DEA | \(O(HW+TH+WT)\); compatible with standard 2D CNNs |

The two representations are complementary rather than redundant. RasEPC emphasizes sparse point efficiency, whereas DEV emphasizes richer 3D cues recoverable from tri-plane projections.

## 4. Supervision, inference pipeline, and dataset ecology

In the 2022 EV-Pose formulation, the full inference chain is: raw event stream, rasterized event point cloud, fixed-size point sampling, 3D backbone encoding, two linear decoders for \(x\) and \(y\), 2D keypoint prediction by argmax, and triangulation into 3D pose using camera projection matrices [2206.04511]. The training loss for RasEPC is Kullback–Leibler divergence between predicted and target 1D vectors. Training uses Adam for 30 epochs on a single RTX 3090, with latency measured on an NVIDIA Jetson Xavier NX using batch size 1 [2206.04511].

The later EV-Pose study broadens the dataset ecology. In addition to DHP19, it introduces EV-3DPW, a synthetic event-based dataset derived from 3DPW using the ESIM event simulator. EV-3DPW adapts the event resolution to a DAVIS-346-style setting, \(256\times480\) or \(480\times256\), crops each sample to separate each person instance, and follows 3DPW splits with 23,475 training samples and 40,145 test samples [2311.04591]. The paper also derives EV-JAAD from the JAAD driving dataset for zero-shot qualitative evaluation and reports qualitative validation on outdoor collected driving and mobile robot event data [2311.04591].

The initial DHP19 study compares two label-generation strategies for event windows: Mean Label, defined as the average 3D joint position over the event window, and Last Label, defined as the label closest to the last event in the window. Mean Label performs better across all three point-cloud backbones, which the authors attribute to its use of both early and late events and its robustness to instantaneous Vicon labeling noise [2206.04511].

A recurrent theme is that EV-Pose is not restricted to a single network family. It is better understood as a representation-centric program: point-cloud backbones for RasEPC and 2D CNNs with tri-plane fusion for DEV.

## 5. Empirical results and ablation structure

On DHP19, the headline result of the original EV-Pose paper is that PointNet with 2048-point RasEPC input achieves **82.46 mm** in MPJPE\(_{3D}\) with **12.29 ms** latency on Jetson Xavier NX [2206.04511]. The same study reports the following DHP19 comparison for point-cloud backbones: PointNet, **82.46 mm** and **12.29 ms**; DGCNN, **77.32 mm** and **127.96 ms**; Point Transformer, **73.37 mm** and **497.27 ms**. Event-frame baselines reported in the same table include Pose-ResNet18 at **61.03 mm** and **68.07 ms**, Pose-ResNet50 at **59.83 mm** and **93.56 ms**, MobileHP-S at **64.14 mm** and **48.09 ms**, LeViT-128S at **87.79 mm** and **14.40 ms**, and the DHP19 baseline at **87.90 mm** and **27.55 ms** [2206.04511].

The 2023 paper sharpens the representation argument through ablations. For PointNet with 2048 points on DHP19, raw \((x,y,t)\) input gives **310.65 mm**, raw \((x,y,t,p)\) gives **310.64 mm**, normalized \((x,y,t_{norm})\) gives **89.62 mm**, normalized \((x,y,t_{norm},p_{\pm1})\) gives **86.07 mm**, rasterized \((x,y,t_{avg})\) gives **87.59 mm**, rasterized \((x,y,t_{avg},p_{acc})\) gives **84.58 mm**, and rasterized \((x,y,t_{avg},p_{acc},e_{cnt})\) gives **82.46 mm** [2206.04511]. This establishes that timestamp normalization and richer rasterized statistics are both material contributors.

For DEV on DHP19, the paper reports that DEV improves MPJPE by **18.25%** on the DHP19 backbone compared to the event-frame baseline. With a ResNet18 backbone, event frames with Pose-ResNet50 obtain **5.28 px**, whereas DEV with ResNet18 obtains **4.93 px** [2311.04591]. View ablations further show that \(xy\) alone yields **7.67 / 87.90**, \(xy+xt\) yields **6.42 / 72.45**, \(xy+yt\) yields **6.60 / 74.90**, and \(xy+xt+yt\) yields **6.27 / 71.01**, reported as 2D/3D MPJPE. Fusion ablations compare addition, concatenation, attentional feature fusion, and DEA; the best result is **DEA: 4.93 / 55.53** [2311.04591].

On EV-3DPW, the later paper reports that event-based methods outperform RGB under the same backbone and that 3D event representations outperform 2D event-frame methods in more challenging cases. For Pose-ResNet18, RGB obtains **27.50**, event frame **17.87**, and DEV **15.68**, corresponding to a **12.3% improvement** over the event-frame baseline [2311.04591]. RasEPC is described as low-latency and suitable when computational efficiency matters, while DEV is presented as the most accurate of the two 3D representations [2311.04591].

| Setting | Result | Note |
|---|---|---|
| RasEPC + PointNet on DHP19 | **82.46 mm** MPJPE\(_{3D}\), **12.29 ms** | 2048 points on Jetson Xavier NX |
| RasEPC + Point Transformer on DHP19 | **73.37 mm**, **497.27 ms** | Highest EV-Pose accuracy among point backbones in the 2022 study |
| DEV on DHP19 | **18.25%** MPJPE improvement | Relative to the event-frame baseline on the DHP19 backbone |
| Pose-ResNet18 on EV-3DPW | RGB **27.50**, event frame **17.87**, DEV **15.68** | **12.3%** improvement of DEV over event frame |

These results support a stable conclusion across both papers: preserving 3D spatiotemporal structure improves robustness and accuracy, but the choice between RasEPC and DEV remains a speed-accuracy trade-off rather than a universally dominant design.

## 6. Relation to adjacent work, limitations, and nomenclature

Within event-based human pose estimation, EV-Pose is closely related to "EventHPE: Event-based 3D Human Pose and Shape Estimation" [2108.06819]. EventHPE extends the problem from 2D pose plus triangulated 3D reconstruction to 3D pose and SMPL shape estimation over time using a two-stage event-to-flow-to-shape pipeline. The relation is methodological rather than terminological: both approaches treat event streams as motion-centric signals, but EV-Pose is centered on 3D event representations for keypoint estimation, whereas EventHPE is centered on optical flow and temporal body-shape recovery [2108.06819].

Several limitations are explicit. The original EV-Pose predicts **2D poses first**, then triangulates to 3D; it does **not** perform direct 3D regression and does **not** include temporal continuity optimization based on confidence thresholds, though the paper notes that such optimization could improve results [2206.04511]. The later study states that multi-person use is **top-down** and depends on preprocessed bounding boxes, is **not yet end-to-end multi-person pose estimation**, and that the synthetic outdoor dataset still leaves room for more real outdoor benchmarks [2311.04591]. These constraints are important because they delimit the operational scope of the reported gains.

The name itself requires careful disambiguation. EV-Pose in event-based HPE is distinct from "EvaPose" for egocentric visibility-aware HPE [2602.23618], "EvoPose" for transformer-based 3D pose lifting with kinematic priors [2306.09615], "EvoPose2D" for neuroevolution-based 2D pose estimation [2011.08446], and "BEVPose" for pose-guided multi-modal BEV alignment [2410.20969]. The same string has also been used for a drone-oriented event-camera visual positioning service for landing [2510.00646]. A practical implication is that the term is best interpreted together with its task domain and citation rather than as a unique model name.

In the narrower and historically earlier human-pose sense, EV-Pose is best characterized as a representation-driven event-HPE framework built around the claim that event data should be processed as 3D spatiotemporal structure—either sparse point clouds or decoupled tri-plane voxels—rather than coerced into frame-like encodings [2206.04511; 2311.04591].

Source: https://www.emergentmind.com/topics/ev-pose