SEPose: Synthetic Event-Based Pose Dataset

• SEPose is a synthetic event-based human pose estimation dataset that provides 73,000 event frames and over 350,000 annotated pedestrians for realistic traffic monitoring.
• It utilizes dynamic vision sensors in the CARLA simulator to capture diverse conditions including day, night, rain, fog, and distracted pedestrian scenarios using precise sensor simulation.
• The dataset supports sim-to-real benchmarking with baseline models like RVT and YOLOv8x, highlighting performance gaps that guide further improvements in safety-critical applications.

Searching arXiv for the specified paper and closely related event-based pose-estimation references. I’ll look up arXiv entries relevant to SEPose and the datasets/models it is compared against. SEPose is a synthetic event-based human pose estimation dataset for pedestrian monitoring generated using dynamic vision sensors in the CARLA simulator. It is described as, to the authors’ knowledge, the first large-scale, fully synthetic event-based human pose estimation dataset created expressly for pedestrian monitoring at traffic intersections, and it is framed as a response to the limited availability of data covering safety-critical scenarios such as low light, adverse weather, and distracted walking in realistic traffic-camera views. At its core, SEPose contains 73 000 event frames and corresponding raw event streams, over 350 000 individual pedestrians annotated with full body keypoints, and fixed-camera views spanning urban, suburban, and rural four-way intersections under day-night and weather variation. The dataset is also used to train state-of-the-art models, including RVT and YOLOv8, and to assess sim-to-real generalization on real event-based pose data (Chanda et al., 16 Jul 2025).

1. Research setting and stated motivation

SEPose is situated in event-based pedestrian and traffic monitoring, where event-based sensors are described as promising because their low-latency and high dynamic range allow for improved response time in safety-critical situations caused by distracted walking or other unusual movements (Chanda et al., 16 Jul 2025). The stated problem is that the availability of data covering such scenarios remains limited. The dataset is therefore presented as filling a gap in neuromorphic-vision research for intelligent transportation systems, specifically the lack of publicly available data capturing people in realistic traffic-camera views under challenging conditions.

The design emphasis is fixed pedestrian perception at traffic intersections rather than ego-centric sensing or laboratory capture. The scenes include busy and light crowds, traffic, pedestrian-vehicle interactions, complete traffic signal cycles, and environmental variation across day, night, rain, fog, and twilight. This suggests that SEPose is intended not merely as a pose-estimation corpus, but as a benchmark aligned with traffic-camera deployment conditions in pedestrian safety systems.

2. Dataset composition and scene coverage

SEPose consists of the following components and scene categories (Chanda et al., 16 Jul 2025):

Component	Specification
Event data	73 000 event frames and corresponding raw event streams
Human annotations	over 350 000 individual pedestrians annotated with full body keypoints
Towns	Town03, Town10; Town04, Town05; Town07, Town12
Environment types	urban, suburban, and rural
Traffic setting	busy four-way intersections, complete traffic signal cycles, pedestrian-vehicle interactions
Condition variation	day-night plus weather variations (rain, fog, twilight)

The six CARLA towns are explicitly grouped into urban scenes (Town03, Town10), suburban scenes (Town04, Town05), and rural scenes (Town07, Town12). The number of pedestrians per simulation is sampled from $\mathcal{N}(\mu = 50, \sigma = 15)$, yielding scenes ranging from solo walkers to heavy clusters. A cleaning pipeline rejects any pedestrian whose bounding box area falls below $300\ \text{px}^2$ or is over $50\%$ occluded, with occlusion determined by UE4 ray-casts to each keypoint. After this pruning, the typical frame contains $1$–$8$ visible people, supporting both single-person and multi-person pose estimation.

SEPose does not impose a fixed train/val/test split. A recommended example is an $80/10/10$ split over the 73 000 frames so as to preserve the diversity of towns, lighting, and weather across all sets. A plausible implication is that evaluation protocol remains, at least in part, a matter of experimental design rather than a fully standardized benchmark split.

3. Event-camera simulation and event representation

The synthetic DVS simulation is carried out in CARLA (Unreal Engine 4) using CARLA’s built-in DVS sensor, with $\text{FoV} = 90^\circ$, resolution $= 800 \times 600$, and dynamic range $= 140\ \text{dB}$ (Chanda et al., 16 Jul 2025). The motivation for this sensor choice is consistent with the paper’s emphasis on microsecond latency and dynamic range greater than $120\ \text{dB}$ for difficult lighting conditions.

At each $300\ \text{px}^2$0 simulation step in synchronous mode, a stream of events $300\ \text{px}^2$1 is emitted whenever the change in log-intensity at pixel $300\ \text{px}^2$2 exceeds a contrast threshold $300\ \text{px}^2$3:

$300\ \text{px}^2$4

$300\ \text{px}^2$5

$300\ \text{px}^2$6

Positive polarity $300\ \text{px}^2$7 denotes a rise in brightness, while negative polarity $300\ \text{px}^2$8 denotes a fall. Events are accumulated into $300\ \text{px}^2$9 event frames for visualization and for models that expect tensor inputs.

This event formulation is central to the dataset’s role as an event-based benchmark rather than a frame-based one. It also explains the dual release format: raw event streams for methods operating directly on $50\%$0 and accumulated event frames for architectures that consume tensorized image-like representations.

4. Pose schema, projection pipeline, and annotation format

Each pedestrian is endowed with $50\%$1 keypoints following a COCO-style ordering:

$50\%$2

At runtime, CARLA provides each pedestrian’s skeleton in the world coordinate frame. That skeleton is transformed through the camera’s extrinsic matrix into the sensor frame and then projected via the intrinsic matrix to pixel coordinates $50\%$3 (Chanda et al., 16 Jul 2025). All keypoints $50\%$4 are recorded in COCO-compatible JSON annotation files. Each annotation lists

$50\%$5

together with a center point $50\%$6 used for detection.

The annotation procedure couples simulator-provided 3D skeletons with camera geometry, so the final labels are 2D keypoints in sensor coordinates rather than unprojected kinematic states. This suggests that SEPose is directly usable with standard 2D pose-estimation pipelines while retaining a physically grounded generation process.

5. Baseline models and reported results

Two state-of-the-art architectures are reported as baselines: Recurrent Vision Transformer (RVT) with an event-tensor head trained on raw $50\%$7 tensors, and YOLOv8x trained on $50\%$8 event frames (Chanda et al., 16 Jul 2025). Both models are trained from scratch using standard object-detection and pose-regression losses, specifically an $50\%$9 keypoint loss plus objectness/bounding-box IoU loss, and both are reported to converge within $1$0 epochs. The paper also gives typical settings: learning rate $1$1, batch size $1$2, AdamW optimizer, and a multi-task loss

$1$3

with $1$4.

On held-out SEPose validation data, the reported results are:

Model	mAP	AP_50
RVT	69.9%	98.1%
YOLOv8x	63.2%	94.7%

For sim-to-real generalization, both models are evaluated out-of-the-box, with no domain adaptation, on two real DVS pose datasets: PEDRo and DHP19. RVT’s mAP drops to $1$5 $1$6 on PEDRo, described as a $1$7 change, and to $1$8 $1$9 on DHP19, described as a $8$0 change. YOLOv8x is reported to display a similar pattern. The interpretation given in the paper is that SEPose embeddings transfer meaningful features, but that further closure of the noise/model gap, for example through physics-based sensor noise modeling or event-level augmentation, would likely boost real-world performance.

These results place the dataset in a sim-to-real regime rather than a purely synthetic benchmark regime. The reported gap does not negate transfer; instead, it identifies residual mismatch between synthetic DVS generation and real event-camera data.

6. Release, downstream uses, and nomenclature

SEPose is to be released under a CC BY license at https://github.com/YourOrg/SEPose, including raw event streams in NumPy .npz, $8$1 event frames in PNG, RGB frames in PNG, and COCO-style JSON annotations containing keypoints, bounding boxes, and scene metadata such as town ID, weather, and time-of-day (Chanda et al., 16 Jul 2025). The dataset is explicitly positioned as enabling downstream tasks requiring accurate pedestrian localization under safety-critical conditions, including gait-based intent prediction, distracted-walking detection, low-light anomaly alerts, and multimodal fusion in autonomous systems.

A common point of confusion is nomenclature. SEPose denotes the synthetic event-based human pose estimation dataset for pedestrian monitoring described above. It should not be conflated with “SpyroPose: SE(3) Pyramids for Object Pose Distribution Estimation,” which addresses object pose distribution estimation on $8$2 rather than event-based pedestrian pose estimation (Haugaard et al., 2023). The similarity in names reflects orthographic proximity, not a shared problem formulation.

In sum, SEPose is characterized as a first-of-its-kind benchmark dataset that leverages photorealistic simulation to capture the advantages of event-based sensing—microsecond timestamps and extreme dynamic range—in a fixed-camera pedestrian-monitoring setting where fast, reliable pose estimation is directly linked to safety-critical perception (Chanda et al., 16 Jul 2025).