SynDRA-BBox: Synthetic Railway Detection

• SynDRA-BBox is a synthetic, multimodal dataset designed for railway perception, providing precise 2D/3D bounding boxes and LiDAR, camera, and point-cloud annotations.
• It supports domain adaptation research by enabling Road-to-Rail and Sim-to-Real transfer through a tailored evaluation protocol and hazard-specific scenarios.
• The dataset integrates UE5-based simulation with realistic railway hazards and size-aware sampling to enhance safety-oriented object detection research.

Searching arXiv for SynDRA-BBox and directly related railway detection/domain adaptation papers. SynDRA-BBox is a synthetic, multimodal railway dataset created to support object detection and related vision tasks in rail environments, with particular emphasis on LiDAR-based 3D detection and cross-domain transfer to real railway data. It extends the UE5-based SynDRA framework by adding precise 2D and 3D bounding boxes and point-cloud labels across realistic railway scenes, and it is paired with an evaluation protocol for both Road-to-Rail and Sim-to-Real domain adaptation using modern 3D detectors (Diaz et al., 22 Jul 2025).

1. Definition and research context

SynDRA-BBox was introduced in the context of automatic train operations, where perception systems must operate under railway-specific constraints including long braking distances, narrow operational corridors along tracks, and scene topologies dominated by rails, sleepers, platforms, and catenaries. The motivating problem is the scarcity of public real-world railway datasets with 3D boxes. OSDaR23 is identified as one of the few available datasets, but it is described as having limited temporal diversity and relatively small scale, while label acquisition in rail is costly and constrained by safety and privacy regulations (Diaz et al., 22 Jul 2025).

Within that context, SynDRA-BBox is presented as the first publicly available synthetic railway dataset explicitly supporting both 2D and 3D detection. Its role is not merely to provide synthetic training data, but to make domain adaptation practical in a setting where automotive detectors must generalize to open-field rail corridors, stations, tunnels, and long-range objects in narrow LiDAR frustums. The dataset therefore serves a dual purpose: it is a detection resource in its own right, and it is a controlled source domain for adaptation to real railway perception benchmarks (Diaz et al., 22 Jul 2025).

The dataset’s design is explicitly safety-oriented. The targeted hazards include vehicles at level crossings, pedestrians crossing or walking along tracks, and natural obstacles such as fallen trees and rocks. This focus distinguishes SynDRA-BBox from generic synthetic driving datasets: the scenes, object classes, and sensor configurations are organized around railway operating conditions rather than road traffic alone (Diaz et al., 22 Jul 2025).

2. Dataset composition, scenarios, and annotations

SynDRA-BBox comprises seven distinct level-crossing scenarios plus a bonus station scenario. Each scene contains multiple sequences with a dominant hazard type. The train starts approximately 350 meters before the crossing in each sequence, approaching hazards at varying times and distances. The obstacles include car, bus, truck, pedestrians crossing left-to-right or right-to-left, pedestrians walking along the track in the same or opposite direction, fallen trees with leaves or bare branches, and fallen rocks obstructing main or adjacent tracks (Diaz et al., 22 Jul 2025).

The dataset is multimodal. It provides LiDAR point clouds from two configurations, RGB images from two camera fields of view, and inherits depth and semantic information from SynDRA. The paper also states that annotations are supplied in JSON and that parsing and visualization scripts are included (Diaz et al., 22 Jul 2025).

Aspect	Specification	Role
LiDAR	Livox TELE-15 and Velodyne HDL-64E	Narrow-frustum long-range and wide-scanning evaluation
Cameras	30° and 90° FoV at 2464×1600	Tele and wide RGB observation
Annotations	2D boxes, 3D boxes, point-cloud labels, semantics	Detection and cross-modal analysis

The TELE-15 configuration is described as telephoto with a narrow field of view of approximately $15^\circ$, aligned with long-range railway stopping requirements, whereas the HDL-64E provides wide scanning and denser returns. The RGB cameras are rendered at $2464 \times 1600$ with $30^\circ$ and $90^\circ$ fields of view. Depth images and semantic segmentation are also available from SynDRA, and SynDRA-BBox follows SynDRA’s semantics and conventions (Diaz et al., 22 Jul 2025).

The class taxonomy is railway-specific: car, bus, truck, pedestrian, fallen_tree, and fallen_rock. The 3D boxes are obtained directly from UE5 internal functions and are attached to detection-relevant objects even when LiDAR returns are sparse. The 2D boxes are aligned with the image plane and stored as $(x, y, w, h)$ in JSON. During training and evaluation, objects with fewer than 5 LiDAR points may be filtered to better reflect real detectability (Diaz et al., 22 Jul 2025).

The split protocol is fixed: 70% train, 20% validation, and 10% test, constructed in 10-frame batches with every 6th frame assigned to test, the 3rd and 9th to validation, and the remainder to training. The reported counts are 4,838 training frames, 1,367 validation frames, and 672 test frames. Aggregate label counts are approximately 14,359 car/vehicle instances and 12,107 pedestrian instances (Diaz et al., 22 Jul 2025).

3. Geometric conventions and benchmark protocol

SynDRA-BBox adopts a standard oriented 3D bounding-box parameterization,

$b = (x, y, z, w, l, h, \theta),$

where $(x, y, z)$ is the box center in sensor or world coordinates, $(w, l, h)$ are width, length, and height, and $\theta$ is yaw in the ground plane. For stable regression, yaw can also be encoded as

$r(\theta) = (\cos \theta, \sin \theta).$

The dataset provides point clouds in the LiDAR sensor frame, while standard rigid transforms relate sensor, world, and camera coordinates (Diaz et al., 22 Jul 2025).

The transform convention is written as

$2464 \times 1600$0

with corresponding LiDAR-to-camera mappings derived from UE5 rig metadata. For cross-domain training with Waymo and OSDaR23, SynDRA-BBox points are adjusted to align with ground level because those real datasets use vehicle-ground frame conventions. The experimental detection range is $2464 \times 1600$1 meters, reflecting long-range forward detection with limited lateral span (Diaz et al., 22 Jul 2025).

The evaluation protocol uses standard IoU-based detection metrics:

$2464 \times 1600$2

AP 3D and AP BEV are reported with AP40, using Cars at IoU thresholds 0.7 and 0.5, and Persons at 0.5 and 0.25 (Diaz et al., 22 Jul 2025).

A central benchmark quantity is the Closed Gap:

$2464 \times 1600$3

This metric quantifies how much of the source-to-target performance gap is recovered by adaptation. A direct implication of the formula is that values above $2464 \times 1600$4 occur when an adapted model exceeds Oracle on the reported metric (Diaz et al., 22 Jul 2025).

4. Domain adaptation framework and railway-specific modifications

The benchmark detector is CenterPoint, implemented in OpenPCDet. Source-only and Oracle models are trained for 20 epochs, while the semi-supervised domain adaptation procedure uses a two-stage SSDA3D schedule of 20 epochs for a CutMix stage followed by 20 epochs for a MixUp stage. The source domains are either SynDRA-BBox, Waymo, or both, and the target domain is OSDaR23 (Diaz et al., 22 Jul 2025).

The adaptation procedure modifies SSDA3D for railway constraints. In the inter-domain Point-CutMix stage, the paste order is reversed relative to the original automotive setting: source crops are pasted into target OSDaR23 frames so that the background retains realistic railway topology. A horizontal translation is computed to align the source crop’s center to the nearest target point, which is intended to avoid unrealistic placements inside the narrow TELE-15 frustum. A class-presence constraint requires at least one ground-truth box in the source crop, and multi-source sampling from Waymo and SynDRA-BBox is made size-aware so that the smaller synthetic source is not oversampled (Diaz et al., 22 Jul 2025).

The CutMix objective is written as

$2464 \times 1600$5

where $2464 \times 1600$6 is the mixed point cloud and $2464 \times 1600$7 is the transformed union of labels. The subsequent intra-domain Point-MixUp stage regularizes on unlabeled target frames and pseudo-labels. The paper describes

$2464 \times 1600$8

and summarizes the total objective conceptually as

$2464 \times 1600$9

Teacher-student or adversarial discriminators are explicitly not used in this implementation (Diaz et al., 22 Jul 2025).

Additional railway-specific adjustments are reported. For OSDaR23 target experiments, only the middle TELE-15 LiDAR is retained to align the target domain with SynDRA-BBox TELE-15 distributions and emphasize long-range detection in a narrow frustum. Because SynDRA-BBox initially lacks intensity, a constant intensity of 1 is assigned and normalized in SW$30^\circ$0O experiments to harmonize the feature channels with Waymo and OSDaR23. The detector loss follows CenterPoint defaults, with focal loss for classification and Smooth L1 for regression (Diaz et al., 22 Jul 2025).

5. Experimental results and reported significance

The experiments compare several setups: S-only and W-only, where CenterPoint is trained only on SynDRA-BBox or Waymo and tested on OSDaR23; Oracle, where CenterPoint is trained and tested on OSDaR23; S$30^\circ$1O and W$30^\circ$2O, which use SSDA3D with one source; and SW$30^\circ$3O, which uses both SynDRA-BBox and Waymo as sources with size-aware multi-source sampling (Diaz et al., 22 Jul 2025).

For Cars, AP 3D at IoU 0.5 is reported as 48.26 for Oracle, 70.73 for W$30^\circ$4O (CutMix), 66.26 for S$30^\circ$5O (MixUp), 59.86 for SW$30^\circ$6O (CutMix), and 59.56 for SW$30^\circ$7O (MixUp). For Persons, AP 3D at IoU 0.5 is 20.21 for Oracle, 32.45 for W$30^\circ$8O (CutMix), 29.58 for S$30^\circ$9O (MixUp), and 35.67 for SW$90^\circ$0O (MixUp), with the last value identified as the best among pedestrian 0.5-IoU 3D metrics. For Persons AP BEV at IoU 0.25, Oracle is 53.41 and SW$90^\circ$1O (MixUp) reaches 71.08, surpassing Oracle (Diaz et al., 22 Jul 2025).

The paper highlights particularly large Closed Gap values. Examples include 146.6% for Cars AP 3D at IoU 0.5 under W$90^\circ$2O (CutMix), 137.3% for Cars AP 3D at IoU 0.5 under S$90^\circ$3O (MixUp), and 176.50% for Persons AP 3D at IoU 0.5 under SW$90^\circ$4O (MixUp). In the reported interpretation, these results indicate that combining synthetic railway context with real automotive object diversity can outperform the OSDaR23-only Oracle baseline on several metrics (Diaz et al., 22 Jul 2025).

The qualitative analysis is consistent with that claim. In station scenes, SW$90^\circ$5O detects distant pedestrians missed by Oracle, and multi-source adaptation increases the number of correct detections relative to Waymo-only, with three out of four pedestrians detected versus one. The paper attributes this to complementary benefits: SynDRA-BBox supplies railway topology, level-crossing layouts, and railway-specific hazards, while Waymo supplies diverse real-world texture and occlusion statistics (Diaz et al., 22 Jul 2025).

6. Relation to SynDRA, common misconceptions, and limitations

A recurring source of confusion is the relation between SynDRA-BBox and earlier SynDRA-based railway perception work. The paper “Integrating Object Detection, LiDAR-Enhanced Depth Estimation, and Segmentation Models for Railway Environments” explicitly states that it does not define or benchmark a “SynDRA-BBox” subset. Instead, it uses a SynDRA depth split for monocular depth fine-tuning and an evaluation split for end-to-end detection and distance estimation, and it emphasizes instance masks rather than rectangular boxes when computing obstacle distances in order to avoid background contamination (Giannico et al., 16 Apr 2026).

That distinction is important because SynDRA-BBox is specifically organized around 2D and 3D object detection and domain adaptation, whereas the mask-centric SynDRA pipeline in (Giannico et al., 16 Apr 2026) integrates object detection, track segmentation, and LiDAR-enhanced monocular depth estimation for obstacle-distance computation. SynDRA-BBox inherits SynDRA’s rendering framework and multimodal scene structure, but its benchmark identity is defined by explicit 2D/3D boxes, LiDAR-oriented detection experiments, and Road-to-Rail and Sim-to-Real transfer (Diaz et al., 22 Jul 2025).

The reported limitations are concrete. Coverage is focused on level crossings plus one station, so broader railway geographies such as tunnels, bridges, and urban rail are not yet represented. The synthetic LiDAR lacks intensity and real sensor noise, and pedestrian micro-geometry is less detailed than in real data. Long-range sparse returns remain difficult for small targets such as pedestrians, particularly under narrow fields of view. Class imbalance in the target domain also motivates the use of class-aware CutMix and size-aware sampling. Future work is described as adding more scenes, more weather and night conditions, improved realism for pedestrians and sensors, and potentially additional modalities such as radar and thermal imaging (Diaz et al., 22 Jul 2025).

Taken together, SynDRA-BBox occupies a specific position in railway perception research: it is a synthetic railway source dataset and an adaptation benchmark for 3D detection, rather than a generic SynDRA subset or a distance-estimation benchmark. Its significance lies in making controlled railway-domain transfer experiments possible at a scale and annotation density that real railway datasets currently do not offer (Diaz et al., 22 Jul 2025).