M3DGR Dataset: Multi-Sensor SLAM Benchmark
- M3DGR is a ground-robot SLAM benchmark that integrates a full sensor suite with controlled degradation patterns (visual, LiDAR, wheel, GNSS) to assess robustness.
- It systematically induces challenges like low light, LiDAR degeneracy, wheel slip, and GNSS denial, enabling standardized evaluation of 40 SLAM systems.
- The dataset employs precise external ground truth (Optitrack and RTK) to support analysis of sensor fusion, adaptive switching, and failure modes.
Searching arXiv for papers on M3DGR to ground the article in the relevant literature. M3DGR is a ground-robot SLAM benchmark introduced as a “Multi-sensor, Multi-scenario and Massive-baseline SLAM dataset for Ground Robot”. It was designed as a sensor-rich benchmark with systematically induced degradation patterns including visual challenge, LiDAR degeneracy, wheel slippage and GNSS denial, with the explicit goal of enabling standardized robustness analysis rather than only nominal-condition accuracy measurement. In the original benchmark paper, M3DGR is positioned as a response to the limited sensor coverage, limited configurability, and weak degradation modeling of prior datasets, and as a platform large enough to support evaluation of 40 SLAM systems under controlled corner cases (Zhang et al., 11 Jul 2025).
1. Origin and benchmark rationale
M3DGR was created to address two gaps identified in ground-robot SLAM benchmarking. The first is the lack of standardized, configurable benchmarks for systematically evaluating algorithms under degradations such as poor lighting, geometric degeneracy, wheel slip, and GNSS outage. The second is the lack of rich multi-sensor coverage for robustness analysis, since many earlier datasets fix the sensor setup or emphasize only one modality. The benchmark paper therefore frames M3DGR as a dataset for systematic robustness testing, sensor-selection analysis, evaluation across diverse fusion configurations, and fair comparison of many SLAM systems under corner cases (Zhang et al., 11 Jul 2025).
The original paper explicitly contrasts M3DGR with earlier datasets such as EuRoC, UrbanLoco, OpenLoris-Scene, M2DGR, Ground-Challenge, M2DGR-Plus, and MARS-LVIG, arguing that existing resources often support only a narrow set of sensor types, do not provide configurable rich sensor suites, lack systematic degradation scenarios, or use pseudo-ground-truth from SLAM itself rather than external high-precision tracking. M3DGR is presented as overcoming those limitations through a full sensor suite, controlled degradation scenarios, and accurate ground truth from external tracking (Zhang et al., 11 Jul 2025).
A central implication of this design is that M3DGR is not organized merely as a data repository. It is a benchmark whose sequence construction, ground-truth strategy, and degradation taxonomy are all tailored to failure analysis. The paper’s broader conclusion is that robustness depends not only on adding sensors, but on adaptive sensor selection and degradation-aware switching; M3DGR is intended to make those questions experimentally accessible (Zhang et al., 11 Jul 2025).
2. Robotic platform and sensing stack
M3DGR is built on a differential-drive ground robot with two motorized wheels, which makes wheel odometry both naturally available and directly stressable in wheel-slip scenarios. The platform carries a multi-modal sensing stack consisting of RGB-D camera + IMU, omnidirectional camera, two non-repetitive scanning LiDARs, wheel odometer, and two GNSS receivers / GNSS-related devices, together with external tracking systems for ground truth (Zhang et al., 11 Jul 2025).
The main onboard modalities are specified as follows: Intel RealSense D435i RGB at 640×480, 69° horizontal × 42° vertical, 30 Hz; RealSense D435i Depth at 640×480, 87° horizontal × 58° vertical, 30 Hz; RealSense D435i IMU, 6-axis, 200 Hz; Insta360 X4, 2880×1440, 360°, 15 Hz; Livox Avia, non-repetitive, 450 m range, 70.4° × 77.2°, 15 Hz, with IMU: 200 Hz; Livox MID360, non-repetitive, 40 m range, 360° × 59°, 15 Hz, with IMU: 200 Hz; wheel odometer, 20 Hz; GNSS receiver, BDS/GPS/GLONASS/Galileo, 10 Hz; RTK receiver, 0.8 cm horizontal / 1.5 cm vertical localization accuracy, 15 Hz; and Optitrack mocap, 1 mm localization accuracy, 360 Hz (Zhang et al., 11 Jul 2025).
The recording pipeline uses ROS rosbag with a unified timestamp mechanism. The paper also notes hardware synchronization for several sensors: the RealSense D435i internally synchronizes RGB, depth, and IMU, and both Livox LiDARs have built-in IMUs and use hardware triggering to ensure internal synchronization. Data acquisition and storage are handled by a high-performance Intel NUC with a high-speed NVMe SSD (Zhang et al., 11 Jul 2025).
Later method papers underline why the dual-LiDAR configuration matters for algorithm analysis. In Surfel-LIO, M3DGR is described as a “multi-sensor dataset designed for ground robot localization” with two Livox LiDAR sensors having distinct characteristics: AVIA with a non-repetitive scanning pattern and about 70° field of view, and Mid-360 with 360° horizontal field of view and higher point density. The same paper reports about 24,000 points/scan for AVIA and about 15,000 points/scan for Mid-360, which makes M3DGR useful for analyzing how scan density and scanning pattern affect runtime and state-estimation behavior (Choi et al., 3 Dec 2025).
3. Scenario taxonomy, induced degradations, and dataset scale
M3DGR covers both indoor and outdoor environments and is organized around four controlled degradation categories—Visual challenge, LiDAR degeneracy, Wheel slippage, and GNSS denial—plus Standard sequences. The benchmark table reports 32 sequences total, 11,868.95 m total distance, 14,821 s total duration, and 225.0 GB total data size (Zhang et al., 11 Jul 2025).
| Category | Sequences | Distance / Duration / Size |
|---|---|---|
| Visual challenge | 16 | 4,155.10 m / 5,565 s / 66.4 GB |
| LiDAR degeneracy | 3 | 1,016.28 m / 1,395 s / 23.1 GB |
| Wheel slippage | 7 | 1,008.69 m / 1,401 s / 26.3 GB |
| GNSS denial | 2 | 1,162.39 m / 1,359 s / 23.2 GB |
| Standard | 4 | 4,485.49 m / 5,101 s / 86.0 GB |
| Total | 32 | 11,868.95 m / 14,821 s / 225.0 GB |
The Visual challenge category contains four subtypes: Dark, Varying illumination, Dynamic scenes, and Occlusion. Indoors, low light is induced by turning off room lights and using only a mobile phone flashlight; varying illumination is induced by alternating room lights at regular intervals; dynamic scenes are induced by a person moving in front of the robot; and occlusion is induced by deliberately blocking the camera. Outdoors, the corresponding cases include nighttime dark sequences, changing lighting, pedestrians and cyclists, moving vehicles, and partial camera obstruction. The summary table reports 5 dark sequences, 4 varying-illumination sequences, 3 dynamic sequences, and 4 occlusion sequences, for 16 sequences in total (Zhang et al., 11 Jul 2025).
The LiDAR degeneracy category contains Corridor and Elevator transition sequences. The corridor case is induced by driving through a feature-poor, geometrically repetitive corridor, and the elevator case by moving from a corridor into an elevator, creating a sudden change from structured planar geometry to a sparse vertical space. The benchmark reports 2 corridor sequences and 1 elevator sequence. In both cases, the robot is driven in a loop, returning to the starting position (Zhang et al., 11 Jul 2025).
The Wheel slippage category contains Wheel float, Sharp turn, Grass, and Rough road. These are induced by lifting the driving wheels to force slip, commanding rapid directional changes at high speed, traversing uneven grassy terrain, and driving over roads with abrupt elevation changes. The summary table reports 2 wheel-float sequences, 2 sharp-turn sequences, 2 grass sequences, and 1 rough-road sequence, for 7 sequences in total (Zhang et al., 11 Jul 2025).
The GNSS denial category contains 2 GNSS-denial sequences. In this setup, the robot starts in a GNSS-available area, traverses a GNSS-denied region, and then returns to the starting point. This structure is explicitly designed to expose the behavior of fusion systems when absolute positioning becomes unavailable and then reappears (Zhang et al., 11 Jul 2025).
The paper also lists example sequence characteristics such as Dynamic01: 175 s, 72.22 m, Corridor01: 403 s, 332.32 m, Elevator01: 699 s, 470.64 m, and GNSS-denial01: 609 s, 524.37 m. This suggests that M3DGR is meant to span both short controlled stress tests and longer loop-like traversals in which accumulated drift becomes observable (Zhang et al., 11 Jul 2025).
4. Ground truth, synchronization, and evaluation semantics
Ground truth in M3DGR is generated using external tracking rather than SLAM-derived pseudo-reference. The dataset uses Optitrack mocap for indoor ground truth and RTK for outdoor ground truth. For loop-based or special-case sequences—especially in LiDAR degeneracy and GNSS denial—the paper uses ArUco marker detection to compute the relative pose between start and end, which serves as a quantitative measure of trajectory accuracy when dense full-trajectory truth is harder to obtain in degraded settings (Zhang et al., 11 Jul 2025).
The benchmark’s primary trajectory metric is ATE RMSE, used to compare estimated trajectories against ground truth. In the visual-challenge figure, if a system fails, the paper sets the error to 20 m for plotting purposes. For degradation characterization, the paper also reports visual average keypoints per frame, LiDAR average matching error, wheel drift rate, and GNSS average valid satellite count. The reported values are 117 vs 138 keypoints per frame, 0.14 vs 0.04 LiDAR matching error, 10.24 m vs 2.04 m wheel drift rate, and 4 vs 13 GNSS valid satellites; these values are used to argue that M3DGR is a harder and more realistic robustness benchmark (Zhang et al., 11 Jul 2025).
Later studies adopt compatible but task-specific evaluation conventions on M3DGR. Surfel-LIO evaluates odometry using APE (Absolute Pose Error) RMSE of translational differences after SE(3) alignment, computed with the evo toolkit, under controlled settings including identical 0.5 m voxel resolution and identical 200 m × 200 m local map extent. FAST-LIVGO uses ATE RMSE (m) across open-sky and GNSS-denied sequences, again relying on M3DGR’s externally referenced trajectories (Choi et al., 3 Dec 2025, Chen et al., 17 Jun 2026).
A plausible implication is that M3DGR’s ground-truth design serves two distinct purposes. First, it supports conventional full-trajectory benchmarking where external truth is available. Second, in scenarios deliberately constructed around return-to-start loops or GNSS-degraded regions, it supports relative-drift evaluation that is tightly matched to the underlying failure mode being studied.
5. Benchmark use in SLAM, LIO, and sensor-fusion research
The original M3DGR paper evaluates 40 SLAM systems, spanning vision-based methods, LiDAR-based methods, LiDAR-visual fusion systems, wheel odometry, and GNSS SPP. The main findings are organized by degradation mode. Under Visual challenge, performance degrades steadily as the environment becomes darker, more occluded, or more visually dynamic, and pure vision systems are highly vulnerable to feature loss. Under LiDAR degeneracy, corridor and elevator sequences cause severe drift or outright failure in many LiDAR methods, while continuous-time LiDAR methods perform better than discrete-time methods in general; CT-LIO is highlighted as particularly robust in elevator transition, and VoxelMap, HM-LIO, and Fast-LIVO2 are also reported as more stable than many others. Under Wheel slippage, wheel-integrated systems often drift badly under slip. Under GNSS denial, GNSS-integrated methods degrade substantially without absolute positioning, while adding wheel odometry helps reduce drift (Zhang et al., 11 Jul 2025).
The same paper uses M3DGR to motivate Ground-Fusion++, a modular framework that couples GNSS, RGB-D, IMU, wheel odometry, and LiDAR with adaptive switching under degradation. On Longtime01 it reports ATE RMSE of 7.5 m, compared to 22.5 m for Ground-Fusion, and for LiDAR degradation it reports 1.67 m / 1.73 m on two corridor sequences for the proposed system, compared with 6.14 m / 26.66 m for the LIO subsystem alone. These results are method-specific, but they also show how the dataset is used to study failure-aware fusion rather than only single-modality accuracy (Zhang et al., 11 Jul 2025).
In Surfel-LIO, M3DGR is the sole experimental benchmark. The paper evaluates on nine M3DGR sequences—Dark01–02, Dynamic03–04, Occlusion03–04, and Varying-illu03–05—chosen because they stress low-light indoor conditions, dynamic objects, partial sensor blockage, and rapid lighting changes. The method compares against Fast-LIO2 and Faster-LIO and uses M3DGR to support two claims: that Surfel-LIO is much faster than Fast-LIO2 and Faster-LIO, especially on dense scans, and that it preserves odometry accuracy while improving speed. The authors also emphasize that M3DGR’s dual-sensor configuration lets them analyze how dense AVIA scans and lower-density Mid-360 scans change computational load and estimation quality (Choi et al., 3 Dec 2025).
In FAST-LIVGO, M3DGR is the main public benchmark for a LiDAR-Inertial-Visual-GNSS fusion system. The paper uses Varying-illu03, Varying-illu04, Varying-illu05, Outdoor01, Outdoor04, Dark01, and Dark02 to separate open-sky scenarios from GNSS-denied scenarios. It argues that M3DGR is well suited because it provides RTK ground truth and contains conditions that stress both LIVO degeneracy and GNSS degradation. The reported result is that the full method achieves the lowest ATE RMSE on every reported sequence, while the Ours w/o Rejection ablation fails completely in Dark01 and Dark02, which the paper uses as evidence that tightly coupled fusion is brittle without robust rejection logic (Chen et al., 17 Jun 2026).
Taken together, these later papers show that M3DGR has become more than a one-off benchmark. It is used to evaluate runtime scaling across LiDAR scan patterns, robustness under dark and dynamic conditions, behavior under GNSS corruption or denial, and the interaction between local odometry quality and global consistency.
6. Nomenclature and relation to similarly named datasets
M3DGR should be distinguished from several similarly named datasets. It is not M2DGR. The 2021 paper “M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground Robots” consistently refers to M2DGR, states that the paper is about M2DGR, not M3DGR, and explicitly notes that it never introduces a dataset named M3DGR and does not describe any formal relation or renaming from M2DGR to M3DGR (Yin et al., 2021). M3DGR is therefore a separate benchmark rather than a renamed edition of M2DGR.
It is also unrelated to M³D-Stereo, the stereo image restoration dataset introduced in “M3D-Stereo: A Multiple-Medium and Multiple-Degradation Dataset for Stereo Image Restoration”, which contains 7904 high-resolution stereo image pairs for adverse-condition restoration research in underwater, haze/fog, and low-light settings (Yang et al., 14 Apr 2026). Despite the lexical similarity, that dataset addresses image restoration and stereo matching rather than ground-robot SLAM.
Likewise, M3DGR is unrelated to MD, the multimodal information-extraction dataset introduced in “MD: A Multimodal, Multilingual and Multitask Dataset for Grounded Document-level Information Extraction”, which contains 4,093 documents/videos and supports Named Entity Recognition, Entity Chain Extraction, Relation Extraction, and Visual Grounding (Liu et al., 2024).
Within robotics, the distinctive feature of M3DGR is the combination of a full sensor suite, systematically induced degradation scenarios, externally referenced ground truth, and a benchmark protocol expressly organized around failure modes. This suggests that its main contribution is not only additional data volume, but a reproducible experimental structure for studying how and why ground-robot SLAM systems break under adverse sensing conditions.
