SCAND: Socially Compliant Navigation Dataset

• SCAND is a large-scale multi-modal dataset featuring extensive human–robot trajectory data from diverse real-world environments.
• It supports the development of machine learning models that enforce social norms by capturing authentic human interactions.
• The dataset provides detailed sensor streams, event annotations, and benchmarking metrics for both supervised and imitation learning approaches.

The Socially Compliant Navigation Dataset (SCAND) is a large-scale, multi-modal dataset developed to support data-driven research in socially compliant robot navigation. SCAND provides extensive paired human–robot trajectory data from real-world environments, with a central aim of enabling machine learning models to develop navigation policies that respect social norms—particularly the minimization of disruption to humans in shared spaces. It acts as a canonical reference for both supervised and imitation learning within the context of social navigation, and serves as a foundational resource for benchmarking, quantitative evaluation, and model development targeting social compliance in robot motion among humans (Karnan et al., 2022, Hirose et al., 2023, Raj et al., 2023).

1. Dataset Composition and Structure

SCAND consists of 8.7 hours of continuous, teleoperated robot navigation in both indoor and outdoor environments, corresponding to approximately 40 km of robot travel and encapsulated in 138 demonstration trajectories (Karnan et al., 2022, Hirose et al., 2023, Raj et al., 2023). The dataset encompasses a diverse array of scenarios, including corridors, hallways, office walkways, campus sidewalks, and outdoor paths. Data are organized into sessions ranging from 3 to 15 minutes, yielding around 40–50 session folders. Each session incorporates raw sensor data, trajectory logs, and annotated social interactions, grouped by session in a hierarchical file structure including ROS bag files for synchronized replay (Hirose et al., 2023).

The robots used include both a Boston Dynamics Spot (quadruped) and a Clearpath Jackal (wheeled), providing morphological diversity in platform kinematics. Data are collected entirely under human teleoperation, with the robot driven along pre-planned waypoints while encountering approximately 51 distinct human subjects, whose behaviors were unscripted to reflect authentic human–robot encounters. Each trajectory segment is additionally tagged with event labels covering categories such as “With Traffic,” “Sidewalk,” “Street Crossing,” “Overtaking,” “Passing Conversational Groups,” “Narrow Doorway,” and “Navigating Through Large Crowds” (Karnan et al., 2022).

2. Sensor Modalities, Calibration, and Data Formats

SCAND is multi-modal by design, including the following synchronized and calibrated data streams:

• 3D LiDAR scans (Velodyne VLP-16 or equivalent) at ~10 Hz for dense environmental geometry and bird’s-eye reprojections.
• RGB-D Visual Input from front-facing Kinect or Azure Kinect (20 Hz) and, for Spot, five additional monocular cameras (5 Hz) for all-around situational awareness.
• Odometry via wheel encoders or visual-inertial (VO/VIO) systems, typically at 30 Hz.
• Inertial Measurement Unit (IMU) at 16 Hz (Jackal) or 100 Hz (Spot), capturing egomotion and aiding pose estimation.
• Joystick Commands (“expert” action histories, v and ω) at 10–20 Hz, representing the human-teleoperated action traces.
• Human Detections and Bounding Boxes per frame, with ground-plane projections (using depth sensing), offline person-detection/tracking, and unique IDs.
• Annotations for encounter types—passing, overtaking, side-by-side—and for collision/near-miss events.

File formats comprise raw image sequences (PNG/JPEG), depth images (16-bit PNG), 3D point clouds (PCD), odometry (CSV), and human trajectory tracks (JSON), with all data timestamped under a unified ROS clock. The dataset’s public archives (≈ 50 GB) require no authentication and are distributed under a non-commercial CC BY-NC 4.0 license (Hirose et al., 2023).

3. Social Interaction Scenarios and Annotation Protocol

Socially relevant events in SCAND are opportunistic—no human movement is scripted. Critical interaction types include:

1. Face-on Passing in corridors (∼60% of events),
2. Robot Overtaking slower-moving humans (25%),
3. Human Overtaking the robot (15%).

Approximately 500–800 distinct passing or overtaking events are present, with ≈ 1000 robot trajectory segments (bounded by collision/barrier stops) and ≈ 800 unique human trajectory snippets (2–5 s duration). All human participants provided informed consent, and event tagging encompasses collision or near-miss (binary flag), and encounter type per interaction segment (Hirose et al., 2023).

Counterfactual perturbation labels—quantifying hypothetical changes to human motion in the absence of the robot—are not included by default. However, downstream tasks frequently compute Δ_cf via predictive models, where

$$\Delta_{cf} = \frac{1}{T} \sum_{t=1}^T \bigl\|\,\hat h^{\mathrm{robot}=0}_{t} - \hat h^{\mathrm{robot}}_{t}\bigr\|^2,$$

with $\hat h^{\mathrm{robot}}_{t}$ and $\hat h^{\mathrm{robot}=0}_{t}$ being human position predictions under actual vs. counterfactual conditions (Hirose et al., 2023).

4. Benchmarking, Imitation Learning, and Social Compliance Metrics

SCAND underpins supervised and imitation learning pipelines for social navigation. The dataset facilitates the development of both global trajectory planners and local action-level controllers via behavioral cloning, with a principal objective of minimizing deviation from demonstrator actions under identical sensory input (Karnan et al., 2022, Raj et al., 2023). Key evaluation metrics include:

• Hausdorff distance ($d_H$) between global plan and expert path over a 10 m horizon,
• L₂ norm between joystick command histories,
• Personal space violation metrics (mean ≈ 0.45 s per event in baseline imitation),
• Counterfactual perturbation (Δ_cf ≈ 0.15 m per event with simple linear models).

Baselines established on SCAND include teleoperated-policy imitation (∼75% pass success), with behavioral cloning outperforming classical planners in social compliance and safety in both simulated and real-world deployments (Karnan et al., 2022, Raj et al., 2023).

A hybrid planning paradigm—combining classical geometric planners with learned policies via a gating classifier—demonstrated improved coverage (90% vs. 80–84%) and lower collision rates compared to monolithic approaches (Raj et al., 2023).

5. Extensions, Derivative Benchmarks, and Human-Model Comparison

SCAND has served as a basis for constructing derivative datasets and benchmarks. The Social Navigation Scene Understanding Benchmark (SocialNav-SUB) mines 60 challenging SCAND segments to create 4,968 visual question-answering (VQA) tasks, probing spatial, spatiotemporal, and social reasoning in multi-agent scenarios (Munje et al., 10 Sep 2025). This benchmark enables systematic evaluation of vision-LLMs (VLMs) versus human consensus and rule-based systems, revealing that even state-of-the-art VLMs underperform both algorithmic and human baselines in accurate social scene understanding, particularly for spatial and temporal reasoning tasks.

Additionally, SCAND trajectories have been utilized in benchmarking pipelines for social robot navigation metrics via human ratings, with RNN-based models reaching a mean absolute error (MAE) of ≈ 0.16 against normalized human compliance scores (Bachiller-Burgos et al., 1 Sep 2025).

6. Access, Licensing, and Use Within Broader Datasets

SCAND is publicly accessible via HTTPS download on the SACSoN project site, with no authentication or credentials needed. The dataset is distributed in per-session archives (≈ 50 GB total), and users are required to cite Karnan et al., “Socially Compliant Navigation with Vision and Language” (2022) for research use (Hirose et al., 2023). Its license (CC BY-NC 4.0) limits usage to non-commercial research. SCAND is also directly incorporated into larger multi-source datasets—e.g., contributing “D_real” real-robot trajectories in the SocNav Dataset, which aggregates >7 million navigation samples for training foundation models in social navigation (Chen et al., 26 Nov 2025).

7. Limitations and Opportunities for Future Research

Primary limitations of SCAND include its regional specificity (collected exclusively at one geographic location, with regional walking norms) and lack of explicit per-frame human state annotations (for pose and intent, requiring additional processing for human trajectory extraction) (Karnan et al., 2022). Social interaction tags, while present, are limited to a fixed set of 12; rare and nuanced social contexts may be underrepresented. Expanding collection efforts across other cultures and environments, and supplementing SCAND with explicit human trajectory and semantic annotations, are encouraged directions.

SCAND’s detailed real-world multi-robot and multi-modal structure has proven essential for the data-driven study of robust, safe, and socially intelligent robotic mobility in human environments. It remains a core reference point for policy learning, benchmarking, policy evaluation, and as a foundation for hybrid, multi-modal, and foundation-model approaches in social navigation (Hirose et al., 2023, Raj et al., 2023, Munje et al., 10 Sep 2025, Chen et al., 26 Nov 2025, Bachiller-Burgos et al., 1 Sep 2025).