SocialNav-SUB: Social Navigation Scene Benchmark

• SocialNav-SUB is a composite benchmark concept that integrates multimodal scene representation, social-context inference, and trajectory-conditioned reasoning for evaluating socially compliant navigation.
• It spans diverse environments—from outdoor pedestrian settings to indoor photorealistic scenes—using metrics like traversability and social compliance to assess performance.
• The benchmark synthesizes multiple research tracks by combining explicit scene annotations with language-mediated supervision to foster safe, socially aware robotic navigation.

Searching arXiv for benchmark- and dataset-oriented social navigation papers relevant to SocialNav-SUB. Social Navigation Scene Understanding Benchmark (SocialNav-SUB) denotes a benchmark concept for evaluating scene understanding capabilities that support socially aware navigation, especially in environments shared with pedestrians, dynamic agents, and socially meaningful spatial constraints. Across recent work, the concept is only partially instantiated in any single paper. Instead, the literature distributes its ingredients across several families of resources: outdoor multimodal traversability-and-context systems such as MOSU (Liang et al., 7 Jul 2025), dynamic embodied social-navigation benchmarks such as Falcon’s SocialNav benchmark (Gong et al., 2024), egocentric pedestrian scene-understanding datasets such as SANPO (Waghmare et al., 2023), structured simulation toolkits such as SONATA (Baghel et al., 2020), and static social-compliance scene-scoring datasets such as SocNav1 (Manso et al., 2019). Taken together, these works define SocialNav-SUB less as one fixed benchmark than as a composite research agenda spanning scene representation, social-context inference, trajectory-conditioned reasoning, and downstream evaluation of socially compliant behavior.

1. Benchmark concept and scope

A SocialNav-SUB-style benchmark is centered on scene understanding for social navigation rather than on raw obstacle avoidance alone. The literature repeatedly distinguishes this objective from adjacent tasks. MOSU frames scene understanding as identifying candidate trajectories that are “geometrically feasible,” “semantically traversable,” “socially and contextually compliant,” and “aligned with the current GPS subgoal” (Liang et al., 7 Jul 2025). Falcon’s SocialNav benchmark instead operationalizes scene understanding through navigation under moving humans, partial observability, and future human motion, but does not define standalone scene-understanding labels or perception-only tasks (Gong et al., 2024). SocNav1 isolates a different but closely related problem: estimating whether a robot’s position in a static social scene is socially acceptable, represented by a scalar score in $[0,100]$ (Manso et al., 2019).

These formulations imply three recurring interpretations of SocialNav-SUB. First, scene understanding can be treated as a navigation-oriented multimodal ranking problem, as in MOSU’s trajectory fusion pipeline (Liang et al., 7 Jul 2025). Second, it can be treated as socially aware embodied navigation in dynamic scenes, where understanding is evaluated indirectly through success, compliance, and avoidance metrics, as in Falcon (Gong et al., 2024). Third, it can be treated as structured social-scene scoring over explicit entities and relations, as in SocNav1 and the graph-based benchmarking built on it (Manso et al., 2019, Manso et al., 2019).

The scope of environments also varies systematically. MOSU is most relevant to outdoor campus/on-road pedestrian-scale settings involving sidewalks, roads, crosswalks, vegetation boundaries, narrow passages, and pedestrians (Liang et al., 7 Jul 2025). SANPO is explicitly an outdoor human-egocentric dataset for pedestrian mobility, with classes such as sidewalk, curb, crosswalk, stairs, other walkable surface, inaccessible surface, and obstacle (Waghmare et al., 2023). Falcon’s benchmark is explicitly indoor and photo-realistic, with residences, offices, shops, and gyms derived from HM3D and MP3D (Gong et al., 2024). This suggests that SocialNav-SUB is best understood as a family of benchmark regimes rather than a single environment type.

2. Task formulations across the literature

Recent work provides several concrete task templates that together define the design space of SocialNav-SUB.

MOSU formulates scene understanding as multimodal candidate-trajectory selection. Given LiDAR observations, velocity history, RGB semantics, a GPS subgoal, and VLM-based contextual ranking, it selects a trajectory by maximizing a weighted fusion score (Liang et al., 7 Jul 2025):

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$

where $c_\tau^i$ is geometric confidence, $t_\tau^i$ semantic traversability, $r_\tau^i$ VLM ranking score, and $g_\tau^i$ GPS-subgoal proximity (Liang et al., 7 Jul 2025). In this formulation, scene understanding is not pixel labeling alone but trajectory-conditioned decision support.

Falcon formulates social navigation as future-aware point-goal navigation under human motion. Its benchmark evaluates embodied behavior in scenes populated with moving humans and emphasizes that robots should anticipate future trajectories rather than only react to present obstacles (Gong et al., 2024). The method’s reward design includes a trajectory-obstruction penalty against predicted future human paths, which operationalizes social reasoning as avoidance of soon-to-be-occupied human routes rather than only current collisions (Gong et al., 2024).

SocNav1 and its graph-neural-network follow-up define a static scene-understanding task: predict how socially inconvenient the robot’s presence would be in a structured scene containing humans, objects, walls, and interactions (Manso et al., 2019, Manso et al., 2019). The target is a continuous social score rather than a control action. This scalar-scoring formulation makes the benchmark especially suitable for relational models and for evaluating social-scene interpretation independently of a low-level controller.

MUSON contributes a different formulation: egocentric single-frame socially compliant action selection with explicit reasoning supervision (Liu et al., 28 Dec 2025). It defines four downstream tasks—Socially-Compliant Action Planning, Social-Context Scene Perception, Socially-Aligned CoT Reasoning, and Socially-Grounded Navigation Explanation—using a five-step annotation schema of perception, prediction, reasoning, action, and explanation (Liu et al., 28 Dec 2025). This suggests a benchmark track where scene understanding is judged through structured reasoning consistency and safety-aware action choice rather than trajectory rollouts alone.

LISN-Bench extends the task space to language-instructed social navigation. It defines person-following, person avoidance, region reaching, and region avoidance tasks under natural-language instructions, with semantic region masks and person identities as benchmark primitives (Chen et al., 10 Dec 2025). This indicates that a SocialNav-SUB can plausibly incorporate language grounding when scene understanding must resolve which person or which region matters under instruction (Chen et al., 10 Dec 2025).

3. Scene representations and annotation regimes

The literature supports several distinct representational paradigms for SocialNav-SUB.

3.1 Multimodal trajectory-conditioned representations

MOSU does not build a unified semantic map. Instead, it represents the scene as a set of candidate trajectories with separate multimodal scores for geometry, semantics, context, and goal alignment (Liang et al., 7 Jul 2025). Semantic traversability is computed by segmenting RGB images with Mask2Former, projecting trajectories onto the image, rasterizing them via the Bresenham algorithm, and taking the ratio of traversable pixels to total trajectory pixels (Liang et al., 7 Jul 2025). Its semantic classes follow the GND setup: road, sidewalk, vegetation, building, and others, with only sidewalk and road considered traversable for wheeled robots (Liang et al., 7 Jul 2025).

3.2 Dynamic embodied benchmark state

Falcon’s SocialNav benchmark provides dynamic indoor scenes with procedurally inserted humans in Habitat 3.0. Static scene structure comes from reconstructed HM3D and MP3D environments, while dynamic content consists of human positions and trajectories under egocentric depth sensing (Gong et al., 2024). The benchmark does not define explicit semantic segmentation labels, social zones, or interaction-region masks; instead, scene understanding is implicit in what must be inferred online: traversability, nearby humans, current relative positions, and anticipated future human motion (Gong et al., 2024).

3.3 Dense egocentric perception datasets

SANPO provides an outdoor egocentric representation centered on dense prediction. It contains RGB video, dense depth, sparse depth, visual odometry, and dense panoptic segmentation for synthetic data and a subset of real data (Waghmare et al., 2023). Its 31-category taxonomy includes road, curb, sidewalk, guard rail/road barrier, crosswalk, paved trail, building, wall/fence, hand rail, opening-door, opening-gate, pedestrian, rider, animal, stairs, water body, other walkable surface, inaccessible surface, railway track, obstacle, vehicle, traffic sign, traffic light, pole, bus stop, bike rack, sky, tree, vegetation, and terrain (Waghmare et al., 2023). For SocialNav-SUB, this is a strong perception-side ontology for pedestrian navigation, though it lacks explicit social interaction labels (Waghmare et al., 2023).

RoboSense contributes another egocentric representation, this time robot-mounted and multimodal, with camera, fisheye, and LiDAR data supporting 3D detection, tracking, motion prediction, and occupancy prediction in crowded near-field environments (Su et al., 2024). Its object taxonomy is limited to Vehicle, Cyclist, and Pedestrian, but it offers 1.4M 3D boxes, 216K trajectories, and dense near-range annotations from a slow-moving robosweeper in campuses, parks, sidewalks, and streets (Su et al., 2024). This is scene-understanding support infrastructure for social navigation rather than a social benchmark in the norm-compliance sense.

3.4 Structured graphs and symbolic relations

SocNav1 encodes each scene as JSON with robot, room polyline, humans, objects, links, and a scalar score (Manso et al., 2019). Humans and objects have positions and orientations; links denote human-human or human-object interactions (Manso et al., 2019). Its graph-neural-network extension formalizes two graph constructions over these entities, one with explicit interaction nodes and one with labeled edges, enabling scene-conditioned robot-pose scoring through graph message passing (Manso et al., 2019).

SONATA generalizes this structured representation into a simulation toolkit. It exposes topic streams for humans, walls, goals, objects, and interactions, with explicit symbolic relational annotations such as entity1 id, entity2 id, and interaction type (Baghel et al., 2020). It also defines a temporal graph construction with node types for people, objects, walls, goal, and room, and a 42-dimensional node feature vector

$$h_i^{(0)} = (OH_{t_i} \mid OH_{f_i} \mid ts_i \mid p_i \mid o_i \mid r_i \mid w_i \mid g_i)$$

for graph-based learning (Baghel et al., 2020).

Social 3D Scene Graphs extend this line into 3D human-centric semantics. The proposed graph is

$$\mathcal{G} = (\mathcal{V}, \mathcal{E}), \qquad \mathcal{V} = \mathcal{V}_o \cup \mathcal{V}_h, \qquad \mathcal{E} = \mathcal{E}_s \cup \mathcal{E}_a,$$

where object and human nodes are connected by spatial and activity relations (Bartoli et al., 29 Sep 2025). Its SocialGraph3D benchmark evaluates human activity relation prediction and natural-language query answering over 3D graphs (Bartoli et al., 29 Sep 2025). A plausible implication is that SocialNav-SUB can benefit from graph-based scene representations when the objective includes socially meaningful human-object and human-human semantics.

4. Evaluation protocols and metrics

No single metric suite dominates the literature; instead, benchmark design has split between navigation-centered metrics, dense-prediction metrics, and social-compliance metrics.

4.1 Navigation-centered metrics

MOSU reports traversability, distance to target, and inference time (Liang et al., 7 Jul 2025). Traversability is defined by overlaying a generated trajectory onto the GND traversability map and computing the percentage of fully traversable waypoints along the trajectory (Liang et al., 7 Jul 2025). Distance to target is defined as

$1 - \frac{d_\tau - d_o}{|\tau|},$

where $d_\tau$ is the distance between target and generated trajectory, $$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$0 the distance to the optimal ground-truth trajectory, and $$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$1 the generated trajectory length (Liang et al., 7 Jul 2025). MOSU reports 77 traversability, 73 distance to target, and 2.30 s inference time, outperforming its compared methods in traversability while maintaining strong target progress (Liang et al., 7 Jul 2025).

Falcon’s benchmark uses Success Rate, SPL, STL, Human-Robot Collision Rate, and Personal Space Compliance (Gong et al., 2024). PSC uses a 1.0 m threshold based on a human collision radius of 0.3 m and robot radius of 0.25 m, though no explicit formula is provided (Gong et al., 2024). Falcon achieves roughly 55% success and around 90% PSC on both Social-HM3D and Social-MP3D (Gong et al., 2024).

LISN-Bench expands the metric space with Success Rate, Collision Rate, Path Smoothness, Average Subject Score, and Average Region Score (Chen et al., 10 Dec 2025). Path Smoothness is defined as

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$2

although the paper contains an inconsistency between the textual interpretation and the direction of improvement shown in the results table (Chen et al., 10 Dec 2025). Average Subject Score and Average Region Score quantify compliance with person-centered and region-centered social constraints (Chen et al., 10 Dec 2025).

SocialNav defines scene-semantic compliance metrics directly. In addition to Success Rate, Route Completion, and SPL, it introduces Distance Compliance Rate and Time Compliance Rate (Chen et al., 26 Nov 2025). DCR is

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$3

with an analogous definition for TCR using time (Chen et al., 26 Nov 2025). These metrics measure how much of a successful route lies inside socially compliant regions, making them especially relevant to SocialNav-SUB.

4.2 Perception and scene-understanding metrics

SANPO uses mean Intersection over Union for semantic segmentation, panoptic quality for panoptic segmentation, and a depth inlier metric

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$4

for monocular depth estimation (Waghmare et al., 2023). Its zero-shot results show that standard urban models struggle with pedestrian-egocentric outdoor navigation, especially in distinguishing sidewalk from road (Waghmare et al., 2023).

RoboSense benchmarks multi-view 3D detection, LiDAR 3D detection, multimodal 3D detection, multiple 3D object tracking, motion prediction, and occupancy prediction (Su et al., 2024). Its distinctive contribution is the closest-collision distance proportion criterion, with thresholds $$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$5, reflecting the need to localize near-field collision-relevant geometry rather than merely object centers (Su et al., 2024).

MUSON uses Accuracy, Macro-F1, Collision Rate, and SBERTScore for its reasoning stages (Liu et al., 28 Dec 2025). Collision Rate is defined asymmetrically to penalize aggressive mistakes when the ground truth calls for avoidance:

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$6

and SBERTScore is

$$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$7

Qwen2.5-VL-3B achieves 0.8625 accuracy and 0.0688 collision rate on MUSON (Liu et al., 28 Dec 2025).

SocNav1 itself does not define a canonical machine-learning benchmark metric, but its follow-up graph-neural-network study evaluates mean squared error and reports test MSE $$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$8, compared with human disagreement around $$i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},$$9 and a prior model-based method at $c_\tau^i$0 (Manso et al., 2019).

4.3 Social-competence and human-centered evaluation

SocNavBench standardizes multi-metric evaluation over replayed real pedestrian trajectories, including success rate, failure modes, path quality, motion quality, closest-pedestrian distance, and time-to-collision (Biswas et al., 2021). It is a benchmark for downstream navigation execution rather than scene understanding, but its evaluation philosophy—multi-metric, trade-off-sensitive, and grounded in real human trajectories—is directly relevant to SocialNav-SUB (Biswas et al., 2021).

SocRATES contributes scenario-generation quality metrics rather than scene-understanding metrics: simulability, contextual appropriateness, and alignment with the rough scenario (Marpally et al., 2024). It reports 55% success for unguided generation, 73% for guided generation, and shows that structured prompting materially improves executable social navigation scenario creation (Marpally et al., 2024). This suggests that benchmark quality itself can be evaluated as a first-class object.

5. Benchmark resources and scenario-generation frameworks

Several works are best understood as SocialNav-SUB infrastructure rather than finished benchmark specifications.

SONATA is a benchmark-enabling simulation toolkit rather than a canonical benchmark. Built on CoppeliaSim and PyRep, it supports manual or random generation of dynamic social navigation scenes with explicit symbolic relations, JSON export, and graph-structured representations (Baghel et al., 2020). It also demonstrates supervised control learning with an R-GCN over generated relational-temporal graphs, achieving 76.6% success in a goal-reaching use case (Baghel et al., 2020). Its main value for SocialNav-SUB lies in controllable scenario generation, explicit relation labels, and robot-centered structured data.

SOCIALGYM and SocNavGym are similarly framework-oriented. SOCIALGYM formulates social robot navigation as a POMDP

$c_\tau^i$1

and provides a lightweight 2D simulator with configurable observation, action, and reward spaces, along with benchmark scenarios for social action selection (Holtz et al., 2021). SocNavGym extends this direction with a Gym-style environment exposing structured observations over goals, humans, objects, walls, and relationships, and supports both heuristic and SNGNN-based social rewards (Kapoor et al., 2023). Both are valuable for benchmark prototyping, but they abstract away raw perception and therefore align more with planning and policy evaluation than with scene understanding.

SocRATES addresses another infrastructure layer: automated scenario-based testing. It converts high-level metadata—social context, robot task, rough scenario, and location description—into detailed social navigation scenarios grounded in semantic scene graphs, graph trajectories, and behavior trees (Marpally et al., 2024). Its pipeline is especially useful if SocialNav-SUB is to include curated scenario classes such as elevator negotiation, corridor obstruction, blind intersection emergence, and group navigation (Marpally et al., 2024).

6. Limitations, controversies, and emerging directions

The surveyed literature exposes several unresolved design tensions for SocialNav-SUB.

The first concerns whether scene understanding should be benchmarked directly or only through downstream navigation. Falcon, LISN-Bench, SocialNav, and SocNavBench primarily evaluate behavior, not explicit scene-understanding outputs (Gong et al., 2024, Chen et al., 10 Dec 2025, Chen et al., 26 Nov 2025, Biswas et al., 2021). By contrast, SANPO, RoboSense, MUSON, and SocNav1 offer more explicit supervision for perception, reasoning, or social-scene scoring (Waghmare et al., 2023, Su et al., 2024, Liu et al., 28 Dec 2025, Manso et al., 2019). This suggests that a mature SocialNav-SUB would likely require multiple tracks: perception, reasoning, and embodied execution.

The second tension concerns structured labels versus language-mediated supervision. MOSU operationalizes social understanding through VLM-based trajectory ranking prompts without explicit social labels (Liang et al., 7 Jul 2025). LISN-Bench also grounds semantics through language and VLM tools (Chen et al., 10 Dec 2025). MUSON and SocialNav add structured reasoning or traversability annotations but still rely heavily on generated text or VLM supervision (Liu et al., 28 Dec 2025, Chen et al., 26 Nov 2025). A plausible implication is that future SocialNav-SUB designs may need both symbolic labels and language-grounded annotations to support interpretability and flexible semantics.

A third issue is that many socially relevant factors remain weakly annotated. MOSU does not report social comfort, norm-violation counts, or personal-space metrics (Liang et al., 7 Jul 2025). SANPO lacks interaction labels, proxemics, group behavior, and forecasting targets (Waghmare et al., 2023). Falcon acknowledges that its simulated humans do not exhibit higher-level behaviors such as yielding (Gong et al., 2024). SocNav1 is static and synthetic, with culturally localized annotations from native middle-class residents of Spain (Manso et al., 2019). SocialGraph3D is strong in social semantics but limited to synthetic, mostly static indoor scenes without dynamic crowd behavior (Bartoli et al., 29 Sep 2025). These omissions indicate that SocialNav-SUB remains a partially realized benchmark agenda rather than a solved specification.

The literature also points toward several emerging directions. SocialNav’s separation between a Cognitive Activation Dataset and an Expert Trajectories Pyramid suggests that benchmark design should disentangle cognitive supervision from action demonstrations (Chen et al., 26 Nov 2025). MOSU’s fusion of geometry, semantics, context, and goal alignment suggests that trajectory-conditioned multimodal scoring is a practical benchmark structure for outdoor navigation (Liang et al., 7 Jul 2025). MUSON’s five-step annotation scheme suggests that socially aware scene understanding benefits from explicit intermediate supervision over perception, prediction, reasoning, action, and explanation (Liu et al., 28 Dec 2025). SocialGraph3D indicates that 3D human-centric scene graphs and open-vocabulary query answering may become important substrates for socially aware robotics (Bartoli et al., 29 Sep 2025).

Across these works, SocialNav-SUB emerges as a benchmark concept with partial methodological realization in multiple domains: outdoor multimodal traversability and context, indoor future-aware embodied navigation, egocentric pedestrian scene parsing, structured scene-graph scoring, and reasoning-oriented local action selection. No single benchmark yet unifies all of these dimensions. The current literature therefore supports viewing SocialNav-SUB as an overview target: a benchmark family that would combine socially meaningful scene representations, diverse environment types, explicit and implicit social supervision, and multi-level evaluation of perception, reasoning, and action.