---
title: 'SocialNav-SUB: Social Navigation Scene Benchmark'
url: https://www.emergentmind.com/topics/social-navigation-scene-understanding-benchmark-socialnav-sub
type: topic
---

# SocialNav-SUB: Social Navigation Scene Benchmark

Searching arXiv for benchmark- and dataset-oriented social navigation papers relevant to SocialNav-SUB.
Social Navigation Scene Understanding Benchmark (SocialNav-SUB) denotes a benchmark concept for evaluating scene understanding capabilities that support socially aware navigation, especially in environments shared with pedestrians, dynamic agents, and socially meaningful spatial constraints. Across recent work, the concept is only partially instantiated in any single paper. Instead, the literature distributes its ingredients across several families of resources: outdoor multimodal traversability-and-context systems such as MOSU [2507.04686], dynamic embodied social-navigation benchmarks such as Falcon’s SocialNav benchmark [2409.13244], egocentric pedestrian scene-understanding datasets such as SANPO [2309.12172], structured simulation toolkits such as SONATA [2009.05345], and static social-compliance scene-scoring datasets such as SocNav1 [1909.02993]. Taken together, these works define SocialNav-SUB less as one fixed benchmark than as a composite research agenda spanning scene representation, social-context inference, trajectory-conditioned reasoning, and downstream evaluation of socially compliant behavior.

## 1. Benchmark concept and scope

A SocialNav-SUB-style benchmark is centered on scene understanding for social navigation rather than on raw obstacle avoidance alone. The literature repeatedly distinguishes this objective from adjacent tasks. MOSU frames scene understanding as identifying candidate trajectories that are “geometrically feasible,” “semantically traversable,” “socially and contextually compliant,” and “aligned with the current GPS subgoal” [2507.04686]. Falcon’s SocialNav benchmark instead operationalizes scene understanding through navigation under moving humans, partial observability, and future human motion, but does not define standalone scene-understanding labels or perception-only tasks [2409.13244]. SocNav1 isolates a different but closely related problem: estimating whether a robot’s position in a static social scene is socially acceptable, represented by a scalar score in \([0,100]\) [1909.02993].

These formulations imply three recurring interpretations of SocialNav-SUB. First, scene understanding can be treated as a navigation-oriented multimodal ranking problem, as in MOSU’s trajectory fusion pipeline [2507.04686]. Second, it can be treated as socially aware embodied navigation in dynamic scenes, where understanding is evaluated indirectly through success, compliance, and avoidance metrics, as in Falcon [2409.13244]. Third, it can be treated as structured social-scene scoring over explicit entities and relations, as in SocNav1 and the graph-based benchmarking built on it [1909.02993], [1909.09003].

The scope of environments also varies systematically. MOSU is most relevant to outdoor campus/on-road pedestrian-scale settings involving sidewalks, roads, crosswalks, vegetation boundaries, narrow passages, and pedestrians [2507.04686]. SANPO is explicitly an outdoor human-egocentric dataset for pedestrian mobility, with classes such as sidewalk, curb, crosswalk, stairs, other walkable surface, inaccessible surface, and obstacle [2309.12172]. Falcon’s benchmark is explicitly indoor and photo-realistic, with residences, offices, shops, and gyms derived from HM3D and MP3D [2409.13244]. This suggests that SocialNav-SUB is best understood as a family of benchmark regimes rather than a single environment type.

## 2. Task formulations across the literature

Recent work provides several concrete task templates that together define the design space of SocialNav-SUB.

MOSU formulates scene understanding as multimodal candidate-trajectory selection. Given LiDAR observations, velocity history, RGB semantics, a GPS subgoal, and VLM-based contextual ranking, it selects a trajectory by maximizing a weighted fusion score [2507.04686]:
$$
i^* = \arg\max_{i \in [1, N]} \left\{ \beta_1 c_\tau^i + \beta_2 t_\tau^i + \beta_3 r_\tau^i + \beta_4 g_\tau^i \right\}, \;\;\; \tau = \tau^{i^*},
$$
where \(c_\tau^i\) is geometric confidence, \(t_\tau^i\) semantic traversability, \(r_\tau^i\) VLM ranking score, and \(g_\tau^i\) GPS-subgoal proximity [2507.04686]. In this formulation, scene understanding is not pixel labeling alone but trajectory-conditioned decision support.

Falcon formulates social navigation as future-aware point-goal navigation under human motion. Its benchmark evaluates embodied behavior in scenes populated with moving humans and emphasizes that robots should anticipate future trajectories rather than only react to present obstacles [2409.13244]. The method’s reward design includes a trajectory-obstruction penalty against predicted future human paths, which operationalizes social reasoning as avoidance of soon-to-be-occupied human routes rather than only current collisions [2409.13244].

SocNav1 and its graph-neural-network follow-up define a static scene-understanding task: predict how socially inconvenient the robot’s presence would be in a structured scene containing humans, objects, walls, and interactions [1909.02993], [1909.09003]. The target is a continuous social score rather than a control action. This scalar-scoring formulation makes the benchmark especially suitable for relational models and for evaluating social-scene interpretation independently of a low-level controller.

MUSON contributes a different formulation: egocentric single-frame socially compliant action selection with explicit reasoning supervision [2512.22867]. It defines four downstream tasks—Socially-Compliant Action Planning, Social-Context Scene Perception, Socially-Aligned CoT Reasoning, and Socially-Grounded Navigation Explanation—using a five-step annotation schema of perception, prediction, reasoning, action, and explanation [2512.22867]. This suggests a benchmark track where scene understanding is judged through structured reasoning consistency and safety-aware action choice rather than trajectory rollouts alone.

LISN-Bench extends the task space to language-instructed social navigation. It defines person-following, person avoidance, region reaching, and region avoidance tasks under natural-language instructions, with semantic region masks and person identities as benchmark primitives [2512.09920]. This indicates that a SocialNav-SUB can plausibly incorporate language grounding when scene understanding must resolve which person or which region matters under instruction [2512.09920].

## 3. Scene representations and annotation regimes

The literature supports several distinct representational paradigms for SocialNav-SUB.

### 3.1 Multimodal trajectory-conditioned representations

MOSU does not build a unified semantic map. Instead, it represents the scene as a set of candidate trajectories with separate multimodal scores for geometry, semantics, context, and goal alignment [2507.04686]. Semantic traversability is computed by segmenting RGB images with Mask2Former, projecting trajectories onto the image, rasterizing them via the Bresenham algorithm, and taking the ratio of traversable pixels to total trajectory pixels [2507.04686]. Its semantic classes follow the GND setup: road, sidewalk, vegetation, building, and others, with only sidewalk and road considered traversable for wheeled robots [2507.04686].

### 3.2 Dynamic embodied benchmark state

Falcon’s SocialNav benchmark provides dynamic indoor scenes with procedurally inserted humans in Habitat 3.0. Static scene structure comes from reconstructed HM3D and MP3D environments, while dynamic content consists of human positions and trajectories under egocentric depth sensing [2409.13244]. The benchmark does not define explicit semantic segmentation labels, social zones, or interaction-region masks; instead, scene understanding is implicit in what must be inferred online: traversability, nearby humans, current relative positions, and anticipated future human motion [2409.13244].

### 3.3 Dense egocentric perception datasets

SANPO provides an outdoor egocentric representation centered on dense prediction. It contains RGB video, dense depth, sparse depth, visual odometry, and dense panoptic segmentation for synthetic data and a subset of real data [2309.12172]. Its 31-category taxonomy includes road, curb, sidewalk, guard rail/road barrier, crosswalk, paved trail, building, wall/fence, hand rail, opening-door, opening-gate, pedestrian, rider, animal, stairs, water body, other walkable surface, inaccessible surface, railway track, obstacle, vehicle, traffic sign, traffic light, pole, bus stop, bike rack, sky, tree, vegetation, and terrain [2309.12172]. For SocialNav-SUB, this is a strong perception-side ontology for pedestrian navigation, though it lacks explicit social interaction labels [2309.12172].

RoboSense contributes another egocentric representation, this time robot-mounted and multimodal, with camera, fisheye, and LiDAR data supporting 3D detection, tracking, motion prediction, and occupancy prediction in crowded near-field environments [2408.15503]. Its object taxonomy is limited to Vehicle, Cyclist, and Pedestrian, but it offers 1.4M 3D boxes, 216K trajectories, and dense near-range annotations from a slow-moving robosweeper in campuses, parks, sidewalks, and streets [2408.15503]. This is scene-understanding support infrastructure for social navigation rather than a social benchmark in the norm-compliance sense.

### 3.4 Structured graphs and symbolic relations

SocNav1 encodes each scene as JSON with robot, room polyline, humans, objects, links, and a scalar score [1909.02993]. Humans and objects have positions and orientations; links denote human-human or human-object interactions [1909.02993]. Its graph-neural-network extension formalizes two graph constructions over these entities, one with explicit interaction nodes and one with labeled edges, enabling scene-conditioned robot-pose scoring through graph message passing [1909.09003].

SONATA generalizes this structured representation into a simulation toolkit. It exposes topic streams for humans, walls, goals, objects, and interactions, with explicit symbolic relational annotations such as `entity1 id`, `entity2 id`, and `interaction type` [2009.05345]. It also defines a temporal graph construction with node types for people, objects, walls, goal, and room, and a 42-dimensional node feature vector
$$
h_i^{(0)} = (OH_{t_i} \mid OH_{f_i} \mid ts_i \mid p_i \mid o_i \mid r_i \mid w_i \mid g_i)
$$
for graph-based learning [2009.05345].

Social 3D Scene Graphs extend this line into 3D human-centric semantics. The proposed graph is
$$
\mathcal{G} = (\mathcal{V}, \mathcal{E}), \qquad \mathcal{V} = \mathcal{V}_o \cup \mathcal{V}_h, \qquad \mathcal{E} = \mathcal{E}_s \cup \mathcal{E}_a,
$$
where object and human nodes are connected by spatial and activity relations [2509.24966]. Its SocialGraph3D benchmark evaluates human activity relation prediction and natural-language query answering over 3D graphs [2509.24966]. A plausible implication is that SocialNav-SUB can benefit from graph-based scene representations when the objective includes socially meaningful human-object and human-human semantics.

## 4. Evaluation protocols and metrics

No single metric suite dominates the literature; instead, benchmark design has split between navigation-centered metrics, dense-prediction metrics, and social-compliance metrics.

### 4.1 Navigation-centered metrics

MOSU reports traversability, distance to target, and inference time [2507.04686]. Traversability is defined by overlaying a generated trajectory onto the GND traversability map and computing the percentage of fully traversable waypoints along the trajectory [2507.04686]. Distance to target is defined as
$$
1 - \frac{d_\tau - d_o}{|\tau|},
$$
where \(d_\tau\) is the distance between target and generated trajectory, \(d_o\) the distance to the optimal ground-truth trajectory, and \(|\tau|\) the generated trajectory length [2507.04686]. MOSU reports 77 traversability, 73 distance to target, and 2.30 s inference time, outperforming its compared methods in traversability while maintaining strong target progress [2507.04686].

Falcon’s benchmark uses Success Rate, SPL, STL, Human-Robot Collision Rate, and Personal Space Compliance [2409.13244]. PSC uses a 1.0 m threshold based on a human collision radius of 0.3 m and robot radius of 0.25 m, though no explicit formula is provided [2409.13244]. Falcon achieves roughly 55% success and around 90% PSC on both Social-HM3D and Social-MP3D [2409.13244].

LISN-Bench expands the metric space with Success Rate, Collision Rate, Path Smoothness, Average Subject Score, and Average Region Score [2512.09920]. Path Smoothness is defined as
$$
\text{Curvature Smoothness} = \sum_{i=1}^{n-1} \left| \mathrm{wrap}(\alpha_{i+1} - \alpha_i) \right|,
$$
although the paper contains an inconsistency between the textual interpretation and the direction of improvement shown in the results table [2512.09920]. Average Subject Score and Average Region Score quantify compliance with person-centered and region-centered social constraints [2512.09920].

SocialNav defines scene-semantic compliance metrics directly. In addition to Success Rate, Route Completion, and SPL, it introduces Distance Compliance Rate and Time Compliance Rate [2511.21135]. DCR is
$$
\text{DCR} = \begin{cases} \dfrac{d_{\text{compliant}}}{d_{\text{actual}}}, & \text{if } s = 1 \\ 0, & \text{otherwise} \end{cases}
$$
with an analogous definition for TCR using time [2511.21135]. These metrics measure how much of a successful route lies inside socially compliant regions, making them especially relevant to SocialNav-SUB.

### 4.2 Perception and scene-understanding metrics

SANPO uses mean Intersection over Union for semantic segmentation, panoptic quality for panoptic segmentation, and a depth inlier metric
$$
\mathbb{E}\left[\max\left(\frac{y}{y'}, \frac{y'}{y}\right) \leq 1.25\right]
$$
for monocular depth estimation [2309.12172]. Its zero-shot results show that standard urban models struggle with pedestrian-egocentric outdoor navigation, especially in distinguishing sidewalk from road [2309.12172].

RoboSense benchmarks multi-view 3D detection, LiDAR 3D detection, multimodal 3D detection, multiple 3D object tracking, motion prediction, and occupancy prediction [2408.15503]. Its distinctive contribution is the closest-collision distance proportion criterion, with thresholds \(\{5\%,10\%,20\%\}\), reflecting the need to localize near-field collision-relevant geometry rather than merely object centers [2408.15503].

MUSON uses Accuracy, Macro-F1, Collision Rate, and SBERTScore for its reasoning stages [2512.22867]. Collision Rate is defined asymmetrically to penalize aggressive mistakes when the ground truth calls for avoidance:
$$
\text{CR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(a_{pred}^i \in \mathcal{A}_{go} \cap a_{gt}^i \in \mathcal{A}_{stop}),
$$
and SBERTScore is
$$
S_{bert} = \frac{E(T_{pred}) \cdot E(T_{gt})}{\|E(T_{pred})\| \|E(T_{gt})\|}.
$$
Qwen2.5-VL-3B achieves 0.8625 accuracy and 0.0688 collision rate on MUSON [2512.22867].

SocNav1 itself does not define a canonical machine-learning benchmark metric, but its follow-up graph-neural-network study evaluates mean squared error and reports test MSE \(= 0.03173\), compared with human disagreement around \(0.022\) and a prior model-based method at \(0.12965\) [1909.09003].

### 4.3 Social-competence and human-centered evaluation

SocNavBench standardizes multi-metric evaluation over replayed real pedestrian trajectories, including success rate, failure modes, path quality, motion quality, closest-pedestrian distance, and time-to-collision [2103.00047]. It is a benchmark for downstream navigation execution rather than scene understanding, but its evaluation philosophy—multi-metric, trade-off-sensitive, and grounded in real human trajectories—is directly relevant to SocialNav-SUB [2103.00047].

SocRATES contributes scenario-generation quality metrics rather than scene-understanding metrics: simulability, contextual appropriateness, and alignment with the rough scenario [2412.19595]. It reports 55% success for unguided generation, 73% for guided generation, and shows that structured prompting materially improves executable social navigation scenario creation [2412.19595]. This suggests that benchmark quality itself can be evaluated as a first-class object.

## 5. Benchmark resources and scenario-generation frameworks

Several works are best understood as SocialNav-SUB infrastructure rather than finished benchmark specifications.

SONATA is a benchmark-enabling simulation toolkit rather than a canonical benchmark. Built on CoppeliaSim and PyRep, it supports manual or random generation of dynamic social navigation scenes with explicit symbolic relations, JSON export, and graph-structured representations [2009.05345]. It also demonstrates supervised control learning with an R-GCN over generated relational-temporal graphs, achieving 76.6% success in a goal-reaching use case [2009.05345]. Its main value for SocialNav-SUB lies in controllable scenario generation, explicit relation labels, and robot-centered structured data.

SOCIALGYM and SocNavGym are similarly framework-oriented. SOCIALGYM formulates social robot navigation as a POMDP
$$
M = \langle S, A, T, R, \Omega, O, \gamma \rangle
$$
and provides a lightweight 2D simulator with configurable observation, action, and reward spaces, along with benchmark scenarios for social action selection [2109.11011]. SocNavGym extends this direction with a Gym-style environment exposing structured observations over goals, humans, objects, walls, and relationships, and supports both heuristic and SNGNN-based social rewards [2304.14102]. Both are valuable for benchmark prototyping, but they abstract away raw perception and therefore align more with planning and policy evaluation than with scene understanding.

SocRATES addresses another infrastructure layer: automated scenario-based testing. It converts high-level metadata—social context, robot task, rough scenario, and location description—into detailed social navigation scenarios grounded in semantic scene graphs, graph trajectories, and behavior trees [2412.19595]. Its pipeline is especially useful if SocialNav-SUB is to include curated scenario classes such as elevator negotiation, corridor obstruction, blind intersection emergence, and group navigation [2412.19595].

## 6. Limitations, controversies, and emerging directions

The surveyed literature exposes several unresolved design tensions for SocialNav-SUB.

The first concerns whether scene understanding should be benchmarked directly or only through downstream navigation. Falcon, LISN-Bench, SocialNav, and SocNavBench primarily evaluate behavior, not explicit scene-understanding outputs [2409.13244], [2512.09920], [2511.21135], [2103.00047]. By contrast, SANPO, RoboSense, MUSON, and SocNav1 offer more explicit supervision for perception, reasoning, or social-scene scoring [2309.12172], [2408.15503], [2512.22867], [1909.02993]. This suggests that a mature SocialNav-SUB would likely require multiple tracks: perception, reasoning, and embodied execution.

The second tension concerns structured labels versus language-mediated supervision. MOSU operationalizes social understanding through VLM-based trajectory ranking prompts without explicit social labels [2507.04686]. LISN-Bench also grounds semantics through language and VLM tools [2512.09920]. MUSON and SocialNav add structured reasoning or traversability annotations but still rely heavily on generated text or VLM supervision [2512.22867], [2511.21135]. A plausible implication is that future SocialNav-SUB designs may need both symbolic labels and language-grounded annotations to support interpretability and flexible semantics.

A third issue is that many socially relevant factors remain weakly annotated. MOSU does not report social comfort, norm-violation counts, or personal-space metrics [2507.04686]. SANPO lacks interaction labels, proxemics, group behavior, and forecasting targets [2309.12172]. Falcon acknowledges that its simulated humans do not exhibit higher-level behaviors such as yielding [2409.13244]. SocNav1 is static and synthetic, with culturally localized annotations from native middle-class residents of Spain [1909.02993]. SocialGraph3D is strong in social semantics but limited to synthetic, mostly static indoor scenes without dynamic crowd behavior [2509.24966]. These omissions indicate that SocialNav-SUB remains a partially realized benchmark agenda rather than a solved specification.

The literature also points toward several emerging directions. SocialNav’s separation between a Cognitive Activation Dataset and an Expert Trajectories Pyramid suggests that benchmark design should disentangle cognitive supervision from action demonstrations [2511.21135]. MOSU’s fusion of geometry, semantics, context, and goal alignment suggests that trajectory-conditioned multimodal scoring is a practical benchmark structure for outdoor navigation [2507.04686]. MUSON’s five-step annotation scheme suggests that socially aware scene understanding benefits from explicit intermediate supervision over perception, prediction, reasoning, action, and explanation [2512.22867]. SocialGraph3D indicates that 3D human-centric scene graphs and open-vocabulary query answering may become important substrates for socially aware robotics [2509.24966].

Across these works, SocialNav-SUB emerges as a benchmark concept with partial methodological realization in multiple domains: outdoor multimodal traversability and context, indoor future-aware embodied navigation, egocentric pedestrian scene parsing, structured scene-graph scoring, and reasoning-oriented local action selection. No single benchmark yet unifies all of these dimensions. The current literature therefore supports viewing SocialNav-SUB as a synthesis target: a benchmark family that would combine socially meaningful scene representations, diverse environment types, explicit and implicit social supervision, and multi-level evaluation of perception, reasoning, and action.

Source: https://www.emergentmind.com/topics/social-navigation-scene-understanding-benchmark-socialnav-sub