Walking The Maps: Multimodal Navigation

• Walking The Maps is a research nexus redefining maps beyond static images by integrating walking as a sensing, evidentiary, and navigational process.
• Studies employ multimodal sensors—including accelerometers, cameras, and GPS—to create detailed, annotated graphs that optimize accessible routing and spatial data generation.
• The field bridges digital, paper, and embodied map interactions to improve urban routing, indoor mapping, and robotic navigation through advanced graph-based models.

Walking The Maps designates a research nexus in which maps are not merely consulted before locomotion. In the surveyed literature, walking is alternately the phenomenon to be optimized, the embodied sensing process that generates new spatial data, the evidence from which traversability is inferred, and the behavior executed from abstract, digital, or paper maps. The resulting systems span accessible urban sidewalk routing, walkability visualization, foot-vibration haptic mapping, floor-plan inference from indoor trajectories, citizen-science GPS mapping, pixel-based map navigation benchmarks for large vision-LLMs, behavior-graph navigation for robots, route extraction from paper hiking maps, and continuous video interfaces for street exploration (Bolten et al., 2016, Kim, 2023, Ying-Lei et al., 12 Apr 2025, Mura et al., 2021, Xing et al., 18 Mar 2025, Loo et al., 2024, Sugimoto et al., 2020).

1. Conceptual scope

A central feature of this field is that “map” no longer denotes only a metrically faithful overhead representation. In the accessible-routing literature, the operative map is an annotated sidewalk graph whose edges encode distance, slope, curb-ramp availability, and construction impact. In haptic walking research, the map is a geospatial surface of vibrations associated with ground materials and road conditions. In map-reading benchmarks, the input is a human-readable pixel map whose semantics must be converted into a valid route description. In behavior-based robotics, the map is a topological graph of executable actions rather than a metric occupancy model. In indoor trajectory inversion, the map is a floor plan reconstructed from the latent relation between human movement and room structure (Bolten et al., 2016, Ying-Lei et al., 12 Apr 2025, Xing et al., 18 Mar 2025, Loo et al., 2024, Mura et al., 2021).

This breadth also clarifies a common misconception: walking is not treated solely as path following. Some systems optimize walking, some observe it, some infer structure from it, and some ask machines to “walk” a map by converting symbols into sequential action. The question posed by “Walking = Traversable?” is therefore not rhetorical. That work explicitly tests whether observed human motion can stand in for direct measurement of free space, and answers it with a fused pipeline rather than a simple equivalence (Liang et al., 2023).

2. Walking as sensing, evidence, and map production

In VibWalk, walking becomes a haptic surveying process. The system is a shoe-mounted wearable with two customized accelerometers, one miniature microphone, one miniature RGBD camera, a Raspberry Pi 5, and 3D-printed PLA mounting modules, including vibration transmitter plates attached to the bottom of the shoe. It records wideband vibration signals generated by shoe-ground contact, using ACC over $0\sim 800$ Hz and MIC over $35\sim 18000$ Hz, with explicit coverage of low-frequency lower-limb dynamics below $10$ Hz, middle-frequency vibration features from $10\sim 400$ Hz, and high-frequency waveforms up to 18 kHz. ACC data are processed in 2-second windows with STFT, a 20 Hz high-pass filter, and Teager-Kaiser operator features; MIC data use 1-second windows, 64 Mel filter banks, an 800-sample frame hop, and a 20 Hz high-pass filter. A ResNet-based CNN with 5 BasicBlocks and a 10-layer configuration was selected as the best tradeoff between performance and runtime. In a study with 31 participants walking on 18 ground materials for 100 seconds each, the multimodal MIC + ACC + TKO model achieved a within-user f1-score of 96.1% and a cross-user f1-score of 88.5%; the abstract reports overall identification accuracy exceeding 95% and cross-user accuracy of 87%. GPS-linked classifications are then packaged by a phone and rendered as an interactive HTML map (Ying-Lei et al., 12 Apr 2025).

Walk2Map approaches map generation from a different sensing minimalism: a 2D projection of an indoor walk trajectory, obtainable from data-driven inertial odometry on smartphone IMU readings. The system predicts interior footprint, portals or doors, and furniture in a Discrete Floor Plan Grid. It separates area-related recovery from wall-related recovery, using an image-based Encoder-Decoder for footprint and furniture, and a Dynamic Edge-Conditioned GCN over the closed wall boundary loop for doors. Training is based on Matterport3D scans from 90 indoor environments, with about 50K extra LIFULL HOME’S samples for footprint extraction, and evaluation includes 20 real trajectories from offices and private homes. On simulated trajectories, Walk2Map reports interior footprint above 96% precision, recall, and F1, doors over 70%, and furniture close to 80%; on real trajectories, interior footprint remains above 96% despite domain shift (Mura et al., 2021).

The PfH line of work treats human movement as evidence for traversability. “Walking = Traversable?” uses a robot-mounted third-person monocular camera, ORB-SLAM3 for stationary objects, multiple object tracking for moving humans, and HO3-SLAM for reasoning about human-object occlusion ordering. Traversability is represented on a 2D grid with $10 \text{ cm} \times 10 \text{ cm}$ cells labeled traversable, untraversable, or unknown. The final prediction is the intersection of the PfH and HO3 regions,

$$\mathcal{T}_{final} = \mathcal{T}_{PfH} \cap \mathcal{T}_{HO3}.$$

Experiments on I-, L-, and T-configurations show that the combined method is generally stable, with map quality index scores of 2.35, 18.68, and 15.77 respectively for the full SfM + PfH + HO3 pipeline, where lower is better (Liang et al., 2023).

Citizen-science GPS studies extend this logic from controlled sensing systems to neighborhood-scale operative mapping. In Primer de Maig, Granollers, 72 participants in 19 groups explored a neighborhood through six festive tasks—dance, play, chat, lunch, celebration or toast, and celebration with confetti—using the Wikiloc App on tablet devices. The study shares about 3,532 raw GPS records, 2,981 processed records, and 339 detected stops, after removing 551 records outside the neighborhood boundary. Stop detection is based on

$\Delta_i(t)=t_{i+1}-t_i,$

with a stop defined by $\Delta_i(t)\ge \Delta_c$ and $\Delta_c = 10 \text{ s}$; distance and instantaneous velocity are computed as

$$d(t)=|\vec{r}(t+\Delta(t))-\vec{r}(t)|,\qquad v(t)=\frac{d(t)}{\Delta(t)}.$$

The resulting data are intended not only for description but for operative mapping of walkability, livability, and small-scale urban intervention (Larroya et al., 2024).

3. Accessibility, walkability, and route computation

Urban sidewalk routing makes explicit that a shortest path is not necessarily a traversable path. AccessMap was motivated by the informational gap faced by people with limited mobility, for whom route feasibility depends on steepness, sidewalk existence, curb ramps, construction, and crossability. Its first phase overlaid accessibility information on a map for self-planning; the next phase synthesizes open data into a fully connected, annotated sidewalk graph for variable cost-function accessible routing. The system integrates streets, sidewalks, curb ramps, elevation, and construction permits; it is implemented in Python, with de-noising and graph construction in SQL using PostGIS on PostgreSQL. Seattle-specific T-intersection errors were corrected by detecting 3-way intersections with angles between 170 and 190 degrees and reconnecting sidewalks within 100 feet, fixing 88.2% of the 4,352 T-intersection sidewalks examined. Across-street connections were then generated geometrically and topologically, without relying on explicit crosswalk metadata, and were produced at 88.1% of intersection corners. The resulting graph is routed with PGRouting, whose customizable SQL-based edge cost functions permit user-specific weighting of distance, slope, curb-ramp availability, and barriers such as construction (Bolten et al., 2016).

The Dublin walkability study uses a different but complementary formalization. Walkability is defined as the quality of an urban area measured by the distance required to walk to a range of amenities; unwalkability is the difference between Euclidean distance and Google Maps walking distance. The workflow combines 246,617 address points, about 1,900 amenity points, and 5,362 housing records. After sampling 1,000 random grid points across Dublin and filtering origin–destination pairs to distances between 1.5 km and 2 km, the study reports 4,862 Euclidean-distance calculations and 4,862 Google Maps walking-route computations. Tableau maps show that the largest discrepancies cluster at the urban edge, with examples including schools north of Phoenix Park requiring 4–5 km walking routes, sports clubs near the seashore yielding 5.2–7.6 km routes within the same 2 km radius, retail in Finglas reaching 9 km, and religious facilities showing 4.2–8 km walking distances. The top 20 least walkable areas tend to appear around Ballyfermot, Finglas, Marino, and other peripheral zones. Weighted Least Squares is then used to relate unwalkability to house prices; the final model has $35\sim 18000$0-statistic $35\sim 18000$1, $35\sim 18000$2, and $35\sim 18000$3, with all coefficients and interaction terms reported as significant (Kim, 2023).

Taken together, these studies define accessibility and walkability as properties of network structure, annotation quality, and urban form rather than as simple geometric proximity. This suggests that pedestrian routing becomes substantively different once sidewalks, crossings, slopes, and detours are treated as first-class variables.

4. Reading maps and converting them into actions

MapBench studies whether large vision-LLMs can read a human-readable map image and generate a valid route. It comprises 100 diverse map images and 1,649 path-finding queries drawn from nine scenarios: Zoo, Museum, National Park, Campus, Google Maps, Theme Park, Trail, Urban, and Mall. Each query provides a map image plus start and end landmarks. The key intermediate representation is the Map Space Scene Graph,

$35\sim 18000$4

whose nodes are landmark nodes $35\sim 18000$5 and intersection nodes $35\sim 18000$6, and whose edges encode topological and visual relations such as “connect,” “adjacent,” “observable,” and “unrelated.” MSSG supports both conversion from graph paths to natural-language directions and parsing of generated instructions back into structured paths. The benchmark evaluates zero-shot prompting and a Chain-of-Thought framework with four stages—Localization, Describe Surrounding, Connect Path, and Summarize—and introduces graph- and query-level measures including Elements Index, Meshedness Index, Average Shortest Path Length Index, Query Difficulty Index, and Path Quality Score. Evaluations of Llama-3.2-11B-Vision-Instruct, Qwen2-VL-7B, GPT-4o, and GPT-4o mini show that current LVLMs struggle substantially, with closed-source models generally performing better and open-source models often producing invalid or missing responses (Xing et al., 18 Mar 2025).

Scene Action Maps move from map reading to robotic execution without metric localization. The representation is a behavior graph

$35\sim 18000$7

whose nodes are destination nodes and changepoint nodes, and whose directed edges are labeled with behaviors such as turn-left, go-forward, and turn-right. For any node, each outgoing edge has a unique behavior. Offline map reading is decomposed into node prediction and edge prediction: a CNN named node predicts changepoints from sampled map patches, and a CNN named edge predicts behavior assignments to nearby neighbors, with the Sinkhorn algorithm used as a differentiable constrained matching layer. Planning is performed with Dijkstra’s algorithm over the behavior graph. Online graph localization adapts Graph Localization Networks and adds endpoint prediction to detect when the robot is near the end of its current edge. The full system was deployed on a Boston Dynamics Spot with an Nvidia AGX Xavier and three Intel Realsense d435i RGB-D cameras. It is evaluated on hand-drawn maps, floorplans, and satellite maps, with route difficulty grouped into easy, medium, and hard by number of changepoints and distance (Loo et al., 2024).

RouteExtract addresses the conversion of scanned paper hiking and sightseeing maps into GPS-ready routes. It is a modular pipeline with four stages: georeferencing, segmentation, graph construction, and iterative route generation. Georeferencing assumes manually provided ground control points and estimates an affine transform $35\sim 18000$8; roughly 5–10 GCPs were found sufficient in practice. Segmentation uses a U-Net on a 6-channel input formed by stacking the RGB map image with a sampled trail color vector. The resulting binary mask is skeletonized, transformed into an 8-connected graph, simplified by degree-2 path contraction and node collapsing, and then converted into graph-derived waypoint sequences. Route computation is delegated to an external pedestrian routing engine such as GraphHopper, and iterative refinement compares projected GPX routes to the trail mask via a symmetric Chamfer distance in EPSG:32654. The paper reports a median IoU of 0.763 with an interquartile range of 0.130 for segmentation, and a median Chamfer distance of 13.04 meters with a standard deviation of 11.40 meters when route generation is evaluated in isolation using ground-truth masks (Kremser et al., 15 Sep 2025).

These three lines of work share a common operation: they transform visually encoded map content into a structured representation over which path validity, action sequencing, or route quality can be computed.

5. Interfaces for moving through mapped space

Some systems make maps explorable by approximating the phenomenology of movement rather than only optimizing paths. The Movie Map system organizes omnidirectional street videos into a navigable city interface with four stages—acquisition, analysis, management, and interaction. Videos are captured by physically walking with an omnidirectional camera along streets in both directions; OpenVSLAM estimates camera trajectories independently for each video, and each trajectory is aligned to a common coordinate frame using two manually assigned reference coordinates. Intersections are detected by dividing trajectories into rectangles every 100 frames, testing overlap, and then refining candidate frame pairs through ORB-feature similarity after pose-based rotation. Videos are segmented between intersections, and turning views are synthesized by rotating and blending outgoing and incoming frames so that transitions feel smooth rather than abrupt. In user studies with 16 students, the blended turning view was generally preferred to direct switching or rotation alone, and exploration comfort was rated higher than with Google Street View; in one test area, no participant missed the target with Movie Map, while several missed it or failed to find it within time using GSV (Sugimoto et al., 2020).

GraphMaps transfers the online map metaphor to large graph browsing. It builds a sequence of layers $35\sim 18000$9, where each layer refines the previous one, and assigns each node a zoom level $10$0 in powers of two. Nodes persist across levels, while rails—the straight line segments used to represent edges—belong to the level at which they are created. The viewport zoom is computed as

$10$1

and the rendered layer for a viewport $10$2 is selected by $10$3, where $10$4. Rendering is restricted to entities in the chosen layer that intersect the current viewport. Quotas $10$5 and $10$6 bound the number of nodes and maximal rails that can appear in any view; in an example figure, $10$7 and $10$8. The method is designed to preserve the mental map by keeping vertex positions fixed and edge trajectories stable across zoom levels, while additional interactions permit node highlighting, rail highlighting, and label search. A caveat noted in the case studies is that edge bundling can obscure exact adjacency and loses edge directionality (Nachmanson et al., 2015).

At the urban-navigation end of the spectrum, AccessMap’s first version also belongs to this interface tradition: it overlaid sidewalk-related accessibility information on a map so that users could inspect barriers and accessible features before routing was added (Bolten et al., 2016). Across these systems, movement is represented not only as a shortest-path computation but as a controlled visual or interactional experience.

6. Formal structures, evaluation regimes, and recurring limits

A striking regularity across the literature is the reliance on graph-structured intermediates. AccessMap can be understood as constructing a graph $10$9 whose vertices are sidewalk endpoints, corners, and crossing-related nodes, and whose edges are sidewalk and crossing segments annotated with accessibility-relevant attributes. MSSG represents landmarks and intersections in pixel space for language conversion. Scene Action Maps encode changepoints and destination nodes connected by behavior-labeled edges. RouteExtract simplifies a skeletonized trail mask into a semantic graph retaining leaves and junctions. Walk2Map uses the Discrete Floor Plan Grid with cell labels $10\sim 400$0 and segment labels $10\sim 400$1. PfH uses a 2D traversability grid. GraphMaps uses layered node-and-rail structures rather than a single static node-link rendering (Bolten et al., 2016, Xing et al., 18 Mar 2025, Loo et al., 2024, Kremser et al., 15 Sep 2025, Mura et al., 2021, Liang et al., 2023, Nachmanson et al., 2015).

Evaluation is similarly task-specific but structurally aligned with downstream use. VibWalk reports within-user and cross-user f1-scores for material identification and shows that MIC + ACC + TKO outperforms MIC only, ACC only, and MIC + ACC. MapBench supplements answer validity with graph difficulty and path-quality metrics, including $10\sim 400$2, $10\sim 400$3, $10\sim 400$4, $10\sim 400$5, and $10\sim 400$6. Walk2Map uses precision, recall, and F1 with a 0.25 m correctness threshold. PfH evaluates predicted traversability maps by shortest-path deviation from an oracle map, producing a navigation-centric map quality index in which lower is better. RouteExtract uses IoU for segmentation and Chamfer distance for route-to-trail alignment. The Dublin study evaluates explanatory power with regression diagnostics, including Durbin–Watson $10\sim 400$7, $10\sim 400$8, and White-test evidence of heteroscedasticity, motivating Weighted Least Squares (Ying-Lei et al., 12 Apr 2025, Xing et al., 18 Mar 2025, Mura et al., 2021, Liang et al., 2023, Kremser et al., 15 Sep 2025, Kim, 2023).

The recurring limitations are also consistent. Municipal accessibility data are fragmented and noisy; raw sidewalk segments may be disconnected or misaligned. Paper-map extraction still requires manual GCP specification and is sensitive to unusual styling. LVLM map reading remains weak in structured decision-making, despite Chain-of-Thought prompting. Behavior-graph extraction struggles with long-range edges beyond the receptive field. Walk2Map is trained on synthetic trajectories and is restricted to single-room Manhattan-world environments. Movie Map requires manual reference coordinates and assumptions about route intersections. GraphMaps can preserve stability only at the cost of bundled abstractions that may hide exact directionality (Bolten et al., 2016, Kremser et al., 15 Sep 2025, Xing et al., 18 Mar 2025, Loo et al., 2024, Mura et al., 2021, Sugimoto et al., 2020, Nachmanson et al., 2015).

This suggests that Walking The Maps is less a single method than a convergence of multimodal sensing, graph construction, topological abstraction, and embodied execution. Across the surveyed work, maps become actionable when they are enriched with the attributes, behaviors, and observations that ordinary geometric representations omit.