Remote Assistance in Navigation (RAIN)

• RAIN is a navigation paradigm in which remote guidance from operators or algorithms supplements user autonomy without fully taking over control.
• The system externalizes navigation state into manipulable maps and graphs, supporting decision-making in diverse domains from web browsing to vehicle operation.
• RAIN systems optimize safety and performance by integrating multimodal interfaces, such as haptic feedback, visual cues, and conversational grounding.

Searching arXiv for the core RAIN-related papers to ground the article in current preprints. Remote Assistance in Navigation (RAIN) denotes a class of systems in which navigation is supported by external intelligence rather than being left entirely to an unaided user or an autonomous controller. Across the literature, that intelligence may be a remote human operator, a remote or local planning service, or a shared-control layer that guides action without fully replacing human agency. The term covers web-navigation aids that externalize browsing history as an explicit map of virtual space (Herrouz et al., 2013), haptic-guided tele-navigation for mobile manipulators (Sripada et al., 16 Jun 2025), event-driven remote support for automated vehicles (Majstorovic et al., 2023), assistive wayfinding systems for visually impaired users (Gallo et al., 2013), and a dedicated embodied-dialog setting in which a Navigator collaborates with a remote Guide, released as the RAIN dataset (Han et al., 16 Sep 2025).

1. Definition and conceptual scope

In the broad sense used across these works, RAIN is a navigation paradigm in which guidance is produced through a collaboration between a navigating subject and an external assistive process. That process may expose the structure of a space, recommend or constrain actions, or resolve ambiguity when local perception is insufficient. In the web-navigation literature, this appears as a visited-site map that helps users “get one’s bearings” in a virtual space (Herrouz et al., 2013). In assistive robotics and teleoperation, it appears as shared autonomy, multimodal guidance, or supervisory control (Sripada et al., 16 Jun 2025). In embodied dialog, it appears as cooperative multi-turn communication between a Navigator and a Guide who must infer location from language rather than direct state access (Han et al., 16 Sep 2025).

A recurrent misconception is that remote assistance is interchangeable with remote driving. The automated-driving literature makes a sharper distinction. Remote Assistance (RA) is defined as an event-driven class of teleoperation in which the remote operator provides high-level assistance for a limited time while the vehicle retains the driving task and safety responsibility (Majstorovic et al., 2023). Remote Driving Systems (RDS), by contrast, perform direct and full control of the vehicle over an extended period of time through a Remote Driver at a Remote Control Station (Hans et al., 18 Jul 2025). Within that taxonomy, a RAIN system may be advisory and indirect, as in Remote Assistance Systems (RAS), or may become full remote control when the operational context requires continuous human driving.

The term also has a narrower, benchmark-specific meaning. In DialNav, RAIN is the name of a dataset and task setting for multi-turn, language-mediated navigation in photorealistic indoor scenes, where a remote Guide has global environmental knowledge but cannot directly observe the Navigator’s position (Han et al., 16 Sep 2025). This narrower usage does not replace the broader concept; it operationalizes one important form of it.

2. Core representations and control schemes

A central design pattern in RAIN is the externalization of navigation state into a manipulable representation. The most explicit instance is the graph formulation of navigation space as

$G = (V, E),$

where nodes represent locations or visited pages and directed edges represent transitions (Herrouz et al., 2013). In the web assistant, nodes store URL, title, and dwell time; the resulting map is both geographical and conceptual, because it records movement while also approximating how meaning is organized across documents. A closely related move appears in bronchoscopy, where topological localization places the scope within a generic airway tree and argues that accurate metric localization is not always required, because topological localization with regard to a generic airway model can often suffice to assist the surgeon with navigation (Tomasini et al., 10 Oct 2025).

Once navigation state is externalized, assistance can be attached to it through planning or control. In automated driving, Dynamic Collaborative Path Planning (DCPP) combines graph-based route computation on a modified HD-map with sampling-based path planning on an occupancy grid, and scores candidate routes using

$c(i,j) = w_1\,d(i,j) + w_2\,\frac{1}{p(i)},$

thereby balancing geometric distance and the preference cost of temporarily extending the Operational Design Domain (ODD) (Majstorovic et al., 2023). The same line of work formalizes an expanded ODD $O^+ = O_n \cup \Delta O$, allowing remote operators to authorize controlled deviations from nominal infrastructure or rules in exceptional cases.

In tele-mobile manipulation, assistance is injected directly at the control layer. HARMONI computes navigation-side haptic forces as

$$F_{fmr} = K_{fmr} \big(P^{t+40}_{fmr} - P_{fmr}\big),$$

where a future pose from the DWA trajectory acts as a virtual fixture that pulls the operator toward a safe, obstacle-aware path (Sripada et al., 16 Jun 2025). The human remains in the loop, velocity commands remain human-generated, and autonomy shapes the operator’s decisions through haptics rather than by direct command fusion. This suggests a general RAIN principle: external assistance is often most effective when it augments local decision-making while preserving legible human authority.

3. Interaction modalities and human factors

RAIN systems differ sharply in how assistance is conveyed. One family uses explicit visual structure. The connectivity-aware UGV interface estimates Direction of Arrival (DoA) of the radio signal strength and renders it as a color bar surrounding the video feed, so that green segments indicate directions where signal is strong or improving and red segments indicate directions where signal is weak or worsening (Parasuraman et al., 2017). The design keeps video central and network state peripheral, thereby integrating physical and communication situational awareness in one frame.

Another family uses haptics. HARMONI uses a 7-DoF leader arm as both tele-manipulation device and 2-DoF haptic joystick, with autonomous navigation producing force cues that pull the operator toward safe trajectories (Sripada et al., 16 Jun 2025). In social telepresence navigation, multimodal shared autonomy combines SA-RVO with haptic joystick forces

$$F(t) = K_p \left( v_A^{\text{optimal}}(t) - v_A^{\text{pref}}(t) \right),$$

and visual overlays such as guidance trajectories or steering bars; participants preferred multimodal assistance with a visual guidance trajectory over haptic or visual modalities alone, although it had no impact on navigation performance (Mbanisi et al., 2022). The distinction is important: subjective transparency and cooperation can improve even when path length, time, and proxemic intrusion metrics do not.

Assistive wayfinding systems for visually impaired users place special emphasis on preserving auditory channels. ARIANNA uses a smartphone camera to detect colored paths on the floor and a binary vibrational signal to indicate when the finger touches the path area, deliberately avoiding continuous audio because hearing remains necessary for environmental awareness (Gallo et al., 2013). MR.NAVI takes the opposite route of exploiting audio, but does so through spatial audio and concise scene descriptions generated from object detection, depth sensing, and LLM-based language output on HoloLens 2 (Pfitzer et al., 28 May 2025). The 3D-map RSA prototype adds another modality: a remote sighted agent uses an interactive 3D map with localization and a live camera feed, yielding significantly faster navigational assistance and significantly reduced mental workload relative to baseline RSA (Xie et al., 2022).

A further modality is conversational grounding. R2H frames the helper as a multimodal response generator that “can see and respond,” and evaluates helper quality by downstream navigation performance rather than surface text similarity (Fan et al., 2023). DialNav strengthens this model by requiring the Guide to infer the Navigator’s location from dialog alone, making communication itself part of the navigation problem (Han et al., 16 Sep 2025). A plausible implication is that future RAIN systems will increasingly treat dialog not as an auxiliary interface but as a core sensorimotor channel.

4. Representative domains and system families

RAIN spans digital, physical, and hybrid navigation. In web accessibility, the “Navigation Assistance and Web Accessibility Helper” constructs a directed map of visited pages, keeps that map visible in parallel with browser content, and uses dwell time and sequence reports to reduce disorientation and cognitive overload (Herrouz et al., 2013). Although implemented locally through a proxy server, the architecture already anticipates remote supervision, guided tours, and collaborative review of navigation traces.

In mobility assistance for visually impaired users, the spectrum runs from low-cost environmental instrumentation to mixed reality and VLM-based scene understanding. ARIANNA relies on colored paths and QR codes; the augmented reality navigation system for visual prosthesis combines localization, A$^*$ global planning, DWA local planning, and phosphene-based path overlays; MR.NAVI integrates object detection, depth sensing, obstacle clustering, public-transit APIs, and spatial audio; and recent VLM evaluations show that GPT-4o is currently the strongest among tested models for counting obstacles, relative spatial reasoning, and wayfinding-pertinent scene understanding, while open-source models struggle with nuanced reasoning and complex environments (Gallo et al., 2013, Sanchez-Garcia et al., 2021, Pfitzer et al., 28 May 2025, Li et al., 26 Jan 2026).

In robotics and teleoperation, RAIN includes both direct shared-control systems and supervisory interfaces. HARMONI unifies tele-navigation and tele-manipulation in one torque-level shared-control framework (Sripada et al., 16 Jun 2025). Connectivity-aware UGV teleoperation embeds radio-link gradients into the operator’s visual periphery (Parasuraman et al., 2017). Social telepresence navigation adds proxemics-aware velocity selection and multimodal guidance (Mbanisi et al., 2022). TUM Teleoperation generalizes these concerns into an open-source software stack with standardized ROS 2 interfaces, video, LiDAR, data, and configuration channels, and explicit support for both Remote Driving and high-level Remote Assistance with an automated driving stack such as Autoware (Kerbl et al., 16 Jun 2025).

In automated driving, RAIN becomes tightly coupled to ODD management. Dynamic Collaborative Path Planning allows a remote human operator to authorize system-generated path options that temporarily extend the nominal ODD in urban environments (Majstorovic et al., 2023). A separate framework then argues that the choice between RDS and RAS should be made from the ODD and use-case analysis rather than from architecture alone, using the PEGASUS six-layer model to assess where advisory assistance suffices and where full remote driving is required (Hans et al., 18 Jul 2025). This suggests that, in vehicular RAIN, “navigation assistance” is not a unitary capability but a family of support regimes whose suitability depends on road geometry, temporary manipulations, environmental conditions, and communication infrastructure.

In medical navigation, bronchoscopy introduces a clinically distinct but structurally similar problem. Online topological localization from bronchoscopic video yields a live probability distribution over airway nodes and is explicitly framed as navigation assistance without patient CT scan or extra sensors (Tomasini et al., 10 Oct 2025). The same broad logic applies: remote or algorithmic guidance operates on symbolic topology, not necessarily metric coordinates.

5. Evaluation traditions, benchmarks, and evidence

The evaluation literature is heterogeneous, ranging from qualitative educational deployments to statistically analyzed user studies and benchmark datasets. Representative examples are summarized below.

System or benchmark	Domain	Evaluation note
Navigation Assistance and Web Accessibility Helper	Web navigation	About 100 users, aged 19–22; qualitative benefits for orientation and cognitive load (Herrouz et al., 2013)
HARMONI	Tele-navigation and tele-manipulation	20 participants; smoother trajectories, lower HR for many participants, no significant change in pure tele-navigation completion time (Sripada et al., 16 Jun 2025)
DoA-enhanced UGV interface	Connectivity-aware teleoperation	20 usable datasets; more symbols found, higher RSS gain, longer connected exploration (Parasuraman et al., 2017)
AR navigation for visual prosthesis	Prosthetic vision	12 subjects; RoboticG reduced time, distance, collisions, and verbal interventions (Sanchez-Garcia et al., 2021)
Interactive 3D map for RSA	Remote sighted assistance	13 agents and 1 simulated user; faster assistance and lower workload (Xie et al., 2022)
DialNav / RAIN dataset	Embodied dialog navigation	2,231 episodes and 6,403 segments; joint evaluation of navigation, localization, and dialog (Han et al., 16 Sep 2025)

Several patterns emerge. First, many systems report human-factors gains even when classic navigation metrics move only modestly. HARMONI reports improved trajectory adherence and lower stress without significant change in pure tele-navigation completion time (Sripada et al., 16 Jun 2025). The social telepresence study reports preference, intent understanding, and cooperation gains for multimodal guidance despite no impact on navigation performance (Mbanisi et al., 2022). Second, benchmark-oriented work increasingly evaluates assistance holistically rather than as isolated captioning or path-planning. R2H evaluates helpers by downstream performer success and human judgments of faithfulness (Fan et al., 2023), while DialNav jointly measures success, path efficiency, dialog turns, and localization error (Han et al., 16 Sep 2025).

Third, evaluation difficulty rises sharply with realism. The VLM study on navigation assistance for blindness and low vision shows that even strong models display counting failures in cluttered scenes, biases in relative spatial reasoning, and occasional hallucination in navigation guidance (Li et al., 26 Jan 2026). This supports the view that RAIN evaluation cannot stop at average success: stability across repeated trials, uncertainty behavior, and failure mode structure matter directly for safety.

6. Limitations, controversies, and future directions

A persistent controversy concerns the proper scope of remote support. The automotive literature explicitly rejects the idea that one can select between remote assistance and remote driving on architectural convenience alone; the decisive variables are ODD and use-case complementarity (Hans et al., 18 Jul 2025). For dynamic, temporary manipulations such as construction zones and accident scenes, the paper argues that RDS is generally better suited because a human remote driver can overtake the entire DDT, whereas RAS is more appropriate for structured, intermittent, or strategic decisions. This is not merely a deployment issue; it is a conceptual boundary on what counts as assistance rather than takeover.

A second misconception is that richer localization is always better. The bronchoscopy literature states that accurate metric localization is not always required and that topological localization with regard to a generic airway model can often suffice (Tomasini et al., 10 Oct 2025). This matters beyond medicine. It suggests that many RAIN systems may benefit more from stable symbolic state estimation, route legibility, and ambiguity management than from ever more precise coordinates.

Across domains, the current limitations are domain-specific but structurally similar. Web navigation assistance remains mostly local and only hints at fuller remote collaboration (Herrouz et al., 2013). Haptic shared-control systems are often validated in static-map settings and do not provide formal passivity or stability proofs (Sripada et al., 16 Jun 2025). ARIANNA depends on physical markings and instrumented spaces (Gallo et al., 2013). MR.NAVI reports latency, outdoor sensing difficulties, and the inadequacy of generic walking instructions from public navigation APIs (Pfitzer et al., 28 May 2025). VLM-based assistance remains brittle, with hallucination and spatial bias still limiting safe deployment (Li et al., 26 Jan 2026). DialNav shows that generalization to unseen environments is poor and that modular pipelines propagate errors between localization, question generation, answer generation, and navigation (Han et al., 16 Sep 2025). TUM Teleoperation is explicitly not production-ready and does not yet address cyber-security or full regulatory compliance (Kerbl et al., 16 Jun 2025).

The literature nevertheless points toward a coherent future agenda. One direction is tighter integration of explicit maps, topological state, and multimodal interaction so that both human assistants and autonomous modules operate over the same navigation substrate. Another is adaptive selection among RAS-like and RDS-like support modes based on ODD, network conditions, and task criticality. A third is stronger alignment between assistive interfaces and user capability: haptic-first for some settings, audio-sparse for others, dialog-centered when disambiguation dominates. A plausible implication is that mature RAIN systems will be hybrid rather than monolithic: graph- and topology-aware, uncertainty-sensitive, multimodal, and capable of switching between advisory assistance and direct intervention as the navigational context changes.