---
title: Dual-Level Visual Guidance System
url: https://www.emergentmind.com/topics/dual-level-visual-guidance-system
type: topic
---

# Dual-Level Visual Guidance System

Searching arXiv for the cited papers and closely related work on dual-/multi-level visual guidance.
A dual-level visual guidance system is a design pattern in which visual guidance is split across two interacting levels rather than handled by a single monolithic mapping. Across recent work, the term is used for systems that separate a slow, deliberative, or global layer from a fast, reactive, local, or execution layer, with the two levels coupled by structured interfaces such as visual prompts, semantic embeddings, pixel goals, floorplan patches, motion cues, or AR overlays [2603.22003]. In this usage, the high-level layer typically specifies *what* and *where*, while the low-level layer specifies *how*; the result is a family of architectures for robotics, video understanding, floorplan localization, augmented reality, assistive navigation, visual generation, image restoration, and mixed-reality training [2508.17442].

## 1. Concept and defining characteristics

In robotics, the pattern is stated explicitly as a split between “a slow, deliberative ‘System 2’ that reasons in language and scene semantics, and a fast, reactive ‘System 1’ that executes precise motor control, with a visual prompting interface in between” [2603.22003]. In video understanding, the same structure appears as “global, event-level guidance” together with “local, sub-event-level guidance,” where an LVLM supplies semantic descriptions that steer lower-level spatio-temporal processing [2508.17442]. In floorplan localization, the term refers to “first get depth-based candidates, then use dual-level visual-geometric contrastive signals to pick the correct one,” namely position-level and orientation-level guidance [2601.01822].

This suggests that the phrase does not name a single algorithm. It names a recurrent systems principle: one level provides stable, semantically meaningful, or globally constraining guidance, while another level uses denser, lower-latency, or more spatially specific signals to complete perception, control, or decision making. In some systems the split is architectural and asynchronous, as in fast-slow navigation models [2607.08359]; in others it is representational, as in the use of semantic and structural priors in visual text generation [2501.05892]; and in still others it is user-interface driven, as in the separation between task planning and situated AR rendering [2508.03547].

The high-level layer is generally associated with instruction interpretation, task decomposition, global path selection, semantic summarization, or user-facing strategy selection. The low-level layer is generally associated with precise grounding, local motion, temporal calibration, candidate disambiguation, or visually explicit execution cues. A plausible implication is that dual-level guidance is most useful when a single stage would otherwise have to solve semantically heterogeneous subproblems at once, such as instruction parsing, grounding, spatial disambiguation, and motor generation.

## 2. Architectural patterns and interfaces

A common pattern is **planner–controller decomposition**. In "VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models" [2603.22003], standard VLA policies of the form $a_t = \pi_{\theta}(l, o_t)$ are replaced by a dual-system architecture in which a “System 2 Planner” decomposes the task into subtasks, identifies target objects and target locations, and generates visual prompts, while a “System 1 Controller” consumes raw observations and the prompt image to output action chunks. The controller policy becomes $a_t = \pi_{\theta}(l, o_t, I_{vp}^t)$, and the visual prompt image $I_{vp}^t$ is the System 2 $\rightarrow$ System 1 interface [2603.22003].

A second pattern is **global-semantic and local-refinement coupling**. In "Multi-Level LVLM Guidance for Untrimmed Video Action Recognition" [2508.17442], a Cross-Modal Guidance Branch uses an LVLM to generate a single “Global Event Prompting” narrative and multiple “Temporal Sub-event Prompting” descriptions, which are then injected into the video encoder through adaptive gating, cross-modal attention, and an event graph module. Here the interface is not an image overlay but a set of semantic embeddings and graph relations that modulate visual features at different temporal scales [2508.17442].

A third pattern is **coarse candidate generation followed by local disambiguation**. In "DisCo-FLoc: Using Dual-Level Visual-Geometric Contrasts to Disambiguate Depth-Aware Visual Floorplan Localization" [2601.01822], a Ray Regression Predictor produces a Depth-Aware FLoc Probabilistic Map, Top-$X$ pose candidates are selected, and a second-stage contrastive module evaluates local floorplan patches using position-level and orientation-level negatives. The two levels are fused as $P_{\text{final}(S_i)} = (1 - w)\,P_d(S_i) + w\,P_{\text{CL}(S_i)}$, with $w = 0.5$ by default [2601.01822].

A fourth pattern is **slow global reasoning with fast local control**. In "FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigation" [2607.08359], the slow stream extracts stable semantic priors from a pre-trained VLM, while a Diffusion Transformer fast stream models cross-temporal action distributions; the two branches run asynchronously. In "Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation" [2512.08186], System 2 predicts a mid-term waypoint as a 2D pixel goal and a latent goal representation, while System 1 uses those signals plus high-frequency RGB to generate smooth continuous trajectories at much higher frequency [2512.08186].

In user-interface systems, the interface often becomes a **guidance rendering layer**. "Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Models" [2508.03547] separates a semantic level, where ChatGPT-4o generates step-wise instructions and assigns one of five visual guidance types, from a spatial level, where Gemini, SAM 2.1, ARKit, and SceneKit localize and render highlights, movement animations, hand gestures, tool usage, and timers. "JARVIS: A Just-in-Time AR Visual Instruction System for Cross-Reality Task Guidance" [2604.10108] similarly separates pre-task planning and in-task state verification from the visual renderer that grounds bounding boxes, shape previews, arrows, and embodiment cues into the scene [2604.10108].

## 3. Guidance representations and learning mechanisms

The interface between levels varies by domain, but the representations are consistently structured. In VP-VLA, the planner outputs a visual prompt representation $\psi_t = \{C, B\}$, where $C \in \mathbb{R}^2$ is a crosshair placed at the centroid of the target object and $B \in \mathbb{R}^4$ is a bounding box marking the placement or goal region; these are rendered on the overhead camera view to produce $I_{vp}^t$ [2603.22003]. In DualVLN, the high-level planner outputs a 2D pixel goal in the current image together with a latent goal representation extracted by learnable query tokens appended to the VLM sequence [2512.08186]. In AerialVLA, the “minimalist dual-view perception” uses a vertical mosaic $I_{\text{comp}} = [I_{\text{front}}; I_{\text{down}}]$ so that the front view provides critical cues for obstacle avoidance and target identification, while the down view is essential for precise ground alignment and landing maneuvers [2603.14363].

Training objectives likewise reflect the separation of levels. VP-VLA adds an auxiliary visual grounding loss to ensure that the controller actually uses the prompts: image coordinates are discretized into $N = 1000$ bins, cross-entropy is used for grounding, and the total loss is an action loss plus event-triggered grounding loss with $\lambda = 0.1$ [2603.22003]. ECVT combines classification loss, temporal GIoU regression, a semantic consistency loss between calibrated visual features and LVLM sub-event embeddings, and a temporal calibration loss anchored to an event graph [2508.17442]. DisCo-FLoc combines a ray regression loss for depth-aware candidate generation with a PointInfoNCE-style contrastive loss that includes both position-level negatives and orientation-level negatives [2601.01822].

In diffusion-based generation and restoration, the two levels can be semantic and structural rather than planner and controller. "Beyond Flat Text: Dual Self-inherited Guidance for Visual Text Generation" [2501.05892] defines a Semantic Rectification Branch that injects a flat-text semantic prior and a Structure Injection Branch that injects a glyph-structured prior, both by masked AdaIN and weighted fusion. "HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance" [2505.19742] constructs a positive prompt from a high-quality branch and a negative prompt from residual-noise and HMB-segmentation branches, then combines the resulting predictions by $z_\varepsilon = z_\text{neg} + \lambda_\text{cfg}(z_\text{pos} - z_\text{neg})$ [2505.19742]. This suggests that dual-level guidance need not be hierarchical in time; it can also be hierarchical in representation, with one level correcting content and another constraining structure or degradation.

## 4. Representative instantiations across domains

The pattern appears in a wide range of application areas.

| Domain | Representative system | Two guidance levels |
|---|---|---|
| Vision-language-action robotics | VP-VLA [2603.22003] | System 2 Planner / System 1 Controller |
| Untrimmed video action recognition | ECVT [2508.17442] | Global Event Prompting / Temporal Sub-event Prompting |
| Floorplan localization | DisCo-FLoc [2601.01822] | Depth-aware candidate map / position-orientation contrastive disambiguation |
| AR task guidance | Guided Reality [2508.03547] | Instruction & visual strategy / visual grounding & rendering |
| VLN and aerial navigation | FSD-VLN [2607.08359], DualVLN [2512.08186] | slow semantic planner / fast diffusion-based controller |
| UAV VLA control | AerialVLA [2603.14363] | front-view coarse navigation / down-view precision landing |

Assistive and human-centered systems extend the same principle to perception, instruction, and accessibility. "NavVI: A Telerobotic Simulation with Multimodal Feedback for Visually Impaired Navigation in Warehouse Environments" [2507.15072] has a higher-level path-planning layer and a lower-level proximity/obstacle layer, expressed across visual, audio, and haptic channels. "Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments" [2410.20666] separates a global, symbolic text-map planner from local camera-based localization, hazard reasoning, and low-level robot motion. "VeasyGuide: Personalized Visual Guidance for Low-vision Learners on Instructor Actions in Presentation Videos" [2507.21837] separates a low-level activity recognition pipeline from a high-level activity visualization module that turns detections into personalized highlights and zoom behavior [2507.21837].

Some systems define the two levels perceptually rather than computationally. "Bridging the Gap: Enhancing Gaze-Performance Link in Children with ASD through Dual-Level Visual Guidance in MR-DMT" [2510.03724] uses a perceptual level implemented by salience modulation and a transformational level implemented by kinetic metaphors such as animated fish that travel along limb trajectories. "Beyond Symbols: Motion Perception Cues Enhance Dual-Task Performance with Wearable Directional Guidance" [2601.17799] prototypes a foveal task-focused channel together with a monocular, peripheral motion channel that carries directional guidance on a wearable display [2601.17799]. In these cases, the two levels are split across human perceptual pathways rather than across separate machine subsystems.

## 5. Empirical patterns and reported benefits

A recurring empirical claim is that decoupling levels improves spatial precision, robustness, interpretability, or latency. VP-VLA reports that experiments on the Robocasa-GR1-Tabletop benchmark and SimplerEnv simulation demonstrate that the method improves success rates by 5% and 8.3%, surpassing competitive baselines including QwenOFT and GR00T-N1.6 [2603.22003]. On RoboCasa, ablations show that the full method reaches 53.8% average, compared with 49.4% without grounding loss, 49.5% with grounding at all frames, 47.3% when changing crosshair to point, and 50.8% for direct overlay onto the raw image [2603.22003].

ECVT reports 40.5% average mAP on ActivityNet v1.3 and 67.1% mAP@0.5 on THUMOS14, outperforming leading baselines; ablations show that both GEP-only and TSEP-only improve over a baseline video transformer, while the full dual-level system performs best [2508.17442]. DisCo-FLoc reports that it outperforms the state-of-the-art semantic-based method on two standard visual FLoc benchmarks, and ablations on Structured3D(full) show that adding both position-level and orientation-level negatives improves R@0.5 m from 55.9 to 59.0 and R@1 m 30° from 63.2 to 66.0 [2601.01822].

In navigation, FSD-VLN reports up to 2X higher navigation success rates on unseen scenes than SOTA methods, while cutting single-action inference delay and total task runtime by over 50% [2607.08359]. DualVLN reports that the asynchronous design yields 0.03 s inference for 32 trajectories in parallel in System 1, and that it outperforms prior methods across VLN benchmarks while maintaining real-world long-horizon planning and dynamic adaptability [2512.08186]. AerialVLA reports that on the TravelUAV benchmark it achieves state-of-the-art performance in seen environments and nearly three times the success rate of leading baselines in unseen scenarios, while an ablation shows that a 5-view design degrades unseen-map success relative to the streamlined dual-view setup [2603.14363].

In human-centered interfaces, effects are often reported as reduced search effort or improved dual-task performance. Guided Reality reports a user study with $N=16$ and a reduction in error rate in a printer task from 17.14% to 10.42% [2508.03547]. VeasyGuide reports that low-vision learners improved in detecting instructor actions, with mean success rate increasing from 0.61 to 0.88 and a significant reduction in NASA-TLX dimensions such as mental demand and frustration [2507.21837]. The wearable motion-cue study reports that, in a demanding dual-task scenario, the motion-based approach resulted in significantly more accurate interpretation of directional cues with $p=.008$ and showed a trend towards reduced errors on the concurrent primary task with $p=.066$ [2601.17799]. This suggests that dual-level guidance can improve performance either by redistributing computation across machine modules or by redistributing perceptual load across human channels.

## 6. Limitations, failure modes, and open directions

Across domains, the most common limitation is dependence on the correctness of the higher-level signal. VP-VLA states that performance depends on the correctness of System 2 reasoning and segmentation; if System 2 misidentifies objects or subtask progress, visual prompts will mislead the controller [2603.22003]. ECVT notes dependence on LVLM quality and static offline guidance; if the LVLM misinterprets rare or novel actions, both global and local guidance can be misleading [2508.17442]. DisCo-FLoc depends on accurate calibration and on the quality of candidate generation; repetitive layouts and orientation ambiguity remain challenging even after contrastive disambiguation [2601.01822].

Another limitation is restricted prompt expressiveness. VP-VLA notes that crosshair plus box are 2D and relatively coarse, and suggests richer affordance cues such as keypoints, heatmaps, or 3D prompts [2603.22003]. DualVLN shows robustness to moderate pixel-goal errors but still fails on large or semantically incorrect goals, especially near obstacles [2512.08186]. AerialVLA lacks global memory and can become conservative in highly repetitive environments, which suggests that purely reactive dual-view control may need an additional long-range memory or exploration mechanism [2603.14363].

In user-interface systems, personalization and ecological validity remain open issues. Guided Reality is currently iPad-based and open-loop, with no state-aware correction during rollout [2508.03547]. VeasyGuide uses a motion-based, content-agnostic detector; future systems could align highlights with semantic slide structures or support activity-based navigation [2507.21837]. The ASD MR-DMT system uses static, rule-based guidance rather than real-time eye tracking and motion tracking; the paper explicitly points to adaptive and personalized guidance as future work [2510.03724].

Open directions stated across the literature converge on a few themes: more expressive visual prompts, tighter planner–controller co-training, dynamic prompting or online adaptation, richer graph or 3D scene representations, and stronger mechanisms for detecting unreliable high-level cues [2603.22003]. A plausible implication is that the next generation of dual-level visual guidance systems will move from fixed, one-way guidance interfaces toward closed-loop systems in which the lower level not only executes but also informs, queries, or corrects the higher level.

Source: https://www.emergentmind.com/topics/dual-level-visual-guidance-system