Screen-space Guiding Maps

• Screen-space guiding maps are algorithmically generated visual aids that use mathematical formulations to encode spatial relationships for navigation and perceptual tasks.
• They employ domain-specific constructs to optimize sampling, reduce rendering variance, and support applications like avatar-mediated telepresence and off-screen POI localization.
• Practical implementations involve real-time optimization, perceptual modeling, and interactive rendering techniques to improve situational awareness and task performance.

Screen-space guiding maps are algorithmically generated, context-sensitive visual aids rendered in 2D or projected within a user’s display to support navigation, spatial alignment, or efficient sampling tasks. They operate at the intersection of perceptual user guidance and computational optimization, underlying a range of applications from avatar-mediated telepresence to off-screen point-of-interest (POI) localization and real-time rendering variance reduction. These maps exploit screen-space representations, optimizing both visual communication and task performance by explicitly modeling spatial relationships, perceptual biases, or sampling statistics.

1. Mathematical Formulation and Core Principles

Screen-space guiding maps are defined by domain-specific mathematical constructs that encode spatial or perceptual relationships, usually instantiated as scalar, vector, or feature fields on a 2D projection of the scene or interface.

Avatar-mediated Telepresence:

Guidance is achieved by associating user or avatar placements (poses in 2D Euclidean space, including orientation $\theta$) with "interaction feature" vectors of the form: $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$ where $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$ and likewise for the other angles. For $n$ targets, features are concatenated. The feature similarity between a sampled local placement $q$ and candidate remote placement $q'$ is computed as: $$S(\Phi_q, \Phi_{q'}) = \exp(-2 \|\Phi_q - \Phi_{q'}\|^2)$$ The recommendation score $r_q$ is the maximal $S$ over all collision-free, in-bounds $q'$ in the remote space, adjusted for constraint violation costs (Yang et al., 2022).

Off-screen POI Localization:

In Wedge-style guidance, the cognitive cost of a given geometric configuration (aperture $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$0, leg length $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$1, distance $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$2) is given by the Kullback–Leibler divergence between an ideal observer distribution $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$3 and the empirically modeled human estimate $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$4 (2D normal), yielding: $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$5 Optimization seeks parameters that minimize $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$6 subject to layout constraints (Miyagawa, 2022).

Monte Carlo Path-Guiding:

Guidance is encoded as parametric per-pixel mixture models on the incoming direction hemisphere, $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$7, with all sufficient statistics (means, covariances, mixture weights) maintained in screen-space G-buffers (Derevyannykh, 2021).

2. Sampling, Optimization, and Update Algorithms

Construction and use of screen-space guiding maps require domain-specific sampling and online or offline optimization:

Avatar Telepresence Placement:

• The local room is discretized on a 0.33 m 2D grid, with sample orientations every $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$8. Positions that do not face targets within $$\Phi_{s \rightarrow t} = [\phi^{(R)}_{s\rightarrow t},\, \phi^{(L)}_{s\rightarrow t},\, \phi^{(R)}_{t\rightarrow s},\, \phi^{(L)}_{t\rightarrow s}]$$9 are excluded.
• For each feasible local sample $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$0, a gradient-based search finds the optimal corresponding $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$1 in the remote space maximizing $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$2, subject to collision and out-of-bound penalties.
• Collision penalties are based on Mahalanobis-type distances between poses and object ellipses, while space-constraint penalties are exponential in distance outside the room bounds.

OptWedge POI Indicators:

• User perceptual responses to wedge geometries are empirically modeled (bias $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$3 and scatter $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$4 fitted by Gaussian process regression).
• Parameter optimization is performed for each POI, sometimes incorporating expected user bias directly into indicator positioning.
• Constraints on maximum wedge width and height set by UI layout are strictly enforced.

Monte Carlo Screen-Space Path-Guiding:

• At each pixel, mixture sufficient statistics (moments, weights) are updated per frame via exponential moving averages, blended using EM-style assignments based on the likelihood of each new light sample.
• Neighbor-guided updates further stabilize and generalize learnt statistics across nearby pixels (Derevyannykh, 2021).

3. Visual Encoding and Rendering Techniques

Guidance maps leverage compact, interpretable screen-space visual encodings aligned with their mathematical underpinnings:

Spatial Sector Encoding (Avatar Telepresence):

• Each sampled placement $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$5 is rendered on the floor as a "cone" with apex at $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$6, orientation $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$7, and $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$8 angular width.
• Cones are solid red if $$\phi^{(R)}_{s\rightarrow t} = \mathrm{atan2}(y_t^R-y_s,x_t^R-x_s) - \theta_s$$9 (experimentally set $n$0); otherwise, the rim is color-mapped from yellow ($n$1) to green ($n$2) in HSV.
• Sectors are linearly interpolated for off-grid heading queries.
• All sectors are instanced in a single mesh and colored by a real-time shader (Yang et al., 2022).

Overlaid Remote Geometry (Avatar Telepresence):

• Semi-transparent meshes of remote room elements are spatially aligned with local space according to the primary interaction target's frame, providing direct context for the validity of local placements.

Off-screen Wedge Indicators (OptWedge):

• For each POI, triangular wedges protrude from the visible edge. Their geometric parameters (leg, aperture) are optimized; for multiple POIs, orientation and spacing are constrained to prevent overlap.

G-buffer Textures (Path-Guiding):

• Two RGBA textures encode, per pixel, the moments, mixing weight, EM epoch, and optionally a packing float, supporting real-time dynamic updates and sampling.

Screen-space Minimap (Navigation):

• Minimaps are rendered as small, head-fixed overlays with north-up orientation, current user position, heading, and POIs or turn cues (Varshney et al., 18 Mar 2026).

4. Empirical Evaluation and Human-Centric Insights

Performance of screen-space guiding maps has been rigorously validated through empirical experiments involving both behavioral and physiological measures:

Preservation of Avatar Context (Telepresence):

• Recommendation score $n$3 robustly correlates with human perception of preserved gaze and pointing context (Spearman $n$4, $n$5).
• User studies established that $n$6 is an optimal threshold for suggesting valid placements, balancing recall and cognitive overload.
• Visual overload introduced by transparency overlays is moderate and tolerated by 80% of users (Yang et al., 2022).

Localization Accuracy (OptWedge):

• For nearby POIs ($n$7 m), both Unbiased and Biased OptWedge variants significantly reduce cognitive cost and localization RMSE relative to standard heuristics (Wilcoxon signed-rank, $n$8).
• At larger distances, cognitive model reliability—hence guidance improvement—declines due to under-constrained or noisy user response data (Miyagawa, 2022).

Navigation Efficiency (Screen-space Minimap):

• Minimap guidance provided intermediate support in dense, time-pressured VR mazes: nav-comp $n$9 ([0.18,0.44]), mean time $q$0 s, excess distance $q$1 (see Table below).
• Dwell time on minimap AOI strongly correlates with total workload and stress. Every additional second of minimap inspection reduces composite navigation performance by 0.12 points.
• Cognitive demand is primarily attributed to allocentric→egocentric translation for north-up static maps (Varshney et al., 18 Mar 2026).

Aid	Nav_Comp (M)	Completion Time (s)	Excess Distance (M)
Arrow	0.49 [0.36,0.61]	12.8	0.44 [0.31,0.57]
Minimap	0.31 [0.18,0.44]	13.9	0.55 [0.41,0.68]
Compass	0.15 [0.02,0.28]	14.4	0.74 [0.62,0.87]

5. Design Guidelines and Performance Considerations

Emergent best practices for the construction and deployment of screen-space guiding maps are domain-specific but share several algorithmic and perceptual commonalities:

Avatar Telepresence:

• Always precompute the Q$q$2→$q$3 table only when targets or local geometry change; 80–120 optimization queries (180 iterations each) complete in hundreds of ms on commodity CPUs.
• Recommendations should be rendered as high-contrast floor sectors and, where tolerated by users, co-rendered with transparent overlays of remote geometry for situational clarity.

OptWedge-style Off-screen Cues:

• Use Unbiased OptWedge as default; expand wedge aperture for closer POIs for improved bias/scatter.
• For systematic user underestimation of distance at large ranges, Biased OptWedge can increase localization fidelity by pre-shifting indicator geometry.
• Calibration requires hundreds of user trials over parameter space and careful empirical regression.
• Multiple POIs require explicit geometric (non-overlap) constraints in indicator layout.

Screen-space Path-guiding:

• Maintain only 8 per-pixel parameters for real-time performance, using dynamic EM updates across frames and neighbors.
• Added rendering overhead is ∼1.5 ms@1080p/RTX2070, with net throughput gain (~5%) due to more coherent path tracing; FLIP error reduction up to 4×, and substantial flicker/noise suppression.

VR Navigation Minimap:

• For head-fixed minimaps, a heading-up (egocentric) orientation is recommended to reduce translation demands.
• Minimalist symbology, adaptive highlighting of blocked or turn segments, and careful management of visual complexity can directly improve both navigation efficiency and subjective workload.

6. Cross-Domain Impact and Future Directions

Screen-space guiding maps encapsulate a common theme: encoding action-relevant or perception-relevant information into concise 2D visualizations optimized by rigorous modeling of user behavior, spatial constraints, or statistical properties. Their role spans collaborative MR, UI design for constrained displays, and real-time high-dimensional sampling in rendering.

Future directions include:

• Integration of deeper cognitive and behavioral models (e.g., online adaptation to individual user profiles).
• End-to-end differentiable pipelines linking perceptual outcomes (e.g., navigation speed, localization error) directly to visual encoding parameters.
• Emergence of domain-agnostic screen-space guidance frameworks capable of supporting both task performance and user comprehension in highly dynamic or multi-user environments.

The convergence of perceptual modeling, task-specific optimization, and real-time visualization continues to drive the evolution of screen-space guiding maps as foundational infrastructure for spatial computing and immersive interface design (Yang et al., 2022, Miyagawa, 2022, Derevyannykh, 2021, Varshney et al., 18 Mar 2026).