Perspective-Based Active Grounding

• Perspective-based active grounding is a framework where models adjust visual observations by altering viewpoints to resolve ambiguous linguistic references.
• It employs adaptive methods such as query-aligned camera placement, dynamic cropping, and multi-perspective fusion to refine perceptual evidence.
• The approach integrates perception with reasoning in GUI, 3D, and embodied contexts, using uncertainty-driven selection to enhance grounding accuracy.

Perspective-based active grounding denotes a family of grounding procedures in which an agent or model does not treat perception as fixed, but instead adapts viewpoint, crop, frame selection, or reference frame so that linguistic intent can be resolved against the most informative visual evidence. In the literature, this idea appears in several forms: query-aligned camera placement for zero-shot 3D visual grounding, iterative crop-and-click policies for GUI grounding, active scene exploration for robotics, panoramic scene representations for 360-degree or 3D settings, and neuro-symbolic reasoning over frame-dependent spatial predicates (Li et al., 2024, Wang et al., 4 Sep 2025, Sripada et al., 2024, Jung et al., 24 Dec 2025, Kamali et al., 21 Jun 2026). Despite these differences, the common objective is to ground language under partial observability, clutter, occlusion, or perspective-sensitive spatial relations by selecting or constructing a better perceptual context rather than relying on a single passive observation.

1. Conceptual scope and defining properties

Perspective-based active grounding differs from passive grounding in that the model is allowed to modify the evidential basis of grounding. In GUI settings, this may mean zooming into a region and deciding whether additional focus steps are needed; in 3D visual grounding, it may mean rendering a scene from a query-aligned viewpoint; in robotics, it may mean physically moving a camera or robot to inspect hidden or ambiguous objects; and in multi-perspective spatial reasoning, it may mean explicitly switching frames of reference during inference (Wang et al., 4 Sep 2025, Li et al., 28 May 2025, Zhang et al., 7 Aug 2025, Kamali et al., 21 Jun 2026).

The term “perspective” is used in at least three technically distinct senses across the literature. First, it denotes literal camera viewpoint, as in query-aligned rendering in SeeGround or panoramic viewpoint placement in PanoGrounder (Li et al., 2024, Jung et al., 24 Dec 2025). Second, it denotes egocentric or embodied visual context, as in active object localization from first-person video and embodied 3D grounding from ego-centric RGB-D inputs (Wu et al., 2023, Zhang et al., 5 Jun 2025). Third, it denotes frame-of-reference selection in compositional spatial reasoning, where relations such as “left” or “in front of” must be interpreted with respect to a camera, object, agent, or virtual anchor (Huang et al., 15 Jul 2025, Kamali et al., 21 Jun 2026).

The term “active” is likewise instantiated differently. Some systems actively gather new observations through motion or exploration, such as AP-VLM, MAG-Nav, ChangingGrounding, and PerspAct (Sripada et al., 2024, Zhang et al., 7 Aug 2025, Hu et al., 16 Oct 2025, Patania et al., 11 Nov 2025). Others are active in the sense of adaptive internal perception, using cropping, zooming, or frame selection without changing the external world state, as in LASER, AutoFocus, and UniFunc3D (Wang et al., 4 Sep 2025, Yao et al., 4 May 2026, Lin et al., 24 Mar 2026). This suggests that perspective-based active grounding is best understood as a design principle rather than a single task formulation.

2. Core computational pattern

A recurrent computational pattern is an interleaved perception–reasoning loop in which candidate perceptual actions are evaluated according to their expected contribution to final grounding. LASER formalizes multi-step active perception in GUI grounding by letting the model receive an image region $r_t$ and instruction $I$, then predict either $\text{Crop}(x, y, w, h)$ or $\text{Click}(x, y)$, with behavior expressed as

$$\theta^{*} = \arg \max_{\theta} \mathbb{E}\left[\sum^{T}_{t=0} \log \pi_{\theta}(a_t | s_t)\right], \quad s_t = [I, s_{t-1}]$$

where $s_t$ records instruction and trajectory history (Wang et al., 4 Sep 2025). In this formulation, perception is not a preprocessing step but part of the policy.

A second recurring pattern is explicit quality estimation over perceptual choices. LASER constructs preference data by combining Monte Carlo quality estimation with IoU-based region diversity. Candidate crop trajectories are retained when they are both reliable and non-redundant, using

$$\mathcal{R}_{acc}(a^1_{\text{crop}}, a^2_{\text{crop}}) = \frac{1}{N}\sum_{i=1}^{N}\mathbb{I}(a^{(i)}_{\text{click}} = \hat{a}_{\text{click}})$$

and

$$\mathcal{R}_{div}(r_1^1, r_1^2) = \frac{|r_1^1 \cap r_1^2|}{|r_1^1 \cup r_1^2|}$$

followed by DPO on filtered pairs (Wang et al., 4 Sep 2025). AutoFocus takes a different route, treating token-level perplexity in coordinate generation as an intrinsic measure of spatial uncertainty and converting sampled coordinate hypotheses into an anisotropic Gaussian spatial probability field (Yao et al., 4 May 2026). In both cases, perceptual refinement is conditioned on uncertainty or estimated downstream utility rather than fixed heuristics.

A third pattern is structured aggregation across perspectives. Multi-view or multi-step systems usually require a fusion stage that reconciles per-view or per-crop evidence into a final grounded prediction. ViewSRD uses Cross-modal Consistent View Tokens to preserve viewpoint information across textual and scene modules, then aggregates multi-view predictions in a Textual-Scene Reasoning module (Huang et al., 15 Jul 2025). PanoGrounder grounds the query on each panoramic rendering, lifts per-view masks into 3D, and selects the best-consistent view through cross-view IoU scoring before fitting the final 3D box (Jung et al., 24 Dec 2025). UniFunc3D first performs coarse active spatial-temporal grounding, then refines within dense temporal windows and finally lifts verified 2D masks into 3D using multi-view agreement (Lin et al., 24 Mar 2026).

A common misconception is that active grounding is reducible to exhaustive multi-view enumeration. The surveyed systems do not frame it that way. They generally emphasize adaptive selection, relevance to the query, and reduced redundancy rather than uniform coverage alone (Li et al., 2024, Jung et al., 24 Dec 2025).

3. GUI grounding and adaptive visual focus

In GUI grounding, perspective-based activity is primarily realized as zooming, cropping, and instruction-conditioned focus control over high-resolution screenshots. LASER was proposed to endow VLMs with active perception and perspective-based grounding capabilities in dense GUI layouts, using a self-evolving framework that progresses from naïve perception to adaptive region focusing (Wang et al., 4 Sep 2025). Its preference data construction is notable because it jointly encourages both accuracy and diversity, and its experiments report 47.5% avg accuracy on ScreenSpot-Pro with a Qwen2.5-VL-7B backbone, a +20.7% improvement over its vanilla form, as well as 55.7% with GTA1-7B, establishing a new state-of-the-art among 7B-scale models (Wang et al., 4 Sep 2025).

AutoFocus addresses the same resolution gap from a test-time, training-free perspective. Rather than learning crop trajectories, it samples multiple coordinate hypotheses, computes axial perplexities

$$PPL_x^{(i)} = \exp \left( -\frac{1}{L_x} \sum_{j=1}^{L_x} \log P(t_{x,j} | \mathbf{t}_{x,<j}, \mathcal{I}, \mathcal{T}) \right)$$

and

$$PPL_y^{(i)} = \exp \left( -\frac{1}{L_y} \sum_{j=1}^{L_y} \log P(t_{y,j} | \mathbf{t}_{y,<j}, \mathbf{t}_x, \mathcal{I}, \mathcal{T}) \right),$$

then broadens each sample into an anisotropic Gaussian kernel and derives local and global region proposals for refinement (Yao et al., 4 May 2026). The framework reports consistent improvements across general-purpose and GUI-specialized VLMs on ScreenSpot-Pro and ScreenSpot-V2, with especially strong gains for icon and small-object localization (Yao et al., 4 May 2026).

UI-Ins shifts the locus of “perspective” from the visual crop to the instruction itself. It introduces the Instruction-as-Reasoning paradigm, treating diverse instructions as analytical pathways such as appearance, function, location, and intent, and letting the model select or compose them during reasoning (Chen et al., 23 Oct 2025). The paper reports a 23.3% flaw rate in existing grounding instructions, up to a 76% relative performance improvement from inference-time exploitation of instruction diversity, and state-of-the-art results including 57.0% on ScreenSpot-Pro for UI-Ins-32B (Chen et al., 23 Oct 2025). This suggests that perspective-based active grounding in GUIs can be implemented either through active visual focus or through active selection of linguistic perspectives, and sometimes through both.

4. Query-aligned viewpoints in 3D and panoramic grounding

In zero-shot 3D visual grounding, the central problem is how to present a 3D scene to a 2D VLM in a perspective that preserves the spatial and appearance cues implied by the query. SeeGround addresses this with a Perspective Adaptation Module that dynamically selects viewpoints for rendering. Given a query, the system extracts target objects and anchor or context objects, places the camera at the scene center, directs it toward the anchor, then moves it backward and upward to broaden visible context and reduce occlusion (Li et al., 2024, Li et al., 28 May 2025). On ScanRefer and Nr3D, the method outperforms previous zero-shot baselines by 7.7% and 7.1%, respectively, and in an ablation over viewpoint strategies its query-aligned rendering surpasses Center2Corner, Edge2Center, Corner2Center, and Bird’s Eye View, especially on Hard and View-Dependent queries (Li et al., 28 May 2025).

ViewSRD tackles a related but more linguistic problem: perspective variation in multi-anchor queries. Its Simple Relation Decoupling module decomposes complex expressions into perspective-aware single-anchor statements, and its Multi-view Textual-Scene Interaction module injects shared Cross-modal Consistent View Tokens into both text and scene streams (Huang et al., 15 Jul 2025). The model reports 68.6% accuracy on the View Dep. split of Nr3D and 71.5% on Multi-Anc, outperforming prior methods on queries requiring precise spatial differentiation (Huang et al., 15 Jul 2025). Here perspective is not only where the scene is viewed from, but also how the query is reformulated so that viewpoint-sensitive relations become explicit.

Panoramic methods extend the same logic to scene-scale coverage. PanoGrounder uses structure-aware, active placement of a compact set of panoramic viewpoints, scoring candidate camera locations by

$I$0

and greedily selecting cameras until at least 90% coverage of the scene’s footprint is achieved (Jung et al., 24 Dec 2025). Each panoramic view contains RGB, range, and semantic feature maps, allowing a pretrained 2D VLM to reason over a 360-degree representation that retains long-range object-to-object relations (Jung et al., 24 Dec 2025). PanoAffordanceNet generalizes panoramic grounding to scene-level affordances in 360-degree indoor environments, introducing the Distortion-Aware Spectral Modulator for ERP distortions and the Omni-Spherical Densification Head for topological continuity, with 360-AGD as the first high-quality panoramic affordance grounding dataset (Zhu et al., 10 Mar 2026). On the Hard split of 360-AGD, it reports KLD 1.31, SIM 0.47, and NSS 4.40, substantially better than OOAL and OS-AGDO (Zhu et al., 10 Mar 2026).

The contrast between query-aligned pinhole rendering and panoramic coverage is methodological rather than oppositional. A plausible implication is that the choice depends on whether the dominant bottleneck is disambiguating a local relation around an anchor or preserving global scene-wide relational context.

5. Embodied, egocentric, and memory-driven grounding

In embodied settings, perspective-based active grounding is inseparable from agent motion, egocentric sensing, and partial observability. AP-VLM combines a VLM with a 3D virtual grid overlaid on the scene and an Active Perception Policy that selects new viewpoints and orientations when the current observation is insufficient (Sripada et al., 2024). The system operates on real robotic platforms and is explicitly designed for cases in which fixed views are inadequate, such as objects hidden in mugs or visible only under changed camera orientation (Sripada et al., 2024).

MAG-Nav places the same principle inside language-driven navigation. Its pipeline includes initial VLM-based grounding, perspective-based active grounding by viewpoint optimization, and memory-based reserved grounding through historical memory backtracking (Zhang et al., 7 Aug 2025). The viewpoint planner optimizes a reward over visibility, field of view, distance penalty, and feasibility penalty,

$I$1

and the full system reports a Success Rate of 0.408 and SPL of 0.156 on HM3D, outperforming prior approaches; removing active grounding reduces SR from 0.405 to 0.307 in ablation (Zhang et al., 7 Aug 2025). ChangingGrounding pushes the same idea into dynamic scenes where old reconstructions may be stale. It formulates 3D visual grounding as an active, memory-driven problem, and Mem-ChangingGrounder uses memory retrieval, Omnidirectional Scene Scanner, Spatial Relation Aware Scanner, fallback behavior, and multi-view fusion to ground objects while reducing exploration cost (Hu et al., 16 Oct 2025). On its benchmark, the method reaches 36.8% [email protected], versus 32.4% for Wandering Grounding and 26.0% for Memory-only, while using 8.5 actions rather than 44.2 for Wandering (Hu et al., 16 Oct 2025).

Egocentric and embodied grounding also appear in non-navigation settings. “Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge” treats perspective-based active grounding as localizing and tracking objects undergoing change from a first-person viewpoint using task instructions (Wu et al., 2023). The framework augments GLIP with LLM-extracted pre-conditions, post-conditions, and descriptions, and uses per-object aggregation masking for joint inference, yielding >54% improvements in all standard metrics on TREK-150-OPE-Det, >7% improvements on TREK-150-OPE, and >3% AP on Ego4D SCOD (Wu et al., 2023). DEGround, by contrast, grounds from ego-centric multi-view RGB-D images by sharing DETR queries between detection and grounding and adding a Regional Activation Grounding module plus a Query-wise Modulation module; it reports 62.18% overall accuracy on EmbodiedScan validation, a +7.52% gain over BIP3D (Zhang et al., 5 Jun 2025).

These systems make clear that “active” in embodied grounding often has two layers: the agent actively changes what it sees, and the model actively conditions future perception on memory, symbolic context, or current grounding confidence.

6. Multi-agent, symbolic, and theoretical perspectives

Perspective-based active grounding is not limited to viewpoint geometry; it also includes explicit modeling of whose perspective governs spatial interpretation and how uncertainty should propagate through compositional reasoning. PerspAct extends the Director task with active visual exploration across seven scenarios of increasing perspective-taking complexity, allowing the Matcher agent to move, inspect, and ask clarifying questions while using ReAct-style reasoning (Patania et al., 11 Nov 2025). The results show that explicit perspective cues combined with active exploration improve interpretative accuracy in some scenarios, but both baseline and PerspAct still struggle on deeper epistemic cases such as Distractor and NotThat (Patania et al., 11 Nov 2025). This is a reminder that active vision alone does not solve perspective-taking when ambiguity is fundamentally social or epistemic.

SATURN addresses this limitation by explicitly separating perception from reasoning. It reconstructs an approximate 3D scene, derives soft perspective-aware spatial predicates with respect to anchors $I$2, and composes them in a training-free Pythonic symbolic executor using fuzzy logical operators (Kamali et al., 21 Jun 2026). Spatial predicate scores take the form

$I$3

allowing uncertainty to be preserved through multi-hop reasoning (Kamali et al., 21 Jun 2026). On 3D FORCE, SATURN reports 88.85% overall accuracy and remains stable as reasoning depth and perspective complexity increase; on MindCube it reaches 78.57% overall accuracy, outperforming the strongest baseline by 14 percentage points (Kamali et al., 21 Jun 2026).

At a broader conceptual level, “An Algorithmic Information-Theoretic Perspective on the Symbol Grounding Problem” reframes grounding as information compression, with grounding occurring when

$I$4

The paper argues that any fixed symbolic system is perspective-bound because what it can ground depends on the information already present in the system, and that adapting to new worlds requires genuinely novel information rather than mere inference from existing code (Liu, 2 Oct 2025). This suggests a theoretical interpretation of active grounding: activity is necessary because no fixed perspective is sufficient for all grounding problems.

A further historical antecedent is Situate, an architecture for semantic image retrieval that actively grounds the components of a visual situation through an agent-based search over object proposals and learned spatial relationships (Quinn et al., 2017). Although earlier and not framed in the language of VLMs, it already embodies a central principle of the field: partial detections update the search distribution for remaining entities, so grounding is progressively conditioned on a changing internal perspective (Quinn et al., 2017).

7. Representative systems and emerging directions

The current literature spans multiple input modalities and operational definitions of perspective. The following systems illustrate the breadth of the paradigm.

Domain	Representative mechanism	Representative evidence
GUI grounding	Multi-step crop/click policies, uncertainty-aware zooming, instruction-as-reasoning	LASER on ScreenSpot-Pro and ScreenSpot-v2; AutoFocus on ScreenSpot-Pro and ScreenSpot-V2; UI-Ins on five GUI benchmarks (Wang et al., 4 Sep 2025, Yao et al., 4 May 2026, Chen et al., 23 Oct 2025)
3D visual grounding	Query-aligned rendering, multi-view decomposition, panoramic viewpoint placement	SeeGround on ScanRefer and Nr3D; ViewSRD on view-dependent and multi-anchor queries; PanoGrounder on ScanRefer and Nr3D (Li et al., 28 May 2025, Huang et al., 15 Jul 2025, Jung et al., 24 Dec 2025)
Embodied and egocentric grounding	Robot viewpoint control, memory-guided exploration, ego-centric fusion	AP-VLM on Franka Panda and UR5; MAG-Nav on HM3D and quadruped deployment; DEGround on EmbodiedScan (Sripada et al., 2024, Zhang et al., 7 Aug 2025, Zhang et al., 5 Jun 2025)

Several directions recur across this body of work. One is the move from fixed views to adaptive evidence selection, whether through crop policies, frame sampling, scene rendering, or physical motion (Wang et al., 4 Sep 2025, Lin et al., 24 Mar 2026, Zhang et al., 7 Aug 2025). Another is the shift from hard decisions to uncertainty-aware or soft reasoning, visible in AutoFocus’s perplexity-derived spatial fields, LASER’s Monte Carlo preference construction, and SATURN’s soft predicates (Yao et al., 4 May 2026, Wang et al., 4 Sep 2025, Kamali et al., 21 Jun 2026). A third is the integration of memory and history, as in MAG-Nav and ChangingGrounding, where previous observations are not merely auxiliary but structurally necessary for efficient grounding in large or changing environments (Zhang et al., 7 Aug 2025, Hu et al., 16 Oct 2025).

A common misconception is that perspective-based active grounding is only a robotics problem. The surveyed literature shows otherwise: the same principle is central in GUIs, 3D scene grounding, egocentric video, panoramic affordance grounding, and symbolic multi-agent reasoning (Wang et al., 4 Sep 2025, Li et al., 2024, Wu et al., 2023, Zhu et al., 10 Mar 2026, Patania et al., 11 Nov 2025). Another misconception is that the paradigm is defined by any one mechanism, such as zoom-in search or multi-view rendering. The evidence instead indicates a broader methodological commitment: grounding systems should be able to adapt perception, perspective, or reference frame in response to linguistic demands and current uncertainty.

This suggests that perspective-based active grounding has become a unifying lens for tasks in which the correct referent is not recoverable from a single passive observation. Its technical realizations differ widely, but the central claim is consistent: grounding improves when models can decide not only what to infer, but also what to look at, from where, and under which perspective.