Affordance-VP: Affordance-Guided View Planning

Updated 6 July 2026

Affordance-VP is a design pattern that uses predicted action relevance to select the next observation pose, visual region, or waypoint.
The methodology involves leveraging dense affordance maps, top-k pooling, and heuristic or learned planners to enhance reconstruction, manipulation, and navigation.
Empirical results demonstrate significant gains in hazard avoidance, manipulation accuracy, and navigation efficiency compared to geometry-only or random strategies.

Across the cited literature, Affordance-guided View Planner (Affordance-VP) denotes either a named next-best-view module or a closely related affordance-guided planning interface in which a system first predicts an affordance representation and then uses that representation to choose where to look, where to move, or which visual region should condition control. The exact phrase appears as a module name in ImageManip and GCNGrasp-VP (Li et al., 2023, Tong et al., 17 Jun 2026). Closely related formulations include a guided View Planner for active reconstruction (Yang et al., 2018), affordance-driven active view selection for 3D affordance grounding (Park et al., 14 Jan 2026), an action-aligned visual planning interface inside VLA models (Wang et al., 22 May 2026), Visual Affordances Prompting for continuous vision-language navigation (Chen et al., 2024), visual affordance prediction used as a goal-sampling distribution (Bharadhwaj et al., 2023), and a modular affordance-guided navigation planner that augments classical planning with learned navigability maps (Qi et al., 2020).

1. Terminology and conceptual scope

The term is not used uniformly across papers. In some works, Affordance-VP is a literal next-best-view selector. In ImageManip, it is the module that chooses a second camera pose from a discrete candidate set so as to improve affordance estimation for manipulation (Li et al., 2023). In GCNGrasp-VP, it is the camera planner that uses a dense affordance field as a task-aware information-gain proxy to select the next observation pose without explicit scene reconstruction (Tong et al., 17 Jun 2026).

Other papers use different names for substantially related mechanisms. “Active Object Reconstruction Using a Guided View Planner” defines a guided planner that sequentially predicts the next azimuth angle on a viewing circle using reconstruction improvement and projection consistency as reward signals (Yang et al., 2018). Affostruction frames the problem as selecting the next-best viewpoint that “maximizes visibility of high-affordance regions,” but the planner itself is a greedy deterministic heuristic over rendered affordance mass rather than a separately trained policy (Park et al., 14 Jan 2026). Afford-VLA does not define a separate view planner network; instead, it internalizes task-conditioned affordance as an explicit visual planning interface inside a VLA, implemented by <AFF> tokens, mask decoding, top- $k$ pooling, and direct action conditioning (Wang et al., 22 May 2026).

This variation matters because “affordance-guided” and “view planner” are both overloaded. In the narrower sense, the planner chooses a camera pose. In the broader sense, it chooses a task-relevant visual region, a candidate waypoint, or a cost field for a downstream planner. A plausible implication is that Affordance-VP is best treated as a design pattern rather than a single algorithmic template.

2. Affordance as the planning signal

The common element is the use of an affordance representation that is explicitly action-relevant. The representation, however, differs sharply across domains.

In navigation, “Learning to Move with Affordance Maps” learns a navigability affordance map: a per-pixel binary prediction of whether each visible location is traversable by the agent. The segmentation model maps RGBD input $x$ to a pixelwise affordance prediction $\hat{y}$ , and the resulting map is fused with a geometric occupancy map before planning with A* (Qi et al., 2020). The paper explicitly broadens traversability beyond geometry to include hazards, dynamic actors, and semantic constraints.

In passive visual affordance prediction for robot exploration, the affordance is not a mask but a distribution over plausible future visual states. The model learns

$p_\psi(o_g \mid o_c),$

where $o_c$ is the current scene image and $o_g$ is a plausible future image after interaction. Sampled future images are then used as visual goals for goal-conditioned exploration (Bharadhwaj et al., 2023).

In manipulation-oriented next-view selection, ImageManip defines the affordance map as a per-pixel probability of successful manipulation, with the initial module outputting

$a_p \in [0,1]$

for each pixel $p$ . The highest-affordance point becomes the center for candidate next-view generation (Li et al., 2023). GCNGrasp-VP instead predicts a dense affordance field over a point cloud, then upsamples it and selects the points whose scores exceed the $90$th percentile before clustering them with DBSCAN to identify the target region for camera motion (Tong et al., 17 Jun 2026).

In object-centric active perception, Affostruction predicts a 3D affordance heatmap over reconstructed geometry, decodes it into an affordance-colored mesh, and uses rendered affordance intensity as the viewpoint score (Park et al., 14 Jan 2026). In VLA manipulation, Afford-VLA predicts a task-conditioned affordance mask over image patches,

$G_t = \mathcal{D}_{\mathrm{aff}}(A_t, P_t),$

then selects top- $x$ 0 patches and pools them into an affordance embedding $x$ 1, which is concatenated with contextual hidden states to condition action generation (Wang et al., 22 May 2026).

A common misconception is that affordance always means object-part segmentation. The literature instead uses the term for navigability, future-state reachability, visible-ground masks, 3D functional saliency, per-point task relevance, and patchwise task-conditioned interaction regions.

3. Planning objectives and decision rules

The planning rule is the second axis along which Affordance-VP systems differ. Some are trained policies, some are discriminative view selectors, and some are post hoc heuristics.

In the reconstruction setting of (Yang et al., 2018), the planner is trained with a reward shaped by 3D IoU improvement, projection consistency, and a movement penalty: $x$ 2 The associated policy-gradient loss is

$x$ 3

Here, a good view is one that improves volumetric reconstruction and 2D projection agreement while avoiding redundant motion (Yang et al., 2018).

In ImageManip, viewpoint selection is cast as supervised classification over a discrete candidate set. The view-selection module $x$ 4 is trained from affordance-improvement labels: a candidate view is favorable when the post-fusion affordance confidence $x$ 5 exceeds the initial confidence $x$ 6, and supervision is applied with binary cross-entropy loss $x$ 7. The candidate space is split into nine candidates, with camera distance from the contact point between 2.5 and 4.5, azimuth between $x$ 8 and $x$ 9, and altitude between $\hat{y}$ 0 and $\hat{y}$ 1 (Li et al., 2023).

In Affostruction, the planner is explicitly not entropy-based, mutual-information-based, RL-based, or probabilistic in its decision rule. It samples

$\hat{y}$ 2

candidate poses uniformly on a hemisphere, renders the affordance-colored mesh from each pose, scores each by

$\hat{y}$ 3

and then chooses

$\hat{y}$ 4

This is a greedy max-sum visibility criterion over predicted affordance mass (Park et al., 14 Jan 2026).

In GCNGrasp-VP, the next-best-view objective is

$\hat{y}$ 5

with

$\hat{y}$ 6

and Bayesian-optimized weights $\hat{y}$ 7, $\hat{y}$ 8. The three terms encourage the camera to face the affordance cluster, avoid projected occlusion, and avoid degenerate top-down poses (Tong et al., 17 Jun 2026).

In AO-Planner, low-level motion planning is performed in image space. Grounded SAM produces visible-ground masks, candidate points are scattered inside those masks, an LLM proposes waypoint and path candidates, and a high-level PathAgent selects the final path before deterministic depth-and-intrinsics back-projection converts the chosen path to 3D motion (Chen et al., 2024). In Afford-VLA, the “planner” is internal: top- $\hat{y}$ 9 mask pooling yields the affordance embedding

$p_\psi(o_g \mid o_c),$ 0

which is appended to the hidden states as

$p_\psi(o_g \mid o_c),$ 1

before action prediction (Wang et al., 22 May 2026).

4. System-level instantiations

The main instantiations can be organized by the affordance signal they use and the variable they actually plan over.

System	Affordance signal	Planner output
Guided View Planner (Yang et al., 2018)	reconstruction/projection improvement	next azimuth on a viewing circle
ImageManip (Li et al., 2023)	per-pixel manipulation affordance map	one next view from nine candidates
Affostruction (Park et al., 14 Jan 2026)	3D affordance-colored mesh	argmax over 40 hemisphere poses
AO-Planner (Chen et al., 2024)	visible-ground affordance masks	candidate waypoints and paths
Afford-VLA (Wang et al., 22 May 2026)	task-conditioned affordance mask	top- $p_\psi(o_g \mid o_c),$ 2 patch selection for action conditioning
GCNGrasp-VP (Tong et al., 17 Jun 2026)	dense point-cloud affordance field	next camera pose $p_\psi(o_g \mid o_c),$ 3

Despite these differences, the surrounding pipeline often follows the same order. A first observation produces a coarse affordance estimate; candidate views, patches, or paths are generated around the predicted task-relevant region; a planner scores those candidates with a task-specific criterion; the selected observation is fused or the selected embedding is passed to the action module. ImageManip uses geometric-consistency-based token fusion to combine the initial global view and the selected close-up view into refined depth and affordance maps (Li et al., 2023). Affostruction iterates a reconstruction-affordance cycle in which better geometry improves affordance prediction and better affordance prediction improves view choice (Park et al., 14 Jan 2026). GCNGrasp-VP omits scene reconstruction entirely and uses the affordance field directly as the planning substrate (Tong et al., 17 Jun 2026).

The broader navigation literature shows the same modular pattern outside explicit next-view selection. In “Learning to Move with Affordance Maps,” the learned semantic layer is projected into a 2D plane, fused with geometry as

$p_\psi(o_g \mid o_c),$ 4

and inserted into a global allocentric map $p_\psi(o_g \mid o_c),$ 5 used by A*. The learned module therefore outputs a spatial cost or affordance field rather than direct actions (Qi et al., 2020).

5. Empirical behavior across tasks

Reported gains are strongest when geometry-only or random strategies fail to expose the task-relevant or hazard-relevant region.

In navigation, augmenting frontier exploration with affordance maps improves performance by about 60% in hazard-dense environments, outperforms the PPO-based RL baseline by about 70% in exploration, and improves navigation success rate by about 45% over the geometry-only A* baseline. Even with only one-fifth of the data, the method still yields about 25% improvement, and active sampling beats random sampling by more than 10% at 100k samples (Qi et al., 2020).

In visual goal generation for exploration, the affordance-based method achieves 70% on pushing, 60% on pick and place, and 60% on stacking, compared with 50%, 40%, and 30% for curiosity. The paper reports roughly 25% higher average success than baselines, and in a human perceptual study the model’s samples are preferred 69.8 ± 11.9 versus Pix2Pix and 75.5 ± 10.8 versus CVAE (Bharadhwaj et al., 2023).

In object-centric active view selection, Affostruction achieves 19.1 aIoU on affordance grounding and 32.67 IoU for 3D reconstruction, while the active-view ablation shows especially large gains under low view budgets: from around 4.3 aIoU initially to 9.2 aIoU after 1 additional view, compared with 4.7 for sequential and 6.2 for random. After 4 views, the proposed strategy reaches 12.4 aIoU, compared with 9.1 and 11.0 (Park et al., 14 Jan 2026).

In image-based manipulation, ImageManip reports that the refined depth map improves average AbsRel on seen categories from 0.5 to 0.2, with 0.35 on unseen categories. The ablations show that using a random next view is worse than the best-view strategy, while adding a third view yields only a small improvement over the two-view design (Li et al., 2023).

In task-oriented grasping, GCNGrasp-VP is strongest after a single view adjustment. At $p_\psi(o_g \mid o_c),$ 6 views, the reported mAPs are 98.42 for pan pour, 70.28 for watering can dispense, 76.10 for brush brush, and 50.16 for cup drink; the corresponding numbers for GauSS-MI are 76.59, 65.29, 50.49, and 33.35, and for Active-NGF are 82.95, 67.89, 50.83, and 22.21. Real-world grasp success after one planned movement is 24/24, 14/20, 20/28, and 10/24 on the same four tasks (Tong et al., 17 Jun 2026).

In VLA manipulation, internalized affordance planning also improves downstream control. Afford-VLA achieves 97.4% average success on LIBERO, 78.1% total on LIBERO-Plus, 58.1% average on SimplerEnv, and real-world results of 80% on Cup-to-Plate and 70% on Fork-in-Bowl (Wang et al., 22 May 2026). In continuous VLN, AO-Planner reaches SPL = 16.6% on R2R-CE validation unseen, compared with 11.1% for the prior best zero-shot method A $p_\psi(o_g \mid o_c),$ 7Nav (Chen et al., 2024).

Several limitations recur across the literature. Affostruction is explicit that its planner is greedy, heuristic, model-based, not learned, and not explicitly information-gain based; it also depends on reconstruction quality, lacks explicit uncertainty handling in the planner itself, and restricts optimization to 40 candidate poses on a hemisphere (Park et al., 14 Jan 2026). GCNGrasp-VP depends on affordance quality: under severe occlusion, affordance peaks may drift away from the true task-relevant region, and performance can decrease slightly with three views because of noise accumulation during multi-view feature fusion (Tong et al., 17 Jun 2026). ImageManip shows that more views are not automatically better, since the jump from two views to three views is small while computational cost rises (Li et al., 2023).

A second boundary concerns what the planner is actually optimizing. A common misconception is that Affordance-VP is simply a synonym for uncertainty reduction. That is not the case. GCNGrasp-VP contrasts its task-aware affordance objective with scene-uncertainty-driven baselines, arguing that uncertainty-driven methods may focus on geometrically complex but task-irrelevant regions such as brush bristles rather than the handle (Tong et al., 17 Jun 2026). Affostruction likewise uses visible affordance mass, not entropy or mutual information, as its scoring function (Park et al., 14 Jan 2026).

A third boundary concerns whether the method is truly a view planner. USA-Net is closely related because it learns a single differentiable memory that encodes both semantics and affordance and supports gradient-based navigation planning, with trajectories reported as 5–10% shorter and 10–30% closer to the goal query in CLIP embedding space than comparable grid-based planners. However, it is not framed as a camera-view selection system (Bolte et al., 2023). Similarly, Afford-VLA maps closely onto the Affordance-VP idea, but the paper’s own naming treats it as an internal affordance pathway rather than a separate planner network (Wang et al., 22 May 2026).

Taken together, these works show that Affordance-VP is not a single canonical module. It is a family of architectures in which affordance estimates act as the operative intermediate for selecting viewpoints, waypoints, path segments, or action-conditioning regions. What unifies the family is not the optimizer, the modality, or the supervision protocol, but the structural decision to let predicted action relevance determine what the system observes or attends to next.