Distribution-Robust Affordances

• Distribution-robust affordances are representations of action possibilities that remain valid when deployment conditions differ from training data.
• They integrate intermediate structures such as calibrated uncertainty, geometric recovery fields, and risk-aware value estimators to manage out-of-distribution shifts.
• This design principle spans multiple frameworks—from probabilistic decision-making and Bayesian perception to transferable manipulation priors—ensuring success in dynamic, real-world tasks.

to=arxiv_search.search 大发快三的 天天中彩票公司 大发棋牌json {"query":"\"distribution-robust affordances\" affordance robust manipulation few-shot manipulation task distribution affordances", "max_results": 10} to=arxiv_search.search ￣奇米影视ೆಯ 重庆时时彩的json {"query":"(Ma et al., 13 May 2026) OR (Khetarpal et al., 11 Feb 2026) OR (Achab et al., 2021) OR (Wu et al., 2024) OR (Nasiriany et al., 2024) OR (Ju et al., 2024) OR (Lowin et al., 16 Mar 2026) OR (Xu et al., 8 Dec 2025) OR (Mur-Labadia et al., 2021) OR (Mur-Labadia et al., 2023) OR (Gruszczynski et al., 4 Dec 2025)", "max_results": 15} Distribution-robust affordances are affordance representations, operators, or estimators designed to remain reliable when deployment departs from the narrow conditions present in training or demonstration data. In the current literature, the term does not denote a single formalism. Instead, it spans object-centric action fields for robotic manipulation under pose, viewpoint, clutter, and disturbance shifts; Bellman-consistent safe and risky action summaries under transition uncertainty in dynamic programming; state–option pairs whose intended effects remain valid across a task distribution; uncertainty-aware perceptual affordance estimators; transferable 3D contact-and-trajectory priors for cross-category manipulation; and rough or modal affordance relations over actor–object–environment triples (Ma et al., 13 May 2026, Achab et al., 2021, Khetarpal et al., 11 Feb 2026, Wu et al., 2024, Gruszczynski et al., 4 Dec 2025).

1. Conceptual scope and principal meanings

Affordances are treated in several works as relations between an agent, objects, and possible actions. In Bayesian RGB-based affordance prediction, they are defined as “relationships between an agent, the objects, and the possible future actions in the environment,” specialized to socially acceptable and context-dependent affordances (Mur-Labadia et al., 2021). In the logical line of work, an affordance is formalized as a ternary relation

$\varphi \subseteq A \times O \times E,$

where $(a,o,e)$ means that entity $o$ affords action $\mathsf{Act}_\varphi$ to actor $a$ in environment $e$ (Gruszczynski et al., 4 Dec 2025). In multi-task planning with LLMs, an affordance is a state–option pair $(s,o)$ for which the intended trajectory distribution is close to the realized one, and a distribution-robust affordance is one whose intent remains satisfied with bounded failure probability over a task distribution (Khetarpal et al., 11 Feb 2026). In distributional dynamic programming, the same phrase is used for robust value summaries attached to actions, with worst-case and best-case Bellman AVaR components under an augmented robust MDP interpretation (Achab et al., 2021). In robotic manipulation, the emphasis shifts toward geometric or object-centric structures that specify where and how interaction remains reliable under distribution shift, such as motion fields, spatial affordance fields, transferable contact points, and key interaction poses (Ma et al., 13 May 2026, Xu et al., 8 Dec 2025, Wu et al., 2024, Nasiriany et al., 2024, Ju et al., 2024).

Line of work	Affordance object	Robustness notion
Multi-task LLM planning	State–option pair $(s,o)$	Intent holds with probability at least $1-\delta$ over $T \sim \mathrm{Dist}(\mathcal{T})$
Distributional DP	Tuple of Bellman AVaR values for $(a,o,e)$0	Worst-/best-case value over augmented kernels $(a,o,e)$1
Few-shot manipulation	Object-centric motion field or action field	Online recovery from OOD states into demonstrated manifold
Transferable manipulation priors	3D contact points, trajectories, key poses	Generalization to unseen instances and categories
Bayesian perception	Probability maps or class distributions	Robustness through aleatoric and epistemic uncertainty estimation
Rough-set logic	Ternary relation $(a,o,e)$2 and $(a,o,e)$3	Necessity/possibility under observational granularity

A recurrent theme is that robustness is obtained not by making a single end-to-end predictor invariant to every shift, but by introducing intermediate structure. Depending on the formulation, that structure may be a task-conditioned intent, an object-relative vector field, a pair of coherent risk measures, a dense uncertainty map, a transferable contact prior, or an approximation operator over indiscernibility classes.

2. Probabilistic and decision-theoretic formulations

In the multi-task partial-world-model framework, tasks are MDPs

$(a,o,e)$4

drawn from a task distribution $(a,o,e)$5, while abstract actions $(a,o,e)$6 are associated with temporally extended intents $(a,o,e)$7 (Khetarpal et al., 11 Feb 2026). A $(a,o,e)$8-affordance set $(a,o,e)$9 is a subset of state–option pairs for which intended and realized trajectory distributions differ by at most $o$0. Definition 4 then introduces distribution-robust affordances by requiring that, for every $o$1,

$o$2

The central theorem states that if a deterministic agent is $o$3-optimal, then its policy encodes a partial world model $o$4 on the affordance set, with worst-case error bounded by

$o$5

where

$o$6

This is a notion of robustness over task families rather than a minimax formulation over adversarial model perturbations (Khetarpal et al., 11 Feb 2026).

A different formalization appears in distributional dynamic programming. There, an action’s distribution-robust affordance is derived from a two-atom approximation of the return distribution,

$o$7

where the two atoms are left and right Bellman AVaR values. The resulting fixed point satisfies

$o$8

and admits a robust MDP interpretation over an augmented state space with worst-case substates $o$9 and best-case substates $\mathsf{Act}_\varphi$0 (Achab et al., 2021). For $\mathsf{Act}_\varphi$1-coherent policies,

$\mathsf{Act}_\varphi$2

so each action carries an explicit pessimistic and optimistic value under a structured, non-rectangular uncertainty set. In balanced MDPs, this yields safe and risky tie-breaking operators among expectation-optimal actions, so the same expected return can correspond to distinct robust affordances (Achab et al., 2021).

These two lines of work are often conflated, but they are technically distinct. The task-distribution formulation uses probabilistic guarantees over sampled tasks, whereas the Bellman AVaR formulation uses worst-case and best-case values in an augmented robust MDP. This suggests that “distribution-robust affordance” is best understood as a family resemblance concept rather than a single mathematical definition.

3. Object-centric recovery fields for manipulation under distribution shift

In few-demonstration manipulation, distribution-robust affordances are instantiated as an object-centric motion field plus a local egocentric execution policy. “Sliding into Distribution” separates manipulation into approach/alignment and execution, learns an object-centric motion field from the approach portions of one or two demonstrations, and uses that field to iteratively slide the current state back toward the demonstrated manifold before handing control to a local policy (Ma et al., 13 May 2026). The canonicalization operator is

$\mathsf{Act}_\varphi$3

and in the reported instantiation the object-centric state is the target-object pose in the wrist-camera frame,

$\mathsf{Act}_\varphi$4

The field is trained to imitate a target sliding step

$\mathsf{Act}_\varphi$5

where $\mathsf{Act}_\varphi$6 is the nearest demonstrated state. Far from the demonstrated manifold, the magnitude is large; near convergence it vanishes. The resulting behavior is an online distribution recovery process in object pose space (Ma et al., 13 May 2026).

Once the field norm is small, SID uses an egocentric policy trained with conditional flow matching over action chunks, together with kinematically consistent point-cloud reprojection augmentation and an auxiliary ID-confidence head. The confidence head can re-engage the motion field when observations leave the policy’s support, yielding a closed-loop arbitration between global alignment and local skill execution (Ma et al., 13 May 2026). Across six real-world tasks, SID achieves approximately $\mathsf{Act}_\varphi$7 success under OOD initializations with only two demonstrations, with under a $\mathsf{Act}_\varphi$8 drop under distractors and external disturbances. Reported OOD success rates for the closed-loop variant include $\mathsf{Act}_\varphi$9 on Open Drawer, $a$0 on Pour Water, $a$1 on Hang Tape, $a$2 on Hang Cup, $a$3 on PnP-Box, and $a$4 on Multi-PnP-Box; without egocentric augmentation, average success drops from about $a$5 to about $a$6 (Ma et al., 13 May 2026).

A related but architecturally different use of affordance fields appears in “Affordance Field Intervention.” Here the issue is the “Memory Trap,” in which a VLA reproduces memorized trajectories under OOD changes instead of adapting to the updated scene (Xu et al., 8 Dec 2025). AFI constructs a 3D Spatial Affordance Field $a$7 over a voxelized workspace by combining a target-guidance field and an obstacle-avoidance field,

$a$8

with the target derived from GPT-4o stage decomposition and Grounded-SAM segmentation. A memory trap is detected when the end-effector is quasi-static but still far from the target centroid. AFI then rolls back to the recent historical pose with lowest SAF cost, samples SAF-guided waypoints, asks the VLA for multiple trajectory proposals, and selects the one with minimum cumulative affordance cost

$a$9

The reported gains are an average improvement of $e$0 across $e$1 and $e$2 under OOD scenarios on real-world robotic platforms, and $e$3 on LIBERO-Pro; on Stack Tape, an ensemble of $e$4 and $e$5 with AFI reaches $e$6 success (Xu et al., 8 Dec 2025).

Both SID and AFI make the same structural move: they externalize an affordance-like spatial guidance layer that is recomputed from current geometry rather than relying exclusively on end-to-end imitation. In SID the affordance is an object-centric action field over $e$7; in AFI it is a task-conditioned 3D cost field over the workspace. In both cases, robustness comes from explicit online realignment.

4. Transferable affordances across novel instances and categories

A second major strand studies affordances as transferable intermediate representations rather than online recovery fields. In RT-Affordance, affordances are the robot end-effector poses at key stages of a task, extracted from trajectories at gripper open/close transitions and the final timestep (Nasiriany et al., 2024). The model is hierarchical: an affordance generator predicts an affordance plan $e$8 from language and the initial image,

$e$9

and an affordance-conditioned policy executes actions conditioned on language, current observation, and the plan,

$(s,o)$0

At execution time, the plan is visually overlaid onto the image. This representation is intended to balance under-specified language and over-specified goal images. On novel grasping tasks, RT-2 achieves $(s,o)$1 average success, goal-image-conditioned GC-RT-2 achieves $(s,o)$2, RT-Affordance with oracle affordances achieves $(s,o)$3, and RT-Affordance with predicted affordances achieves $(s,o)$4; on placement and articulated tasks, RT-2 achieves $(s,o)$5, while both oracle and predicted affordance variants achieve $(s,o)$6 (Nasiriany et al., 2024). The affordance predictor also shows robustness to several OOD factors: $(s,o)$7 in-distribution affordance success, about $(s,o)$8 under novel camera view, about $(s,o)$9 under novel background, and a larger but still graceful drop for novel objects (Nasiriany et al., 2024).

AffordDP makes this transfer explicitly geometric. It represents affordances as a static contact point and a dynamic post-contact trajectory,

$(s,o)$0

stores them in an affordance memory $(s,o)$1, retrieves a source affordance via CLIP similarity, maps the contact point with SD-DINOv2 semantic correspondence, and transfers the trajectory by an $(s,o)$2 transform based on Point-SAM part segmentation and ICP (Wu et al., 2024). The dynamic affordance transfer is

$(s,o)$3

AffordDP also adds affordance guidance during diffusion sampling with an adaptive contact loss

$(s,o)$4

In simulation, unified-policy AffordDP reaches $(s,o)$5 on seen PullDrawer instances, $(s,o)$6 on unseen instances, and $(s,o)$7 on unseen categories, whereas DP3 reaches $(s,o)$8, $(s,o)$9, and $1-\delta$0, respectively; in real-world OpenDoor, AffordDP reaches $1-\delta$1 on seen instances and $1-\delta$2 on both unseen instances and unseen categories, while DP and DP3 remain far lower (Wu et al., 2024).

Robo-ABC treats affordances as human contact points extracted from egocentric videos and stored in an affordance memory indexed by CLIP features (Ju et al., 2024). For a target object crop $1-\delta$3, the system retrieves visually or semantically similar memory objects and uses DIFT diffusion features for pixel-level semantic correspondence from source contact point $1-\delta$4 to target point $1-\delta$5. The resulting 2D point is projected into 3D and matched to an AnyGrasp candidate by

$1-\delta$6

On affordance prediction, Robo-ABC obtains SR $1-\delta$7, NSS $1-\delta$8, and DTM $1-\delta$9, compared with $T \sim \mathrm{Dist}(\mathcal{T})$0, $T \sim \mathrm{Dist}(\mathcal{T})$1, and $T \sim \mathrm{Dist}(\mathcal{T})$2 for the strongest listed baseline, LOCATE; on real-robot cross-category object grasping, Robo-ABC achieves $T \sim \mathrm{Dist}(\mathcal{T})$3 success, compared with $T \sim \mathrm{Dist}(\mathcal{T})$4 for VRB (Ju et al., 2024).

These methods differ in representation—key end-effector poses, 3D contact-and-trajectory pairs, or retrieved human contact points—but share the same operational principle: they extract a compact interaction prior that can be transferred across new objects more reliably than a policy conditioned only on raw observations.

5. Uncertainty-aware affordance perception and recursive estimation

Distribution-robust affordances also appear at the perception layer, where the objective is not yet action selection but calibrated estimation of where interaction is possible. In Bayesian RGB affordance prediction, a multiscale local–global CNN predicts object-level affordance labels for actions such as sit, run, and grasp, while representing epistemic and aleatoric uncertainty via Monte Carlo dropout or deep ensembles (Mur-Labadia et al., 2021). The predictive covariance is decomposed as

$T \sim \mathrm{Dist}(\mathcal{T})$5

The paper reports that deep ensembles are marginally better than MC-dropout on the Brier score and the Expected Calibration Error, and explicitly notes that epistemic uncertainty appears in samples out of the distribution, such as objects that appear more rarely in the dataset (Mur-Labadia et al., 2021). This formulation does not provide a robust control law by itself, but it supplies calibrated uncertainty signals that can be used for abstention, active learning, or cautious downstream planning.

The same logic is pushed to the spatial level in Bayesian affordance segmentation. There, Mask R-CNN is converted into a Bayesian model with MC-dropout in the encoder and heads, and each affordance instance is represented by a predictive mean mask together with per-pixel aleatoric and epistemic variance maps (Mur-Labadia et al., 2023). The spatial variance decomposition is

$T \sim \mathrm{Dist}(\mathcal{T})$6

The authors introduce the Probability-based Mask Quality measure and report $T \sim \mathrm{Dist}(\mathcal{T})$7 for Bayesian Mask R-CNN ResNeXt-101 (MC Enc-FC), compared with $T \sim \mathrm{Dist}(\mathcal{T})$8 for the previous deterministic best on IIT-AFF. Aleatoric variance concentrates on contours due to camera noise, while epistemic variance appears in visually challenging pixels such as occlusions, light artefacts, and ambiguous regions (Mur-Labadia et al., 2023). The robust-affordance interpretation here is local and probabilistic: a region is affordant only insofar as its predicted affordance probability remains high while epistemic uncertainty remains low.

A more explicitly deployment-oriented estimator is the coupled-particle-filter method for robust affordance estimation. It factorizes affordances into graspability and movability, represents each as a recursive particle-filter belief over space, and couples the filters by cross-modal density so that regions supported by both survive resampling (Lowin et al., 16 Mar 2026). The coupling term is

$T \sim \mathrm{Dist}(\mathcal{T})$9

and the final particle weight combines modality-specific and cross-modal support. Evaluated on the RBO dataset, the method outperforms Where2Act, Hands-as-Probes, and HRP by $(a,o,e)$00, $(a,o,e)$01, and $(a,o,e)$02 in precision, and reaches a $(a,o,e)$03 real-world success rate; in cluttered tabletop scenes it achieves $(a,o,e)$04 success, and in cluttered IKEA shelf scenes $(a,o,e)$05, while the listed baselines remain at or below $(a,o,e)$06 (Lowin et al., 16 Mar 2026). Here robustness comes not from a single stronger predictor, but from coupling complementary estimators with different error modes.

Across these perception-centric works, robustness is operationalized through calibrated uncertainty, temporal belief maintenance, or estimator coupling. A plausible implication is that perception-level distribution robustness is most effective when uncertainty is structured enough to be acted upon, rather than merely reported.

6. Rough, modal, and methodological perspectives

The logical treatment of affordances offers a non-probabilistic but rigorous account of robustness under changing information. Given Pawlak information systems for actors, objects, and environments, each with its own indiscernibility relation, a rough affordance is defined as a pair

$(a,o,e)$07

where

$(a,o,e)$08

and

$(a,o,e)$09

The lower approximation captures affordances that hold for all indiscernible refinements of the observed actor–object–environment triple; the upper approximation captures affordances that hold for at least one such refinement (Gruszczynski et al., 4 Dec 2025). Modal operators such as

$(a,o,e)$10

and

$(a,o,e)$11

then support possibility- and sufficiency-style reasoning about environments, with parallel operators for actors and objects (Gruszczynski et al., 4 Dec 2025). This framework does not provide statistical distribution robustness, but it formalizes a closely related notion: stability of affordances under observational coarsening.

One common misconception is that all work on distribution-robust affordances belongs to classic distributionally robust optimization. The literature does not support that reading. The multi-task LLM framework explicitly states that it does not formulate classic DRO; instead it defines robustness probabilistically over a task distribution (Khetarpal et al., 11 Feb 2026). The Bellman AVaR framework is a robust MDP construction with a specific non-rectangular uncertainty set in an augmented state space (Achab et al., 2021). SID and AFI manage OOD states by online geometric recovery rather than by minimax training (Ma et al., 13 May 2026, Xu et al., 8 Dec 2025). Bayesian affordance perception focuses on calibrated uncertainty and OOD indicators, not on robust control objectives (Mur-Labadia et al., 2021, Mur-Labadia et al., 2023). The logical account of affordances is qualitative and approximation-theoretic, not probabilistic (Gruszczynski et al., 4 Dec 2025).

The limitations reported across the literature are correspondingly diverse. SID’s capture region is limited to the sampled pose domain and depends on reliable pose estimation; transparent or reflective objects and heavy motion can degrade canonicalization (Ma et al., 13 May 2026). The LLM partial-model framework offers no guarantee under adversarial or strongly shifted task distributions (Khetarpal et al., 11 Feb 2026). RT-Affordance improves “how” to perform known interaction types but does not generalize to completely novel skills (Nasiriany et al., 2024). AffordDP depends on the quality of foundation-model correspondence and ICP-based part registration, both of which can fail under ambiguity or poor geometry (Wu et al., 2024). Coupled particle filters cannot resolve shared systematic biases when both base estimators are wrong in the same region (Lowin et al., 16 Mar 2026). Bayesian segmentation and classification provide uncertainty-aware perceptual substrates, but do not themselves define robust downstream policies (Mur-Labadia et al., 2021, Mur-Labadia et al., 2023). The logic of affordances leaves open a complete proof theory and any extension to explicitly probabilistic information systems (Gruszczynski et al., 4 Dec 2025).

Taken together, these works define distribution-robust affordances not as a single algorithmic recipe but as a design principle: action possibilities should be represented in a form that remains meaningful when the state distribution shifts. Depending on the application, the appropriate robust object may be an intent-satisfying option, a risk-aware action value, an object-centric correction field, a transferable 3D interaction prior, a calibrated uncertainty map, or a rough modal relation over actor, object, and environment.