Affordance Transfer Learning (ATL)

• Affordance Transfer Learning (ATL) is a framework that uses intermediate, action-relevant representations to generalize across views, modalities, and tasks.
• ATL methods extract key features like end-effector poses or latent equivalence classes to enable cross-view, cross-embodiment, and multi-modal transfer without dense supervision.
• By integrating heterogeneous data sources, ATL reduces reliance on task-specific demonstrations and improves generalization and performance in robotics applications.

Affordance Transfer Learning (ATL) denotes a family of methods that treat affordances as transferable structure for perception, reasoning, and control. In this literature, an affordance is not merely an object label or a generic feature descriptor; it is an action-relevant representation that can encode where interaction should occur, how an effect can be produced, or which part of an object supports a task. ATL uses that representation to move knowledge across views, modalities, datasets, object instances, embodiments, or tasks, thereby reducing dependence on dense robot demonstrations or task-specific supervision. Recent work spans robot manipulation, weakly supervised affordance grounding, human-object interaction detection, substitute-tool reasoning, and task-oriented handover, with affordances instantiated as key poses, contact regions, latent equivalence classes, dense correspondence maps, or task-specific metric spaces (Nasiriany et al., 2024).

1. Conceptual scope and defining properties

ATL is unified by a recurrent design choice: the transferable unit is neither a full policy nor a raw observation, but an intermediate representation that is sufficiently structured to capture action possibilities and sufficiently compact to generalize. RT-Affordance states this explicitly by proposing affordances as an intermediate policy representation that is “more structured than language alone” but “less brittle / high-dimensional than full goal images or trajectories,” with the specific representation being an affordance plan composed of key robot end-effector poses (Nasiriany et al., 2024). In this sense, ATL is a representational strategy for decomposing task understanding, spatial planning, and execution.

The same principle appears in visual grounding. “Grounded Affordance from Exocentric View” formulates cross-view affordance grounding as learning affordance knowledge from exocentric human-object interactions and transferring it to egocentric object views using only affordance labels during training, explicitly naming the problem “cross-view affordance knowledge transfer” (Luo et al., 2022). LOCATE and LoopTrans sharpen this formulation by transferring object-part knowledge from interaction images to inactive-object images under weak supervision, while AffCorrs shows that even a single annotated affordance region can be transferred by correspondence rather than task-specific retraining (Li et al., 2023, Tang et al., 20 Oct 2025, Hadjivelichkov et al., 2022).

A second defining property is that ATL is often object-centric but not object-identity-bound. “Cross-Embodied Affordance Transfer through Learning Affordance Equivalences” formalizes affordances as tuples such as $(effect,(object,action))$ and $(effect,(agent,(object,action)))$, then defines object equivalence, action equivalence, and agent equivalence as shared affordance relations in a common latent space (Aktas et al., 2024). The implication is that transfer should track functional equivalence, not superficial similarity. A closely related idea appears in HOI detection, where HOIs are factorized into affordance/verb and object components so that novel interactions can be composed by combining known affordances with novel objects (Hou et al., 2021).

A third property is heterogeneity of supervision. ATL methods routinely combine robot trajectories, human videos, web data, static annotated images, or pretrained visual features. RT-Affordance bridges robot trajectories, large web datasets, and cheap in-domain affordance images; RAM builds an affordance memory from robotic data, human-object interaction data, and custom data; AFT-Handover transfers a proxy exemplar’s annotated affordance field to a novel object-task pair through LLM reasoning and point-cloud correspondence (Nasiriany et al., 2024, Kuang et al., 2024, Tulbure et al., 5 Feb 2026).

2. Representations and formal formulations

The most explicit ATL formalism in manipulation is RT-Affordance. The robot trajectory dataset is written as

$$\mathcal{D} = \{l, \{(o_i, e_i, g_i, a_i)\}_{i=0}^T\},$$

where $l$ is language, $o_i$ image observations, $e_i$ end-effector pose, $g_i$ gripper state, and $a_i$ action. The affordance plan is

$q = (e_{t_1}, e_{t_2}, ..., e_{t_n}),$

a sequence of key end-effector poses selected at critical timesteps such as gripper open-to-close, close-to-open, or final-timestep events. The downstream policy is conditioned on affordances,

$\pi(a \mid l, o, q),$

or, after visual rendering of the affordance onto the image,

$(effect,(agent,(object,action)))$0

To remove the need for human specification at test time, RT-Affordance learns an affordance predictor

$(effect,(agent,(object,action)))$1

This is a canonical ATL decomposition: infer a structured affordance plan, then execute a policy conditioned on that plan (Nasiriany et al., 2024).

Cross-embodied ATL uses a different abstraction. In Affordance Blending Networks, action trajectories, effect trajectories, and object observations are encoded into latent vectors and blended into a shared affordance latent:

$(effect,(agent,(object,action)))$2

with convex-combination constraints on the coefficients. This latent affordance space is meant to represent an effect-object-action relation regardless of whether the evidence came from one object or another, one robot or another, or one action variant or another. Transfer then operates through “Affordance Equivalence,” meaning that latent clusters should correspond to shared functional relations rather than individual exemplars (Aktas et al., 2024).

Visual grounding papers formalize ATL at the level of regions and correspondences. “Grounded Affordance from Exocentric View” uses exocentric features $(effect,(agent,(object,action)))$3 and egocentric features $(effect,(agent,(object,action)))$4, then combines NMF-based affordance invariance mining, dense cross-view feature transfer, and affordance co-relation preserving losses:

$(effect,(agent,(object,action)))$5

with $(effect,(agent,(object,action)))$6, $(effect,(agent,(object,action)))$7, and $(effect,(agent,(object,action)))$8 (Luo et al., 2022). LOCATE similarly converts exocentric interaction embeddings into compact prototypes and aligns a selected object-part prototype with egocentric pooled features via a cosine embedding loss, while LoopTrans closes the loop by refining exocentric activation from egocentric localization through denoising distillation (Li et al., 2023, Tang et al., 20 Oct 2025).

A separate line treats affordances as task-specific similarity structure. The regularized metric-learning approach learns, for each affordance, a transform $(effect,(agent,(object,action)))$9 such that positive examples cluster and negatives separate:

$$\mathcal{D} = \{l, \{(o_i, e_i, g_i, a_i)\}_{i=0}^T\},$$0

Here, the learned transform is itself interpreted as an affordance abstraction, and transfer is mediated by invariant sensory-grounded features rather than symbolic labels (Hjelm et al., 2019).

3. Transfer regimes and major methodological families

A useful way to organize ATL is by the axis along which knowledge is transferred.

Transfer regime	Representative papers	Transfer object
Cross-modality / heterogeneous supervision	RT-Affordance, RAM, AFT-Handover	Affordance plans, 2D/3D affordances, affordance textures
Cross-view grounding	Grounded Affordance from Exocentric View, LOCATE, LoopTrans, AffCorrs	Affordance regions or object-part correspondences
Cross-embodiment / cross-instance transfer	Affordance Equivalence, SemFM	Shared latent affordance space, dense semantic correspondence
Experience- or task-function transfer	SAGAT, tool-affordance transfer, invariant metric learning	Ranked grasp transforms, p-tool task score, task-specific metric

In manipulation, one major family uses affordances as execution interfaces. RT-Affordance is hierarchical: an affordance generator predicts an affordance plan from language and the initial scene, then an affordance-conditioned policy executes actions conditioned on language plus the plan (Nasiriany et al., 2024). RAM is retrieval-based rather than predictor-based: it builds a unified affordance memory from robotic, HOI, and custom data, retrieves a source demonstration hierarchically given a language instruction and target observation, transfers 2D affordance by dense correspondence, and lifts it into executable 3D affordance $$\mathcal{D} = \{l, \{(o_i, e_i, g_i, a_i)\}_{i=0}^T\},$$1 (Kuang et al., 2024). GAT-Grasp replaces language with human gesture as the query modality: a pointing gesture localizes the target region, a grasp gesture retrieves similar human grasp interactions from a memory bank, and dense correspondence transfers the contact point while a hand-to-gripper mapping transfers orientation (Wang et al., 8 Mar 2025).

Another family centers on exocentric-to-egocentric transfer. “Grounded Affordance from Exocentric View” addresses interaction bias both between regions and between views through AIM, CFT, and ACP (Luo et al., 2022). LOCATE makes the transfer more part-centric by first localizing interaction areas, clustering them into human/object-part/background prototypes, selecting the object-part prototype with PartSelect, and using that prototype to guide egocentric grounding (Li et al., 2023). LoopTrans argues that one-way exocentric-to-egocentric transfer is insufficient in complex scenarios and introduces a closed-loop in which shared activation, pixel-level localization, and denoising distillation refine each other bidirectionally (Tang et al., 20 Oct 2025). AffCorrs is structurally different: it is unsupervised and one-shot, using pre-trained DINO-ViT descriptors and cycle-consistent correspondences to transfer an annotated affordance part from a support image to a target image (Hadjivelichkov et al., 2022).

Cross-instance and cross-embodiment ATL emphasize functional equivalence. Affordance Blending Networks create a shared affordance representation spanning multiple agents and objects, enabling both action generation from effect trajectories and direct imitation across robots (Aktas et al., 2024). SemFM, by contrast, is geometric-semantic: it identifies semantically corresponding functional regions across meshes, selects mutually exclusive semantic anchors, and propagates those constraints with a functional map to obtain dense correspondence, after which a source affordance region is mapped to the target as $$\mathcal{D} = \{l, \{(o_i, e_i, g_i, a_i)\}_{i=0}^T\},$$2 (Dong et al., 16 Feb 2026).

Older ATL formulations already contained the same core idea under different machinery. The substitute-tool framework with 3D vision data learns task-specific geometric priors from simulated web models, encodes tools as a 25-dimensional p-tool representation, and transfers those priors to novel objects by task-driven projection of grasp region, action region, and orientation (Abelha et al., 2017). SAGAT stores prior task-affordance experiences in a library and transfers grasp selection to new objects by forward simulating task outcomes from candidate grasps and ranking them with a KL-divergence-based confidence measure (Ardón et al., 2020).

4. Applications and empirical evidence

The strongest empirical case for ATL in robot manipulation comes from RT-Affordance. On a benchmark of novel household objects unseen in the robot trajectory datasets, the paper reports average success of 28% for RT-2, 24% for GC-RT-2, 76% for RT-A (Oracle Aff), and 68% for RT-A (Ours). On additional tasks including placing fruit into receptacles, closing articulated objects, and turning a sink faucet, the language-conditioned baseline reaches only 3%, while RT-A achieves 70%. The affordance prediction model also shows graceful degradation under novel object instances, novel camera views, and novel backgrounds, with OOD performance staying within about 10% of in-distribution performance overall (Nasiriany et al., 2024).

Cross-view grounding papers provide another consistent empirical pattern: localized affordance transfer improves both seen and unseen generalization. On AGD20K, LOCATE reports Seen performance of KLD 1.226, SIM 0.401, NSS 1.177 and Unseen performance of KLD 1.405, SIM 0.372, NSS 1.157, outperforming Cross-view-AG+ and an AffCorrs adaptation; the gains are especially marked on the unseen split (Li et al., 2023). LoopTrans extends this trend on AGD20K, HICO-IFF, EPIC-Kitchens, and OPRA: on AGD20K-Seen it reports KLD 1.088, SIM 0.445, NSS 1.322, and on AGD20K-Unseen KLD 1.247, SIM 0.403, NSS 1.315, with reported average improvements of about 6.7% over WSMA on AGD20K, 10.5% over WSMA on HICO-IFF, and about 11.3% over LOCATE on image benchmarks (Tang et al., 20 Oct 2025). The cross-view grounding model of 2022 likewise reports best performance on KLD, SIM, and NSS relative to saliency methods, WSOL methods, Hotspots, and Cross-view-AG, including stronger Unseen performance and cleaner heatmaps in cluttered, multi-object, and long-tail settings (Luo et al., 2022).

Zero-shot retrieval-based ATL also has strong application-level results. RAM reports an average success rate of 52.62% in IsaacGym evaluations over 13 tasks and more than 70 objects in 10 categories, outperforming Where2Act, VRB, and a modified Robo-ABC baseline (Kuang et al., 2024). GAT-Grasp, evaluated in cluttered real-world scenes, reports average success rates of 40.56% for GPT-4o, 32.22% for Qwen-VL, 34.44% for Robo-ABC, 40.00% for RAM, and 51.67% for GAT-Grasp, with ablations showing large drops when pointing gesture, affordance transfer, or rotation mapping are removed (Wang et al., 8 Mar 2025). In task-oriented handover, AFT-Handover reports an average successful robot grasp rate of 83% with GPT-4o and 86% with GPT-5, compared with about 62% for OS-TOG; substituting ground-truth affordances yields 95%, treated as an upper bound. In a user study, 71.43% of participants preferred AFT-Handover, with significantly fewer regrasps before tool use (Tulbure et al., 5 Feb 2026).

Cross-instance and cross-embodiment ATL is also empirically credible, though its evaluation is more heterogeneous. Affordance Equivalence shows latent clustering by functional class in insertion, graspability, and rollability experiments in CoppeliaSim/V-REP, including direct imitation on a UR-10 with a Robotiq 3-finger gripper and cross-embodiment transfer across UR-10, Baxter, and KUKA LBR4+ settings (Aktas et al., 2024). SemFM reports affordance-transfer IoU substantially above classical FM-WKS baselines and competitive with multi-view GPT-4o prompting, while running in about 8.69–11.45 s per transfer versus about 31.2–53.6 s for the VLM baselines (Dong et al., 16 Feb 2026). Earlier experience-based ATL also shows measurable gains: SAGAT improves stand-alone grasp-affordance detectors from 79.2% average success to 85.4%, with the largest task-specific gain being 11.67% for pouring (Ardón et al., 2020).

5. Recurrent challenges, misconceptions, and limitations

A common misconception is that ATL is reducible to object recognition or language conditioning. The literature argues otherwise. RT-Affordance explicitly places affordances between underspecified language and over-specified goal images: “pick the dustpan” does not specify where to grasp, whereas a full goal image may be brittle or hard to learn from (Nasiriany et al., 2024). The HOI literature reaches a similar conclusion from a different angle: the transferable component is an affordance/verb representation that can be recombined with object representations, not a monolithic HOI label (Hou et al., 2021).

A second misconception is that any correspondence suffices for transfer. Multiple papers show that transfer quality is sharply constrained by semantic or geometric alignment. AFT-Handover reports that cross-class transfer works best when source and target share functional similarity: hammer-to-screwdriver is close to the class-matched baseline, whereas mug-to-knife is much worse (Tulbure et al., 5 Feb 2026). RAM finds geometric retrieval essential, with the largest ablation drop occurring when viewpoint alignment is removed (Kuang et al., 2024). SemFM argues that pure geometry is insufficient because manipulation-relevant regions are often small, localized, and semantically defined, so semantic anchors are needed before dense propagation (Dong et al., 16 Feb 2026).

A third challenge is noise and occlusion in source observations. Cross-view grounding papers repeatedly identify the human body, background clutter, scale variation, and viewpoint change as major obstacles. “Grounded Affordance from Exocentric View” names this interaction bias and addresses it by separating affordance-specific features from person-specific habits and by preserving affordance co-relations across views (Luo et al., 2022). LoopTrans makes the more forceful claim that one-way transfer is structurally inadequate when interaction regions are fully occluded by the human body, motivating its closed-loop refinement (Tang et al., 20 Oct 2025).

A fourth limitation is over-transfer. Cross-embodied affordance transfer depends on the new situation truly sharing the same affordance. The rollability experiments explicitly note that generalizing a learned push action to other directions can become incorrect when the new object’s behavior depends on direction (Aktas et al., 2024). Retrieval-based systems exhibit the same failure mode when retrieval is semantically plausible but functionally mismatched. This suggests that ATL is strongest when the latent or retrieved affordance class is tightly tied to the downstream effect.

A fifth limitation is infrastructural rather than conceptual. Many methods depend on strong pretrained modules or auxiliary estimators: DINO-ViT and DINOv2 features, CLIP, SigLip2Vision, SAM3D, LangSAM, hand-pose estimators, grasp planners, or point-cloud correspondence networks (Li et al., 2023, Wang et al., 8 Mar 2025, Dong et al., 16 Feb 2026, Tulbure et al., 5 Feb 2026). This does not negate transfer, but it does mean that ATL performance is often coupled to the robustness of upstream perception and alignment.

6. Trajectory of the field and relation to neighboring topics

The development of ATL can be read as a progression from task-specific geometric transfer to scalable multimodal representations. The 2017 substitute-tool framework already framed transfer as task-driven projection of learned geometric priors onto novel objects, using 3D point clouds, superquadrics, and simulated web models (Abelha et al., 2017). The 2019 metric-learning work shifted emphasis toward sensory-grounded invariants and interpretable task-specific transforms, while the 2020 SAGAT pipeline introduced experience-based one-shot transfer of task-aware grasps (Hjelm et al., 2019, Ardón et al., 2020). By 2021, ATL in HOI detection treated affordances as compositional factors for zero-shot interaction synthesis (Hou et al., 2021). The 2022–2023 grounding papers then established cross-view weakly supervised affordance transfer as a substantial subfield, culminating in part-centric and closed-loop transfer strategies (Luo et al., 2022, Hadjivelichkov et al., 2022, Li et al., 2023, Tang et al., 20 Oct 2025). Recent robotics work extends ATL to internet-scale or proxy-based knowledge sources, zero-shot embodiment-agnostic execution, and direct human-robot interaction (Kuang et al., 2024, Nasiriany et al., 2024, Wang et al., 8 Mar 2025, Tulbure et al., 5 Feb 2026, Dong et al., 16 Feb 2026).

This trajectory suggests a broad convergence. Across manipulation, grounding, and HOI, affordances increasingly function as a shared intermediate language between perception and action. In some systems that language is explicit and geometric, as with affordance plans, p-tools, or semantic anchors; in others it is latent, as with affordance equivalence spaces; in still others it is retrieved from memory or transferred as a dense field. A plausible implication is that ATL’s lasting contribution is methodological rather than task-specific: it provides a principled way to couple rich external supervision to downstream embodied behavior without requiring end-to-end robot data for every new object, view, or embodiment.

The acronym itself is not unique. Outside affordance research, ATL also names “appliance transfer learning” in non-intrusive load monitoring and “autonomous transfer learning” for many streaming processes (DIncecco et al., 2019, Pratama et al., 2019). Within robotics and embodied vision, however, Affordance Transfer Learning now denotes a distinct and increasingly coherent research program centered on transferable action-relevant representations.