Motion Transferability Framework Overview

• Motion Transferability Framework is a set of formal methods that evaluate if motion models, representations, or perturbations remain effective when transferred across different environments or modalities.
• It spans diverse applications including shared-space simulation, trajectory prediction, action recognition, and teleoperation, using metrics like ADE, KL divergence, and accuracy drops.
• Studies indicate that targeted adaptation—focusing on motion cues and structural compatibility—can mitigate performance degradation from environmental, cultural, or contextual shifts.

Searching arXiv for the cited motion-transferability papers to ground the article in the current record. Motion Transferability Framework denotes a family of formal methods for determining whether a motion model, motion representation, or motion-sensitive perturbation learned in a source setting remains effective in a target setting with different cultural norms, traffic conditions, spatial layout, embodiment, dataset composition, or model architecture. In shared-space simulation, transferability is defined as “The ability of a shared-space motion model, originally calibrated for one environment (source), to reproduce realistic agent behaviour in a different environment (target)—differing in cultural norms, traffic conditions or spatial layout—with no or only minimal adaptation” (Johora et al., 2022). In trajectory prediction, the same idea is operationalized through a directional score $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$ computed from latent scene embeddings (Westny et al., 29 Jun 2026), whereas in action recognition it is quantified by the degradation from KnownContext to UnknownContext on coarse and fine motion classes (Abdullah et al., 31 Jul 2025).

1. Scope and principal formulations

Across the literature, motion transferability is not restricted to one task class. It appears in shared-space motion simulation, trajectory prediction, action recognition, adversarial example generation, teleoperation, and learning from demonstration. What transfers depends on the problem: continuous interaction dynamics, latent scene statistics, high-level motion concepts, perturbation patterns, or cross-embodiment mappings. The evaluation criteria therefore differ across domains, but they are uniformly framed as source-to-target generalization under shift rather than in-domain accuracy alone (Johora et al., 2022).

Domain	Transfer object	Main criteria
Shared-space motion modeling	Source-calibrated GSFM applied to a new environment	$\mathrm{ADE}$, $\Delta v$, $\epsilon_{\rm dec}$
Video action recognition	Coarse and fine motion classes across disjoint contexts	$D_{\mathrm{abs}}$, $D_{\mathrm{rel}}$, $\mathrm{HM}$
Trajectory prediction	Dataset-to-dataset transfer via latent Gaussian similarity	$\tau_{s\to t}=-D_{KL}(G_t\|G_s)$
Teleoperation	Human/robot pair compatibility and chain mapping	$T_{A\to B}$
Adversarial transfer	Cross-model transfer success or fooling rate	Transfer success rate, fooling rate

A central distinction in this literature is between direct behavioral transfer and metric-based prediction of transfer. The former asks whether a model trained on source data still reproduces target behavior, as in the enhanced GSFM case study (Johora et al., 2022). The latter estimates transferability before retraining or adaptation, as in latent scene embeddings for trajectory datasets (Westny et al., 29 Jun 2026) or the pre-training compatibility metric for human-to-humanoid chain mapping (Stanley et al., 2022).

2. Shared-space motion models and norm-aware adaptation

The most explicit formal transferability criterion is given for shared-space motion models in "On Intercultural Transferability and Calibration of Heterogeneous Shared Space Motion Models" (Johora et al., 2022). A model $M_S$ calibrated on source data $\mathrm{ADE}$0 is considered transferable to target data $\mathrm{ADE}$1 if, without any or with only nominal adjustment, it achieves error measures on $\mathrm{ADE}$2 comparable to its source performance. With $\mathrm{ADE}$3, the criterion is

$\mathrm{ADE}$4

where $\mathrm{ADE}$5 is a small tolerance determined a priori, for example $\mathrm{ADE}$6 m.

The underlying model is an enhanced Game-Theoretic Social Force Model (GSFM) with three parameter classes: social-force parameters $\mathrm{ADE}$7, safety-distance parameters $\mathrm{ADE}$8, and game-theoretic payoff weights $\mathrm{ADE}$9 (Johora et al., 2022). Calibration proceeds in two phases. First, SFM and safety parameters are optimized with a Genetic Algorithm whose chromosome is one candidate set of the 12 SFM+safety parameters and whose fitness is the mean Euclidean position discrepancy over training scenarios. Second, payoff weights are calibrated by feature extraction, backward elimination with a multinomial logit model retaining only features with $\Delta v$0-values $\Delta v$1, payoff-matrix definition for a two-player Stackelberg game, and a second Genetic Algorithm whose fitness rewards agreement between real and SPNE-computed strategies.

The motion equations combine continuous-force dynamics and game-triggered interaction. For pedestrians,

$\Delta v$2

where pairwise repulsion takes the form

$\Delta v$3

Game interaction is formulated as a Stackelberg leader–follower problem solved by subgame-perfect Nash equilibrium (Johora et al., 2022).

The German-to-China case study makes the transfer problem concrete. The source environment used the HBS dataset with 1 115 pedestrians, 331 cars, $\Delta v$4 s, and 104 interactions split into calibration and validation. Calibration on Germany produced $\Delta v$5 m, $\Delta v$6 m, $\Delta v$7 m/s, and $\Delta v$8 m/s, with decision-accuracy of 90.27%/88.37% for cars and 80.71%/84.48% for pedestrians on train/test. Direct transfer to the Chinese DUT dataset degraded to $\Delta v$9 m, $\epsilon_{\rm dec}$0 m/s, $\epsilon_{\rm dec}$1 m, and $\epsilon_{\rm dec}$2 m/s. After integrating social norms extracted from target video clips, pedestrian ADE improved to 3.69 m and car ADE to 4.77 m, corresponding to $\epsilon_{\rm dec}$3 and $\epsilon_{\rm dec}$4 respectively (Johora et al., 2022).

The recommended workflow for a new target environment is explicit: data collection and annotation; source-model baseline; gap analysis; norm extraction; norm integration; optional re-calibration of payoff weights only; and validation. Best practices include keeping the GSFM architecture modular, using a minimal set of norm-features, leveraging backward-elimination to keep the payoff model parsimonious, and always reporting pre-/post-transfer metrics on a held-out subset (Johora et al., 2022).

3. Dataset-level transferability in trajectory prediction

In trajectory prediction, the transferability problem is formulated at the dataset level in "Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddings" (Westny et al., 29 Jun 2026). Each multi-agent driving scene, consisting of past trajectories and an optional HD map, is encoded into a low-dimensional code $\epsilon_{\rm dec}$5 by an encoder built from an inter-agent Graph-GRU and a lane-graph map encoder. The model is trained jointly with a reconstruction head and a forecasting head under

$\epsilon_{\rm dec}$6

Scene-level codes are aggregated into dataset-level Gaussians $\epsilon_{\rm dec}$7. Transferability from source $\epsilon_{\rm dec}$8 to target $\epsilon_{\rm dec}$9 is then defined directionally as

$D_{\mathrm{abs}}$0

with $D_{\mathrm{abs}}$1 compared against Maximum Mean Discrepancy and the 2-Wasserstein distance. On 24 datasets and 552 zero-shot train-test pairs using a modified QCNet predictor, Spearman’s rank correlation between $D_{\mathrm{abs}}$2 and zero-shot minADE$D_{\mathrm{abs}}$3 at 3 s was $D_{\mathrm{abs}}$4 with $D_{\mathrm{abs}}$5 confidence interval $D_{\mathrm{abs}}$6. The corresponding values for MMD, Wasserstein, and a simple $D_{\mathrm{abs}}$7 centroid distance were approximately $D_{\mathrm{abs}}$8, $D_{\mathrm{abs}}$9, and $D_{\mathrm{rel}}$0 (Westny et al., 29 Jun 2026).

The same framework was also used for source selection. When a single source was chosen for each target by minimizing $D_{\mathrm{rel}}$1, the mean rank of the selected source among 23 candidates was 3.00, the top-3 rate was 83.3%, the absolute minADE gap to the oracle best source was 0.100, and the relative gap was 21.0%. Closest speed, largest dataset, and random selection all performed substantially worse (Westny et al., 29 Jun 2026).

A complementary trajectory-prediction study, "Transfer Learning Study of Motion Transformer-based Trajectory Predictions" (Ullrich et al., 2024), analyzes adaptation after pretraining rather than transferability estimation before training. Using a Motion Transformer pretrained on WOMD and transferred to CarMaker simulation, it compares multi-task learning, feature reuse, and fine-tuning. Full fine-tuning achieved the best target performance, with CMD mAP 0.6611, target minADE 0.6508, target minFDE 1.2165, and target MissRate 0.0782, but reduced source WOMD mAP to 0.1774. Encoder-only fine-tuning gave CMD mAP 0.4968 with 0.73 d of additional CMD-only training, compared with 0.96 d for full fine-tuning. Feature reuse required only 0.33 d of extra training but reached CMD mAP 0.3785, while multi-task learning failed to improve over source-only transfer on the target domain (Ullrich et al., 2024).

Taken together, these studies separate two questions that are often conflated: which source is intrinsically compatible with a target dataset, and which adaptation strategy is computationally acceptable once transfer is attempted. The former is answered through latent distributional similarity; the latter through explicit performance-versus-training-time trade-offs (Westny et al., 29 Jun 2026).

4. Motion concepts and context shifts in video understanding

"Punching Bag vs. Punching Person: Motion Transferability in Videos" formalizes motion transferability as recognition of the same coarse motion under disjoint fine contexts (Abdullah et al., 31 Jul 2025). Two sets, $D_{\mathrm{rel}}$2 and $D_{\mathrm{rel}}$3, share coarse classes but have disjoint fine classes. This yields CoarseMotion-KC and CoarseMotion-UC for known and unknown context, and FineMotion-KC and FineMotion-UC for multimodal models. The metrics are absolute drop,

$D_{\mathrm{rel}}$4

relative drop, and harmonic mean,

$D_{\mathrm{rel}}$5

The benchmark consists of three datasets. Syn-TA is synthetic, built from Blender-rendered 3D object motions with 20 coarse activities and 100 fine classes. K400-TA adapts 205 Kinetics-400 fine classes into 41 coarse motions. SSv2-TA reorganizes Something-Something-v2 into 26 coarse and 149 fine classes. All are preprocessed by frame sampling at 8–16 frames per clip, resizing to $D_{\mathrm{rel}}$6, ImageNet normalization, and standard augmentations (Abdullah et al., 31 Jul 2025).

The quantitative results establish strong context dependence. Averaged across 13 models, coarse-motion performance dropped from 59.3% to 36.0% on Syn-TA, from 82.0% to 53.2% on K400-TA, and from 53.3% to 31.3% on SSv2-TA, with harmonic means of 50.7, 64.6, and 42.4 respectively. Fine-motion transfer, averaged over the 5 multimodal models, was worse: Syn-TA fell from 84.6% to 31.8%, K400-TA from 86.9% to 52.5%, and SSv2-TA from 62.0% to 26.6%, yielding harmonic means of 37.4, 64.6, and 40.9 (Abdullah et al., 31 Jul 2025).

The analysis isolates several failure modes. All models suffer large drops from KnownContext to UnknownContext, typically with $D_{\mathrm{rel}}$7 points. Multimodal models struggle more with fine-grained unknown actions than with coarse ones. The bias-free Syn-TA proves as challenging as real-world datasets, and in some settings more challenging. Larger backbones improve transferability when spatial cues dominate but struggle with intensive temporal reasoning, while reliance on object and background cues hinders generalization. Replacing Syn-TA scenes with plain backgrounds improves harmonic mean by 8–12 points, and in K400-TA the text encoder in EZ-CLIP can exploit novel object tokens such as “goat” rather than motion structure (Abdullah et al., 31 Jul 2025).

The same work also evaluates a disentanglement strategy. A dual-branch extension of EZ-CLIP jointly learns coarse and fine embeddings, with residual fusion of fine-branch features into the coarse branch, yielding $D_{\mathrm{rel}}$8–$D_{\mathrm{rel}}$9 points HM on Syn-TA and SSv2-TA, including an increase from 67.9 to 72.0 on Syn-TA (Abdullah et al., 31 Jul 2025).

5. Adversarial transferability and motion-aware attacks

A distinct branch of the literature studies transferability of adversarial perturbations against motion-sensitive recognition models. "Boosting Adversarial Transferability using Dynamic Cues" introduces temporal prompts into a frozen Vision Transformer so that a source image model can capture video dynamics without full architectural conversion (Naseer et al., 2023). The forward pass augments standard spatial tokens with a temporal class token and a learnable transformation $\mathrm{HM}$0 over frame-wise patch tokens. The adversarial objective maximizes

$\mathrm{HM}$1

combining supervised video-head loss and a self-supervised feature-consistency term on the spatial head.

The transfer effect is substantial. Under single-step FGSM at $\mathrm{HM}$2, a baseline DeiT-T surrogate transferred to TimesFormer on UCF101 with accuracy dropping from 90.6% to 75.1%; after adding temporal prompts and attacking both heads, the drop became 90.6% to 64.5%. For image-to-image transfer on ImageNet, adding scale-space prompts at $\mathrm{HM}$3, $\mathrm{HM}$4, $\mathrm{HM}$5, and $\mathrm{HM}$6 raised the DIM-attack fooling rate of DeiT-B on a 5k-sample subset to 86.6%. Increasing temporal tokens from 8 to 16 to 32 further improved transfer, reducing TimesFormer accuracy on UCF from 55.5% to 21.5% (Naseer et al., 2023).

"Boosting Adversarial Transferability for Skeleton-based Action Recognition via Exploring the Model Posterior Space" addresses skeleton-based HAR, where the difficulty is attributed to a rugged and sharp loss landscape (Diao et al., 2024). The proposed PDBA attack keeps the pretrained surrogate fixed, appends a tiny Bayesian MLP $\mathrm{HM}$7, and rewrites the attack objective in Bayesian form by averaging predictions over an approximate posterior $\mathrm{HM}$8. A second Bayesian layer explores Gaussian neighborhoods $\mathrm{HM}$9 around posterior samples to approximate worst-case directions. Motion dynamics are enforced by fitting first-order and second-order time-varying autoregressive models to the clean skeleton sequence and combining position, velocity, and acceleration gradients in the attack update,

$\tau_{s\to t}=-D_{KL}(G_t\|G_s)$0

with $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$1 used in practice (Diao et al., 2024).

On HDM05, NTU-60, and NTU-120, the average untargeted transfer success rates reached 35.9%, 45.5%, and 35.7%, compared with MI-FGSM at 3.3%, 6.0%, and 10.4%, SMART at 2.9%, 5.5%, and 9.3%, and SMI at 3.8%, 4.8%, and 6.1%. The paper further reports targeted transfer on NTU-60 of about 9% for PDBA versus about 6% for SMART and about 5% for MI-FGSM, as well as strong transfer against TRADES and BEAT defenses (Diao et al., 2024).

These adversarial frameworks redefine motion transferability as cross-model generalization of perturbations rather than cross-domain preservation of nominal behavior. Yet their central mechanism remains the same: transfer improves when the source representation captures temporal structure rather than only static appearance (Naseer et al., 2023).

6. Cross-embodiment mapping, transfer chains, and reproducibility

In teleoperation, "Transferability-based Chain Motion Mapping from Humans to Humanoids for Teleoperation" introduces Synergy Mapping via Dual-Autoencoder (SyDa) and a transferability metric for selecting mapping chains between agents (Stanley et al., 2022). Two parallel autoencoders encode agents $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$2 and $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$3 into a shared latent dimension $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$4, called the common synergy space, and are trained with reconstruction losses plus the alignment term

$\tau_{s\to t}=-D_{KL}(G_t\|G_s)$5

The transferability metric combines length ratio, manipulability mismatch, and sensor noise: $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$6

Transfer chains are selected by maximizing the product of edge transferabilities, equivalently minimizing the sum of edge costs $\tau_{s\to t}=-D_{KL}(G_t\|G_s)$7 along a path. In experiments with H1, H2, and a Pepper robot R1, pairwise SyDa mapping errors were 0.0691 m for H2$\tau_{s\to t}=-D_{KL}(G_t\|G_s)$8R1, 0.0718 m for H1$\tau_{s\to t}=-D_{KL}(G_t\|G_s)$9R1, 0.1368 m for R1$T_{A\to B}$0H1, and 0.1595 m for R1$T_{A\to B}$1H2, with corresponding transferability scores 0.2358, 0.2335, 0.2002, and 0.1991. SyDa gave approximately $T_{A\to B}$2 better bidirectional accuracy than direct models. For two-hop chains, H2$T_{A\to B}$3H1$T_{A\to B}$4R1 achieved 0.0720 m error and $T_{A\to B}$5, outperforming H1$T_{A\to B}$6H2$T_{A\to B}$7R1 at 0.1137 m and 0.1071 (Stanley et al., 2022).

A related but stricter notion of transferability appears in "Guided Learning from Demonstration for Robust Transferability" (Sukkar et al., 2023). Here the goal is to determine the largest region

$T_{A\to B}$8

such that demonstrations within $T_{A\to B}$9 are reproducible on the target robot, in the sense that the distortion between task-space motion and configuration-space motion is bounded by $M_S$0. The reproducible region is extracted by the Hausdorff Approximation Planner on a discretized grid in $M_S$1, and then exposed to the demonstrator through a real-time GUI that renders forbidden voxels in red. The framework guarantees bounded-length lifted trajectories: $M_S$2

The empirical results combine robot validation and a user study. In a Sawyer-to-UR5 setup, unguided demonstrations that traversed forbidden voxels led to large wrist-angle swings and collisions, whereas guided demonstrations yielded smooth, collision-free UR5 motions. In a study with 9 users demonstrating a mock weld task with a VICON-tracked tool, unguided users produced a reproducible demonstration only 44% of the time within three tries, while guided users achieved 100% success, all within two tries. Subjective measures showed confidence increasing by 30% and surprise at reproduction decreasing by 40%, with ease of use approximately unchanged (Sukkar et al., 2023).

7. Recurrent findings, misconceptions, and research directions

Several recurrent empirical findings emerge across these frameworks. First, source performance is not a reliable proxy for target performance. In the shared-space case, a model calibrated on Germany degraded sharply when moved to China until target social norms were added (Johora et al., 2022). In video understanding, all models suffered large KnownContext-to-UnknownContext drops, and fine-grained unknown actions were harder than coarse unknown actions (Abdullah et al., 31 Jul 2025). In Motion Transformer transfer, full fine-tuning improved CMD performance but catastrophically forgot WOMD (Ullrich et al., 2024). In trajectory prediction, simple heuristics such as speed similarity or source dataset size were inferior to latent $M_S$3 for predicting cross-dataset transfer (Westny et al., 29 Jun 2026).

Second, transferability is repeatedly improved by restricting adaptation to the parts of the system that encode the relevant structure of the shift. In GSFM, social-norm adjustments are localized to payoff-feature mappings, view range, minimum distance, and specific interaction components (Johora et al., 2022). In MTR transfer, encoder-only fine-tuning is a practical compromise between compute and target performance (Ullrich et al., 2024). In adversarial transfer, temporal prompts or motion-aware Bayesian smoothing improve black-box success without retraining the full model (Naseer et al., 2023, Diao et al., 2024). In video understanding, disentangling coarse and fine embeddings improves transfer on temporally challenging datasets (Abdullah et al., 31 Jul 2025).

A common misconception is that larger models or richer modalities automatically imply better transfer. The motion-transferability benchmark reports that larger models improve transferability when spatial cues dominate but struggle with intensive temporal reasoning, and that multimodal models can overuse object tokens or background cues (Abdullah et al., 31 Jul 2025). The adversarial literature reaches an analogous conclusion from the opposite direction: transfer improves not from scale alone but from smoothing the loss landscape and injecting dynamic cues into the surrogate (Diao et al., 2024, Naseer et al., 2023).

This suggests that motion transferability is best understood as a structural compatibility problem. A plausible implication is that the relevant structure depends on the task: cultural yielding norms in mixed traffic, scene-distribution geometry in trajectory prediction, temporal ordering in video understanding, posterior flatness in adversarial transfer, or kinematic and workspace compatibility in cross-embodiment mapping. The surveyed frameworks therefore do not converge on a single universal metric, but they do converge on a shared methodological pattern: define the source-target discrepancy in motion-relevant terms, measure it explicitly, and adapt only the components required to preserve the motion semantics under shift.