Adversarial Interaction Priors in Control

• Adversarial Interaction Priors (AIP) are adversarially learned or chosen regularizers that enforce valid geometric or reactive interaction patterns in control tasks.
• AIP leverage techniques such as discriminator training on latent interaction representations or state transitions to guide policies toward expert-like behaviors.
• Empirical studies show that incorporating AIP significantly improves task success rates and generalization across varied object scales and multi-agent scenarios.

Searching arXiv for the cited papers and closely related work on Adversarial Interaction Priors. Adversarial Interaction Priors (AIP) are adversarially learned or adversarially chosen priors that regularize interaction behavior rather than only endpoint task achievement. In the most explicit formulation, AIP are “an adversarially trained prior over interaction geometry, not over full-body motion,” and they are used to reward policies whose interaction latents remain within the manifold of “valid geometric interactions” under reinforcement learning (Lin et al., 25 Feb 2026). Closely related work learns discriminator-based priors over multi-agent reaction transitions for physics-based characters (Younes et al., 2023), while a minimax-Bayes line treats the prior itself as an adversarial distribution over teammate scenarios in ad hoc teamwork (Villin et al., 4 Feb 2025). Taken together, these formulations suggest that AIP is not a single fixed architecture but a research direction in which interaction structure is represented, regularized, and optimized adversarially.

1. Conceptual definition and scope

In current arXiv usage, AIP is most clearly defined in embodied-control settings where a policy must interact with objects or other agents without relying on fixed reference motions. In "LessMimic," the central claim is that absolute joint configurations vary with object geometry, whereas the local geometric relationship between robot and object surface—captured by an interaction latent $z_t$—can remain consistent across geometries and therefore serve as a transferable supervision signal (Lin et al., 25 Feb 2026). The prior is adversarial because a discriminator is trained to distinguish expert-like interaction latents from policy-generated latents, and its output is turned into a reward during RL.

A related but distinct formulation appears in "MAAIP: Multi-Agent Adversarial Interaction Priors for imitation from fighting demonstrations for physics-based characters," where an interaction prior is a discriminator trained on multi-actor interaction clips, while a separate motion prior is trained on single-actor motion clips (Younes et al., 2023). Here, the interaction prior measures whether a character’s behavior is interaction-consistent relative to the interaction dataset. The paper explicitly frames this as an extension of AMP and MAGAIL to multi-agent fighting interactions with physics-based characters.

These definitions share a common separation between motion style and interaction style. In LessMimic, AIP regularize “interaction validity in the geometric domain,” whereas AMP regularizes motion naturalness (Lin et al., 25 Feb 2026). In MAAIP, the dual-prior structure makes the same separation operational: the single motion prior captures individual motor skills, and the interaction prior captures reactive interaction patterns between agents (Younes et al., 2023). This suggests that AIP are best understood as priors over contact geometry, reactive state transitions, or partner-conditioned behavior, rather than as generic motion priors.

2. Formal mechanisms

Across the literature, AIP are instantiated through three main mathematical mechanisms: a discriminator on interaction latents, a discriminator on reactive multi-agent transitions, or an adversarial distribution over teammate scenarios. The unifying role is to bias learning toward interaction-consistent regions of behavior space rather than to prescribe a single trajectory.

Formulation	Interaction representation	Adversarial object
LessMimic (Lin et al., 25 Feb 2026)	DF-based latent $z_t$ from $\Phi(\mathbf{x}_t)$, $\nabla\Phi(\mathbf{x}_t)$, $\mathbf{v}_t^{\text{norm}}$, $\mathbf{v}_t^{\text{tan}}$	LSGAN discriminator over reference and policy latents
MAAIP (Younes et al., 2023)	Reactive transition $(o_t,\;o_{t+1}^{\text{self}})$	Per-agent GAIL-style discriminator with gradient penalty
Ad hoc teamwork (Villin et al., 4 Feb 2025)	Scenario prior $\beta \in \Delta(\Sigma(B))$	Adversarial prior over teammate scenarios

In LessMimic, the discriminator is trained with the least-squares GAN objective

$$\mathcal{L}_D = \mathbb{E}_{z \sim \mathcal{B}_{\text{ref}}}\left[(D(z)-1)^2\right] + \mathbb{E}_{z \sim \pi}\left[(D(z)+1)^2\right],$$

and its output defines the interaction prior reward

$$r_{\text{interact}}(z_t)=\max\left(0,\,1-0.25(D(z_t)-1)^2\right).$$

The composite RL reward is

$z_t$0

The AIP term is therefore a learned regularization term in PPO that biases the policy toward expert-like regions of interaction latent space (Lin et al., 25 Feb 2026).

In MAAIP, the interaction prior for agent $z_t$1 is a discriminator $z_t$2 over reactive transitions, with reward

$z_t$3

The total imitation reward is

$z_t$4

where $z_t$5 is the single motion prior reward and $z_t$6 is optional task-specific control. The discriminator objectives are standard GAIL losses on state transitions with gradient penalties, rather than LSGAN (Younes et al., 2023).

In the ad hoc teamwork formulation, the adversarial object is not a discriminator but a prior over scenarios. For utility,

$z_t$7

and for regret,

$z_t$8

Here $z_t$9 is an adversarial interaction prior over teammate configurations. The resulting saddle-point policy optimizes worst-case utility or worst-case regret over the scenario set $\Phi(\mathbf{x}_t)$0 (Villin et al., 4 Feb 2025).

3. Distance-field AIP in long-horizon humanoid interaction

The most explicit AIP implementation currently described on arXiv is LessMimic. Its interaction representation is distance-field based. For each important humanoid link at time $\Phi(\mathbf{x}_t)$1, the method computes the DF distance $\Phi(\mathbf{x}_t)$2, DF gradient $\Phi(\mathbf{x}_t)$3, and a decomposition of link velocity into normal and tangential components,

$\Phi(\mathbf{x}_t)$4

The per-timestep feature is

$\Phi(\mathbf{x}_t)$5

and a history window

$\Phi(\mathbf{x}_t)$6

is encoded by a VAE into a latent $\Phi(\mathbf{x}_t)$7 (Lin et al., 25 Feb 2026).

The policy conditions on this latent, while the AIP discriminator also receives $\Phi(\mathbf{x}_t)$8. This architectural choice is central: AIP operate only on $\Phi(\mathbf{x}_t)$9, not on full state, actions, raw joint configurations, or raw contact variables. The paper stresses that AIP therefore constrain geometric interaction statistics—distance evolution, gradient patterns, and normal-versus-tangential velocity patterns—rather than directly constraining joint angles. This is what permits new joint configurations for new geometries while preserving interaction validity.

AIP are used only in the discriminative post-training stage. The training pipeline is: interaction skill pre-training with BC plus DAgger from a ResMimic teacher, discriminative post-training with RL plus AIP, and visual-motor distillation to a depth-based policy (Lin et al., 25 Feb 2026). In post-training, Table A.2 assigns the “Interaction Style” reward a weight of 2.0, larger than the “Motion Style” weight of 1.0. The implementation details list $\nabla\Phi(\mathbf{x}_t)$0, $\nabla\Phi(\mathbf{x}_t)$1, $\nabla\Phi(\mathbf{x}_t)$2, $\nabla\Phi(\mathbf{x}_t)$3, and $\nabla\Phi(\mathbf{x}_t)$4 environment steps.

Empirically, AIP are the main mechanism behind scale generalization and long-horizon viability in LessMimic. A single policy achieves $\nabla\Phi(\mathbf{x}_t)$5 success across object scales from $\nabla\Phi(\mathbf{x}_t)$6 to $\nabla\Phi(\mathbf{x}_t)$7 on PickUp and SitStand, attains $\nabla\Phi(\mathbf{x}_t)$8 success on 5 task instances trajectories, and remains viable up to 40 sequentially composed tasks (Lin et al., 25 Feb 2026). The ablation “Ours - AIP” collapses PickUp at small scales, reduces Carry from $\nabla\Phi(\mathbf{x}_t)$9 to $\mathbf{v}_t^{\text{norm}}$0, and drops long-horizon success at $\mathbf{v}_t^{\text{norm}}$1 from $\mathbf{v}_t^{\text{norm}}$2 to $\mathbf{v}_t^{\text{norm}}$3. These numbers localize AIP’s contribution more sharply than qualitative claims alone.

4. Multi-agent interaction priors for physics-based characters

MAAIP places AIP in a multi-agent imitation-learning setting. The method assumes two unstructured datasets: a single-actor dataset $\mathbf{v}_t^{\text{norm}}$4 and a multiple-actors interaction dataset $\mathbf{v}_t^{\text{norm}}$5 (Younes et al., 2023). From $\mathbf{v}_t^{\text{norm}}$6, the system learns a single motion prior $\mathbf{v}_t^{\text{norm}}$7 on observation transitions

$\mathbf{v}_t^{\text{norm}}$8

while from $\mathbf{v}_t^{\text{norm}}$9, it learns one interaction discriminator per agent,

$\mathbf{v}_t^{\text{tan}}$0

where $\mathbf{v}_t^{\text{tan}}$1. The interaction prior therefore models how an agent reacts to the opponent’s current state.

The policy architecture uses parameter sharing for homogeneous agents,

$\mathbf{v}_t^{\text{tan}}$2

and training is done with MAPPO under CTDE, with a centralized value function and discriminator-based rewards (Younes et al., 2023). Observations are explicitly engineered: each agent observes self and opponent features in $\mathbf{v}_t^{\text{tan}}$3, and actions are target joint angles for a PD controller with total dimension 28.

The interaction prior is not a latent-variable model. The paper states that no explicit latent variables or sequence-level probabilistic models are used; the priors are purely discriminative functions on $\mathbf{v}_t^{\text{tan}}$4 pairs. This is an important methodological contrast with LessMimic, where the prior acts on a VAE-encoded DF latent. It suggests that AIP can be either latent-space or transition-space objects, depending on how interaction structure is represented.

The empirical domain is fighting: boxing and QwanKiDo. With pure imitation, the reported behaviors include approaching an opponent while guarding, evading and blocking attacks, moving around an opponent while maintaining safe distance, and attacking when opportunities arise (Younes et al., 2023). In controlled tasks, the method adds rewards for damage minimization, damage maximization, or heading control. The balance between motion prior and interaction prior is critical: the paper reports that too strong $\mathbf{v}_t^{\text{tan}}$5 can make agents ignore interaction, whereas too strong $\mathbf{v}_t^{\text{tan}}$6 can make agents overuse interaction-type motions even when inappropriate. It also reports that an LSGAN-style objective yielded worse interactive behavior and more mode collapse than the standard cross-entropy GAIL loss plus gradient penalty in this multi-agent setting.

5. Distributional AIP and relation to adversarial motion priors

A broader interpretation of AIP appears in the minimax-Bayes approach to ad hoc teamwork. Here, uncertainty about partners is encoded by a prior $\mathbf{v}_t^{\text{tan}}$7 over scenarios $\mathbf{v}_t^{\text{tan}}$8, where each scenario specifies how many focal players there are and which background policies fill the remaining slots (Villin et al., 4 Feb 2025). The adversary then chooses the prior itself to minimize utility or maximize regret. This converts the training distribution over teammates into an optimized object rather than a fixed design choice.

The theoretical consequences are explicit. The maximin solution satisfies optimal worst-case utility over training scenarios, and the minimax-regret solution satisfies optimal worst-case regret. The paper also proves continuity bounds over $\mathbf{v}_t^{\text{tan}}$9-nets of scenario sets, yielding out-of-distribution guarantees of the form

$(o_t,\;o_{t+1}^{\text{self}})$0

and

$(o_t,\;o_{t+1}^{\text{self}})$1

for utility and regret respectively (Villin et al., 4 Feb 2025). This extends AIP from learned discriminators to adversarial distributions over partner behavior.

The immediate conceptual precursor is AMP. The robotics AMP literature uses a discriminator-based style reward learned from motion data, then combines it with task reward in RL (Escontrela et al., 2022). Aerial-humanoid locomotion work likewise uses AMP as a learned prior over motion snippets, with a discriminator $(o_t,\;o_{t+1}^{\text{self}})$2 that distinguishes dataset transitions from policy-generated transitions, and a style reward

$(o_t,\;o_{t+1}^{\text{self}})$3

that is added to the task reward (L'Erario et al., 2023). Both papers explicitly describe AMP as a data-driven substitute for complex reward engineering. LessMimic makes the connection explicit by stating that AIP play the same conceptual role as AMP but in distance-field interaction latent space (Lin et al., 25 Feb 2026). MAAIP operationalizes the same distinction by pairing a motion prior with an interaction prior (Younes et al., 2023). The literature therefore treats AIP less as a rejection of AMP than as a specialization from motion naturalness to interaction validity.

6. Misconceptions, limitations, and terminological ambiguity

A recurrent misconception is that AIP prescribe exact trajectories. The most explicit AIP paper argues the opposite: reference motions are no longer valid under procedural geometry randomization, and AIP are needed precisely because they provide a reference-free, task-agnostic signal about whether an interaction is geometrically plausible (Lin et al., 25 Feb 2026). Another misconception is that AIP must be generative latent-variable models. MAAIP shows that an interaction prior can be entirely discriminative and operate on state transitions with no explicit latent variables (Younes et al., 2023).

The main limitations currently reported are support limitations, modality limitations, and optimization limitations. LessMimic’s AIP depend on the quality of the reference buffer $(o_t,\;o_{t+1}^{\text{self}})$4, are designed for rigid objects, and are not yet replaced by a “visual AIP” trained directly from depth at distillation time (Lin et al., 25 Feb 2026). MAAIP notes mode collapse, sparse interaction data, limited generalization to unseen interaction modes, and the fact that current priors operate on single-step transitions rather than long-horizon strategy (Younes et al., 2023). The minimax-Bayes approach identifies conservatism as a real failure mode: in small games, a worst-case prior can collapse onto defecting partners and thereby suppress practice of beneficial cooperative behaviors; it also requires either full-information computation or expensive estimation of $(o_t,\;o_{t+1}^{\text{self}})$5 for regret minimization (Villin et al., 4 Feb 2025).

A separate source of confusion is the word “adversarial.” "Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions" studies targeted perturbations that alter predicted reactions in skeleton-based interaction models, using temporal smoothness and perceptual equivalence constraints, but it is an attack on interaction prediction rather than a control prior used to regularize RL or imitation learning (Koren et al., 2021). The acronym is also overloaded outside this literature: "AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Prompt" uses AIP to mean “Adversarial Instructional Prompt” (Chaturvedi et al., 18 Sep 2025), and "AIP: Agent Identity Protocol for Verifiable Delegation Across MCP and A2A" uses AIP to mean “Agent Identity Protocol” (Prakash, 25 Mar 2026). In contemporary embodied-control and multi-agent imitation literature, however, AIP refers specifically to adversarial priors over interaction geometry, reactive transitions, or teammate configurations.