InterActor: Modular AI Interaction

• InterActor is a modular component that mediates interactions in AI systems by enabling cross-context attention, iterative feedback, and structured data fusion.
• It underpins diverse architectures such as trajectory prediction, text-to-motion synthesis, and ad generation by integrating multiple data sources.
• Empirical results demonstrate its impact through improved prediction accuracy, motion realism, and GUI control performance across various domains.

An InterActor, sometimes stylized as Interactor or InterActor module, refers to a class of model components, architectural interfaces, or agentic blocks that mediate and coordinate interaction within complex information-processing, generative, or agentic AI systems. While the term appears across multiple domains and is instantiated with domain-specific mechanisms, common features include cross-contextual attention or coordination, iterative or agentic feedback, and the fusion or transfer of structured information between decoupled system modules or agents. Recent literature presents specialized InterActor architectures for trajectory prediction, multi-agent RL, text-to-interaction generation, programmatic logical inference, ad description synthesis, and instruction-conditioned multimodal reasoning, each with rigorously specified objectives, dataflows, and optimization regimes.

1. Semantics and Formal Interfaces

The InterActor abstraction first arose in the logic programming community as a generalization of logic engines in continuation-passing Prolog systems, notably BinProlog, Jinni Prolog, and Lean Prolog (Tarau, 2011). Here, an InterActor (or Interactor) is defined as a reentrant, first-class Prolog engine—a logic interpreter with a well-defined API permitting instantiation, suspension, resumption, and bidirectional exchange of arbitrary data and continuations between a client and an engine. The API includes operations for engine creation (new_engine), answer retrieval (get), resource cleanup (stop), yielding intermediate results (return), and message exchange (to_engine, from_engine). At the formal level, the InterActor operates as a labeled transition system in which state transitions preserve Prolog's semantics while enabling mixed-initiative and coroutine-like control structures.

More broadly, modern InterActor modules are architecturally specified neural blocks or agents mediating between representations, learning objectives, or modalities—for example, as cross-attentional fusion modules, agentic RL agents, or token-compressing interaction subnets. These instantiations retain the original spirit of an interface that enables controlled, interpretable, and often bidirectional coordination among subsystems.

2. Architectural Realizations and Domain-Specific Designs

Trajectory Prediction: Cross-Context Attention

In vehicular trajectory prediction, as exemplified by GC-GAT (Gulzar et al., 15 Apr 2025), the InterActor module is a deeply structured neural block positioned between a context encoder and a multimodal trajectory decoder. Its principal role is to fuse static scene features (lane-graph embeddings), dynamic context (neighbor agent embeddings), the target agent's embedding, and learned goal proposals into K mode-specific fused context vectors via multi-head cross-attention. The module proceeds in three sub-steps: (a) target-to-lane and target-to-neighbor cross-attention, (b) goal proposal encoding with Gumbel-Softmax selection and lane aggregation, and (c) final multimodal fusion through trainable mode embeddings and an array of cross-attention heads over context, lane, neighbor, and goal features.

Text-to-Interaction Diffusion

In text-to-two-person motion synthesis (Wu et al., 7 Oct 2025), the InterActor is realized as an alternating stack of word-level conditioning blocks (motion tokens cross-attend to word/token embeddings) and motion–motion interaction blocks (intra- and inter-agent attention), permitting both fine-grained semantic alignment with prompt tokens and spatiotemporal synchrony between generated motion streams. Adaptive interaction loss is introduced to enforce contextually relevant inter-person joint coupling, maximizing both fidelity and realism in generated human–human motions.

Agentic RL and Iterative Creation

In agentic RL settings for ad description synthesis (Wei et al., 14 Jun 2026), an Interactor is instantiated as a generative policy capable of multi-turn iterative interaction with a complex environment. At each turn, the policy produces either "think" (internal reasoning), "retrieve" (external web content), or "create" (description generation) actions, receiving structured scalar and textual reward signals from generative reward models (GenRMs) evaluating knowledge capacity, landing-page consistency, and attractiveness (CTR prediction). The iterative process is formalized as an MDP where the environment produces reward vectors and rationales at each interaction, enabling gradient-based policy optimization with RL objectives and generative feedback.

RL-Guided Low-Level GUI Control

In the staged workflow of SWIRL (Lu et al., 27 Aug 2025), the Interactor is a vision-language transformer agent responsible for grounding high-level plans into executable GUI actions based on current screen state, instruction tokens, and action history. Training proceeds via staged, interleaved policy optimization in a decoupled multi-agent setting, ensuring monotonic improvement and convergent behavior via a formalized clipped-surrogate RL objective.

Instruction-Guided Multimodal Reasoning

In the context of autonomous driving MLLMs (Zhai et al., 2024), the instruction-guided InterActor is a plug-and-play module for compressing, aligning, and fusing multi-camera and BEV visual tokens with language instructions before LLM decoding. The module selects instruction-relevant tokens using cosine similarity, performs token-wise cross-attention for context preservation, and delivers compact, semantically-aligned representations to downstream LLMs, facilitating world knowledge-enhanced perception and reasoning.

3. Mathematical Formulations, Algorithms, and Hyperparameters

In each engineering instantiation, the InterActor module is specified with precise mathematical operations, typically involving cross-attention, token selection, and agentic policy gradients.

• Cross-Attention (GC-GAT, Text2Interact):

$$\text{Attention}(Q,K,V) = \mathrm{softmax}\left( \frac{QK^\top}{\sqrt{d_k}} \right) V$$

Queries, keys, and values are constructed from embeddings derived from scene context, tokenized instructions, or candidate interactions.

• Goal Proposal with Gumbel-Softmax (GC-GAT):

$$g_\text{soft}^{(i,k)} = \frac{\exp((g_\text{enc}^{(i,k)} + G_{ik})/\tau)}{\sum_{j=1}^K \exp((g_\text{enc}^{(i,j)} + G_{ij})/\tau)}$$

$$h_\text{goal} = \sum_{i=1}^V \sum_{k=1}^K g_\text{soft}^{(i,k)} h_\text{lane}^{(i)}$$

• Adaptive Interaction Loss (Text2Interact):

$$L_\text{AdaInteract} = \sum_{i=1}^N \sum_{j=1}^N \frac{1}{d_{ij} + \epsilon} \| d_{ij} - \hat{d}_{ij} \|_2$$

This loss emphasizes accurate modeling of close joint pairs between two interacting agents.

• RL Optimization Objectives (SWIRL, Ad Description Interactor):

Policy and value updates employ clipped-surrogate or group-normalized policy objectives, with regularization via KL penalty and batchwise or rollout-based advantage normalization. Rewards in the ad description Interactor combine binary and continuous dimensions with tuned weights: $$R_t = w_{KC}r_t^{KC} + w_{LP}r_t^{LP} + w_{CTR}r_t^{CTR}$$, with $w_{KC}=1, w_{LP}=2, w_{CTR}=5$ (Wei et al., 14 Jun 2026).

• Token-Compression and Selection (Instruction-Guided InterActor):

Visual tokens are pruned via top-$k$ instruction similarity, with $k$ tuned in ablation (e.g., $k=90$ per view optimal).

4. Practical Applications and Empirical Results

InterActor modules are integrated at the core of SOTA systems across multiple domains:

• Vehicular Trajectory Prediction: In GC-GAT, the InterActor enables robust fusion of heterogeneous context, yielding high MinADE and low MissRate on nuScenes (Gulzar et al., 15 Apr 2025).
• Two-Person Motion Generation: InterActor boosts text-alignment and physical plausibility, achieving superior R-Precision and user study gains over previous text-to-motion systems (Wu et al., 7 Oct 2025).
• GUI Control Agents: SWIRL's Interactor achieves ~85% low-level action accuracy across GUI benchmarks, with a nearly 10 point jump in step success rate (SR) due to interleaved agent training (Lu et al., 27 Aug 2025).
• Autonomous Driving Reasoning: Instruction-Guided InterActor achieves up to +9 BLEU4 and +100 CIDEr on Object-Level Risk Assessment, with reliable performance across planning and VQA tasks (Zhai et al., 2024).
• Ad Description Generation: The Interactor framework provides +0.715 informativeness and +0.872 faithfulness on test data, significantly outperforming both single-turn and other RL baselines; +0.74% ad revenue uplift and +7pp human-rated quality in live deployment (Wei et al., 14 Jun 2026).
• Programmatic Logic Engines: Interactors in Prolog support efficient source-level emulation of classic built-ins, with per-engine overhead of a few microseconds and near-zero amortized per-answer cost in Lean Prolog (Tarau, 2011).

A summary of representative empirical metrics is presented below:

Domain	Metric (SOTA interactor)	Ablation/Comparison
Vehicular Prediction	MinADE/MissRate: best on nuScenes	-
Text2-Motion (InterActor)	R-P@1: 0.483, FID/MMDist: SOTA	InterMask: R-P@1=0.449
GUI Control (SWIRL)	SR: 78.81 (AndroidControl-Low)	non-stage2: 68.87
Autonomous Driving (ORA)	BLEU4: 49.08, CIDEr: 344.55, mAP: 15.68	no-InterActor BLEU4: 45.86, mAP: 0
Ad Description (Interactor)	Info.: 0.715, Faith.: 0.872, Attract.:12.4%	w/o reasoning Info: 0.636, Faith:0.706

5. Training Methodologies and Optimization Regimes

InterActor modules employ large-scale, domain-aligned training protocols:

• End-to-end Differentiable Learning: InterActor parameters are optimized jointly with encoders and decoders using Adam or AdamW, minimum trajectory error regression losses, or group-normalized RL objectives.
• Iterative and Staged Learning: In SWIRL and ad generation settings, training is staged, with either alternation (single-agent RL updates per agent with others frozen) or agent–environment feedback loops guided by reward models, human or synthetic evaluators, or external retrieval sources (Lu et al., 27 Aug 2025, Wei et al., 14 Jun 2026).
• Specialized Data Curation: Datasets often blend real-world (e.g., nuScenes, proprietary ad logs), synthetic (e.g., InterCompose-generated two-person motions), and automatically grounded or annotated samples. Fine-tuning and ablation confirm the necessity of both large-scale breadth and high-precision feedback for best results (Zhai et al., 2024, Wu et al., 7 Oct 2025).

6. Limitations, Ablations, and Theoretical Guarantees

Ablation studies across domains isolate the effect of the InterActor. Removal or simplification consistently reduces informativeness, coupling, or grounding metrics. Theoretical guarantees (notably in SWIRL) provide monotonicity and convergence of InterActor policy updates, ensuring that alternated optimization does not degrade overall system performance (Lu et al., 27 Aug 2025).

Practical limitations involve the dependence on high-quality auxiliary evaluators (e.g., GenRMs, human raters), potential computational cost from complex cross-attentional blocks, and the need for domain-specific architecture tuning (e.g., token selection $k$, balance of reward weights).

7. Significance and Prospects

The InterActor paradigm exemplifies a general design philosophy in modern AI system architecture: decoupling representation, reasoning, and control modules while enabling fine-grained, semantically precise, and high-bandwidth interaction. Across logic programming, neural trajectory prediction, multimodal reasoning, and agentic RL, the InterActor serves to coordinate, condense, and align disparate data sources and objectives. The diversity of successful applications and the performance gains observed in ablation and fielded studies suggest growing importance for InterActor-like design patterns in future modular and agentic AI architectures (Gulzar et al., 15 Apr 2025, Wu et al., 7 Oct 2025, Lu et al., 27 Aug 2025, Wei et al., 14 Jun 2026, Zhai et al., 2024, Tarau, 2011).