Interaction-Driven Dynamic Embeddings

• Interaction-driven dynamic embeddings are vector representations that update in real time with observed interactions, capturing the evolving states of entities.
• They employ methods such as coupled recurrent updates, projection for temporal drift, attention over history windows, and temporal walks to model fine-grained temporal dynamics.
• These approaches demonstrate significant improvements in prediction metrics and scalability across applications like recommendation systems, dynamic network analysis, and adaptive dialogue agents.

Interaction-driven dynamic embeddings are vector representations of entities that are updated explicitly in response to observed interactions over time, rather than relying solely on static properties or fixed snapshots. This paradigm has become foundational across dynamic network analysis, recommender systems, dialogue agents, and domains characterized by frequently evolving relationships or actions. Interaction-driven approaches treat each (user, item), (node, node), or more general multi-entity interaction as an event that co-evolves the latent states ("embeddings") of the involved entities, enabling models to track and predict behavior with fine temporal granularity.

1. Mathematical Frameworks for Interaction-Driven Dynamic Embeddings

Interaction-driven dynamic embeddings formalize the evolving property of entities (e.g., users, nodes, items) via update operators triggered by interactions, frequently parameterized by neural networks or other parametrized mappings. Iconic frameworks include:

• Coupled Recurrent Update: Each interaction between entities $a$ and $b$ at time $t$ triggers joint updates to both embeddings:

$$\mathbf{e}_a(t) = f_a(\mathbf{e}_a(t^-), \mathbf{e}_b(t^-), \Delta_a, f_{ab}), \quad \mathbf{e}_b(t) = f_b(\mathbf{e}_b(t^-), \mathbf{e}_a(t^-), \Delta_b, f_{ab})$$

$\Delta_{(\cdot)}$ are time gaps, $f_{ab}$ are interaction features. This coupled update is widely implemented via RNNs in models like JODIE (Kumar et al., 2019, Kumar et al., 2018).

• Projection for Temporal Drift: Embeddings are projected forward in time via parameterized linear or non-linear transformation, analytically modeling the expected drift in absence of interaction:

$$\widehat{\mathbf{e}}(t+\Delta) = \text{Proj}(\mathbf{e}(t), \Delta)$$

as in JODIE, via elementwise scaling (Kumar et al., 2019).

• Mutual Attention via History Windows: Instead of maintaining strict recursion, some models aggregate representations from recent histories of interactions and align them using attention mechanisms. DeePRed constructs short-term embeddings from a $k$-window history over stationary "long-term" embeddings, then combines user/item histories via multi-way attention (Kefato et al., 2020).
• Temporal Walks: In graph settings, temporal walks sample sequential, time-respecting paths over edge streams, ensuring that each sampled sequence respects the partial temporal ordering of the interactions (Lee et al., 2019). This guides Skip-Gram–type objectives in continuous-time dynamic embeddings (CTDNE).

The shared principle is that time-varying interaction events define the atomic currency of update, with embedding dynamics reflecting the structure and order of the interaction sequence.

2. Algorithmic Paradigms and Core Update Mechanisms

The algorithmic ecosystem for interaction-driven dynamic embeddings encompasses several families, often selected according to the application domain and data structure.

• Mutually-Recursive RNNs: JODIE advances the field by employing mutually-recursive RNNs to update user/item embeddings directly on each event, then projecting the embedding forward in continuous time to support real-time prediction. Its t-Batch algorithm enables efficient, dependency-respecting mini-batching (Kumar et al., 2019, Kumar et al., 2018). This structure is effective for interaction networks with unavoidable recursion between entity states.
• Non-Recursive Mini-Batching via Long-Term Proxies: DeePRed avoids full recursion by employing fixed, stationary embeddings as proxies for short-term dynamics. It aggregates recent histories with recurrent units (GRU) and attention, which allows trivial mini-batching and dramatically improved computational efficiency, outperforming JODIE in both accuracy and training time (Kefato et al., 2020).
• Temporal Walk-Based Embeddings: In edge stream scenarios, e.g., CTDNE, random walks are redefined to respect edge timestamps, producing only time-respecting node sequences. The context window for Skip-Gram targets is selected from these temporal walks, providing fine granularity and full compliance with event order (Lee et al., 2019).
• High-Order or Heterogeneous Updates: In healthcare and financial domains, embedding architectures often combine (a) per-interaction updates for each entity via MLP/RNNs, (b) explicit modeling of heterogeneous interaction types, and (c) static attribute or attribute-graph–informed components. Co-evolving neural architectures are used for multiple entity types with type-specific update networks (Jang et al., 2023, Chitsazan et al., 2021).
• Attention/Transformer-Based Temporal Modeling: Attention-based temporal predictors, as in DANE-ATT, aggregate high-order, activeness-weighted neighborhood embeddings at each time step and predict future embeddings by self-attention over recent time slices, thus reducing reliance on RNN recursion and increasing parallelism (Xu et al., 2020).
• Interaction Embeddings in KGs: Crossover models like CrossE generate triple- or interaction-specific embeddings by gated fusion of entity and relation general embeddings, producing adaptive representations for each query (Zhang et al., 2019).

3. Application Domains and Design Patterns

Interaction-driven dynamic embeddings underpin models in a variety of data regimes:

• Temporal Recommendation Systems: Models like JODIE and DeePRed learn dynamic user and item embeddings to predict next interactions. JODIE achieves gains of up to 22.4% in inverse-rank metrics on Reddit/Wikipedia-like sources (Kumar et al., 2019, Kefato et al., 2020).
• Dynamic Graph Representation: CTDNE and DANE-ATT provide frameworks for node embeddings that capture both contemporaneous structure and the history of time-stamped (possibly attributed) interactions, enabling link prediction, node classification, and anomaly detection in evolving networks. These methods yield AUC improvements of 7–12% over static or windowed baselines (Lee et al., 2019, Xu et al., 2020, Xie et al., 2020).
• Health and Finance Analytics: Dynamic customer embeddings (DCE) and heterogeneous co-evolving networks (DECEnt) provide architectural blueprints for highly multivariate, cross-entity interaction streams, fusing temporal, event-type, and side-information into dynamic vectors used for predictive risk scoring, intent detection, and anomaly classification (Jang et al., 2023, Chitsazan et al., 2021).
• Dialogue and Collaborative Agents: Dynamic knowledge graph embedding models, where each conversational exchange triggers graph updates, allow agents to update their latent world representations continuously and inform policy for both utterance generation and world state estimation (He et al., 2017).
• Interactive/Adaptive Embedding Spaces: Interactive re-fitting enables humans to locally supervise the geometry of word embeddings by adjusting selected clusters or distances via explicit constraints, leading to dynamically shaped vector spaces in response to user steering (Powell et al., 2020).
• Social Neuroscience: Contrastive embedding frameworks applied to raw multi-subject EEG ("hyperscanning") learn representations that distinguish role, dyad, and behavioral context by explicitly encoding interaction-driven neural dynamics in paired time series (Glushanina et al., 27 Sep 2025).

4. Training Objectives, Optimization, and Cold Start

The loss functions in interaction-driven dynamic embeddings are tuned to the downstream task and embedding structure:

• Next-Action/Next-Interaction Prediction: Supervised objectives often take the form of minimizing the distance (e.g., squared error) or maximizing similarity between the predicted embedding of the next item/interactant and the true (static or dynamic) embedding at the prediction time (Kumar et al., 2019, Kefato et al., 2020).
• Consistency and Regularization: Smoothness penalties (e.g., $\|\mathbf{e}(t) - \mathbf{e}(t^-)\|^2$) are used to discourage erratic changes (Kumar et al., 2019, Jang et al., 2023). In contrast, Laplacian regularizers ensure locality or structure in the embedding manifold.
• Negative Sampling: Skip-Gram–style objectives on temporal walks or event windows are optimized with negative sampling to scale to large, sparse graphs (Lee et al., 2019, Xie et al., 2020).
• Multi-Loss or Unsupervised Autoencoding: Models in domains with complex or unlabeled interactions (e.g., healthcare) employ unsupervised reconstruction, temporal consistency, and domain-specific Laplacian losses (Jang et al., 2023).
• Cold-Start Mitigation and Modular Extension: As new entities emerge, dynamic embedding tables are efficiently extended by copying past weights, initializing new entries via category-average or nearest-neighbor in metadata space, and then resuming SGD. This yields greater stability and better area-under-curve (AUC) than retraining or default token mapping (Gomes et al., 2024).

5. Computational Complexity and Scalability

Interaction-driven dynamic embeddings introduce unique computational and scaling challenges:

• Recursive Dependencies: Approaches like JODIE necessitate specialized batching schemes (e.g., t-Batch) to preserve dependencies and allow parallelization, resulting in 8–9x speedup over naïve implementations (Kumar et al., 2019).
• Non-Recursive Histories: DeePRed's design enables trivial minibatching, reducing per-epoch runtime by 10–20x versus recursive RNN-based schemes (Kefato et al., 2020).
• Continuous-Time or Streaming Data: Temporal walk–based or event-driven models operate in $O(\text{batch size} \times d)$ per batch step, with real-time updates of 1–10 ms per interaction, and scale to high event rates (Lee et al., 2019, Xie et al., 2020).
• Modular Extension: Incremental embeddings with table extensions require $O(m\cdot d)$ time and memory per block update (for $m$ new entities), preserving the per-update training cost of static table methods but dispensing with retraining from scratch (Gomes et al., 2024).
• Side-Information and Heterogeneity: Models integrating node attributes or static graphs for multiple entity types include further overhead for encoding and co-evolution modules, but retain tractability via modularization (Jang et al., 2023).

6. Empirical Results and Application Impact

Interaction-driven dynamic embeddings have shown substantial empirical superiority over static or window-based alternatives across multiple tasks:

Model/Domain	Metric	Baseline	Dynamic Embedding	Gain	Reference
Temporal RecSys	MRR (Reddit)	0.603	0.726 (JODIE)	+20%	(Kumar et al., 2019)
Temporal RecSys	MRR (Reddit)	0.726	0.828 (DeePRed)	+14%	(Kefato et al., 2020)
Dynamic Network	AUC (Avg.)	—	+11.9% (CTDNE)	+11.9% vs. static	(Lee et al., 2019)
Knowledge Graph	MRR (FB15k-237)	0.291	0.299 (CrossE)	+5–8 points	(Zhang et al., 2019)
Healthcare Risk	Macro-F1	0.15	0.223 (DECEnt)	+48.1%	(Jang et al., 2023)
Dynamic Customer	AUROC	0.689	0.758 (DCE)	+0.069	(Chitsazan et al., 2021)
Lifelong Learning	AUC	0.662	0.710 (Unknown-Token)	+0.048	(Gomes et al., 2024)

These methods improve not only predictive metrics (e.g., link prediction, next-item recommendation, risk scoring), but also enable interpretable analytics and real-time applications.

7. Structural Variants, Generalization, and Limitations

Interaction-driven dynamic embeddings exhibit adaptability and several extension paths:

• Cross-Domain Generalization: Techniques originally designed for transaction logs generalize to knowledge graphs (Zhang et al., 2019), dense dyadic neural time series (Glushanina et al., 27 Sep 2025), and multimodal healthcare records (Jang et al., 2023).
• Cold-Start and Lifelong Learning: Modular embedding extension, initialization heuristics, and integration with continual-learning regularizers address the challenges of entity churn and concept drift (Gomes et al., 2024).
• Heterogeneous and Multimodal Embeddings: Co-evolving networks and attention over high-order neighborhoods accommodate complex domains with diverse interaction types, simultaneous entity streams, and attribute graphs (Jang et al., 2023, Xu et al., 2020).
• Limitations: Recursive RNN models present computational scaling barriers; heuristic initialization of new embeddings may not fully resolve entity diversity; long-term manifold drift can accumulate; in some settings, attention and transformer-based temporal modeling can mitigate but not completely bypass the need for explicit recursion or history expansion (Kefato et al., 2020, Xu et al., 2020).

A plausible implication is that as interaction data volume and entity vocabulary growth accelerate, the field is likely to converge on non-recursive, attention-based, and modularly extensible architectures that maintain the fidelity and adaptability of interaction-driven dynamic embeddings while scaling efficiently. A further trend is the integration of human-in-the-loop or side-information driven controls (as in interactive refitting and metadata-driven initialization) to enhance personalization and robustness.