Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks (2101.05974v5)

Abstract: Temporal networks serve as abstractions of many real-world dynamic systems. These networks typically evolve according to certain laws, such as the law of triadic closure, which is universal in social networks. Inductive representation learning of temporal networks should be able to capture such laws and further be applied to systems that follow the same laws but have not been unseen during the training stage. Previous works in this area depend on either network node identities or rich edge attributes and typically fail to extract these laws. Here, we propose Causal Anonymous Walks (CAWs) to inductively represent a temporal network. CAWs are extracted by temporal random walks and work as automatic retrieval of temporal network motifs to represent network dynamics while avoiding the time-consuming selection and counting of those motifs. CAWs adopt a novel anonymization strategy that replaces node identities with the hitting counts of the nodes based on a set of sampled walks to keep the method inductive, and simultaneously establish the correlation between motifs. We further propose a neural-network model CAW-N to encode CAWs, and pair it with a CAW sampling strategy with constant memory and time cost to support online training and inference. CAW-N is evaluated to predict links over 6 real temporal networks and uniformly outperforms previous SOTA methods by averaged 10% AUC gain in the inductive setting. CAW-N also outperforms previous methods in 4 out of the 6 networks in the transductive setting.

• The paper introduces Causal Anonymous Walks (CAWs) to capture temporal network dynamics without relying on explicit node identities.
• It presents a CAW-based neural network (CAW-N) that encodes temporal motifs efficiently while supporting online training with constant time and memory cost.
• Experiments on six real-world networks show that CAW-N improves link prediction accuracy by approximately 15% in inductive settings.

Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks

The paper "Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks" presents a novel methodology for learning representations of temporal networks, focusing on capturing dynamic laws inherent in these structures. Temporal networks are a critical abstraction used to model various real-world systems with dynamic interactions, such as social networks and communication systems. This paper specifically addresses the challenge of inductive learning, which aims to generalize the learned representations to unseen parts of the network during training.

Key Contributions

1. Causal Anonymous Walks (CAWs): The authors propose a new method called Causal Anonymous Walks to capture network dynamics without relying on specific node identities. This method involves sampling temporal walks and anonymizing node identities based on the frequency and positions of nodes in these walks. The CAWs aim to encode temporal motifs in the network, which are patterns of interactions occurring within a restricted time span.
2. Causality and Anonymization in CAWs: CAWs incorporate a novel anonymization strategy that only retains the relative identities of nodes based on their presence and positions in the walks. This strategy helps in establishing correlations between different network motifs. The causal aspect is achieved by backtracking adjacent links over time, thus implicitly capturing the dynamics of temporal motifs.
3. Neural Network Model CAW-N: The authors develop a neural network model, CAW-N, designed to encode these anonymous walks efficiently. This model utilizes RNNs for encoding the sequences generated by CAWs and includes a set-based anonymization process to ensure inductive learning capabilities. It supports efficient online training and inference with a constant memory and time cost.

Experimental Evaluation

The effectiveness of the proposed CAW-N is demonstrated through extensive experiments on six real-world temporal networks. The experiment results indicate that CAW-N considerably outperforms state-of-the-art methods in both inductive and transductive settings, specifically for link prediction tasks. It shows superior performance by improving prediction accuracy by approximately 15% on average over six networks in inductive settings.

Implications and Future Work

The introduction of CAWs and CAW-N represents a significant advancement in modeling temporal networks without relying on explicit node identities, overcoming a major limitation in existing methods that are not truly inductive. By effectively capturing the network dynamics through implicitly learned temporal motifs, this approach opens avenues for future research, including:

• Generalizing to Higher-Order Structures: The methodology can be extended to predict more complex, high-order structures beyond dyadic interactions, such as cliques or communities, which are important in understanding the organization of many biological and social systems.
• Interpretable Neural Networks: Leveraging neural network interpretation techniques on CAW-N can potentially lead to the automatic discovery of meaningful temporal patterns or motifs, providing insights into the dynamics of complex systems.
• Broader Applicability: The approach could be adapted for various applications, including dynamic recommender systems, anomaly detection in cyber-physical systems, and modeling of evolving biological networks, showcasing its flexibility and scope.

Overall, the proposed framework demonstrates a robust methodology for learning inductive representations in temporal networks, emphasizing the importance of causality and correlation between network motifs for improved understanding and prediction of network dynamics.