Learning Granger Causality from Instance-wise Self-attentive Hawkes Processes (2402.03726v2)

Abstract: We address the problem of learning Granger causality from asynchronous, interdependent, multi-type event sequences. In particular, we are interested in discovering instance-level causal structures in an unsupervised manner. Instance-level causality identifies causal relationships among individual events, providing more fine-grained information for decision-making. Existing work in the literature either requires strong assumptions, such as linearity in the intensity function, or heuristically defined model parameters that do not necessarily meet the requirements of Granger causality. We propose Instance-wise Self-Attentive Hawkes Processes (ISAHP), a novel deep learning framework that can directly infer the Granger causality at the event instance level. ISAHP is the first neural point process model that meets the requirements of Granger causality. It leverages the self-attention mechanism of the transformer to align with the principles of Granger causality. We empirically demonstrate that ISAHP is capable of discovering complex instance-level causal structures that cannot be handled by classical models. We also show that ISAHP achieves state-of-the-art performance in proxy tasks involving type-level causal discovery and instance-level event type prediction.

• The paper introduces a novel framework that leverages Neural Point Processes to infer instance-level Granger causality in asynchronous event sequences.
• It integrates self-attention into traditional Hawkes processes, enabling direct computation of instance-specific causal strengths.
• Empirical results demonstrate improved type-level causal discovery and event prediction across synthetic and real-world datasets.

Learning Granger Causality from Instance-wise Self-attentive Hawkes Processes

The paper introduces a novel framework utilizing Neural Point Processes (NPPs) for the instance-wise inference of Granger causality in asynchronous, interdependent, multi-type event sequences. This method, termed Instance-wise Self-Attentive Hawkes Processes (ISAHP), represents a significant stride forward from traditional Hawkes processes, with a particular emphasis on capturing instance-level causal relationships, a task that classical models often neglect.

Hawkes processes have long served as a foundational tool for modeling event sequences displaying temporal self-excitement and mutual excitation. These processes typically rely on a linear model with additive events, which, while effective in simpler cases, muscle into a limitation when attempting to address the intricacies of real-world event sequences where nonlinear effects, such as synergistic interactions, become significant. Classical methods, particularly those assuming linear relationships, often fall short in environments rich with complex, interwoven dependencies.

ISAHP introduces a deep learning-based design that maintains the interpretability inherent in additive intensity frameworks but extends the capability by incorporating a self-attentive architecture, drawing insights from the success of transformers. This architecture allows ISAHP to directly compute instance-level causal strength by integrating self-attention mechanisms into the parametric structures of its intensity function. Essentially, this marks a significant departure from older baselines that typically aggregate or decompose interactions into coarser type-level inferences.

Numerous empirical evaluations underscore ISAHP's versatility and improved performance. The inclusion of both synthetic and real-world datasets, such as the Synergy and MemeTracker (MT) datasets respectively, demonstrate the model's robustness across contexts. The results reflect ISAHP's superior capability in not only outstripping existing algorithms in the precision of type-level causal discovery but also in predicting next event-types with a compelling accuracy increase over its competitors.

ISAHP's remarkable performance benchmarks are grounded in its finely-tuned architecture that allows for rich representation of event instances without resorting to heuristics or post-hoc attributions such as those seen in RPPN or CAUSE models. For instance, while CAUSE relies on post-training attribution methodologies, ISAHP accomplishes direct inference without requiring additional computational overhead.

The implications of this research stretch beyond immediate performance gains. By placing a marker in both the theoretical and practical landscapes, ISAHP opens avenues for exploring more dynamic interactions in event-driven systems, potentially serving as a pivotal tool in fields such as finance, health monitoring, and social network analysis where event causality is paramount. Furthermore, one might contend that the contiguous design of the self-attentive mechanisms in ISAHP might pave the way for future expansions into the predictive domains of non-linear dynamics and control systems.

While ISAHP introduces substantial innovations to the field, an aspect for future contemplation and development could involve refining its adaptability to varied scales of event data. As applications of artificial intelligence expand, embedding such systems into the massive throughput of IoT, and other data streams will require both scaling efficiencies and further robustness in event characterization.

In summary, ISAHP not only challenges the current boundaries for causal inference tasks through its advanced framework but also lays a foundation for the next progression in causal discovery technologies, resonating well with the ongoing evolutions within the AI ecosystem.