Fully Neural Network based Model for General Temporal Point Processes (1905.09690v3)

Abstract: A temporal point process is a mathematical model for a time series of discrete events, which covers various applications. Recently, recurrent neural network (RNN) based models have been developed for point processes and have been found effective. RNN based models usually assume a specific functional form for the time course of the intensity function of a point process (e.g., exponentially decreasing or increasing with the time since the most recent event). However, such an assumption can restrict the expressive power of the model. We herein propose a novel RNN based model in which the time course of the intensity function is represented in a general manner. In our approach, we first model the integral of the intensity function using a feedforward neural network and then obtain the intensity function as its derivative. This approach enables us to both obtain a flexible model of the intensity function and exactly evaluate the log-likelihood function, which contains the integral of the intensity function, without any numerical approximations. Our model achieves competitive or superior performances compared to the previous state-of-the-art methods for both synthetic and real datasets.

• The paper introduces a fully neural network-based model to flexibly represent temporal point process intensity functions, avoiding traditional parametric limitations.
• The model utilizes a cumulative hazard function network and its derivative to accurately evaluate the log-likelihood function without numerical approximations.
• Experiments on synthetic and real datasets show the neural network model outperforms state-of-the-art methods in predictive performance.

Overview of Neural Network Models for Temporal Point Processes

The presented paper offers a novel approach to modeling temporal point processes using a fully neural network-based model, diverging from the traditional parametric frameworks that often limit the expressive power of such models. Temporal point processes are instrumental in modeling discrete event sequences across diverse applications, from social network communications to web user interactions. The authors propose a recurrent neural network (RNN) model that flexibly represents the intensity function of a point process without imposing a specific functional form.

Novel Approach and Methodology

The crux of the paper lies in its innovative approach to modeling the intensity function. Unlike preceding models that assume a parametric form, often exponential or constant, the proposed model utilizes a feedforward neural network to represent the integral of the intensity function. Subsequently, the intensity function is obtained as the derivative of this integral. This methodology not only enhances the model's flexibility but also facilitates an exact evaluation of the log-likelihood function without reliance on numerical approximations.

The authors employ a cumulative hazard function network to implement this model. The cumulative hazard function governs the monotonic increase in event probability over time, ensuring the coherence and positivity essential for real-world applicability. This network structure allows the model to capture complex dependencies on event history, offering a significantly more robust predictive capability.

Comparative Analysis and Results

Through extensive experimentation with both synthetic and real datasets, the paper demonstrates the superiority of the proposed neural network-based model compared to existing state-of-the-art models, including those based on constant, exponential, and piecewise constant forms. Notable findings include:

• Synthetic Data Performance: The neural network model consistently displayed competitive or superior predictive performance across diverse data-generating mechanisms, including stationary and non-stationary renewal processes, self-correcting processes, and Hawkes processes.
• Real Data Performance: Evaluations using practical datasets, such as finance and emergency call records, confirmed the model's robustness and adaptability, especially in scenarios featuring significant variability in inter-event intervals.

Implications and Future Directions

The implications of these findings are profound, potentially influencing the trajectory of temporal point process modeling in several areas:

• Practical Applications: The flexibility and accuracy of the model hold promise for enhanced predictions in financial transactions, social behavior analytics, and other domains where real-time event forecasting is critical.
• Theoretical Advancements: The approach paves the way for further exploration into neural network architectures, facilitating continuous refinement of intensity function modeling without traditional parametric constraints.

Looking forward, the model's efficacy opens avenues for extending its architecture to incorporate event marks, enhancing its applicability to marked temporal point processes. Additionally, its integration with reinforcement learning could unveil new pathways for dynamic event process prediction in complex adaptive systems.

In conclusion, this paper presents a robust advancement in modeling temporal point processes, leveraging neural network innovations to transcend limitations of traditional methodologies, with promising implications across theoretical and practical domains.