Intensity-Free Learning of Temporal Point Processes (1909.12127v2)

Abstract: Temporal point processes are the dominant paradigm for modeling sequences of events happening at irregular intervals. The standard way of learning in such models is by estimating the conditional intensity function. However, parameterizing the intensity function usually incurs several trade-offs. We show how to overcome the limitations of intensity-based approaches by directly modeling the conditional distribution of inter-event times. We draw on the literature on normalizing flows to design models that are flexible and efficient. We additionally propose a simple mixture model that matches the flexibility of flow-based models, but also permits sampling and computing moments in closed form. The proposed models achieve state-of-the-art performance in standard prediction tasks and are suitable for novel applications, such as learning sequence embeddings and imputing missing data.

• The paper introduces intensity-free learning by directly modeling inter-event time distributions, bypassing traditional conditional intensity functions.
• It leverages normalizing flows and a mixture of log-normal models to enhance flexibility and enable closed-form solutions for sampling and moment computation.
• Extensive experiments demonstrate state-of-the-art performance in prediction tasks, paving the way for applications like sequence embedding and missing event imputation.

Overview of "Intensity-Free Learning of Temporal Point Processes"

This paper, authored by Shchur et al., addresses the modeling of event sequences occurring at irregular intervals, using Temporal Point Processes (TPPs). The traditional method of learning in TPPs involves parameterizing the conditional intensity function, which often results in trade-offs regarding flexibility, efficiency, and ease of use. The authors propose an alternative methodological framework that eschews intensity-based modeling in favor of directly modeling the conditional distribution of inter-event times.

Key Contributions

1. Normalizing Flows in TPPs: The paper presents a novel approach by employing normalizing flows to establish models tailored for learning in TPPs. Normalizing flows create flexible probability distributions by transforming simple ones, enabling the construction of expressive models that can approximate complex distributions of inter-event times. The authors leverage this property to enhance the flexibility and theoretical soundness of TPP models.
2. Introduction of a Mixture Model: A significant proposal in the paper is a simple mixture model rivaling state-of-the-art techniques. This model offers the dual benefit of flexibility akin to flow-based models, while retaining computational advantages such as closed-form solutions for sampling and moment computation.
3. Performance and Applications: Through extensive experimentation, the proposed models demonstrate state-of-the-art performance in standard prediction tasks, outperforming or equaling established methods. The models also cater to novel applications beyond predictions, such as sequence embeddings and data imputation with missing events using advanced sampling methods.

Technical Insights

• Modeling Approach: The shift from intensity-based to direct inter-event time modeling via neural density estimation techniques stands out as a core advancement. The research aligns with the broader objectives of neural density estimation, integrating it effectively within the TPP framework.
• Model Structure: The use of normalizing flows and a mixture of log-normal distributions exemplifies the methodological versatility, supporting closed-form evaluations essential for real-time applications and theoretical exploration of TPPs.
• Comparison with Existing Techniques: A series of experiments benchmark the suggested methodologies against established models such as the Neural Hawkes process and RMTPP. The mixture model, in particular, offers compelling evidence of its strengths in every category: flexibility, closed-form solutions, ease of sampling, and evaluability of moments, which are highlighted through quantitative metrics.

Implications and Future Work

The implications of this paper are profound, suggesting a paradigm shift in how temporal data modeling is approached. By discarding the traditional intensity function in favor of directly estimating the distribution of inter-event times, the models not only enhance flexibility and accuracy but also expand the utility of TPPs to applications previously underserved.

While the paper establishes significant groundwork, potential future research avenues could explore:

• Extending the models to multivariate TPPs, where multiple event types interact, necessitating even more sophisticated modeling of dependencies.
• Investigating alternative base distributions beyond log-normal for mixture models to further generalize and refine model capabilities.
• Exploring hybrid models that could capitalize on the strengths of both intensity and intensity-free approaches, optimizing specific applications.

In conclusion, Shchur et al. present a comprehensive and technically robust framework that challenges conventional approaches toward temporal event modeling, showcasing both promising results and novel application domains. The discourse initiated by this paper has the potential to guide future enhancements and adaptations of TPP methodology, further integrating advanced machine learning techniques with practical real-world applications.