ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction Based on Large Language Model

Abstract: In the realm of event prediction, temporal knowledge graph forecasting (TKGF) stands as a pivotal technique. Previous approaches face the challenges of not utilizing experience during testing and relying on a single short-term history, which limits adaptation to evolving data. In this paper, we introduce the Online Neural-Symbolic Event Prediction (ONSEP) framework, which innovates by integrating dynamic causal rule mining (DCRM) and dual history augmented generation (DHAG). DCRM dynamically constructs causal rules from real-time data, allowing for swift adaptation to new causal relationships. In parallel, DHAG merges short-term and long-term historical contexts, leveraging a bi-branch approach to enrich event prediction. Our framework demonstrates notable performance enhancements across diverse datasets, with significant Hit@k (k=1,3,10) improvements, showcasing its ability to augment LLMs for event prediction without necessitating extensive retraining. The ONSEP framework not only advances the field of TKGF but also underscores the potential of neural-symbolic approaches in adapting to dynamic data environments.

• The paper introduces ONSEP, a neural-symbolic framework that integrates dynamic causal rule mining (DCRM) and dual history augmented generation (DHAG) for improved event prediction in temporal knowledge graphs.
• It leverages a long short-term bi-branch retriever to merge short-term and long-term event histories, achieving superior Hit@1 scores on datasets like ICEWS14 and ICEWS05-15.
• The framework's dynamic causal inference and adaptability provide practical benefits for applications such as crisis monitoring and financial forecasting.

ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction

Introduction to ONSEP

The "Online Neural-Symbolic Event Prediction" (ONSEP) framework provides an innovative solution for event forecasting in temporal knowledge graphs (TKG). Previous methodologies encountered challenges like static short-term history reliance, which hampered their ability to assimilate evolving data patterns dynamically. ONSEP distinguishes itself by leveraging a dual-layered approach that integrates dynamic causal rule mining (DCRM) and dual history augmented generation (DHAG), significantly enhancing the predictive accuracy of LLMs without the need for extensive retraining.

Figure 1: Comparison of ONSEP and ICL Frameworks for Event Prediction with Schematic Overview of ONSEP's Core Components and Operational Processes.

Dynamic Causal Rule Mining (DCRM)

ONSEP's DCRM module dynamically constructs and updates causal rules from real-time data, accommodating new causal relationships efficiently. This is achieved using a semantic-driven rule learning process where LLMs evaluate potential cause-effect interactions, updating a causal rule base (CRB) with confidence scores. The DCRM process not only improves the adaptability of event prediction models but also ensures that the rule base remains relevant in dynamically changing data environments.

Dual History Augmented Generation (DHAG)

The DHAG module enhances ONSEP by integrating both short-term event patterns and long-term causal trajectories. It employs a long short-term bi-branch retriever (LSTBBR) to concurrently retrieve historical contexts from both short- and long-term event histories. This dual approach, combined with hybrid model inference (HMI), allows ONSEP to capitalize on the LLM's generative capabilities with superior contextual insights, thereby augmenting the accuracy of event predictions.

Figure 2: Performance of ONSEP in terms of Hit@1 across various DHAG ensemble weights $\lambda$ of DHAG. The underlying LLM is InternLM2-7B, processing input histories of length 100. $\lambda$ represents the weight given to long-term causal event chains. This illustrates how varying $\lambda$ influences the integration of short-term and long-term reasoning contexts within ONSEP.

Experimental Results

Implementing ONSEP with the InternLM2-7B model across datasets like ICEWS14, ICEWS05-15, and ICEWS18 demonstrated remarkable performance enhancements. ONSEP notably outperformed traditional in-context learning (ICL) methods with significant gains in Hit@1 scores, showcasing its effectiveness in dynamic TKG environments. This was especially evident when employing a balanced ensemble weight in the DHAG module, optimizing the fusion of short-term and long-term historical contexts.

Figure 3: Performance comparison across various model series under ONSEP and ICL methods with the percentage improvement indicated in red above the green bars.

Practical Implications and Future Directions

ONSEP's framework provides a robust solution for real-time event prediction applications such as crisis monitoring, financial forecasting, and social analytics. By enabling LLMs to update causal inferences dynamically, ONSEP offers significant advancements in the field of TKGF. Future research could focus on refining the integration between DCRM and DHAG or exploring the applicability of ONSEP across different domains and models to further capitalize on its promising capabilities.

Conclusion

By integrating neural and symbolic reasoning, ONSEP provides a powerful, adaptive framework for dynamic event prediction in temporal knowledge graphs. Its innovative approach ensures that LLMs remain responsive to new data patterns, effectively bridging the gap between static data models and real-world applications. This framework not only improves predictive accuracy but also sets a foundation for future advancements in event forecasting methodologies.