Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events (2104.07635v1)

Published 15 Apr 2021 in cs.CL, cs.AI, and cs.LG

Abstract: Tracking entities throughout a procedure described in a text is challenging due to the dynamic nature of the world described in the process. Firstly, we propose to formulate this task as a question answering problem. This enables us to use pre-trained transformer-based LLMs on other QA benchmarks by adapting those to the procedural text understanding. Secondly, since the transformer-based LLMs cannot encode the flow of events by themselves, we propose a Time-Stamped LLM~(TSLM model) to encode event information in LMs architecture by introducing the timestamp encoding. Our model evaluated on the Propara dataset shows improvements on the published state-of-the-art results with a $3.1\%$ increase in F1 score. Moreover, our model yields better results on the location prediction task on the NPN-Cooking dataset. This result indicates that our approach is effective for procedural text understanding in general.

Citations (23)

Summary

We haven't generated a summary for this paper yet.