Encode and evaluate simultaneous events as partial orders

Develop an evaluation that represents simultaneous events as equivalence classes or weak chronological orders and determine how this changes chronology-reconstruction and chronicle evaluation.

Background

The benchmark forces simultaneous events into an arbitrary total order. This causes chronicle tellings that differ only in the order of simultaneous events to receive errors, including zero exact-match scores for the chronicle category.

A weak-order representation and metrics such as Kendall’s tau-b or equivalence-class exact match would evaluate these cases according to their actual temporal ambiguity rather than an arbitrary linearization.

References

We have not yet encoded the chronology as a weak order and scored it with Kendall's $\tau_b$ or equivalence-class exact match, as our own Recommendation~2 advises.

— Can LLMs Tell Stories Out of Order? A Corpus and Benchmark for Nonlinear Narrative Generation  (2610.21468 - Raiyan et al., 15 Oct 2026) in Limitations, paragraph “Simultaneity is scored as error”