Organization and broader functions of narrative mechanisms

Characterize how narrative representations are internally organized in language models, how they relate to narrative theory, and what broader functions they serve.

Background

The paper identifies attention heads, MLP neurons, and sparse-autoencoder features whose activations correlate with textual narrativity and shows that ablating selected components can affect prediction loss, generation narrativity, and narrative-related benchmark performance. However, these results do not determine how the components are organized internally, whether their structure corresponds systematically to established theories of narrative, or what functions beyond the tested tasks they support. The abstract explicitly identifies these issues as unresolved questions.

References

These findings provide causal evidence that selected sets of narrative-associated components contribute to model behaviour, while their internal organisation, relation to narrative theory, and broader functions remain open questions.

— Language Models Have Interpretable Narrative Mechanisms  (2610.12989 - Liu et al., 9 Oct 2026) in Abstract

An open question is whether attention and FFN components support distinct aspects of narrative representation. Prior work interprets FFNs as key value memories that associate input patterns with output-relevant values \citep{geva-etal-2021-transformer}, whereas attention heads selectively move information between token positions \citep{vaswani2017attention,elhage2021mathematical} and mediate prediction-specific information-flow routes \citep{ferrando-voita-2024-information}. How do these mechanisms interact to construct narrative representations?

— Language Models Have Interpretable Narrative Mechanisms  (2610.12989 - Liu et al., 9 Oct 2026) in Section Discussion, subsection “Narrative Components”

Additionally, the NVC method itself may be biased towards finding certain types of features or components. The observed large SAE-ablation effect does not by itself establish that residual-stream representations are more important than attention heads or neurons. The exploration of these alternative methods, such as using entropy instead of correlation and probability instead of value \citep{tang2024language} is left for future work.

— Language Models Have Interpretable Narrative Mechanisms  (2610.12989 - Liu et al., 9 Oct 2026) in Section Discussion, subsection “Narrative Components”

Part of the gap lies in the specification: published reverse chronology is a rule applied to scenes, our prompts stated it on events, and whether models find the scene-level shape once scenes are marked is the most germane open question.

— Can LLMs Tell Stories Out of Order? A Corpus and Benchmark for Nonlinear Narrative Generation  (2610.21468 - Raiyan et al., 15 Oct 2026) in Discussion, Section 6