Mechanisms underlying in-context learning in large language models

Characterize the mechanisms that enable large language models to perform in-context learning during inference without any additional weight updates when examples are provided in the prompt.

Background

The paper studies how transformer-based models might achieve in-context learning (ICL) by introducing contextual blocks that generalize transformer blocks. It proves that contextual layers can induce implicit low-rank updates to the first MLP layer based on the prompt, proposing a concrete mechanism for implicit adaptation at inference time.

Despite presenting this mechanism and empirical verification in a controlled setting, the authors emphasize that the broader underlying mechanisms enabling ICL in LLMs are not fully understood, which motivates the explicit open problem stated in the abstract.

References

One of the most striking features of LLMs (LLM) is their ability to learn in context. Namely at inference time an LLM is able to learn new patterns without any additional weight update when these patterns are presented in the form of examples in the prompt, even if these patterns were not seen during training. The mechanisms through which this can happen are still largely unknown.

— Learning without training: The implicit dynamics of in-context learning  (2507.16003 - Dherin et al., 21 Jul 2025) in Abstract

It also leaves open how the recovered change must be realized inside the model.

— When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL  (2608.13385 - Li, 13 Aug 2026) in Section 2, “A Unified Formulation of Implicit M-ICL”

Fourth, the analysis is purely behavioral; connecting these patterns to internal model mechanisms remains an open challenge.

— Recursive Reasoning or Statistical Extrapolation? In-Context Learning in Multi-Agent Interdependent Decision-Making  (2609.18591 - Liu et al., 16 Sep 2026) in Section 7, Limitations