The Attribution Contract: Feature Attribution for Generative Language Models

Abstract: Feature attribution methods promise to identify which input features matter for a model output. In generative LLMs, however, it is often unclear what should count as a feature in the first place. In autoregressive LLMs, earlier generated tokens are both outputs of the model and inputs to later predictions. In diffusion LLMs, generation proceeds through iterative denoising or unmasking rather than fixed left-to-right prediction, so local explanation may target a state of diffusion rather than a next token. We argue that this ambiguity is not merely an implementation detail, but a conceptual limitation of carrying classifier-era feature attribution directly into generative language modeling. We introduce the Attribution Contract, a specification for feature-attribution claims that names what output is being explained, which features are eligible to receive attribution, what generative process is assumed, what is held fixed, and what model score is being attributed. The contract clarifies why the same attribution method can answer different questions depending on how it is instantiated. We argue that many disagreements about feature attribution in generative LLMs are not disagreements about attribution algorithms, but about unstated explanatory contracts. Using autoregressive and diffusion LLMs as case studies, we show when attribution to earlier generated tokens, intermediate states, or denoising stages is informative, when it is misleading, and why feature-attribution methods in generative LLMs should be evaluated as method-contract pairs.