Language Models are Injective and Hence Invertible (2510.15511v3)

Abstract: Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer LLMs mapping discrete input sequences to their corresponding sequence of continuous representations are injective and therefore lossless, a property established at initialization and preserved during training. Second, we confirm this result empirically through billions of collision tests on six state-of-the-art LLMs, and observe no collisions. Third, we operationalize injectivity: we introduce SipIt, the first algorithm that provably and efficiently reconstructs the exact input text from hidden activations, establishing linear-time guarantees and demonstrating exact invertibility in practice. Overall, our work establishes injectivity as a fundamental and exploitable property of LLMs, with direct implications for transparency, interpretability, and safe deployment.

• The paper shows that decoder-only Transformer language models are almost surely injective, ensuring that distinct prompts yield unique last-token activations.
• It provides a rigorous proof using real-analytic properties and measure-zero collision arguments, validated by billions of empirical comparisons across diverse datasets.
• The SipIt algorithm efficiently reconstructs the exact input in linear time, with significant implications for interpretability, privacy, and regulatory compliance.

Injectivity and Invertibility in Transformer LLMs

Introduction and Motivation

The paper "LLMs are Injective and Hence Invertible" (2510.15511) rigorously analyzes the mapping from discrete input sequences to continuous hidden representations in decoder-only Transformer LLMs. Contrary to prevailing assumptions that non-linearities, normalization, and attention mechanisms induce information loss, the authors establish that these models are almost surely injective: distinct prompts yield distinct last-token representations for essentially all parameter settings and throughout training. This injectivity is not only a theoretical property but is operationalized via the SipIt algorithm, which reconstructs the exact input sequence from hidden activations in provable linear time.

Mathematical Foundations of Injectivity

The core technical contribution is a proof that the mapping from input sequences to last-token hidden states in decoder-only Transformers is real-analytic and injective almost everywhere in parameter space. The argument leverages the following facts:

• Real-analyticity: All architectural components (embeddings, LayerNorm with $\varepsilon > 0$, causal attention, MLPs with analytic activations, residual connections) are real-analytic functions of their parameters. The composition of these blocks preserves real-analyticity.
• Measure-zero collision sets: For any pair of distinct prompts, the set of parameters causing their representations to collide is a zero set of a real-analytic function, and thus has Lebesgue measure zero.
• Initialization and training: Standard parameter initializations (Gaussian, uniform, Xavier/Glorot) avoid measure-zero sets with probability one. Gradient-based training (SGD, mini-batch GD) preserves absolute continuity of the parameter distribution, so training cannot induce collisions.

Formally, for a finite vocabulary $\mathcal{V}$, context length $K$, and embedding dimension $d$, the map

$$(\mathrm{s}, \bm{\theta}) \mapsto \mathbf{r}(\mathrm{s}; \bm{\theta}) \in \mathbb{R}^d$$

is real-analytic, and for almost all $\bm{\theta}$, $$\mathbf{r}(\mathrm{s}; \bm{\theta}) \neq \mathbf{r}(\mathrm{s}'; \bm{\theta})$$ for $\mathrm{s} \neq \mathrm{s}'$.

Empirical Validation: Collision Search

The theoretical results are substantiated by extensive empirical tests. The authors perform billions of pairwise comparisons of last-token representations across 100,000 prompts sampled from diverse datasets (Wikipedia, C4, The Pile, Python code) and six state-of-the-art models (GPT-2, Gemma-3, Llama-3.1, Mistral-7B, Phi-4-mini, TinyStories-33M). No collisions are observed; all pairwise distances are orders of magnitude above the collision threshold ($10^{-6}$).

Figure 1: Minimum pairwise distances between last-token states across layers and models, demonstrating strict separation and absence of collisions.

Further analysis reveals that separation margins typically increase with model depth and scale, and stabilize with sequence length, indicating robust injectivity across architectures and input sizes.

Operationalizing Injectivity: The SipIt Algorithm

Injectivity is made operational via SipIt, an algorithm that reconstructs the exact input sequence from hidden activations. SipIt exploits the causal structure of Transformers: the hidden state at position $t$ depends only on the prefix and the current token. At each step, SipIt enumerates candidate tokens, computes the corresponding hidden state, and matches it to the observed activation. The process is repeated sequentially, recovering the full prompt in at most $T|\mathcal{V}|$ steps for a sequence of length $T$.

The correctness of SipIt is guaranteed by the strict separation margin: for almost all parameter settings, the observed hidden state at each position uniquely identifies the corresponding token. Empirically, SipIt achieves 100% exact recovery with linear-time performance, outperforming brute-force and gradient-based prompt discovery methods.

Figure 2: SipIt inversion times as a function of model depth, showing efficient and stable scaling across layers.

Implications for Interpretability, Privacy, and Deployment

The injectivity and invertibility of Transformer LLMs have significant implications:

• Interpretability and Probing: Last-token hidden states are unique identifiers of the input prompt, providing a sound foundation for mechanistic interpretability and causal analysis. Any failure of probing or inversion methods cannot be attributed to information loss in the model.
• Privacy and Compliance: Hidden states are isomorphic to the input prompt. Systems that store or transmit activations are effectively handling user text, with direct consequences for privacy, data deletion, and regulatory compliance. The results contradict claims that model weights or activations are not personal data.
• Transparency and Auditing: The ability to reconstruct inputs from activations enables stronger transparency and auditing tools, facilitating safe deployment and accountability.

Limitations and Failure Modes

While injectivity holds almost surely, it can be deliberately broken by pathological parameter choices (e.g., identical embeddings for distinct tokens, tied positional encodings, quantization, non-analytic activations). These scenarios require engineered parameter settings and do not arise under standard initialization or training.

Figure 3: Exhaustive collision search among the closest prompt pairs, confirming local injectivity even under stress-test conditions.

Future Directions

The analysis opens several avenues for further research:

• Extension to Multimodal Models: Generalizing injectivity guarantees to multimodal architectures (vision, music, etc.) remains an open problem.
• Robustness to Noise and Quantization: Studying approximate inversion under noisy or quantized activations will clarify the practical limits of invertibility.
• Regulatory Impact: Bridging technical insights with evolving legal frameworks will be crucial for responsible AI deployment.

Conclusion

This work establishes that decoder-only Transformer LLMs are structurally injective: distinct prompts yield distinct last-token representations under all practical parameter settings and training procedures. The SipIt algorithm leverages this property for efficient, exact prompt recovery from hidden activations. These findings have direct implications for interpretability, privacy, and safe deployment, and provide a rigorous foundation for future research on the invertibility and transparency of deep LLMs.