SipIt: Inverting Transformer Hidden States

• SipIt algorithm is an inversion method for autoregressive Transformers that accurately reconstructs input token sequences from their last-token hidden representations.
• It leverages a one-step mapping process based on rigorous injectivity proofs, with empirical collision tests on models like GPT-2, Gemma-3, and Llama-3 confirming its effectiveness.
• SipIt offers practical insights for model transparency, privacy auditing, and causal attribution by enabling precise recovery of input data from internal representations.

The SipIt algorithm is an inversion procedure for autoregressive LLMs, specifically designed to reconstruct the exact input token sequence from the internal (hidden) representations generated by a trained Transformer. This approach is grounded in a rigorous mathematical proof that the mapping from input sequences to last-token hidden states is injective—i.e., distinct inputs almost surely yield distinct internal representations. SipIt operationalizes this invertibility, providing a provably correct and efficient method with direct consequences for model transparency, privacy, and interpretability.

1. Mathematical Foundation: Injectivity of LLMs

The theoretical underpinning of SipIt is the result that the mapping from a discrete token sequence $s \in \mathcal{V}^{\leq K}$ to the last-token hidden representation $r(s;\theta)$ in a standard Transformer is almost surely injective for random parameter initializations and after standard training. The central argument exploits real-analyticity: each layer (including embeddings, self-attention, and nonlinearities) composes to yield a global map whose zero-sets form measure-zero sets in parameter space.

For any two sequences $s \neq s'$, the difference $h(\theta) = \|r(s;\theta) - r(s';\theta)\|^2$ is a real-analytic function that is not identically zero (witnessed by parameter choices such as disjoint embeddings). Thus, $\Pr[r(s;\theta) = r(s';\theta)] = 0$, and by union bounding over all sequence pairs, all representations are pairwise distinct almost surely. This holds irrespective of non-injectivity in individual components (e.g., LayerNorm), as the global composition preserves separation.

2. Empirical Verification of Injectivity

Empirical evaluation involved large-scale "collision tests" across major LLMs such as GPT-2, Gemma-3, and Llama-3. For each model, billions of pairwise distances between last-token representations were measured across up to 100,000 distinct prompts sampled from mixed datasets. No collisions were detected at precision thresholds several orders of magnitude above $10^{-6}$, even under adversarial prompt constructions. Margin analysis showed that separation between hidden states increases with model depth and size, confirming that the injectivity property is robust in practice and not merely an artifact of parameter choice.

3. SipIt Algorithm: Sequential Inversion by One-Step Mapping

SipIt ("Sequential Inverse Prompt via ITerative updates") inverts the forward autoregressive mapping by leveraging the injectivity of the per-token state update. For a sequence $s = (s_1, \dots, s_T)$, the hidden state at position $t$, denoted $h_t(s_{1:t}; \theta)$, is computed by the Transformer given the prefix $s_{1:t}$. To invert, SipIt recovers $s_1$ from $h_1$, then $s_2$ given $h_2$ and the reconstructed prefix, and so on.

The key procedure defines the "one-step map":

$F(v; \pi, t) = h_t(\pi \oplus v)$

where $\pi = (s_1, \dots, s_{t-1})$ and $v \in \mathcal{V}$. Given an observed state $\hat{h}_t$, SipIt searches for the unique $v$ such that:

$\|\hat{h}_t - F(v; \pi, t)\|_2 \leq \epsilon$

with $\epsilon$ set to less than half the minimal observed separation margin (guaranteed positive by theory and experiment). Iterating for $t = 1$ to $T$ reconstructs the full prompt. The search policy can be random enumeration or use heuristics (e.g., gradient-guided candidate ranking). In the worst case, the complexity is $O(T|\mathcal{V}|)$, but in practice, fewer candidate checks are needed.

4. Practical Implementation and Performance

SipIt is implemented to interface with pre-trained LLMs, requiring only black-box access to the per-token state computation. In practice:

• SipIt reconstructs input prompts from internal hidden states with 100% accuracy across a diverse set of prompts and models.
• Empirical runtime is dramatically lower than exhaustive search, and considerably more efficient than black-box optimization or brute-force baselines (see benchmarking results in the source).
• The algorithm is robust to prompt length, model scaling, and minor perturbations, as long as $\epsilon$ is appropriately chosen.

A representative pseudocode sketch is:

for t in range(1, T+1): prefix = recovered_tokens[:t-1] for v in vocabulary: candidate_state = model.forward(prefix + [v]) if np.linalg.norm(observed_states[t] - candidate_state) <= epsilon: recovered_tokens.append(v) break

5. Implications for Transparency, Privacy, and Model Auditing

The invertibility result, operationalized via SipIt, means that internal states (especially the last-token embedding) encode a lossless copy of the input sequence. This has several ramifications:

• Transparency and interpretability: Researchers can "decode" precisely what information the model is currently representing at any point. Information loss at the hidden state level cannot be attributed to the model's internal operations, only to subsequent analysis or probing methods.
• Auditing and privacy: Caching or transmitting internal representations (as might be done in multi-stage or persistent workloads) is logically equivalent to handling user input, with all associated privacy, regulatory, and security implications.
• Causal attribution: Any output can be causally traced without ambiguity to the explicit input sequence, since no two distinct inputs share a hidden state.

6. Scope, Limitations, and Future Applications

SipIt is applicable under the conditions satisfied by standard causal–decoder LLMs: primarily, strict causality (i.e., each $h_t$ is determined solely by $s_{1:t}$), parameter genericity, and sufficiently large separation margin. The technique directly extends to newer and larger models as well as tasks relying on the preservation of prompt information throughout the network stack.

Limitations include the linear dependence on vocabulary size for each inversion step and the need for access to the network at the appropriate hidden state layers. However, ablation studies in the source demonstrate that practical factors (such as increasingly large margins) further reduce collision risk, and optimizations in candidate enumeration (e.g., gradient guidance) can reduce practical runtime.

A plausible implication is that future model designs or privacy-preserving architectures must consider this injectivity: any exposure of intermediate states carries the same risk profile as direct exposure of plaintext prompts.

7. Summary

SipIt provides an explicit, provably correct algorithm for reconstructing input token sequences from Transformer hidden states, built upon measure-theoretic injectivity results and validated through large-scale empirical studies. Its sequential inversion schema offers practical and efficient inversion for both scientific introspection and audit scenarios. The existence and effectiveness of SipIt have broad impact on interpretability, transparency, and privacy in the deployment and study of LLMs (Nikolaou et al., 17 Oct 2025).