Every Language Model Has a Forgery-Resistant Signature (2510.14086v1)

Abstract: The ubiquity of closed-weight LLMs with public-facing APIs has generated interest in forensic methods, both for extracting hidden model details (e.g., parameters) and for identifying models by their outputs. One successful approach to these goals has been to exploit the geometric constraints imposed by the LLM architecture and parameters. In this work, we show that a lesser-known geometric constraint--namely, that LLM outputs lie on the surface of a high-dimensional ellipse--functions as a signature for the model and can be used to identify the source model of a given output. This ellipse signature has unique properties that distinguish it from existing model-output association methods like LLM fingerprints. In particular, the signature is hard to forge: without direct access to model parameters, it is practically infeasible to produce log-probabilities (logprobs) on the ellipse. Secondly, the signature is naturally occurring, since all LLMs have these elliptical constraints. Thirdly, the signature is self-contained, in that it is detectable without access to the model inputs or the full weights. Finally, the signature is compact and redundant, as it is independently detectable in each logprob output from the model. We evaluate a novel technique for extracting the ellipse from small models and discuss the practical hurdles that make it infeasible for production-scale models. Finally, we use ellipse signatures to propose a protocol for LLM output verification, analogous to cryptographic symmetric-key message authentication systems.

• The paper establishes that language model outputs lie on a high-dimensional ellipsoid, creating a natural and forgery-resistant signature.
• It introduces a verification protocol using inverse affine transformations to confirm if logprobs reside on the expected ellipsoid.
• Practical implications include enhanced model forensics and regulatory compliance, though extraction is infeasible for large-scale models.

Forgery-Resistant Ellipse Signatures in LLMs

Introduction

This paper establishes that the outputs of modern LMs are subject to a geometric constraint: they lie on the surface of a high-dimensional ellipsoid (hyperellipse) determined by the model’s final normalization and linear layers. This constraint is not merely a mathematical curiosity; it constitutes a robust, naturally occurring, and self-contained signature that can be used to identify the source model of a given output. The authors demonstrate that this "ellipse signature" is computationally hard to forge without access to the model parameters, distinguishing it from previously studied linear signatures and text-based watermarks. The paper further proposes a cryptographically inspired protocol for output verification based on these signatures.

Figure 1: LLM logits are subject to constraints that force them to lie on a high-dimensional ellipse, which serves as a forgery-resistant signature for model identification.

Mathematical Formulation of Ellipse Signatures

The final layers of typical LMs consist of a normalization (RMSNorm or LayerNorm) followed by a linear or affine transformation. The normalization maps hidden representations onto the surface of a $d$-dimensional sphere, and the subsequent linear transformation stretches and rotates this sphere into a $d$-dimensional ellipsoid embedded in the $v$-dimensional vocabulary space.

Figure 2: The final layers of a LLM map normalized representations onto a sphere, which is then stretched and rotated into an ellipsoid by the linear layer.

Formally, for a hidden state $\mathbf{h} \in \mathbb{R}^d$, the normalized output $\hat{\mathbf{h}}$ is mapped to logits via an affine transformation:

$$\mathbf{z} = \mathbf{A} \hat{\mathbf{h}} + \mathbf{b}$$

where $\mathbf{A}$ encodes rotation and scaling, and $\mathbf{b}$ is a bias. The set of possible output logits thus forms a $d$-dimensional ellipsoid in $\mathbb{R}^v$.

APIs typically expose log-probabilities (logprobs), which are centered versions of logits due to the invariance of softmax to scalar addition. The centering operation preserves the ellipsoid constraint, enabling signature verification directly on logprobs.

Signature Verification and Model Identification

To verify whether a logprob vector $\mathbf{\ell}$ was generated by a specific model, one applies the inverse affine transformation and checks if the result lies on the unit sphere:

$$\left\| \mathbf{A}^+ (\mathbf{\ell} - \mathbf{b}) \right\|_2 \approx 1$$

where $\mathbf{A}^+$ is the pseudoinverse of $\mathbf{A}$. If the transformed logprob lies on the sphere, it is highly likely to have originated from the model.

Empirical evaluation across several open-weight models (Olmo 2 7B, Llama 3.1, Qwen 3 8B, GPT-OSS) demonstrates that the mean distance to the model ellipse is orders of magnitude smaller for the generating model than for others, even when projecting outputs onto each other's column spaces.

Figure 3: Mean distance to the model ellipse for logprobs from several models, showing clear separation and identification capability.

Forgery Resistance and Extraction Complexity

Unlike linear signatures, which can be forged by extracting linear constraints from API outputs, ellipse signatures are substantially harder to forge. The extraction of the model ellipse requires:

• Sample Complexity: $O(d^2)$ outputs to uniquely determine the ellipsoid parameters, where $d$ is the hidden size.
• API Query Complexity: $O(d^3 \log d)$ queries for typical API constraints.
• Computational Complexity: Ellipsoid fitting algorithms require $O(d^6)$ time and $O(d^4)$ space.

For production-scale models (e.g., Llama 3 8B, $d=4096$), the required number of samples and computational resources render ellipse extraction infeasible in practice.

Figure 4: Predicted and true values for bias, singular values, and rotation angles for a small model, demonstrating high accuracy of parameter recovery with sufficient samples.

Figure 5: Distance between predicted and true parameters decreases with more samples, but exhibits diminishing returns due to normalization smoothing.

Figure 6: Extrapolated running time for ellipsoid extraction, showing infeasibility for large models.

Cryptographic Analogy: Message Authentication Codes

The forgery resistance and ease of verification of ellipse signatures enable a protocol analogous to symmetric-key message authentication codes (MACs). Here, the model ellipse acts as the secret key, and logprobs are the message-tag pairs. Verification consists of checking whether the logprobs lie on the secret ellipse.

Figure 7: Comparison of traditional MAC systems and the proposed ellipse-based verification system for LLMs.

This protocol allows third-party verification of model outputs without revealing model parameters or inputs, supporting forensic analysis, regulatory compliance, and accountability.

Implementation Details

The paper provides a detailed algorithm for extracting ellipse parameters from model outputs, leveraging ellipsoid-specific fitting methods (e.g., semidefinite programming via cvxpy and the mosek solver). The approach is robust to normalization smoothing for sufficiently large models, as the distribution of hidden state norms is tightly concentrated near 1.

Figure 8: Overview of intermediate representations and parameter recovery in the final LM layers.

Figure 9: LayerNorm output space and dimensionality reduction via isometric transforms.

Figure 10: Model representations and mappings for LMs with LayerNorm, showing the $d-1$ dimensional ellipse.

Figure 11: Distribution of L2 norms of hidden states for small and large models, illustrating the effect of normalization smoothing.

Comparison to Existing Methods

Ellipse signatures differ fundamentally from text-based watermarks, backdoor fingerprints, and linear signatures. They are:

• Naturally Occurring: Present in all models with final normalization layers.
• Self-Contained: Verifiable without access to model inputs or full weights.
• Compact and Redundant: Detectable in any single logprob output.
• Forgery-Resistant: Extraction and forgery are computationally infeasible for large models.

Existing methods either require intentional implementation, access to multiple outputs, or are vulnerable to forgery via API extraction.

Practical and Theoretical Implications

Ellipse signatures provide a new tool for model forensics, enabling robust identification of model outputs even in closed-weight, API-protected settings. The protocol supports regulatory and accountability frameworks by allowing trusted third parties to verify the provenance of outputs. However, the security guarantees are polynomial rather than cryptographic, and the protocol requires access to logprobs, which is not universally available.

Future work may focus on identifying stronger constraints with cryptographic hardness, extending signature protocols to settings with limited API access, and developing signatures that are difficult to remove or modify.

Conclusion

Ellipse signatures constitute a robust, naturally occurring, and forgery-resistant method for associating outputs with their generating LLMs. Their unique combination of properties fills a previously unaddressed niche in the landscape of output verification systems. While practical limitations exist for large-scale models, the theoretical framework and empirical results presented in this paper lay the groundwork for future research in model forensics, security, and accountability.