AloePri: Privacy-Preserving LLM Inference

• AloePri is a privacy-preserving large language model inference method that obfuscates both input and output data through coordinated transformations.
• It employs covariant obfuscation to jointly transform data and model parameters, ensuring negligible accuracy loss and efficient operation in standard LMaaS settings.
• The method demonstrates strong resistance to inversion and substitution attacks while integrating seamlessly with heterogeneous xPU clusters and existing serving optimizations.

AloePri is a privacy-preserving LLM inference method for industrial applications that operates in the standard LLM as a Service setting. It protects both input and output data by covariant obfuscation, a scheme in which data and model parameters are jointly transformed so that the server performs inference in an obfuscated domain while the client retains the ability to recover plaintext outputs. The method is designed to minimize accuracy and efficiency losses, to run on large-scale clusters of heterogeneous legacy xPUs, and to remain compatible with existing LLM infrastructures such as vLLM and SGLang. In current arXiv usage, AloePri refers to the method introduced in “Towards Privacy-Preserving LLM Inference via Collaborative Obfuscation” (Lin et al., 2 Mar 2026); the exact string “AloePri” does not appear in “Aloe: A Family of Fine-tuned Open Healthcare LLMs” (Gururajan et al., 2024).

1. Deployment setting and design objectives

AloePri targets cloud-hosted LLM inference in which a client sends a prompt to a remote server and receives a generated response. The motivating concern is that prompts can contain sensitive personal, medical, financial, or proprietary data, while responses may also reveal confidential information. In addition, internal model states such as hidden states, attention scores, and the KV-cache are typically resident in accelerator memory and are observable to a curious cloud provider or to an attacker with access to the runtime (Lin et al., 2 Mar 2026).

The method is explicitly framed around three industrial constraints. First, accuracy and efficiency degradation should be negligible relative to plaintext inference, excluding both large quality drops and substantial latency overhead. Second, the system should run on existing heterogeneous xPU clusters without secure-hardware assumptions. Third, it should preserve the standard LMaaS interface—effectively token in, token out—so that existing serving optimizations such as KV cache management, tensor parallelism, and P/D disaggregation remain usable. This positioning is central to the report’s claim that cryptographic approaches, trusted-execution approaches, and earlier obfuscation schemes do not simultaneously satisfy all three constraints in practical industrial deployments (Lin et al., 2 Mar 2026).

The protected objects in the base setting are the input prompt tokens, the output tokens, and internal states correlated with user data. By contrast, model weights are not protected from the server in the base configuration, although the report discusses possible extensions that would combine covariant obfuscation with FHE or TEE mechanisms in scenarios where model confidentiality is also required. This suggests that AloePri is primarily a data-privacy mechanism for remote inference rather than a general secure-computation framework.

2. Threat model and adversarial capabilities

AloePri assumes an honest-but-curious server. The server is expected to execute the designated model and return outputs correctly, but it may inspect obfuscated inputs and outputs, full obfuscated weights, and all intermediate states. The report also considers external attackers who can obtain the same information through service compromise or memory dumping (Lin et al., 2 Mar 2026).

The adversary’s capabilities are organized into several attack classes. One class consists of obfuscation-recovery attacks that exploit algebraic structure, including Vocabulary-Matching Attack, which attempts to recover permutation-like transformations by matching sorted matrix statistics, and Invariant Attacks, which exploit invariants of linear blocks such as gate matrices or Q/K structures. A second class consists of training-based inversion attacks: Internal State Attack optimizes new inputs to reproduce observed hidden states; Inversion Model Attack trains a model to map obfuscated embeddings back to plaintext embeddings or tokens; and nearest-neighbor inversion matches obfuscated embeddings to a reference corpus. A third class consists of token-frequency attacks such as Token Frequency Matching Attack and Substitution Deciphering Attack, which treat the transformation as a substitution cipher and exploit token-frequency regularities (Lin et al., 2 Mar 2026).

Formally, the central secret is a token permutation $\tau \in S_n$. Prompts and responses are obfuscated by applying $\tau$ to token indices, and the model weights are transformed to remain compatible with that permutation. The server therefore observes only $\tau x$, $\tau y$, obfuscated weights, and obfuscated internal states. Privacy is quantified through information leakage, expressed as mutual information $\mathcal{I}(\tau;\text{observations})$, and through attack success rates interpreted in a PAC-privacy framework. The report is explicit that the target is practical privacy against strong but constrained attackers, not unconditional cryptographic security (Lin et al., 2 Mar 2026).

3. Covariant obfuscation as the underlying framework

The conceptual core of AloePri is covariant obfuscation. Let $f : X \times \Theta \to Y$ denote the inference function of an LLM, with $X$ the token-input space, $\Theta$ the parameter space, and $Y$ the output space. A covariant obfuscation is defined as a quintuple

$(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$

where $\tau$0 obfuscates data, $\tau$1 transforms parameters, $\tau$2 obfuscates outputs, $\tau$3 de-obfuscates outputs, and $\tau$4 is inference in the obfuscated space. The defining conditions are a commutation condition,

$\tau$5

and a de-obfuscation condition,

$\tau$6

Data-only obfuscation appears as the special case in which model and output transformations are identities (Lin et al., 2 Mar 2026).

The report’s theoretical claim is that, for any data-only obfuscation $\tau$7, there exists a covariant obfuscation $\tau$8 with the same obfuscation error $\tau$9 but strictly smaller information leakage, $\tau x$0. The significance of this theorem is not that it yields cryptographic secrecy, but that it formalizes the intuition that coordinated transformations of data and parameters produce a better privacy–utility tradeoff than perturbing inputs alone (Lin et al., 2 Mar 2026).

AloePri instantiates this framework by choosing

$\tau x$1

together with a model transformation $\tau x$2 that permutes, perturbs, and linearly transforms the embedding layer, output head, attention blocks, feed-forward layers, and normalization-residual structure. The objective is to realize

$\tau x$3

with small error $\tau x$4. This places the entire inference computation into an obfuscated coordinate system while preserving functional behavior up to bounded approximation error (Lin et al., 2 Mar 2026).

4. Algorithmic construction and inference protocol

AloePri consists of an offline model-obfuscation phase and an online inference phase. Offline, the client starts from a plaintext model, generates the secret permutation $\tau x$5 together with key matrices and their inverses, transforms all weight matrices to produce $\tau x$6, uploads $\tau x$7 to the server, and retains $\tau x$8 and the seeds that generate the key material locally. Online, the client tokenizes the prompt, maps token IDs through $\tau x$9, detokenizes the result back into an obfuscated string, and sends that string through a normal LMaaS API. The server tokenizes this obfuscated text under the obfuscated vocabulary, runs ordinary autoregressive inference with the obfuscated model, and returns an obfuscated response string. The client then applies $\tau y$0 and detokenizes to recover the plaintext output. The report emphasizes that this requires no extra online interaction and no deviation from standard text-based APIs (Lin et al., 2 Mar 2026).

The algebraic glue across layers is provided by key matrices $\tau y$1 and $\tau y$2, constructed so that $\tau y$3. These matrices expand the hidden-state representation from dimension $\tau y$4 to $\tau y$5, with $\tau y$6 a small expansion size such as $\tau y$7, while preserving functional correctness through cancellation across adjacent linear maps. The report describes a concrete generation scheme based on a near-orthogonal base matrix $\tau y$8, low-rank expansion matrices $\tau y$9 and $\mathcal{I}(\tau;\text{observations})$0, and an orthogonal matrix $\mathcal{I}(\tau;\text{observations})$1, yielding

$\mathcal{I}(\tau;\text{observations})$2

and

$\mathcal{I}(\tau;\text{observations})$3

These matrices are not shared with the server; only their effects, absorbed into weight tensors, are present in the deployed model (Lin et al., 2 Mar 2026).

The embedding and output head are transformed through Gaussian noise, token permutation, and key-matrix multiplication. With vocabulary size $\mathcal{I}(\tau;\text{observations})$4, embedding matrix $\mathcal{I}(\tau;\text{observations})$5, and output head $\mathcal{I}(\tau;\text{observations})$6, AloePri first perturbs both matrices by Gaussian noise scaled by coefficients such as $\mathcal{I}(\tau;\text{observations})$7 and $\mathcal{I}(\tau;\text{observations})$8, then applies the permutation matrix $\mathcal{I}(\tau;\text{observations})$9 corresponding to $f : X \times \Theta \to Y$0, and finally maps into and out of the expanded hidden space:

$f : X \times \Theta \to Y$1

$f : X \times \Theta \to Y$2

The stated purpose is to break the exact algebraic relationships exploited by vocabulary-matching and invariant-based attacks while preserving inference semantics (Lin et al., 2 Mar 2026).

Attention layers are obfuscated through intra-head and inter-head transformations. Intra-head transformations include RoPE-compatible $f : X \times \Theta \to Y$3 rotations, per-block scalings, block permutations, value-output coupling matrices, and key matrices applied to the $f : X \times \Theta \to Y$4, $f : X \times \Theta \to Y$5, $f : X \times \Theta \to Y$6, and $f : X \times \Theta \to Y$7 projections. Inter-head transformations permute K/V heads and groups of Q/O heads. The report states that this design covers standard attention variants including MHA, MQA, GQA, and MLA. Feed-forward layers are transformed by per-FFN permutations, diagonal scalings, and key matrices; for dense FFNs the transformed weights satisfy an exact functional relation in the obfuscated space, and for MoE models the router is also normalized, permuted, and transformed so that expert identities are hidden while routing semantics are maintained. RMSNorm is approximated in the expanded space using a scale factor $f : X \times \Theta \to Y$8 derived from the expected norm change under the hidden-state transformation, with the normalization weights fused into adjacent linear layers so that the runtime operator graph remains unchanged. Residual connections are handled by a summation composition theorem, and the entire layer stack by sequential and parallel composition theorems, producing an obfuscation of the full model (Lin et al., 2 Mar 2026).

Component	Transformation	Intended effect
Embedding and head	Noise, token permutation, key matrices	Break row/column matching and preserve token-level compatibility
Attention	RoPE-compatible rotations, scalings, block permutations, head permutations, key matrices	Hide Q/K/V/O structure while maintaining attention computation
FFN and router	Per-layer permutations, scalings, key matrices, expert permutation	Obfuscate nonlinear blocks and expert identities with functional equivalence

5. Privacy guarantees and empirical resistance to attack

The report develops both information-theoretic and empirical security analyses. For information leakage, AloePri models the action of the token-permutation group $f : X \times \Theta \to Y$9 on data and outputs, together with a parameter-transformation group $X$0 acting on the weights. It then bounds the mutual information

$X$1

by a sum of static leakage from obfuscated weights and dynamic leakage from observed obfuscated inputs. For static leakage, Theorem 5.1 gives an upper bound involving the obfuscated parameter covariance and a KL term related to decoding success for the parameter group action. The report’s interpretation is that stronger noise and more group averaging decrease static mutual information. It also gives PAC-style bounds on the probability of recovering an entry of the token permutation from static observations, with the upper bound depending on a matrix $X$2 formed from the embedding and head weights and their noise scales $X$3 (Lin et al., 2 Mar 2026).

The empirical security analysis focuses on the attack classes defined in the threat model. Under Vocabulary-Matching Attack, AloePri’s Text Token Recovery Success Ratio is reported as typically under $X$4–$X$5, often much lower on large models, with $X$6 on Deepseek-V3.1; PII recovery success ratio on PUPA is reported as at most $X$7 across models. Against Invariant Attacks, an ablation study shows that noise alone remains insufficient because Internal State Attack can still recover $X$8–$X$9 of tokens in internal states; adding key matrices reduces hidden-state leakage to $\Theta$0 TTRSR but still leaves attention leakage; and adding head and block permutations reduces ISA TTRSR to $\Theta$1 on both attention scores and hidden states. On Qwen2.5-14B, AloePri yields $\Theta$2 TTRSR for nearest-neighbor inversion, Inversion Model Attack, and Internal State Attack on inputs, while invariant-based recovery obtains about $\Theta$3 tokens with cosine similarity around $\Theta$4, which the report characterizes as low semantic fidelity. By contrast, DP-Forward recovers $\Theta$5 of tokens under IMA and SGT recovers $\Theta$6 under IMA in the same evaluation setting (Lin et al., 2 Mar 2026).

Frequency-based attacks are treated separately because the client–server protocol superficially resembles a substitution cipher. Under Token Frequency Matching Attack, Top-10 and Top-100 recoverable tokens are reported as at most $\Theta$7 and $\Theta$8 in the zero-knowledge setting, and $\Theta$9 and $Y$0 in a distribution-aware setting using the same dataset family. Substitution Deciphering Attack achieves BLEU-4 at most $Y$1, which the report interprets as far from meaningful text recovery. CLUB-based simulations of mutual information between plaintext and obfuscated embeddings further place AloePri below SANTEXT, RANTEXT, DP-Forward, and SGT on estimated mutual information, supporting the claim that covariant obfuscation offers a better privacy–accuracy tradeoff than data-only methods (Lin et al., 2 Mar 2026).

Attack class	AloePri outcome
Vocabulary-Matching Attack	TTRSR typically under $Y$2–$Y$3; $Y$4 on Deepseek-V3.1
Training-based inversion	On Qwen2.5-14B, NN, IMA, and ISA recover $Y$5 of input tokens
Frequency-based attacks	TFMA recovery remains low; SDA BLEU-4 is at most $Y$6

6. Accuracy, efficiency, and systems compatibility

AloePri is evaluated on dense models including Qwen2.5 and Qwen3, Llama3-8B, Deepseek-R1-Distill-14B and -32B, on the MoE model Qwen3-MoE-30B-A3B, and on Deepseek-V3.1-Terminus with $Y$7B parameters. Benchmarks include MMLU, C-Eval, HumanEval, PIQA, IFEval, SST2 with LoRA-tuning, and PUPA. Across models and tasks, the reported accuracy loss is $Y$8–$Y$9. For Deepseek-V3.1-Terminus, plaintext versus AloePri results are MMLU $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$0, C-Eval $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$1, HumanEval $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$2, PIQA $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$3, and IFEval $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$4. For Qwen2.5-14B, MMLU changes from $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$5 to $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$6, PIQA from $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$7 to $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$8, IFEval from $(\phi_X, \phi_\Theta, \phi_Y, \psi_Y, \tilde f),$9 to $\tau$00, and SST2 from $\tau$01 to $\tau$02. The report contrasts these results with SANTEXT, RANTEXT, DP-Forward, and SGT, which either exhibit much larger performance loss or remain highly invertible under attack (Lin et al., 2 Mar 2026).

The offline obfuscation phase incurs a one-time client-side cost. On a single CPU host, reported runtimes are approximately $\tau$03 minutes for R1-Distill-14B, $\tau$04 minutes for R1-Distill-32B, $\tau$05 minutes for Qwen3-MoE-30B-A3B, and about $\tau$06 minutes, or roughly $\tau$07 hours, for Deepseek-V3.1-Terminus. The report presents this as acceptable for deployment because obfuscation is performed once per model rather than once per inference request (Lin et al., 2 Mar 2026).

Online overhead is reported as negligible. On vLLM, Time To First Token and Time Per Output Token differ from plaintext by only a few milliseconds. For R1-Distill-14B at concurrency $\tau$08, plaintext TTFT and TPOT are $\tau$09 ms and $\tau$10 ms, versus $\tau$11 ms and $\tau$12 ms for AloePri. For Deepseek-V3.1-Terminus at concurrency $\tau$13, plaintext TTFT and TPOT are $\tau$14 ms and $\tau$15 ms, versus $\tau$16 ms and $\tau$17 ms for AloePri. Increasing the expansion size $\tau$18 to $\tau$19 increases TPOT by less than $\tau$20, and the report attributes this to the fact that only standard floating-point matrix multiplications are introduced and asymptotic complexity is unchanged (Lin et al., 2 Mar 2026).

The systems argument is that AloePri functions as a drop-in model. It was integrated with an industrial LMaaS system using vLLM $\tau$21 and CUDA $\tau$22 across multi-node GPU clusters, with no required changes to PagedAttention, KV-cache management, distributed tensor parallelism, batching logic, or request APIs. This supports the paper’s claim that AloePri satisfies the industrial triad of accuracy, efficiency, and compatibility at scales up to $\tau$23B parameters (Lin et al., 2 Mar 2026).

7. Relation to prior approaches, limitations, and open problems

The report positions AloePri against three broad families of prior work. Relative to MPC- and FHE-based systems such as SIGMA, Iron, Secure Transformer Inference, BumbleBee, and MPCFormer, AloePri does not provide cryptographic security in the online path, but avoids the extreme runtime overheads and incompatibilities with streaming, KV caching, and dynamic batching that the report attributes to those methods. Relative to TEE-based approaches such as PipeLLM, ccAI, Slalom, GOTen, SOTER, and GroupCover, it requires no trusted hardware and no frequent CPU–GPU transfers. Relative to obfuscation and DP-style schemes such as SANTEXT, RANTEXT, ProSan, SGT, DP-Forward, STIP, PrivacyRestore, and ObfusLM, AloePri preserves the normal text-based API and is presented as substantially more resistant to inversion and structure-recovery attacks. The report’s claim that AloePri is the first method with practical applicability is explicitly tied to three properties: less than $\tau$24 accuracy loss with effectively unchanged latency, support for standard CUDA GPU clusters and heterogeneous legacy xPUs, and direct compatibility with infrastructures such as vLLM and SGLang (Lin et al., 2 Mar 2026).

The limitations are equally explicit. AloePri assumes an honest-but-curious server rather than an actively malicious one, and it does not address active tampering with the model pipeline. The client must securely store $\tau$25 and the seeds used to generate key matrices; compromise of the client leaks all historical and future prompts for that deployment. Side-channel leakage, including timing, memory-access patterns, and microarchitectural channels, is out of scope. Access-pattern leakage such as expert-selection behavior is not explicitly masked. The dynamic-leakage term $\tau$26 is not tightly characterized for long-lived deployments, so the theoretical account of repeated usage remains incomplete. In the base setting, AloePri protects user data but not model intellectual property from the client; extending protection to server-only models would require combining the method with FHE or TEE mechanisms. Finally, the empirical evaluation concerns attacks that are practically implementable rather than ideal adversaries with unbounded compute (Lin et al., 2 Mar 2026).

The open problems identified in the report follow from these limitations. They include stronger formal security models for dynamic leakage over full interaction transcripts, extension of covariant obfuscation to multimodal LLMs and other neural architectures, joint model-IP and data privacy through combinations with FHE or GPU-TEE, adaptive selection of hyperparameters such as $\tau$27, $\tau$28, $\tau$29, and $\tau$30, and the addition of side-channel defenses. A plausible implication is that AloePri should be understood as a systems-oriented privacy mechanism that closes a practical deployment gap, while leaving several classical secure-inference questions unresolved.