TRACE-RPS: Defense Against Attribute Inference

• TRACE-RPS is a defense framework that minimizes attribute inference by using targeted anonymization (TRACE) and suffix optimization (RPS) in a two-stage process.
• TRACE employs attention-based privacy vocabulary extraction and inference chain generation to selectively anonymize sensitive text while preserving meaning.
• RPS uses discrete, inference-time suffix optimization to steer LLM responses toward refusal or misattribution, reducing inference accuracy from around 50% to below 5%.

TRACE-RPS is a proactive, user-side defense framework against attribute inference attacks on LLMs. It combines TRACE—Textual Revision via Attention and Chain-based Editing—with RPS—Rejection-Oriented Perturbation Search—to address two distinct failure modes of privacy protection: residual attribute leakage after anonymization, and continued inference from benign-looking prompts even when the text has been edited. In the framework’s formulation, user text is first selectively anonymized at word and reasoning-chain level, then optionally augmented with an optimized suffix that steers the target model toward refusal behavior under attribute-inference prompts (Yan et al., 12 Feb 2026).

1. Threat model and formal problem

TRACE-RPS is defined for attribute inference attacks in which an adversary queries a frozen LLM with ordinary user-generated text and asks it to infer sensitive attributes such as age, gender, location, place of birth, income, occupation, education, and relationship status. The paper formalizes the dataset as

$D=\{(u_i,t_i)\},$

where $u_i$ is a user and $t_i$ is the associated text, and the target attribute set as

$A=\{a_1,a_2,\dots,a_k\}.$

Each user has an attribute vector

$y_u=(y_u^1,y_u^2,\dots,y_u^k).$

The attacker uses a prompt template

$$P(t)=\text{Prefix}+F_{\text{fmt}}(t)+\text{Suffix}, \tag{1}$$

and the model outputs inferred attributes

$$\hat{A}_u=M(P(t))=\{(a_j,\hat{y}_u^j)\}_{j=1}^k. \tag{2}$$

The attacker’s objective is to maximize

$$\arg\max \; \mathbb{E}_{(u,t)\sim D}\sum_{j=1}^{k}\mathbb{I}[\hat{y}_u^j=y_u^j]. \tag{3}$$

The framework is motivated by two limitations of prior anonymization defenses. First, they are described as coarse-grained, because they do not localize the precise words or phrases driving inference. Second, anonymization alone does not prevent an LLM from inferring attributes from residual contextual evidence through its own reasoning. TRACE-RPS therefore treats defense as both a text transformation problem and a model-behavior suppression problem (Yan et al., 12 Feb 2026).

The defended text is written as

$\bar t = T_{\text{def}}(t), \tag{4}$

and the defense objective is

$$\arg\min \; \mathbb{E}_{(u,t)\sim D}\sum_{j=1}^{k}\mathbb{I}[\hat y_u^j=y_u^j]-\lambda \, \mathbb{I}[\hat y_u^j=\text{Reject}], \tag{5}$$

where $u_i$0 weights the value of inducing refusal. A defense succeeds if the model either predicts the wrong attribute or emits a refusal output.

2. TRACE: fine-grained anonymization through attention and inference chains

TRACE is the first stage of the framework. Its function is to identify privacy-leaking textual elements at fine granularity and rewrite only those segments most responsible for attribute inference. The method combines two signals: privacy vocabulary extraction and privacy inference chain generation (Yan et al., 12 Feb 2026).

For privacy vocabulary extraction, TRACE constructs an attribute-specific prompt

$u_i$1

where $u_i$2 is an attribute query. A pretrained causal LLM processes this prompt, and TRACE uses the final-layer attention vector corresponding to the last token as a proxy for token contribution to inference. If the tokenized text is

$u_i$3

with attention scores

$u_i$4

then for a word $u_i$5 composed of tokens $u_i$6, TRACE defines

$u_i$7

The privacy vocabulary is the top-$u_i$8 set

$u_i$9

The implementation uses $t_i$0 on the Synthetic dataset and $t_i$1 on SynthPAI.

Attention alone is not treated as sufficient. TRACE also prompts an adversarial model to generate an explicit Inference Chain: a step-by-step explanation of how the attribute can be inferred from the text, with cited evidence spans and reasoning. This chain is intended to expose semantic dependencies that high-attention words may not fully capture.

The final anonymization stage is guided jointly by the privacy vocabulary $t_i$2 and the inference chain $t_i$3: $t_i$4 Operationally, the anonymizer is instructed to change as little as possible, preserve meaning, and only generalize, not invent new facts. The paper gives “my husband and I” $t_i$5 “my partner and I” as a valid edit, and “my husband and I” $t_i$6 “my wife and I” as invalid. TRACE therefore functions as targeted semantics-preserving generalization rather than unconstrained paraphrase.

TRACE is iterative. It stops when attacker confidence drops below 2, when the text no longer changes between iterations, or when 5 iterations have been reached. The paper’s ablation on the Synthetic dataset reports Top@1 inference accuracy of 26.86 with both vocabulary extraction and inference chain enabled, compared with 31.43 when vocabulary extraction is removed, 31.62 when inference chain is removed, and 37.14 when both are removed, indicating that the two signals are complementary (Yan et al., 12 Feb 2026).

3. RPS and MPS: inference-time refusal and misattribution by suffix optimization

RPS is the second stage of TRACE-RPS and is the framework’s explicit inference-prevention component. It does not rewrite the original content. Instead, it appends an optimized discrete suffix $t_i$7 to the text so that, under an attribute-inference prompt, the model begins its response with a refusal pattern (Yan et al., 12 Feb 2026).

If the defended text is

$t_i$8

RPS optimizes the first two generated tokens. Its scoring function is

$t_i$9

where $A=\{a_1,a_2,\dots,a_k\}.$0 is the rejection-token set and $A=\{a_1,a_2,\dots,a_k\}.$1. In experiments,

$A=\{a_1,a_2,\dots,a_k\}.$2

Stage 1 anchors the first token: $A=\{a_1,a_2,\dots,a_k\}.$3 Starting from an initial suffix $A=\{a_1,a_2,\dots,a_k\}.$4, the algorithm mutates a contiguous token span with random replacements and keeps the new suffix if $A=\{a_1,a_2,\dots,a_k\}.$5 improves. Stage 1 stops when

$A=\{a_1,a_2,\dots,a_k\}.$6

is reached or the iteration budget is exhausted.

Stage 2 shapes the second token toward refusal: $A=\{a_1,a_2,\dots,a_k\}.$7 The full objective becomes

$A=\{a_1,a_2,\dots,a_k\}.$8

and the final suffix is

$A=\{a_1,a_2,\dots,a_k\}.$9

Stage 2 stops when

$y_u=(y_u^1,y_u^2,\dots,y_u^k).$0

is reached or the iteration budget is exhausted.

The paper emphasizes that RPS is lightweight relative to longer prompt-optimization schemes because each candidate suffix is scored using only one or two generated tokens and their log-probabilities. It does not fine-tune model weights, use soft prompts, or require gradient-based optimization. It is a discrete, test-time input-optimization method.

Because some highly instruction-following models remain answer-generative even after refusal-oriented optimization, the paper introduces MPS—Misattribute-Oriented Perturbation Search—as a fallback. Instead of inducing refusal, MPS optimizes toward an incorrect target attribute: $y_u=(y_u^1,y_u^2,\dots,y_u^k).$1 where $y_u=(y_u^1,y_u^2,\dots,y_u^k).$2 is a chosen wrong value for attribute $y_u=(y_u^1,y_u^2,\dots,y_u^k).$3. In experiments, MPS uses 500 iterations. The paper presents MPS chiefly for Qwen2.5-7B-Instruct, where refusal induction is harder (Yan et al., 12 Feb 2026).

4. End-to-end workflow, implementation assumptions, and access regimes

TRACE-RPS is a sequential framework: $y_u=(y_u^1,y_u^2,\dots,y_u^k).$4 The paper also allows $y_u=(y_u^1,y_u^2,\dots,y_u^k).$5 when RPS is applied without prior TRACE anonymization. The two stages are not jointly trained and are not embedded in an outer optimization loop.

The workflow is as follows. First, raw user text $y_u=(y_u^1,y_u^2,\dots,y_u^k).$6 is processed by TRACE. This includes constructing an attribute-inference query, extracting attention-based privacy vocabulary, generating an inference chain, and producing a minimally edited anonymized text $y_u=(y_u^1,y_u^2,\dots,y_u^k).$7. Second, if the target model is accessible at the logit level, RPS optimizes a suffix $y_u=(y_u^1,y_u^2,\dots,y_u^k).$8 through two-stage random search and outputs $y_u=(y_u^1,y_u^2,\dots,y_u^k).$9. The resulting defended text is what gets published or shared (Yan et al., 12 Feb 2026).

The framework is deliberately hybrid with respect to model access.

TRACE is broadly usable, including for closed-source models, because it relies on an anonymizer and an attention model for vocabulary extraction rather than logit access to the target inference model. In the reported implementation, the attention extraction model is Llama2-7B-Chat, and the anonymization model is GPT-3.5-Turbo or GPT-4o.

RPS, by contrast, requires access to model logits during optimization and is therefore mainly applicable to open-source / white-box or semi-white-box settings. The optimization model is the same as the target inference model. All reported decoding is deterministic: temperature 0 and greedy decoding. RPS uses 10,000 iterations, batch size 1, and one or two NVIDIA RTX 3090 GPUs. The rejection tokens, thresholds, and $$P(t)=\text{Prefix}+F_{\text{fmt}}(t)+\text{Suffix}, \tag{1}$$0 are fixed across experiments (Yan et al., 12 Feb 2026).

This architecture implies an important practical distinction. TRACE can function as a standalone defense when only API access is available. TRACE-RPS, in the full sense, is strongest when the attacker-facing model is introspectable enough to permit suffix search.

5. Empirical performance, transfer behavior, and utility

The evaluation uses three datasets: the Synthetic dataset from Staab et al. (2024), SynthPAI from Yukhymenko et al. (2024), and a real-world Reddit dataset constructed following the PersonalReddit methodology. The attacker task is zero-shot chain-of-thought attribute inference from user comments. Baselines include No Defense, Azure Language Services, Dou-SD, D-Defense, and FgAA. Target models include open-source systems—Llama2-7B-Chat, Llama2-13B-Chat, Llama3.1-8B-Instruct, DeepSeek-R1-Distill-Qwen-7B, Qwen2.5-7B-Instruct—and closed-source systems—GPT-3.5-Turbo, GPT-4o, and Gemini 2.5 Pro (Yan et al., 12 Feb 2026).

The headline result is that on open-source models, TRACE-RPS reduces attribute inference accuracy from around 50% to below 5%, often to nearly zero. On the Synthetic dataset, for Llama2-7B-Chat, inference accuracy drops from 53.71 with no defense to 1.71 with RPS, 0.19 with TRACE-RPS using GPT-3.5-Turbo as anonymizer, and 1.52 with TRACE-RPS using GPT-4o as anonymizer. For Llama3.1-8B-Instruct, the same dataset drops from 57.14 to 0 with RPS and 0 with TRACE-RPS. On SynthPAI, Llama2-13B-Chat falls from 44.58 to 0 with RPS and 0.09 with TRACE-RPS using GPT-4o.

For closed-source models, only TRACE is applicable because RPS requires logits. Even so, TRACE materially reduces inference accuracy. On the Synthetic dataset, GPT-4o drops from 71.24 with no defense to 28.38 or 26.67, depending on anonymizer. On SynthPAI, Gemini 2.5 Pro drops from 67.54 to 41.82 or 34.74. These results show that the anonymization stage alone provides measurable privacy protection when optimization-based refusal is unavailable.

The framework also reports strong performance on the stricter Attack Success Rate (ASR) metric,

$$P(t)=\text{Prefix}+F_{\text{fmt}}(t)+\text{Suffix}, \tag{1}$$1

which assumes the attacker may guess randomly after a refusal. The paper further evaluates prompt variation robustness using 100 GPT-4-generated prompt variants, cross-model transfer, and multi-model suffix optimization. It reports that jointly optimized suffixes over Llama2-7B-Chat, Llama2-13B-Chat, and Llama3.1-8B-Instruct generalize well, including to unseen Llama3.2 variants (Yan et al., 12 Feb 2026).

Utility is measured separately for TRACE and RPS. TRACE is assessed with an LLM-based judge over meaning preservation, readability, and hallucination; RPS is assessed with Sentence-BERT semantic similarity using paraphrase-MiniLM-L6-v2. The paper reports that RPS preserves very high semantic similarity because it leaves the original text untouched and only adds a suffix, with values such as 98.17 for Llama2-7B-Chat and 98.27 for DeepSeek-R1-Distill. TRACE introduces a moderate utility cost, but using GPT-4o as anonymizer improves utility substantially, with a reported utility score of 86.65.

6. Limitations, misconceptions, and broader significance

TRACE-RPS is not a training-time privacy model, not a fine-tuning method, and not a generic prompt-injection defense. TRACE is a prompting-based anonymization system; RPS is a discrete, instance-level suffix optimization procedure. This distinction matters because the framework’s strong results do not come from retraining the protected model (Yan et al., 12 Feb 2026).

Several limitations are explicit. RPS requires logit access, so it cannot be directly applied to many closed-source APIs. Model behavior also matters: highly instruction-following systems such as Qwen2.5-7B-Instruct may not reliably refuse, which is why the paper introduces MPS. TRACE can degrade utility because it edits the original text, even though its prompts attempt to minimize semantic change. TRACE also relies on final-layer attention as an attribution signal, and the paper does not claim that attention is a perfect causal explanation of privacy leakage.

The paper evaluates adaptive attacks as well. SuffixDrop-k, which removes trailing characters, and LLMSanitize, which uses GPT-4o to strip suspicious patterns, both reduce RPS effectiveness, although the defended models remain materially better than the no-defense baseline. Another practical limitation is stylistic: appended suffixes may appear anomalous in publicly shared text.

A common misconception is that anonymization alone is sufficient. The framework’s own results argue against that interpretation. TRACE reduces direct and indirect evidence in the text, but low-cardinality attributes such as gender or broad income class can remain inferable under anonymization alone. RPS addresses this by suppressing response generation rather than only rewriting evidence. A complementary misconception is that refusal-inducing suffixes are always enough. The Qwen results show that this is not uniformly true, which is why the paper includes misattribution-based fallback.

In the paper’s formulation, TRACE-RPS is best understood as a two-layer privacy defense: TRACE weakens the evidence surface available to the attacker, and RPS weakens the model’s willingness to operationalize whatever evidence remains. This suggests a broader significance beyond attribute inference alone: the framework treats privacy defense not merely as sanitization of text, but as joint control over what information is exposed and how a model is permitted to reason from it (Yan et al., 12 Feb 2026).