PromptPET: Privacy-Utility Optimized Prompt Obfuscation

Abstract: Privacy is an important challenge when users interact with AI chatbots, since users may share sensitive information, explicitly or implicitly, and AI chatbots can use this information for user profiling. In this paper, we aim to protect user privacy via a user-side mechanism that transforms sensitive information in a user prompt, while preserving enough information to elicit a useful response from the chatbot. This approach faces an inherent tradeoff between protecting privacy (i.e., avoiding profiling) and preserving utility (i.e., getting personalized and task-specific responses). To that end, we consider, evaluate, and compare four different obfuscation actions, namely redaction, abstraction, replacement, and a novel noising/denoising scheme that we introduce. Additional novel insights include: utilizing a data type taxonomy to both identify and obfuscate sensitive information and explicitly taking into account the utility of chat responses in making the obfuscation decision. First, we systematically optimize and evaluate each obfuscation action independently in terms of the privacy-utility tradeoff it achieves. Second, we propose PROMPTPET, an LLM-based agent that selects the best obfuscation action for each sensitive part of the prompt, using a reinforcement-learning inspired rule optimizer, applied for the first time in this context. Using a real-world chat dataset, we show that PROMPTPET matches the best privacy-utility tradeoff attainable by any single obfuscation action and significantly outperforms prior state-of-the-art approaches.

• The paper presents a principled framework for prompt obfuscation that balances privacy protection with response utility in LLM-based systems.
• It leverages a three-stage process—sensitive unit detection, importance scoring, and learned action decisions using dual-agent reinforcement learning—to determine optimal obfuscation actions.
• Empirical evaluations show up to 3.3× more privacy per utility cost and a dominant performance over baselines, affirming its robust, adaptive privacy-preserving strategy.

Privacy-Utility Optimized Prompt Obfuscation: An Authoritative Analysis of PromptPET

Motivation and Problem Definition

PromptPET addresses the critical challenge of privacy in user–AI agent interactions. As LLM-based chatbots increasingly serve both personal and professional tasks, user prompts frequently contain sensitive information. Providers leverage these prompts for profiling, often inferring fine-grained attributes from non-PII data using taxonomies such as IAB Audience Taxonomy. Conventional server-side privacy controls are insufficient, necessitating user-side mechanisms that obfuscate disclosure while maintaining response utility. PromptPET proposes a principled strategy to optimize the privacy–utility tradeoff via prompt transformations executed prior to provider exposure.

Figure 1: PromptPET architecture showing user-side interception and obfuscation of sensitive prompt components before exposing queries to AI agents.

Methodological Framework

PromptPET decomposes the obfuscation process into three key stages: sensitive unit detection, importance estimation, and a learned action decider for prompt transformation.

Figure 2: The end-to-end PromptPET pipeline, including detection, scoring, decision, query transformation, and post-processing.

Sensitive Unit Detection and Importance Scoring

Sensitive units—atomic fragments within the prompt mapped to hierarchical data types—are detected via instruction-tuned LLMs. Each unit is scored for its contribution to response utility, enabling selective obfuscation. Empirical analysis reveals a right-skewed importance score distribution, with the majority of sensitive content highly relevant for agent responses.

Figure 3: Empirical CDF of sensitive unit importance scores, demonstrating that most units are of high salience.

Obfuscation Actions

PromptPET defines four transformation actions:

• Redact: Remove the unit, yielding maximal privacy but destroying utility for task-critical content.
• Abstract: Generalize to higher taxonomy nodes, reducing specificity but retaining coarse context.
• Replace: Substitute with plausible same-category alternatives, misleading profile inference but potentially distorting intent.
• Noise: Inject contextually plausible decoy values alongside the original unit, preserving utility while obscuring exact user attributes. Decoys are filtered from responses via lightweight denoising.

  Figure 4: Noise action workflow: sensitive unit pairing, decoy generation, and prompt template expansion for multi-option queries.

Rule Optimization: Reinforcement Learning via Dual-Agent LLMs

Action selection is governed by a decision rule set, learned iteratively by two cooperating LLM agents: the Obfuscation Action Decider and the Rule Optimizer. The optimizer refines rule sets based on privacy–utility performance feedback. Optimization uses an objective function $$\alpha \cdot \text{privacy} + (1 - \alpha) \cdot \text{utility}$$ penalized for verbosity, favoring compact and interpretable rule sets.

Figure 5: Rule optimization loop, illustrating feedback-driven learning of robust and adaptive obfuscation policy sets.

Empirical Evaluation

Evaluation leverages WildChat-1M: real user-chatbot logs filtered for prompts containing sensitive content. PromptPET’s effectiveness is validated in two experimental axes: single-action tradeoff analysis and adaptive rule optimization.

Privacy–Utility Tradeoff Analysis

Each obfuscation action is systematically evaluated at varying thresholds of unit importance. Noise delivers superior privacy–utility tradeoffs, achieving a $0.77$ mean privacy and $0.76$ mean utility at maximal strength ($\tau=1.0$), with $73.7\%$ of queries landing in the high-privacy/high-utility region.

Figure 6: Per-query distribution of privacy–utility outcomes for different obfuscation actions and thresholds.

Figure 7: Aggregate privacy–utility tradeoff curves across all actions; Noise consistently forms the upper envelope except at low privacy.

Single-action upper bounds empirically confirm that no single transformation is universally optimal; per-query analysis indicates that Noise is optimal for $58.3\%$ of queries under full obfuscation, but removal/generalization actions are preferred for others.

Adaptive Rule-Based Obfuscation

PromptPET’s rule optimizer produces a policy set that selectively applies each action to maximize the tradeoff frontier. The deployed system predominantly assigns Noise ($84.8\%$), with Abstract and Redact supplementing for low-importance units or peripheral labels. The learned rule set ensures the final system tracks the best single-action performance at every privacy level and consistently outperforms prior state-of-the-art baselines.

Figure 8: PromptPET’s tradeoff curve (orange) against prior baselines and single-action upper bounds, establishing dominance in both privacy and utility.

Strong Numerical Claims

PromptPET delivers 3.3× more privacy per unit utility cost and achieves 1.4× higher privacy overall relative to the strongest prior baseline, with a $0.78$ privacy score at reasonable utility, clearly surpassing both LLM-based minimization and contextual reformulation methods.

Failure Modes and Practical Implications

PromptPET’s residual privacy-miss and utility-collapse modes are attributable to incomplete sensitive unit detection (especially for implicit or contextual signals) and occasional destructive Redact application to high-importance units. These are rare ($1.1\%$ simultaneous failure) and suggest clear avenues for future enhancements: improved contextual detection and more precise utility-aware rule refinement.

Client-side latency (median $11.6$\,s per query) is dominated by local LLM-based detection, but is opt-in and can be mitigated via model compression or specialization.

Theoretical Implications and Future Directions

PromptPET establishes a foundation for systematic, taxonomy-driven privacy preservation in NLP systems. Its reinforcement learning-inspired rule optimization demonstrates policy learning as a paradigm for privacy–utility control. The noise injection/denoising methodology is novel in this domain, providing both robust protection and utility preservation unmatched by earlier minimization or redaction approaches.

These insights motivate further research in multi-turn agentic interactions, longitudinal contextual signal detection, and more granular privacy quantification beyond taxonomy-driven exposure metrics.

Conclusion

PromptPET introduces a unified framework for user-side prompt obfuscation, optimizing the privacy–utility tradeoff across a diverse taxonomy of sensitive information. Through systematic evaluation and adaptive action selection, it matches the empirical upper bounds of single-action strategies and decisively outperforms previous state-of-the-art methods, achieving substantial privacy gains with minimal utility degradation. This research advances practical privacy protection for LLM-based agents, indicating that user-side disclosure control—when guided by learned, interpretable policies—can provide robust resistance to profiling in evolving AI platforms (2607.02932).