Mask-Free Privacy Extraction and Rewriting: A Domain-Aware Approach via Prototype Learning

Abstract: Client-side privacy rewriting is crucial for deploying LLMs in privacy-sensitive domains. However, existing approaches struggle to balance privacy and utility. Full-text methods often distort context, while span-level approaches rely on impractical manual masks or brittle static dictionaries. Attempts to automate localization via prompt-based LLMs prove unreliable, as they suffer from unstable instruction following that leads to privacy leakage and excessive context scrubbing. To address these limitations, we propose DAMPER (Domain-Aware Mask-free Privacy Extraction and Rewriting). DAMPER operationalizes latent privacy semantics into compact Domain Privacy Prototypes via contrastive learning, enabling precise, autonomous span localization. Furthermore, we introduce a Prototype-Guided Preference Alignment, which leverages learned prototypes as semantic anchors to construct preference pairs, optimizing a domain-compliant rewriting policy without human annotations. At inference time, DAMPER integrates a sampling-based Exponential Mechanism to provide rigorous span-level Differential Privacy (DP) guarantees. Extensive experiments demonstrate that DAMPER significantly outperforms existing baselines, achieving a superior privacy-utility trade-off.

• The paper presents DAMPER, a framework that uses contrastive prototype learning to localize and rewrite sensitive spans without manual masking.
• It employs a dual-stage architecture with offline training and online inference to achieve strong span-level differential privacy while preserving contextual utility.
• Empirical evaluations in medical and legal domains show that DAMPER outperforms existing methods, offering robust privacy protection and enhanced domain adaptability.

Mask-Free Privacy Extraction and Rewriting via Prototype Learning: A Domain-Aware Privacy Architecture

Motivation and Limitations of Existing Privacy Rewriting Frameworks

Client-side privacy rewriting of user queries is essential for deploying LLM-based applications in domains such as healthcare and law, where privacy-sensitive content must be sanitized before transmission to untrusted cloud services. Existing methodologies are fundamentally limited: full-text rewriting with DP mechanisms degrades non-sensitive context and utility, while span-level approaches typically require explicit privacy masks, static dictionaries, or unreliable prompt-based LLM localization, which either demand untenable user effort or exhibit poor precision and recall in open-domain, high-stakes settings.

The key challenges are to (1) localize sensitive spans in a mask-free, domain-adaptive fashion, and (2) privatize only those spans with strong DP guarantees, preserving both privacy and domain utility. This work proposes DAMPER (Domain-Aware Mask-free Privacy Extraction and Rewriting), which leverages contrastively learned privacy prototypes to guide autonomous localization and rewriting—operationalizing domain privacy semantics without human annotation or static mask creation.

DAMPER Architecture: Two-Stage Domain-Aware Prototype Learning

DAMPER consists of an offline training phase and a plug-and-play online inference phase, decoupled from downstream LLMs and tailored for efficient client deployment.

Figure 1: DAMPER system overview. (A) Offline: contrastive prototype learning and prototype-guided rewriting alignment. (B) Online: autonomous domain and privacy span localization, DP-constrained rewriting.

Offline Training: Annotated privacy spans are used to train a discriminative span encoder under a multi-domain InfoNCE objective, yielding domain- and privacy-aware span embeddings. The clustering of these embeddings with FINCH produces compact, domain-contingent privacy prototypes that serve as semantic anchors for both localization and rewriting. DAMPER constructs a preference dataset for Direct Preference Optimization (DPO): candidate rewrites are scored for privacy (semantic distance from the original span) and utility (proximity to domain prototypes), explicitly navigating the privacy–utility tradeoff in annotation-free fashion.

Online Inference: At inference, TextChunker decomposes input queries into semantically coherent spans. The system infers the global domain and localizes privacy spans by computing each span's maximum similarity to the domain's prototypes, applying an adaptive threshold via the Otsu criterion. Only spans deemed sensitive are rewritten using a sampling-based Exponential Mechanism with bounded per-token sensitivity, achieving formal span-level DP guarantees (see Section 4.2 in the paper).

Prototype Learning for Domain-Aware Privacy Localization and Rewriting

Traditional encoders trained for general language understanding poorly separate domain-specific privacy concepts in embedding space, resulting in poor cluster boundaries (Figure 2, left). The DAMPER contrastive span encoder reshapes the embedding manifold, bringing together in-domain privacy spans and separating them from both non-private context and out-of-domain privacy spans, enabling discriminitative, domain-coherent prototype formation (Figure 2, right).

Figure 2: T-SNE plots—left: base encoder $g(\cdot)$ shows entangled span clusters; right: domain/privay-separable clusters with trained span encoder $h(\cdot)$.

Prototypes serve a dual role: guiding localization by semantic affinity and anchoring preference-based rewriting for style preservation. The composite reward function combines semantic obfuscation and domain consistency, with the tradeoff modulated by $\alpha$ (see Figure 3 for the effect on privacy–utility balance).

Figure 3: Impact of reward trade-off hyperparameter $\tau_1$ on privacy–utility metrics.

Rigorous Span-Level Differential Privacy with the Exponential Mechanism

Differential privacy at the span level is operationalized by applying the Exponential Mechanism to only localized privacy spans. At each decoding step, the rewriter outputs a logit vector over the vocabulary, which is clipped and sampled with temperature calibrated so that per-token and overall privacy costs are strictly bounded by design. The DP guarantee is only over spans changed, not the full text, allowing strong privacy with minimal semantic degradation outside sensitive regions.

Empirical Evaluation Across Medical and Legal Domains

DAMPER is evaluated on Pri-DDXPlus (medical), Pri-SLJA (legal), and Pri-Mixture (multi-domain) benchmarks. Performance is measured by downstream utility (accuracy), semantic consistency (BERTScore), and LLM-judge human parity scoring. DAMPER consistently outperforms full-text and span-level baselines—including DP-Paraphrase, DP-MLM, DP-Prompt, and PrivacyRestore—especially in multi-domain settings, where domain confusion strongly diminishes the accuracy of dictionary- or mask-dependent methods.

Figure 4: Recall on Pri-DDXPlus when trained with only the top $k\%$ most frequent annotated privacy spans; DAMPER exhibits strong robustness to rare/unseen privacy concepts.

Ablation and robustness analysis confirm that both the contrastively trained prototypes and the DPO rewriting alignment provide large, complementary gains. Notably, DAMPER maintains accuracy even under automatic (mask-free) span localization, demonstrating resilience relative to oracle-masked baselines.

Privacy Risk Assessment Against Embedding Inversion and Prompt Injection

The scheme's formal guarantees are borne out in adversarial settings. Under embedding inversion attacks (EIA) and prompt injection attacks (PIA), DAMPER achieves parity or improved privacy preservation relative to other span-level defenses for a wide range of privacy budgets, maintaining low ROUGE-L recall for sensitive regions.

Figure 5: Embedding Inversion Attack (EIA) leakage on ROUGE-L, showing strong DP-constrained defense up to high privacy budgets.

Qualitative Analysis and Case Studies

Case analyses in medicine (Figures 8 and 9) and law (Figures 10 and 11) show high localization fidelity, with prototype-driven detection identifying legal and clinical spans overlooked by prompt-based baselines. Recall-oriented errors (false positives) mainly cause conservative over-sanitization, whereas precision failures (false negatives) directly expose private attributes, underscoring the need for high-recall localization strategies in deployment.

Figure 6: Medical case—DAMPER localizes symptom/history spans with zero false negatives, avoiding privacy leakage missed by prompt-based detection.

Figure 7: Legal case—comprehensive span detection by DAMPER prevents privacy breach via unmasked legal elements.

Implications and Future Directions

DAMPER provides a general, domain-adaptive privacy rewriting framework that is annotation-free at deployment, supports multi-domain query streams, and delivers strong, rigorously bounded privacy guarantees with minimal context distortion. By decoupling client-side sanitization from downstream LLMs, it enables safer integration of foundation models for regulated and privacy-sensitive domains, and supports practical auditing for compliance.

Further research should (1) extend the paradigm to more finely-graded, cross-domain fusion queries, (2) explore richer or hierarchical prototype structures to handle subdomain privacy nuances, and (3) investigate active learning or user-in-the-loop mechanisms for continuous adaptation as privacy standards evolve.

Conclusion

DAMPER introduces a prototype-driven, mask-free privacy extraction and rewriting framework that addresses the scalability and adaptability demands of real-world, multi-domain LLM deployment. Through contrastive embedding, prototype-guided localization, and rigorous DP span-level rewriting, the system achieves state-of-the-art privacy–utility tradeoffs, resilience to rare/unseen privacy spans, and robustness against adversarial privacy attacks, establishing a comprehensive new baseline for client-side privacy protection in high-stakes NLP applications (2604.10145).