Federated Security for LLMs

• Federated Security for LLMs is a framework integrating federated learning, cryptography, and differential privacy to protect sensitive data in multi-party LLM training.
• Key methodologies include secure aggregation, trusted execution environments, and adversarial defenses that mitigate privacy-leakage, data poisoning, and prompt injection attacks.
• These strategies balance privacy, utility, and computational overhead, proving critical for deploying secure LLMs in healthcare, finance, military, and cross-enterprise applications.

Federated security for LLMs encompasses the suite of architectures, protocols, and defense strategies designed to enable collaborative LLM training or inference across multiple organizations, institutions, or devices—while preserving the confidentiality of local data, protecting model integrity, and defending against privacy-leakage, poisoning, and inference attacks. Leveraging principles from federated learning, cryptographically secure computation, robust aggregation, differential privacy, and trusted-execution environments, these frameworks are increasingly critical for deploying LLMs in sensitive domains such as healthcare, finance, military, and cross-enterprise AI services.

1. Threat Taxonomy in Federated LLM Security

Federated LLMs introduce unique vulnerabilities that surpass traditional federated learning and centralized LLM deployments. Core adversarial and privacy threats include:

• Membership inference attacks (MIA): Determining whether a target instance $x$ was part of any client’s private training set by analyzing model updates or gradients. However, empirical results suggest attack advantage on LLMs in FL is weak due to large corpora and near one-pass semantics (Jiang et al., 13 May 2025).
• Gradient inversion/data reconstruction: Recovering exact input batches or token sequences $X$ from gradient vectors, exploiting the high expressiveness of transformer gradients (Han et al., 2023, Jiang et al., 13 May 2025). Attack formulations include DLG-style optimization and analysis-based reconstructions.
• Prompt injection/jailbreaking: Crafting adversarial inputs $p$ that bypass LLM filters, leak confidential data, or subvert the intended behavior of downstream applications (Lee et al., 30 Jan 2025, Jayathilaka, 15 Nov 2025, Gill et al., 6 Sep 2025).
• Data and model poisoning: Attackers manipulate local datasets (label flips, backdoor triggers) or submit malicious updates $\Delta\theta$ to degrade, misalign, or backdoor the global model (Pang et al., 17 Feb 2025, Han et al., 2023).
• Cross-silo/model inversion and long-tailed leakage: Attackers with partial knowledge mount coordinated attacks, exploiting uncommon or over-memorized data within client datasets (Jiang et al., 13 May 2025).

Most federated LLM security frameworks adopt the honest-but-curious server assumption and consider both non-Byzantine and Byzantine client settings, necessitating defenses at multiple protocol and model layers.

2. Secure Aggregation and Distributed Learning Protocols

Cryptographically secure aggregation is foundational in federated security for LLMs, aiming to shield individual model updates or gradients from the aggregation server and peers.

Key mechanisms and protocols:

• Secure Aggregation: Implemented via pairwise masking (SecAgg) (Sarwar, 27 Feb 2025), additive homomorphic encryption (Paillier) (Luo et al., 22 Jun 2025), or fully homomorphic encryption (CKKS) (Mia et al., 6 Jun 2025). Clients transmit encrypted or masked low-rank updates (e.g., LoRA adapters) such that only the aggregated sum is revealed, never any single $\Delta w_k$.
• Trusted Execution Environments (TEE): Both client- and server-side TEEs (Intel SGX/TDX) encapsulate sensitive parameter adaptations (LoRA or P-tuning) and decrypt/encrypt model slices, allowing split fine-tuning and feature masking (Huang et al., 18 Jan 2024).
• Split Learning and Activation Masking: FL-LLaMA and related schemes decouple LLM layers between server and client, with added Gaussian noise at layer boundaries to ensure $(\varepsilon,\delta)$-differential privacy for activations and gradients (Zhang et al., 21 May 2025).
• Parameter-Efficient Fine-Tuning (PEFT): LoRA is widely deployed to restrict the exposed update subspace, reducing the attack surface for inference and reconstruction (Sarwar, 27 Feb 2025, Mia et al., 6 Jun 2025). Pruning low-magnitude adapter entries (FedShield-LLM) further limits leakage and reduces ciphertext size.

Framework	Encryption/Privacy	Aggregation	Efficient Fine-Tuning
FedMentalCare (Sarwar, 27 Feb 2025)	SecAgg (masks), TLS	FedAvg	LoRA
FedShield-LLM (Mia et al., 6 Jun 2025)	Fully HE (CKKS)	FHE sum, pruning	LoRA + pruning
FL-LLaMA (Zhang et al., 21 May 2025)	Gaussian noise on activations	Split learning	LoRA/PEFT
Split-TLLM (Huang et al., 18 Jan 2024)	TEE + one-time pad (OTP)	Layer split + TEE	LoRA, P-tuning v2, SPF
BinaryShield (Gill et al., 6 Sep 2025)	LDP (randomized response)	X-service signature	N/A (threat detect)

These protocols demonstrate that federated security for LLMs requires careful interplay between cryptographic primitives, parameter-efficient training, and noise-based differential privacy to maintain both confidentiality and usability.

3. Robustness Against Adversarial and Byzantine Attacks

Federated LLMs are exposed to sophisticated adversaries capable of model poisoning, data poisoning, and targeted misalignment. Key robustness optimization frameworks include:

• FedEAT (Federated Embedding-space Adversarial Training): Each client applies projected gradient ascent in the embedding space to generate adversarial perturbations, solving the min–max objective

$$\min_{w}\;\sum_{j=1}^{m_i} \max_{\|\delta_{i,j}\|_p\le\epsilon} \mathcal{L}(f(w,z_{i,j}+\delta_{i,j}),y_{i,j}),$$

and the server aggregates client updates via geometric median (Weiszfeld’s algorithm), achieving significant reductions (3–4 pp) in attack success rate with ≤2% drop in clean accuracy (Pang et al., 17 Feb 2025).

• Robust Aggregation Algorithms: Techniques such as Krum, m-Krum, coordinate-wise median, trimmed mean, and geometric median reduce the influence of outlier or malicious updates (Han et al., 2023). Smoothed Weiszfeld variants and feature-based anomaly detectors help tolerate up to $f$ Byzantine clients per round.
• Differential Privacy in Aggregation: Adding calibrated Gaussian or Laplace noise to parameter updates guarantees $(\epsilon, \delta)$-DP, further mitigating reconstruction and membership inference—even though this reduces utility proportional to the noise scale (Jiang et al., 13 May 2025).
• Adaptive LLM-in-the-Loop Risk Assessment: In edge cloud federations, an LLM monitors encrypted update metadata, dynamically adjusts aggregation weights (softmax on node performance), triggers SMC only under detected risk, and flags drift for robust operation (Luo et al., 22 Jun 2025).

These approaches layer protocol, cryptographic, and model-level defenses to create resilient federated LLM training pipelines.

4. Privacy-Preserving Threat Detection and Cross-Service Intelligence

Prompt injection attacks and cross-boundary threats necessitate federated, privacy-aware detection and intelligence sharing systems:

• Federated Embedding-Based Prompt Injection Detection: Encoded prompt embeddings (e.g., SBERT/MiniLM) are classified via logistic regression trained in a federated pipeline (FedAvg), such that only model parameters are transmitted—never raw text or embeddings. This achieves detection performance on par with centralized training and is suitable for scalable, privacy-preserving deployment (Jayathilaka, 15 Nov 2025).
• BinaryShield for Cross-Service Threat Intelligence: Suspicious prompts are transformed through multi-stage pipelines: aggressive PII redaction, semantic embedding, binary quantization (sign mapping), and randomized response (LDP). Resulting binary fingerprints are non-invertible and suitable for cross-organization sharing, achieving F1 = 0.94 on paraphrase attacks with storage and search efficiency gains (64×, 38×, respectively) (Gill et al., 6 Sep 2025).
• Red/Blue Team Wargaming (Military FL Context): Artificial red-teams generate adversarial prompts or malicious weights, blue-teams develop client- and server-side mitigation protocols, and QA-LLMs perform continuous assurance through anomaly detection and consistency monitoring (Lee et al., 30 Jan 2025).

These architectures allow for federated detection, sharing, and mitigation of new classes of LLM attacks, while maintaining regulatory compliance and user privacy.

5. Challenges in Privacy, Heterogeneity, and Utility Trade-Offs

Federated security for LLMs must be robust to a spectrum of system, data, and model heterogeneities:

• Model Heterogeneity: Clients may fine-tune different subsets of parameters (e.g., LoRA ranks, PEFT modules, split-points in split learning), requiring aggregation methods (dimension-wise Krum, partial coordinate handling) that accommodate partial updates (Jiang et al., 13 May 2025).
• Privacy–Utility Trade-offs: Differential privacy and aggressive noise injection yield formal $(\epsilon, \delta)$-DP but incur explicit drops in fine-tuning accuracy, especially at scale. Homomorphic encryption via FedShield-LLM achieves cryptographic privacy with no noise, thus no utility loss—but at higher computational cost (Mia et al., 6 Jun 2025). Table-based evaluations consistently show LoRA+DP/HE+robust aggregation as optimal for balancing privacy and downstream performance (Jiang et al., 13 May 2025, Sarwar, 27 Feb 2025, Mia et al., 6 Jun 2025).
• Scalable Communication and Computation: Parameter-efficient updates (LoRA, QLoRA) reduce upload/download overhead by 10–100×, making FL feasible for edge and cross-device scenarios (Sarwar, 27 Feb 2025). Pruning and quantization further improve scalability and attack resistance (Mia et al., 6 Jun 2025).

Defense Mechanism	Privacy Guarantee	Utility Impact	Computational Overhead
Differential Privacy	$(\epsilon, \delta)$-DP	Moderate-High drop	Low
Homomorphic Encryption	IND-CPA (CKKS/Paillier)	None	Moderate-High
PEFT + Secure Aggregation	Parameter minimization	Low-Moderate drop	Low-Moderate

Designers must tune privacy budgets, aggregation policies, and adapter sizes to an application’s regulatory context and service-level requirements.

6. Specialized Paradigms and Domain-Specific Frameworks

Emerging deployment contexts require tailored federated security protocols:

• Privacy-Enhancing Multi-Agent Systems (Federated MAS): EPEAgent interposes at the conversation and memory retrieval layers in multi-agent LLM systems, enforcing fine-grained, dynamic data minimization on user profiles and conversational context. This achieves up to 97.6% privacy with <2% utility loss in financial/medical settings by field-level access control, not DP-noise (Shi et al., 11 Mar 2025).
• Regulated Domain (e.g., Healthcare, Military): Frameworks such as FedMentalCare blend LoRA, secure aggregation, and explicit auditability for HIPAA/GDPR compliance (Sarwar, 27 Feb 2025). In military contexts, policy-driven protocols, multi-layer governance, and periodic red/blue cycles address the risk of prompt injection and multi-client collusion (Lee et al., 30 Jan 2025).
• Edge Cloud FL with LLM-in-the-Loop: Hierarchical schemes embed LLMs within the SMC control loop, dynamically determining aggregation strategies based on real-time risk signals and metadata (Luo et al., 22 Jun 2025).

These studies illustrate that robust federated LLM security depends not only on generic FL defenses, but also on application-tailored privacy policies, protocol engineering, and context-aware data flow minimization.

7. Evolving Best Practices and Future Directions

Comprehensive system designs for federated LLM security follow layered, heterogeneity-aware, and adaptive principles (Jiang et al., 13 May 2025, Han et al., 2023):

• Defense-in-depth: Combine secure aggregation, differential privacy (or homomorphic encryption), robust/Byzantine-tolerant aggregation, and client-side sanitization (prompt filters, DP-opt-in, anomaly detectors).
• Parameter-efficient and privacy-aware adapters: Favor LoRA/adapter-based updates (with quantization/pruning) to minimize exposed bits and communication cost without significant utility loss.
• Continuous adversarial testing and threat modeling: Red/blue team cycles, anomaly-based defense activation, and audit logging for monitoring potential attack or data-leakage incidents.
• Scalable, regulation-aligned communication protocols: Secure channels (TLS/IPsec), privacy-preserving fingerprinting (e.g., BinaryShield), minimal reporting across compliance boundaries, and explicit opt-in/consent for clients (Gill et al., 6 Sep 2025).
• Open challenges: Adaptive adversary resilience, federated unlearning (removal of client data history), model IP protection (watermarks), and formal privacy guarantees for dynamic model components (Jiang et al., 13 May 2025).

As federated LLM deployment continues to scale, ongoing research targets certified defenses in embedding space, client-side anomaly detection, scalable FHE acceleration, and application of federated security principles to multimodal and continual learning LLMs (Pang et al., 17 Feb 2025, Mia et al., 6 Jun 2025).

---

References

• "FedEAT: A Robustness Optimization Framework for Federated LLMs" (Pang et al., 17 Feb 2025)
• "FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework" (Sarwar, 27 Feb 2025)
• "A Federated Splitting Framework for LLMs: Security, Efficiency, and Adaptability" (Zhang et al., 21 May 2025)
• "A Fast, Performant, Secure Distributed Training Framework For LLM" (Huang et al., 18 Jan 2024)
• "PluralLLM: Pluralistic Alignment in LLMs via Federated Learning" (Srewa et al., 13 Mar 2025)
• "Privacy-Preserving Prompt Injection Detection for LLMs Using Federated Learning and Embedding-Based NLP Classification" (Jayathilaka, 15 Nov 2025)
• "Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation" (Lee et al., 30 Jan 2025)
• "Federated Learning-Based Data Collaboration Method for Enhancing Edge Cloud AI System Security Using LLMs" (Luo et al., 22 Jun 2025)
• "Cross-Service Threat Intelligence in LLM Services using Privacy-Preserving Fingerprints" (Gill et al., 6 Sep 2025)
• "FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs" (Han et al., 2023)
• "Federated LLMs: Feasibility, Robustness, Security and Future Directions" (Jiang et al., 13 May 2025)
• "FedShield-LLM: A Secure and Scalable Federated Fine-Tuned LLM" (Mia et al., 6 Jun 2025)
• "Privacy-Enhancing Paradigms within Federated Multi-Agent Systems" (Shi et al., 11 Mar 2025)