Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs

Abstract: Federated fine-tuning (FFT) has emerged as a privacy-preserving paradigm for collaboratively adapting LLMs. Built upon federated learning, FFT enables distributed agents to jointly refine a shared pretrained LLM by aggregating local LLM updates without sharing local raw data. However, FFT-based LLMs remain vulnerable to model manipulation threats, in which adversarial participants upload manipulated LLM updates that corrupt the aggregation process and degrade the performance of the global LLM. In this paper, we propose an Augmented Model maniPulation (AugMP) strategy against FFT-based LLMs. Specifically, we design a novel graph representation learning framework that captures feature correlations among benign LLM updates to guide the generation of malicious updates. To enhance manipulation effectiveness and stealthiness, we develop an iterative manipulation algorithm based on an augmented Lagrangian dual formulation. Through this formulation, malicious updates are optimized to embed adversarial objectives while preserving benign-like parameter characteristics. Experimental results across multiple LLM backbones demonstrate that the AugMP strategy achieves the strongest manipulation performance among all competing baselines, reducing the global LLM accuracy by up to 26% and degrading the average accuracy of local LLM agents by up to 22%. Meanwhile, AugMP maintains high statistical and geometric consistency with benign updates, enabling it to evade conventional distance- and similarity-based defense methods.

• The paper introduces AugMP, a novel attack leveraging graph representation learning and augmented Lagrangian duality to generate adversarial updates during federated fine-tuning.
• Empirical results show up to 26% global accuracy reduction on models like Pythia, with malicious updates stealthily mimicking benign statistics to evade detection.
• Ablation studies confirm that both the GRL module and augmented Lagrangian penalty are critical for maximizing model degradation and evading conventional defenses.

Graph Representation Learning for Adversarial Model Manipulation in Federated LLM Fine-Tuning

Problem Motivation and Threat Landscape

Federated fine-tuning (FFT) of LLMs enables distributed adaptation of foundational models at scale, holding promise for privacy-preserving AI on edge devices. However, these decentralized aggregation protocols expose FedLLMs to manipulated model updates from adversarial participants, threatening model utility despite localized data retention. Conventional defenses against model manipulation typically leverage geometric consistency metrics—Euclidean distance or cosine similarity—to flag anomalous updates during aggregation. Yet, these distance-based defenses have proven vulnerable to attacks that closely mimic benign update statistics.

Figure 1: (a) Standard FFT training cycle; (b) impact of adversary contributing manipulated updates in FedLLMs.

The AugMP Manipulation Strategy: Concept and Architecture

The paper introduces Augmented Model maniPulation (AugMP), an attack methodology that synthesizes malicious updates for FedLLMs while explicitly preserving benign-like statistical and geometric features. The attack is characterized by two main technical elements: (1) a graph representation learning (GRL) component, and (2) an iterative optimization algorithm based on augmented Lagrangian duality.

The GRL module utilizes a variational graph autoencoder (VGAE) to encode correlation structures among benign local updates, constructing a feature correlation graph in parameter space. The latent representations derived from the VGAE are then used, via a graph spectral transformation (GST) mechanism, to generate malicious local updates that closely track the topological patterns of benign updates. The augmented Lagrangian formulation enforces adherence to geometric constraints (distance and similarity) while optimizing the adversarial objective: maximization of the global test loss post-aggregation.

Figure 2: The AugMP adversarial pipeline employs GRL with VGAE encoding and GST-based synthesis to craft benign-looking but adversarial updates.

Attack Optimization and Workflow

The attack is formalized as a constrained optimization problem: maximizing evaluation loss with respect to the global model subject to geometric proximity and alignment constraints. The malicious update is iteratively optimized via augmented Lagrangian dual ascent, where quadratic penalties for distance and similarity violations maintain stealthiness. The adversary observes a subset of peer updates (as realistic in federated settings), reconstructs parameter feature graphs, and manipulates the aggregation process with statistically indistinguishable yet adversarial local increments.

Empirical Evaluation: Effectiveness and Stealthiness

The paper conducts comprehensive empirical evaluation across multiple LLM backbones (DistilBERT, Pythia, Qwen2.5) and two text classification tasks (AG News, Yahoo! Answers). Baselines for comparison include ALIE (distribution-constrained shift) and Gaussian random model poisoning (RMP). Notable findings include:

• Global accuracy reduction is consistently greater for AugMP than for either baseline attack, with up to 26% accuracy reduction observed on Pythia and substantial local accuracy drops (up to 22%) in benign agents.

  Figure 3: AugMP consistently induces greater reductions in global testing accuracy across LLM architectures compared to ALIE and RMP baselines.

• Propagation of model degradation leads to system-wide decline in local agent performance due to the broadcasted compromised global model.

  Figure 4: Local average accuracy also degrades under AugMP, demonstrating propagation of adversarial effects.

• Attack stealthiness: AugMP-generated malicious updates exhibit Euclidean distances and cosine similarity statistics that are nearly indistinguishable from benign updates, thus enabling evasion of conventional aggregation-level defenses.

  Figure 5: Distributions of Euclidean distances for AugMP (overlapping benign), versus clear separation for other attack baselines.

  

  Figure 6: Cosine similarity statistics for AugMP remain within benign ranges, contrasting with distinguishable ALIE and RMP.

  

  Figure 7: The manipulated FedLLMs (Qwen2.5 backbone) classify business news as unrelated categories, providing plausible yet fabricated explanations.

Architecture Ablations and LoRA Parameter Sensitivity

The ablation study quantifies the individual contribution of the augmented Lagrangian penalty as well as the GRL framework. Removal of either leads to significant reduction in manipulation effectiveness and increased detectability by geometric defenses.

Figure 8: Ablation demonstrates that both AL penalty and GRL framework are critical for maximum performance degradation and stealthiness.

Sensitivity experiments across different LoRA ranks and scaling factors reveal that parameter-efficient fine-tuning dimensions impact both model vulnerability and attack efficacy. The attack effectiveness increases with greater tunable parameter diversity on small backbones; for larger LLMs, increasing trainability can paradoxically improve robustness due to enhanced adaptation, though at the cost of higher baseline accuracy loss under attack.

Theoretical and Practical Implications

The paper explicitly demonstrates that threats to FedLLMs can be constructed to evade widely adopted geometric anomaly defenses, challenging current assumptions underpinning robust federated aggregation. By exploiting fine-grained parameter correlation structures among benign updates—rather than simply mimicking aggregate statistical measures—AugMP closes the gap between attack stealth and adversarial impact in distributed LLM training.

From a theoretical standpoint, these findings signal the need for aggregation and defense mechanisms capable of reasoning beyond first- and second-order parameter statistics, particularly in the high-dimensional, highly correlated parameter spaces of LLMs. Practical implications include the requirement for new anomaly detection schemes, possibly leveraging deeper semantic or task-based consistency checks, explainability-driven diagnostics, or alternative cryptographically rooted aggregation architectures.

Conclusion

This work develops a graph-representation-augmented attack (AugMP) which can significantly and stealthily degrade FedLLMs by leveraging the structural properties of benign update populations. By achieving substantial accuracy degradation while remaining difficult to detect using distance- and similarity-based methods, it exposes fundamental security limitations in FedLLMs aggregation. The demonstrated attack highlights the necessity for research into advanced detection, defense, and aggregation frameworks suitable for robust, scalable, and private federated adaptation of LLMs.