Fed-DLoRA: Efficient Wireless Federated Learning with Dynamic Low-Rank Adaptation

Abstract: Federated learning (FL) offers a promising distributed learning paradigm for internet of vehicles (IoV) applications. However, it faces challenges from communication overhead and dynamic environments. Model compression techniques reduce computing and communication burden yet create trade-offs between compression ratios and vehicle participation strategies. In this paper, we propose a lightweight FL algorithm named federated learning with dynamic low-rank adaptation (Fed-DLoRA), which is combined with low-rank adaptation (LoRA) to effectively reduce parameters and communication costs while enhancing training efficiency. The convergence analysis of Fed-DLoRA is conducted through stochastic gradient descent optimization coupled with singular value decomposition. This analysis establishes the theoretical relationships among LoRA rank, vehicular scheduling strategies and the model's convergence characteristics. Building on these insights, we formulate a joint optimization problem aimed at maximizing system performance. To address this problem, we propose an adaptive rank, bandwidth and vehicle selection (ARBVS) algorithm that integrates enumeration with greedy optimization strategies. The algorithm provides efficient rank selection and resource scheduling strategies for each FL communication round, thereby achieving effective performance improvements for the FL system. Experimental results demonstrate that Fed-DLoRA achieves superior performance compared to conventional federated learning approaches, exhibiting enhanced accuracy, faster convergence, and improved communication efficiency.

• The paper introduces Fed-DLoRA, which integrates LoRA modules with adaptive vehicular scheduling to achieve scalable, communication-efficient FL in IoV scenarios.
• It employs a hybrid ARBVS algorithm for joint optimization of rank, bandwidth, and device selection, leading to enhanced convergence and accuracy.
• Empirical results show significant improvements, including up to 77.49% reduction in uplink traffic and faster convergence compared to conventional FL methods.

Fed-DLoRA: A Parameter-Efficient Federated Learning Framework with Dynamic Low-Rank Adaptation

Introduction and Motivation

The proliferation of intelligent connected vehicles (ICVs) necessitates scalable, communication-efficient distributed machine learning paradigms tailored for the Internet of Vehicles (IoV). Traditional Federated Learning (FL) is hindered by substantial uplink communication overhead and localized computational constraints, both exacerbated in dynamic vehicular mobile environments with frequent device churn and heterogeneous channel conditions. Model compression strategies such as pruning and sparsification have been explored, but their efficacy is bounded by trade-offs in accuracy and participation. Low-Rank Adaptation (LoRA), originally devised for efficient fine-tuning of LLMs, offers a promising alternative by decoupling optimization from full-rank weight matrices and leveraging trainable low-rank modules. However, prior federated LoRA frameworks assume static device populations and stable channel conditions, leading to suboptimal performance in IoV contexts characterized by rapid mobility and fluctuating wireless resources.

Figure 1: The framework of Fed-DLoRA.

Fed-DLoRA Architecture

Fed-DLoRA integrates LoRA modules into the FL pipeline for parameter-efficient optimization. Model weights in local client models are decomposed into frozen backbone matrices and trainable low-rank factors, drastically reducing the volume of trainable and transmittable parameters. Federated updates consist exclusively of these low-rank components, alleviating both computational and communication bottlenecks. The central Base Station (BS) leverages adaptive rank, bandwidth, and vehicle selection (ARBVS) to dynamically configure LoRA rank, communication allocations, and participant sets per round, accommodating practical mobility and resource constraints. The aggregation procedure uniformly combines low-rank updates from selected ICVs, iteratively minimizing the global loss.

Figure 2: Schematic diagram of LoRA structure.

Mathematical Analysis and Convergence Characterization

The convergence dynamics of Fed-DLoRA are governed by the interplay between LoRA rank selection, vehicular scheduling, and communication resource allocation. Theoretical analysis is grounded in stochastic gradient descent optimization augmented by Singular Value Decomposition (SVD), which quantifies the approximation gap between full-rank and low-rank gradient updates. The low-rank constraint imposed by LoRA is mathematically encoded by retaining only the largest $r$ singular values during backpropagation, introducing an explicit error term in the central model's gradient update.

The bound on expected loss reduction per communication round depends on the chosen rank, the number of participating ICVs, and variance in local gradients. The SVD-based derivation formalizes the Frobenius norm of the error as $$\left \| \Delta G_l \right \|_F = \sqrt{\sum_{i=r+1}^{k_l} \sigma_{l, i}^2} \le M \sqrt{k_l - r}$$, indicating that higher LoRA ranks directly improve convergence fidelity but demand increased communication resources. This establishes a tight theoretical linkage between adaptive scheduling and system performance, substantiating the design of the ARBVS algorithm.

Adaptive Joint Optimization: ARBVS Algorithm

The Fed-DLoRA system addresses the mixed-integer nonlinear programming problem underlying dynamic rank, bandwidth, and vehicle selection with a hybrid enumeration-greedy ARBVS algorithm. For each candidate rank, ARBVS calculates minimal required bandwidth per device and sorts ICVs to maximize participation under current resource constraints. This joint optimization facilitates efficient bandwidth allocation, maximized ICV participation, and judicious rank selection, directly improving global training efficiency and convergence rates.

Experimental Validation

Experiments are conducted in a single-base-station simulated C-V2X scenario, leveraging CIFAR-10 and CIFAR-100 datasets in both IID and non-IID configurations. All methods utilize a parameter-reduced ResNet18 variant, with LoRA modules applied to convolutional and fully connected layers. Baselines include standard FedAvg, FedPT (with partial weight freezing), and FedRA (random LoRA aggregation).

Accuracy and Convergence Rate

Fed-DLoRA consistently yields superior test accuracy and convergence speed compared to baselines, especially in the non-IID regime where increased ICV participation mitigates class imbalance. For instance, on CIFAR-10 IID, Fed-DLoRA achieves $66.50\%$ accuracy at the 20th round, outperforming FedAvg ($51.01\%$), FedPT ($56.48\%$), and FedRA ($57.87\%$).

Figure 3: Test accuracy on CIFAR-10 with IID and non-IID dataset.

Figure 4: Test accuracy on CIFAR-100 with IID and non-IID dataset.

Latency and Communication Efficiency

Fed-DLoRA displays marked reductions in total training time to target accuracy. With increasing ICVs, it maintains robust convergence rates, primarily bottlenecked by communication bandwidth rather than client population. Communication cost analysis shows that Fed-DLoRA reduces uplink traffic by $77.49\%$, $51.55\%$, and $33.90\%$ relative to FedAvg, FedPT, and FedRA, respectively, for achieving $50\%$ accuracy on CIFAR-10 IID. These reductions directly translate to improved energy efficiency, critical for battery-constrained vehicular devices.

Figure 5: Comparison of time to reach target accuracy for different numbers of ICVs.

Figure 6: FL Communication Efficiency Comparison Experiment.

ICV Scheduling and Ablation Study

ARBVS algorithm validation demonstrates its capacity for maximizing test accuracy via adaptive selection of LoRA rank, bandwidth, and client subsets per round. Comparative studies with random scheduling indicate that performance improvements scale with both LoRA rank and number of participating ICVs.

Figure 7: ICV Scheduling Comparison Experiment.

Implications and Future Directions

Fed-DLoRA fundamentally advances FL for IoV applications by tightly coupling parameter-efficient updating with adaptive, resource-aware scheduling. The theoretical analysis offers rigorous guarantees on convergence, linking rank selection and scheduling directly to empirical outcomes. Practically, Fed-DLoRA enables broader participation, lower latency, and substantial communication savings, enhancing the scalability and feasibility of distributed learning in high-mobility vehicular contexts.

Future work will explore extending Fed-DLoRA to LLMs, incorporating joint optimization of LoRA-based local fine-tuning, aggregation, and wireless resource allocation to facilitate ubiquitous intelligence for LLM-enhanced IoV systems.

Conclusion

Fed-DLoRA introduces a robust, parameter-efficient FL framework leveraging LoRA for high-mobility IoV scenarios. The joint optimization of rank, bandwidth, and ICV selection via the ARBVS algorithm is theoretically justified and empirically validated, resulting in strong accuracy, fast convergence, and significant communication savings relative to classical and LoRA-based baselines. The implications substantiate the approach for scalable, energy-efficient FL deployments in dynamic vehicular networks, with prospective adaptations toward LLMs and broader IoT systems.