Outer convergence of HiFedProx

Establish convergence guarantees for HiFedProx with high-order client regularization under finite local optimization budgets, partial client participation, and equal-weight server averaging, beyond the affine reference subproblem.

Background

The paper derives an exact affine client-model response showing that increasing the regularization exponent compresses disparities in client displacement magnitudes. However, the authors emphasize that this calculation is only a reference subproblem and does not constitute a convergence analysis for the full federated method. In particular, for exponents greater than two, equal averaging of exact client solutions is generally not equivalent to taking a common gradient step on an averaged high-order envelope, because the scaling between a client displacement and its envelope gradient depends on the displacement norm.

The implemented HiFedProx procedure additionally uses finite local optimization budgets, stochastic minibatches, same-minibatch Armijo backtracking, and partial client participation. Establishing convergence for this complete outer federated process is therefore left unresolved.

References

The affine formula describes a reference subproblem rather than a complete federated convergence result. Even with exact client solutions and full participation, equal averaging for $p>2$ is not generally a common-step gradient method on the average high-order envelope: the factor relating a client displacement to its envelope gradient depends on that displacement norm. Finite local budgets and partial participation introduce further errors. We therefore use the analysis to explain the radial mechanism and the role of backtracking, while leaving the outer convergence to future work.

— Beyond Conventional Federated Learning via High-Order Regularization  (2609.09904 - Kabgani et al., 9 Sep 2026) in Section 4, “Geometry and Local Safeguard,” paragraph following the displacement diagnostics discussion