Architecture and hyperparameter sensitivity of ORDERS

Determine the sensitivity of ORDERS-R1 to the shared-body, private-adapter, and classifier architecture and to the layer-selection, Transformer, graph-neural-network, split, decay, and positive-alignment-weight choices, including whether clients benefit from larger departures of private residual features.

Background

ORDERS-R1 is evaluated using only one shared-body/private-adapter/classifier split instantiated with a small convolutional neural network for CIFAR-10 and an LSTM for Sent140. The study does not sweep layer selections, alternative architectures such as Transformers or graph neural networks, the shared/private split, the geometric-decay parameter, or the positive alignment coefficient.

The alignment penalty constrains the private residual representation and can discourage useful client-specific departures. The paper therefore identifies the effect of allowing larger private departures, as well as the broader architecture and hyperparameter sensitivity of ORDERS-R1, as unresolved and requiring separate evaluation with training and validation data.

References

Architecture and hyperparameter sensitivity remain unresolved. We evaluate one body/adapter/classifier split in a small CNN and an LSTM. No layer-selection mechanism, Transformer, GNN, split sweep, decay sweep, or positive-β sweep is present. Alignment explicitly limits local residual features; whether a client benefits from a larger departure should be assessed with training/validation data in a separate study.

— ORDERS: An Empirical Study of Norm-Rank Aggregation for Personalized Federated Learning  (2610.10361 - Khan, 7 Oct 2026) in Section 7, Discussion and limitations