PAC-MCoFL-p: Scalable MARL in Federated Learning

• The paper introduces a parameterized conjecture generator that replaces exhaustive action search, reducing computational complexity while ensuring provable error bounds and convergence.
• PAC-MCoFL-p integrates Pareto Actor-Critic, expectile regression, and ternary Cartesian action decomposition to optimize client assignment, adaptive quantization, and resource allocation under privacy constraints.
• Scalability is achieved as PAC-MCoFL-p maintains constant memory and runtime even with increasing service providers, addressing the intractability issues of prior methods.

Searching arXiv for PAC-MCoFL-p and closely related work to ground the article in current papers. PAC-MCoFL-p is a scalable multi-agent reinforcement learning algorithm for communication and computation co-optimization in non-cooperative federated learning services. It is introduced as a variant of PAC-MCoFL within a game-theoretic setting in which service providers act as agents and jointly optimize client assignment, adaptive quantization, and resource allocation under privacy constraints and competing interests (Tan et al., 22 Aug 2025). The method combines Pareto Actor-Critic principles, expectile regression, ternary Cartesian action decomposition, and a parameterized conjecture generator. Its central purpose is to replace the exhaustive conjecture of other agents’ joint actions with a tractable approximation that substantially reduces computational complexity while retaining provably bounded error and convergence guarantees (Tan et al., 22 Aug 2025).

1. Position within non-cooperative federated learning

Federated learning in multi-service provider ecosystems is described as being fundamentally hampered by non-cooperative dynamics, where privacy constraints and competing interests preclude the centralized optimization of multi-service provider communication and computation resources (Tan et al., 22 Aug 2025). Within this setting, PAC-MCoFL is formulated as a game-theoretic multi-agent reinforcement learning framework in which service providers act as autonomous agents.

The original PAC-MCoFL framework seeks decentralized co-optimization of resource allocation strategies, specifically client selection, adaptive quantization, and bandwidth/CPU assignment, while accounting for the competitive, non-cooperative tendencies of multiple service providers sharing limited resources (Tan et al., 22 Aug 2025). PAC-MCoFL-p is the scalable variant of this framework. Its introduction is motivated by the observation that the original conjecture mechanism requires exhaustive enumeration of all possible joint actions of other agents, which becomes computationally prohibitive as the number of service providers increases.

This suggests that PAC-MCoFL-p should be understood not as an alternative objective formulation, but as an architectural refinement intended to preserve the Pareto-oriented, decentralized optimization logic of PAC-MCoFL while removing the principal scalability bottleneck.

2. Core architecture and algorithmic components

PAC-MCoFL-p is characterized by five explicit components: agents, an actor-critic framework, an expectile regression critic, ternary Cartesian action decomposition, and a parameterized conjecture generator (Tan et al., 22 Aug 2025).

Each service provider is modeled as an autonomous agent. Every agent maintains separate actor and critic neural networks. The critic network utilizes expectile regression to capture asymmetric risk preferences among service providers, and the framework integrates Pareto Actor-Critic principles with expectile regression, enabling agents to conjecture optimal joint policies to achieve Pareto-optimal equilibria while modeling heterogeneous risk profiles (Tan et al., 22 Aug 2025).

To manage the high-dimensional action space, the method uses ternary Cartesian decomposition, also described as ternary Cartesian action decomposition (TCAD), which facilitates fine-grained control (Tan et al., 22 Aug 2025). The action dimensions explicitly include client selection, quantization, bandwidth, and CPU frequency, and each action dimension is represented through discrete increments

$\psi_m \in \{-1,0,1\}.$

The corresponding action space is

$\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$

According to the source description, this restricts joint action changes per round to a finite set rather than the entire space, thereby reducing complexity from exponential to constant per-agent (Tan et al., 22 Aug 2025).

The distinctive addition in PAC-MCoFL-p is the parameterized conjecture generator. This module replaces brute-force conjecture of others’ joint actions with direct generation or approximation via a neural network, making the Pareto Actor-Critic approach scalable and practical for large systems (Tan et al., 22 Aug 2025).

3. From exhaustive conjecture to parameterized conjecture generation

In the original PAC-MCoFL formulation, each agent must exhaustively enumerate all possible joint actions of other agents to determine optimal coordinated actions. The complexity of this brute-force policy conjecture scales as

$O(|\mathcal{A}|^{R-1}),$

where $R$ denotes the number of service providers (Tan et al., 22 Aug 2025). This is identified as the central reason that the original approach becomes intractable for large-scale systems.

PAC-MCoFL-p addresses this by introducing a parameterized conjecture generator. For a particular agent $r$, the generator, parameterized by $\varphi_r$, takes as input the current agent action $a_{r,t}$, observation $o_{r,t}$, and a hidden state $h$ derived from $o_{r,t}$, and outputs a distribution over other agents’ joint actions:

$\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$0

This is given as Equation (36) in the paper summary (Tan et al., 22 Aug 2025).

The generator is trained end-to-end through a compound objective,

$\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$1

where $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$2 is agent $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$3's action-value function, $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$4 is a KL divergence regularizer for temporal consistency, $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$5 is a moving average of observed policy distributions from others, and $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$6 is a regularization coefficient (Tan et al., 22 Aug 2025).

By inserting this generator in place of the brute-force maximization, the actor and critic updates become fully differentiable and tractable (Tan et al., 22 Aug 2025). A plausible implication is that PAC-MCoFL-p preserves the conjectural structure of the parent framework while replacing a combinatorial search routine with a learned surrogate distribution over opponents’ joint behavior.

4. Expectile regression and equilibrium behavior

The critic in PAC-MCoFL-p employs expectile loss,

$\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$7

where $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$8 controls risk preference (Tan et al., 22 Aug 2025). The description associates $\mathcal{A}' = \prod_{m=1}^4 \{-1,0,1\}.$9 with aggressive behavior and $O(|\mathcal{A}|^{R-1}),$0 with conservative behavior. The use of expectile regression is said to broaden the range of equilibria and enable agents to model their risk attitudes directly (Tan et al., 22 Aug 2025).

The framework is therefore not limited to a single neutral equilibrium concept. Instead, it accommodates heterogeneous risk profiles among service providers while still targeting Pareto-optimal equilibria through Pareto Actor-Critic principles (Tan et al., 22 Aug 2025). In the non-cooperative federated learning setting, this is significant because service providers may not evaluate communication, computation, and task progress through the same utility lens.

The critic update is summarized in simplified form as

$O(|\mathcal{A}|^{R-1}),$1

with the temporal-difference error including maximization over conjectured policies provided by the generator for PAC-MCoFL-p (Tan et al., 22 Aug 2025). This locates the parameterized conjecture generator directly inside the Bellman-style update mechanism rather than treating it as an external heuristic.

5. Theoretical guarantees

The framework is presented with both a provable error bound and convergence guarantees (Tan et al., 22 Aug 2025).

For the error bound, the source states that if the $O(|\mathcal{A}|^{R-1}),$2-function is Lipschitz continuous, then the expected error in the $O(|\mathcal{A}|^{R-1}),$3-value under the generator satisfies

$O(|\mathcal{A}|^{R-1}),$4

where $O(|\mathcal{A}|^{R-1}),$5 is the Lipschitz constant of the $O(|\mathcal{A}|^{R-1}),$6-function (Tan et al., 22 Aug 2025). The interpretation given is that the performance gap between the generator-based conjecture and the true exhaustive maximum is explicitly bounded by the KL divergence between the distributions; by learning, this divergence, and thus the performance gap, can be driven small.

For convergence, PAC-MCoFL and PAC-MCoFL-p with expectile regression are stated to converge to a Nash equilibrium point for multi-agent reinforcement learning stochastic games under standard assumptions: finite state and action space, bounded reward, and existence of unique Nash equilibrium (Tan et al., 22 Aug 2025). The $O(|\mathcal{A}|^{R-1}),$7-function updates converge almost surely to the Nash $O(|\mathcal{A}|^{R-1}),$8-value, even with expectile regression critics (Tan et al., 22 Aug 2025). The convergence argument relies crucially on the TCAD mechanism to keep the action space finite.

These guarantees place PAC-MCoFL-p within a relatively structured class of decentralized learning algorithms: its scalability modification is not only empirical but also tied to a formal approximation statement and a convergence result.

6. Scalability, memory behavior, and empirical performance

The principal empirical distinction between PAC-MCoFL and PAC-MCoFL-p is scalability. PAC-MCoFL is described as out-of-memory or infeasible for more than three service providers, with GPU memory scaling exponentially. By contrast, PAC-MCoFL-p avoids exhaustive joint action enumeration, and its memory cost and run-time become practically independent of the number of service providers, as illustrated by the following values reported in the source summary (Tan et al., 22 Aug 2025).

Method	R=2	R=3
MAPPO	512MB	498MB
PAC-MCoFL	536MB	2132MB
PAC-MCoFL-p	496MB	490MB

Method	R=4	R=5
MAPPO	524MB	524MB
PAC-MCoFL	OOM	OOM
PAC-MCoFL-p	508MB	508MB

The summary explicitly states that PAC-MCoFL-p is the only variant feasible for $O(|\mathcal{A}|^{R-1}),$9 (Tan et al., 22 Aug 2025).

In terms of optimization quality, PAC-MCoFL achieves approximately 5.8% and 4.2% improvements in total reward and hypervolume indicator, respectively, over the latest multi-agent reinforcement learning solutions (Tan et al., 22 Aug 2025). The tabulated results reproduced in the source give PAC-MCoFL total reward $R$0 and HVI $R$1, while PAC-MCoFL-p yields total reward $R$2 and HVI $R$3 (Tan et al., 22 Aug 2025). PAC-MCoFL-p is reported as slightly less effective than PAC-MCoFL but still outperforming other methods, which is presented as confirmation of the theoretical bounded-error claim.

Additional ablation and robustness statements in the source note that ablating TCAD yields 7.6% lower total reward and 14.2% lower HVI, and that the method is robust to data heterogeneity, maintains a performance margin over other methods under non-IID settings, and extends to CIFAR-10, FashionMNIST, and MNIST deployments across differing service providers (Tan et al., 22 Aug 2025). These are empirical claims reported in the source summary rather than general properties of the algorithm in the abstract.

7. Interpretation, scope, and relation to adjacent PAC terminology

PAC-MCoFL-p belongs to the federated learning and multi-agent reinforcement learning literature rather than to other uses of the acronym “PAC” in coding theory, learning theory, or model theory. The arXiv corpus includes unrelated “PAC” usages such as Probably Approximately Correct Federated Learning (Zhang et al., 2023), PAC codes in coding theory (You et al., 2023), model-theoretic PAC structures (Hoffmann, 2018), and distributional PAC-learning in circuit complexity (Karchmer, 2023). PAC-MCoFL-p specifically denotes the scalable parameterized variant of Pareto Actor-Critic for communication and computation co-optimization in non-cooperative federated learning services (Tan et al., 22 Aug 2025).

Its application domain is heterogeneous, large-scale, non-cooperative federated learning ecosystems in which privacy constraints and competing interests prevent centralized resource optimization (Tan et al., 22 Aug 2025). Its optimization targets are client assignment, adaptive quantization, and resource allocation, while its methodological identity is determined by the conjunction of Pareto Actor-Critic learning, expectile regression, TCAD, and a parameterized conjecture generator.

A common misconception would be to interpret PAC-MCoFL-p merely as a lightweight implementation of PAC-MCoFL. The source material supports a stronger characterization: it is a scalable variant with a specific neural conjecture mechanism, an explicit KL-based error bound, and convergence guarantees to a Nash equilibrium point under stated assumptions (Tan et al., 22 Aug 2025). Another plausible implication is that the method operationalizes a trade-off between exact joint-action conjecture and tractable approximation, accepting a bounded sub-optimality gap in exchange for the ability to function in scaled deployments where the original exhaustive formulation is infeasible.

In this sense, PAC-MCoFL-p occupies the algorithmic point at which Pareto-oriented conjectural MARL becomes deployable beyond very small multi-service-provider settings.