FairSkillMARL: Fairness in Heterogeneous MARL

• FairSkillMARL is a fairness framework for heterogeneous multi-agent reinforcement learning that combines workload balance and skill–task alignment to enhance team coordination and safety.
• It integrates fairness as a penalty in reward shaping, using metrics like the Gini index and alignment ratio to evaluate and adjust agent task assignments.
• Empirical results in healthcare settings show that its tunable multi-objective approach effectively manages trade-offs between performance, fairness, and efficiency.

Searching arXiv for the named papers and closely related FairSkillMARL/fair-MARL work to ground the article in current literature. FairSkillMARL denotes a fairness framework for heterogeneous multi-agent reinforcement learning in which fairness is defined as a dual objective: workload balance and skill–task alignment. In the formulation introduced for healthcare collaboration, the framework targets teams whose members have heterogeneous expertise and must coordinate on temporally structured, order-dependent procedures under operational constraints such as fatigue and recharge. Rather than treating fairness as equal task counts alone, FairSkillMARL penalizes both workload disparity and mismatch between assigned tasks and agent proficiency, and integrates that penalty into standard MARL training through reward shaping (Ekpo et al., 26 Aug 2025). Closely related work also places “fair skill allocation” within a broader constrained resource-allocation setting, where decentralized value estimation is coupled to centralized allocation under global limits, providing a second formal route for FairSkillMARL-style problems (Kumar et al., 6 Feb 2025).

1. Conceptual definition and problem setting

FairSkillMARL addresses settings in which heterogeneous agents must coordinate on shared procedures while expertise matters for both safety and efficiency. The motivating domain is emergency and acute-care workflow, where healthcare workers may differ in proficiency for CPR, rescue breaths, or AED-related subtasks. In that setting, a fairness notion based only on equal workload can itself be unfair: highly skilled agents may be overused, creating burnout risk, while less skilled agents may be assigned mismatched tasks, increasing completion time and error. The framework therefore defines fairness as a composite of two components: workload balance and skill–task alignment (Ekpo et al., 26 Aug 2025).

The underlying decision process is a Dec-POMDP $(I,S,A,T,R,\Omega,O,\gamma)$. The base team reward is not replaced by a separate fairness objective; instead, fairness appears as a penalty added to the reward signal. This makes FairSkillMARL a shaping-based framework rather than a new MARL solver. Its intended use is with heterogeneous teams, structured tasks, and constraints such as shared-task alternation, rather than with exchangeable agents performing interchangeable work (Ekpo et al., 26 Aug 2025).

A central misconception addressed by the framework is that “fairness” and “equal work” are equivalent. In the reported healthcare setting, equalized subtask counts can still produce skill–task mismatch, and this mismatch can harm patient safety and throughput. FairSkillMARL therefore treats expertise alignment as part of fairness itself, not merely as an auxiliary efficiency term (Ekpo et al., 26 Aug 2025).

2. Mathematical formulation of the fairness objective

The workload-balance term is defined by a Gini index over per-agent task counts or effort units. Let $n$ be the number of agents, $x_i$ the number of subtasks assigned to agent $i$, and $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$. Then

$$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$

Here, $L_1=0$ corresponds to perfectly balanced workload, and larger values indicate greater inequality (Ekpo et al., 26 Aug 2025).

The skill–task alignment term is defined relative to the best available team expertise for each subtask. Let $\mathcal{T}$ be the set of subtasks in an episode, let $i_t$ be the agent assigned to subtask $t\in\mathcal{T}$, and let $n$0 denote agent $n$1’s skill level for subtask $n$2. The framework uses

$n$3

The alignment ratio measures how close the realized assignment is to the team’s best-possible skill per task, while $n$4 penalizes deviation from that optimum. Lower $n$5 therefore means better skill–task alignment (Ekpo et al., 26 Aug 2025).

These two components are combined as

$n$6

with $n$7 controlling the trade-off between workload balance and skill alignment. Training uses the shaped reward

$n$8

where $n$9 scales the fairness penalty and the base team reward is

$x_i$0

The heuristic $x_i$1 measures subgoal progress, so the final training signal simultaneously rewards task progress and penalizes composite unfairness (Ekpo et al., 26 Aug 2025).

This construction makes the fairness–efficiency trade-off explicit. Increasing $x_i$2 penalizes disparity more strongly; changing $x_i$3 redistributes pressure between equalized effort and expertise-congruent assignment. A plausible implication is that FairSkillMARL is best viewed as a multi-objective shaping mechanism embedded inside standard MARL pipelines rather than as a standalone fairness metric.

3. Environment model and operational constraints

The principal evaluation environment is MARLHospital, a healthcare-inspired Dec-POMDP wrapper over a PDDL planner, built on Robotouille with EPyMARL integration for MARL baselines. Each episode involves 3 healthcare workers. Skill vectors are embedded in the state, and the environment supports three team configurations: uniform teams with identical skills, specialized teams in which each agent is most proficient at a particular subtask but can perform others less efficiently, and interdependent teams in which agents can only perform subsets of subtasks (Ekpo et al., 26 Aug 2025).

The state includes agent 2D positions $x_i$4, skill levels, energy $x_i$5, and ongoing actions such as treatment. The action space contains discrete move actions, pick/place/stack/unstack, treatment actions including compress_chest and give_rescue_breaths, and noop. Tasks include CPR/chest compressions, rescue breaths, and procedurally configurable AED-like steps. Transition dynamics are standard Dec-POMDP dynamics with order-dependent subgoals that must be satisfied in sequence. Episodes have a horizon of 50 timesteps, and evaluation uses success rate, defined as goal reached (Ekpo et al., 26 Aug 2025).

A distinctive part of the setting is its explicit energy-constrained scheduling model. Each agent has energy $x_i$6. Costly actions consume energy; non-costly actions, including rest or noop, recharge at a constant rate. A task is executable only if the agent has sufficient energy, otherwise rest/noop is enforced. In the reported runs, the cost for compressions is 3 and the recharge rate is 1, with a baseline condition in which the compression cost is 0. Shared-task alternation emerges because agents must recharge, modeling real CPR guidance to switch providers approximately every 2 minutes (Ekpo et al., 26 Aug 2025).

Within an episode, fairness measurement tracks workloads $x_i$7, assigned-skill terms $x_i$8, and the derived values $x_i$9, $i$0, and $i$1. The framework also reports agent contribution shares as percentages of subtasks and their range, defined as max–min contribution. Across training, cumulative $i$2 is used to visualize disparity trends (Ekpo et al., 26 Aug 2025).

4. Integration with MARL algorithms and empirical behavior

FairSkillMARL is integrated into four standard MARL methods: IQL, MAPPO, VDN, and QMIX. The integration mechanism is direct reward shaping: the original reward $i$3 is replaced with $i$4. Per-episode tallies are maintained to compute workload and alignment terms, and shaped rewards can be updated either online through running estimates or at episode end by distributing penalties across the trajectory. Off-policy methods are trained for 2M environment timesteps, on-policy methods for 20M, with 4 random seeds and evaluation over 100 episodes per seed (Ekpo et al., 26 Aug 2025).

In the MARLHospital benchmark, CTDE methods consistently outperform independent or on-policy baselines on both chest-compression and rescue-breath tasks, and VDN is reported to excel overall. Forced cooperation lowers success rates, reflecting coordination burden and non-interchangeability. Energy constraints can also aid learning by imposing structured alternation that simplifies coordination and credit assignment; in specialized teams with energy, MAPPO reaches 0.97 success on chest compressions (Ekpo et al., 26 Aug 2025).

The reported fairness analysis emphasizes the trade-off between workload-only metrics and skill-aware fairness. In specialized teams with moderate fairness penalty $i$5, the reported setup includes: “Workload-only $i$6” with success $i$7, Gini $i$8, shares $i$9, and range $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$0; FairSkillMARL $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$1 with success $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$2, Gini $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$3, shares $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$4, and range $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$5; and FEN with success $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$6, Gini $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$7, shares $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$8, and range $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$9. The accompanying interpretation states that, with moderate penalty, FEN improves success and balance, whereas FairSkillMARL prioritizes skill alignment and skews work toward the most proficient agent, raising the contribution range (Ekpo et al., 26 Aug 2025).

Under strong fairness penalty $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$0, the paper reports that FEN achieves success $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$1 and lower range $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$2, while FairSkillMARL reaches success $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$3; the interpretation is that the alignment term can conflict with strong penalties, reducing both performance and balance when the system is over-constrained. In equal teams with $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$4, FairSkillMARL $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$5 yields success $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$6, Gini $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$7, shares $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$8, and range $$L_1 \;=\; \frac{\sum_{i=1}^{n}\sum_{j=1}^{n}|x_i-x_j|}{2\,n^2\,\bar{x}}.$$9, while FEN has success $L_1=0$0, Gini $L_1=0$1, and an extreme underutilization of Agent 2 at $L_1=0$2 with range $L_1=0$3, which is interpreted as unfair specialization under equal skills (Ekpo et al., 26 Aug 2025).

The ablation on $L_1=0$4 at $L_1=0$5 reports workload-only success $L_1=0$6, Gini $L_1=0$7, shares $L_1=0$8, range $L_1=0$9; FEN success $\mathcal{T}$0, Gini $\mathcal{T}$1, shares $\mathcal{T}$2, range $\mathcal{T}$3; and FairSkillMARL results of $\mathcal{T}$4 for $\mathcal{T}$5, $\mathcal{T}$6 for $\mathcal{T}$7, and $\mathcal{T}$8 for $\mathcal{T}$9. The reported takeaway is that $i_t$0 offers a favorable balance between performance and fairness under moderate penalty, whereas pure workload balancing $i_t$1 can degrade performance and raise disparity. Statistical evidence is also reported: for specialized teams, at $i_t$2, FairSkillMARL $i_t$3 outperforms workload-only in coordinated fairness $i_t$4, while at $i_t$5, skill alignment with strong penalties hurts performance $i_t$6 (Ekpo et al., 26 Aug 2025).

5. Relation to broader fair-MARL and skill-allocation formulations

A broader formalization of fair skill/resource allocation is provided by DECAF, which models Distributed Evaluation, Centralized Allocation (DECA) as a constrained multi-agent MDP and is explicitly proposed as a route to “FairSkillMARL via DECAF.” In that mapping, agents are learners, resources are training modules, tutoring slots, equipment, or mentoring sessions, actions encode allocations including a null action, utility corresponds to expected performance or skill gain, and accumulated skill metrics $i_t$7 can represent cumulative skill attainment or normalized skill rate. The central allocator solves an ILP at each time step, while fairness is learned through Double DQN-based methods: joint weighted optimization (JO), split optimization with separate utility and fairness estimators (SO), and fair-only optimization as online policy perturbation of a black-box utility model (FO). The framework supports variance, $i_t$8-fairness, generalized Gini, and maximin fairness objectives, and the reported summary identifies SO as especially attractive because it permits online control of the utility–fairness trade-off without retraining (Kumar et al., 6 Feb 2025).

FairSkillMARL also sits within a longer line of welfare-oriented fair MARL. One influential formulation optimizes a social welfare function $i_t$9 over user utilities and introduces the SOTO architecture, which separates self-oriented and team-oriented actor–critic heads. That approach supports GGF, proportional fairness, and $t\in\mathcal{T}$0-fairness, and treats fairness as welfare optimization over expected discounted returns rather than as environment-specific reward penalties. Its importance for FairSkillMARL lies in the architectural separation between efficiency-sensitive and equity-sensitive control, which offers a generic fairness credit-assignment mechanism under decentralized policies (Zimmer et al., 2020).

Related work in decentralized navigation shows a different operationalization of fairness: min–max fair goal assignment combined with a fairness reward based on the reciprocal of the coefficient of variation of cumulative traveled distances. There, fair task allocation and fair navigation are jointly learned under CTDE, and the reported outcome is almost perfect goal coverage with improved fairness and modest efficiency loss. This line is not skill-alignment-based, but it clarifies that fair MARL can target equalized burden rather than equalized task counts, which is conceptually adjacent to FairSkillMARL’s emphasis on competence-sensitive burden sharing (Aloor et al., 2024).

A further connection concerns robustness. Decentralized robust fairness under graded contention has been studied without a central allocator, using a permutation-equivariant cross-attention policy trained by PSRO. That work shows that robust fairness in decentralized settings depends on the existence of contest leverage: when contested resources still return a positive share to cooperators, decentralized policies can resist free-riding; when leverage vanishes, such as in winner-take-all settings, robustness fails. This does not define FairSkillMARL directly, but it frames an important boundary condition for fairness mechanisms that rely on decentralized coordination rather than on centralized assignment (Savcı, 4 Jun 2026).

6. Interpretation, trade-offs, and limitations

The main interpretive claim associated with FairSkillMARL is that equal workload alone can be unfair or suboptimal in heterogeneous teams. When skills differ, low workload inequality does not imply low skill–task misalignment. Assigning complex tasks to less skilled agents can increase time and error while still appearing “fair” under pure workload metrics. FairSkillMARL’s $t\in\mathcal{T}$1 term is intended to capture exactly this failure mode (Ekpo et al., 26 Aug 2025).

At the same time, the framework makes the price of fairness explicit. Increasing $t\in\mathcal{T}$2 penalizes disparity more strongly; moderate $t\in\mathcal{T}$3 can improve coordination and balance, but excessive $t\in\mathcal{T}$4 may conflict with alignment and reduce returns. The reported empirical picture is therefore not monotone: stronger fairness shaping does not necessarily improve either success or balanced contribution, and under strong penalties simpler utilization-parity shaping such as FEN can outperform FairSkillMARL in both success and balance. This suggests that the framework should be treated as a tunable multi-objective intervention rather than as a universally dominant fairness criterion (Ekpo et al., 26 Aug 2025).

The reported limitations are likewise specific. Skill representation is simplified and discrete; clinical performance variability and outcome-linked risk are abstracted; the alignment metric $t\in\mathcal{T}$5 uses static skill scores; and the paper provides empirical evidence but no formal optimality or convergence guarantees for fairness. Scaling to larger teams, richer procedures, and more healthcare scenarios is left open, as is the possibility of replacing reward shaping with constrained RL or Pareto-optimal multi-objective MARL (Ekpo et al., 26 Aug 2025).

A final misconception concerns centralization. FairSkillMARL does not require a single architectural pattern. In healthcare, it is implemented as reward shaping inside standard MARL methods; in DECAF, fair skill allocation is handled by centralized ILP-based allocation over learned local value estimates; and in decentralized robust-fairness work, central allocation is absent altogether. A plausible implication is that “FairSkillMARL” is best understood not as one algorithm but as a family of fairness formulations for heterogeneous multi-agent coordination in which expertise, burden, and constrained allocation interact.