FairMarket-RL: Fairness-Aware Market Learning

• The paper introduces FairMarket-RL, a hybrid LLM+MARL framework that uses episodic fairness critiques to augment traditional economic incentives.
• It employs distinct fairness signals—such as buyer fairness and seller peer fairness—to balance profit-driven behavior with equitable market outcomes in both microgrid and electricity market settings.
• Empirical evaluations demonstrate that LLM-guided fairness shaping boosts demand fulfillment and aligns sellers' behavior, outperforming no-LLM baselines in key fairness metrics.

Searching arXiv for the FairMarket-RL paper and closely related market-fairness RL work. FairMarket-RL denotes a line of fairness-aware market-learning frameworks in which reinforcement-learning agents operate inside decentralized trading environments and receive additional fairness guidance from an external critic rather than relying only on profit or efficiency rewards. In its original formulation, FairMarket-RL is a hybrid LLM+MARL framework for peer-to-peer trading in which a LLM evaluates completed trading episodes and returns fairness scores that are injected into the reward used by Independent Proximal Policy Optimization (IPPO) agents (Jadhav et al., 28 Jun 2025). A later extension applies the same design logic to peer-to-peer electricity markets under a continuous double auction, partial observability, discrete price-quantity actions, and storage dynamics, again using LLM-guided fairness shaping to complement economic incentives rather than replace them (Jadhav et al., 26 Aug 2025). Across both formulations, the central design pattern is to treat fairness as a training-time evaluative signal layered onto decentralized market learning, rather than as a fixed hard constraint or a purely post-hoc audit (Jadhav et al., 28 Jun 2025).

1. Origin, scope, and conceptual position

The original FairMarket-RL paper situates the framework in decentralized peer-to-peer markets, with the motivating case being microgrid trading among multiple sellers and buyers (Jadhav et al., 28 Jun 2025). Its premise is that plain RL optimizes the reward it is given and therefore tends to learn profit-maximizing behavior that can worsen inequity when rewards are purely monetary, while handcrafted fairness penalties are brittle, manually tuned, and difficult to transfer across market conditions (Jadhav et al., 28 Jun 2025). FairMarket-RL responds by inserting an instruction-tuned LLM into the reward loop as a fairness critic, so that the learned policy is still optimized by RL but receives episodic fairness assessments grounded in a structured summary of the realized market outcome (Jadhav et al., 28 Jun 2025).

This design places FairMarket-RL in a distinctive position relative to prior market-learning work. Earlier fairness-aware pricing systems typically encoded fairness directly in the reward, as in dynamic pricing with Jain’s index (Maestre et al., 2018), or learned an interpretable regulatory schedule that shifted firms’ incentives through taxation (Thibodeau et al., 30 May 2025). Related work in equitable marketplace mechanism design optimized exchange fees under a weighted inequality measure of participant profits (Dwarakanath et al., 2022). FairMarket-RL instead uses an external fairness critic that evaluates completed episodes and returns normalized fairness scores that are then folded back into MARL training (Jadhav et al., 28 Jun 2025). This suggests a modular separation between economic interaction, fairness evaluation, and policy optimization.

A subsequent extension preserves the FairMarket-RL name while moving from a simple peer-to-peer trading protocol to a continuous double auction for residential electricity trading with rooftop-PV prosumers, pure consumers, a price-taking grid, and optional battery storage (Jadhav et al., 26 Aug 2025). In that version, the fairness critic evaluates grid reliance, seller-side equity, and price-spread fairness after each hourly trading slot and shapes PPO rewards through ramped coefficients (Jadhav et al., 26 Aug 2025). The extension broadens the framework from a proof-of-concept microgrid market to a more structured electricity-market environment.

2. Original 2025 formulation: finite-horizon turn-based market game

In the original paper, the market is modeled as a finite-horizon, turn-based game

$$G = \left(S,\{A_S^{(i)}\}_{i=1}^{N_S}, \{A_B^{(j)}\}_{j=1}^{N_B}, T, R \right),$$

with seller agents $S_1,\dots,S_{N_S}$, buyer agents $B_1,\dots,B_{N_B}$, a deterministic transition function, and terminal rewards for all agents (Jadhav et al., 28 Jun 2025). The global state at time $t$ is

$$s_t = [I_1(t), \dots, I_{N_S}(t), D_1(t), \dots, D_{N_B}(t), p_1(t), q_1(t), \dots, p_{N_S}(t), q_{N_S}(t), o_t],$$

where seller inventories, buyer residual demands, current seller offers, and a stage indicator jointly define the market configuration (Jadhav et al., 28 Jun 2025).

The observation structure is asymmetric. A seller observes its own inventory and all buyers’ demands but not previous sellers’ offers, whereas a buyer observes the full state when its turn arrives (Jadhav et al., 28 Jun 2025). Sellers therefore move under incomplete information about rival quoting, while buyers react to the posted market. Seller $i$ chooses a price-quantity offer

$$(p_i, q_i) \in \{1,\dots,10\} \times \{0,\dots,I_i(t)\}$$

subject to $q_i \le I_i(t)$, and buyer $j$ chooses an allocation vector

$$b^{(j)}(t) = \left(b_1^{(j)}(t), \dots, b_{N_S}^{(j)}(t)\right)$$

subject to

$S_1,\dots,S_{N_S}$0

(Jadhav et al., 28 Jun 2025). After all buyers act, inventories and residual demands update deterministically: $S_1,\dots,S_{N_S}$1 Each episode consists of a single trading round unfolding in seller and then buyer stages, followed by reward computation and fairness scoring (Jadhav et al., 28 Jun 2025).

The game is mixed-motive rather than purely cooperative or purely competitive. Sellers compete on price, profits, and market share; buyers prefer lower cost and fulfilled demand; and all agents share a penalty tied to unmet demand (Jadhav et al., 28 Jun 2025). This structure is important because FairMarket-RL is not merely learning a static fair allocation rule. It is learning decentralized strategic behavior inside a market whose native incentives do not automatically align with equitable outcomes.

3. Fairness critic, fairness metrics, and reward shaping

The defining feature of FairMarket-RL is the LLM-guided reward shaping loop. After each episode, the full outcome ledger is serialized into a deterministic prompt, sent to the LLM fairness critic $S_1,\dots,S_{N_S}$2, and mapped to fairness scores that are added to the agents’ rewards (Jadhav et al., 28 Jun 2025). The original framework uses two fairness signals: Fairness-To-Buyer (FTB) and Fairness-Between-Sellers (FBS), both in $S_1,\dots,S_{N_S}$3 (Jadhav et al., 28 Jun 2025).

FTB is buyer-indexed and is intended to reward outcomes that satisfy buyer demand, keep effective prices reasonable, reduce buyer shortfalls, and avoid overcharging (Jadhav et al., 28 Jun 2025). FBS is a global score reflecting fairness among sellers, including balanced seller margins, profit parity, anti-monopoly behavior, and penalties for lopsided sales shares or excessive markups (Jadhav et al., 28 Jun 2025). The paper is explicit that it does not provide closed-form mathematical definitions of FTB and FBS; they are operational outputs of the LLM critic computed from episode summaries rather than hand-written formulas in the paper text (Jadhav et al., 28 Jun 2025).

The raw seller reward is

$S_1,\dots,S_{N_S}$4

where $S_1,\dots,S_{N_S}$5 is total unmet demand and $S_1,\dots,S_{N_S}$6 is seller $S_1,\dots,S_{N_S}$7's unsold inventory (Jadhav et al., 28 Jun 2025). The raw buyer reward is

$S_1,\dots,S_{N_S}$8

The fairness-shaped rewards are

$S_1,\dots,S_{N_S}$9

for sellers and

$B_1,\dots,B_{N_B}$0

for buyers, where $B_1,\dots,B_{N_B}$1 denotes seller $B_1,\dots,B_{N_B}$2’s share of total units sold (Jadhav et al., 28 Jun 2025). Thus buyer fairness enters as an average seller-side bonus and an individual buyer-side bonus, while peer fairness among sellers is distributed in proportion to realized sales share.

The fairness coefficients are staged. $B_1,\dots,B_{N_B}$3 ramps from 0 to 1 during the first 20% of training, while $B_1,\dots,B_{N_B}$4 ramps from 30% to 80% of training (Jadhav et al., 28 Jun 2025). The rationale is to let agents first learn basic profitable market functioning, then buyer fairness, and only later seller peer fairness. This suggests a curriculum-like approach in which fairness shaping is delayed until economic competence is established.

The later electricity-market extension uses the same architectural idea but with three different fairness scores: Fairness-to-Grid (FTG), Fairness-Between-Sellers (FBS), and Fairness-of-Pricing (FPP), again normalized to $B_1,\dots,B_{N_B}$5 and produced by an LLM from the post-slot trade ledger (Jadhav et al., 26 Aug 2025). FTG measures grid reliance, FBS measures equitable sharing of trade opportunity among sellers, and FPP measures price-spread fairness; the paper gives their economic semantics but not explicit algebraic formulas (Jadhav et al., 26 Aug 2025). The prosumer reward becomes

$B_1,\dots,B_{N_B}$6

where $B_1,\dots,B_{N_B}$7 is raw prosumer profit and the $B_1,\dots,B_{N_B}$8-coefficients follow a piecewise linear ramp schedule over episodes (Jadhav et al., 26 Aug 2025).

4. Learning architecture and empirical findings

The original FairMarket-RL uses Independent Proximal Policy Optimization. Each agent has its own actor and critic, and the framework is described as fully decentralized, with complexity scaling as $B_1,\dots,B_{N_B}$9 (Jadhav et al., 28 Jun 2025). The paper reports a training horizon of 20,000 episodes and discount factor $t$0, but does not provide PPO clip ratio, learning rate, architecture widths, or other optimization details in the text provided (Jadhav et al., 28 Jun 2025). The key ablation is a No-LLM baseline obtained by setting the fairness-shaping coefficients to zero (Jadhav et al., 28 Jun 2025).

In the reported two-seller/one-buyer microgrid case study, the framework achieves over the final 2,000 episodes: 92.1% of episodes with full demand met, average FTB of 0.88, average FBS of 0.87, seller margins of 24–26%, and maximum seller sales share of 57% (Jadhav et al., 28 Jun 2025). The paper sets success criteria of at least 90% total demand met, average FTB and FBS at least 0.80, seller margins in 20–30%, and no seller above 60% total sales share, so these outcomes satisfy the stated targets (Jadhav et al., 28 Jun 2025). Training dynamics show that when buyer fairness shaping begins around episode 2,000, sellers reduce prices and buyer return improves, and when peer fairness shaping saturates around episode 4,500, seller reward curves converge toward each other (Jadhav et al., 28 Jun 2025).

The no-LLM ablation is the strongest causal evidence for the critic’s contribution. With fairness shaping disabled, fairness metrics plateau around 0.35–0.40, demand fulfillment drops to about 70%, and Seller 2 outperforms Seller 1 by roughly 35% in average profit (Jadhav et al., 28 Jun 2025). This supports the interpretation that the fairness critic changes not only scalar reward values but the emergent market equilibrium.

The electricity-market extension moves from episodic single-round trading to hourly trading in a continuous double auction over horizons of 30 or 90 days, using PPO with fairness-shaped rewards (Jadhav et al., 26 Aug 2025). The environment models true loads and PV outputs as

$t$1

with battery state evolution

$t$2

and local observations

$t$3

under 5% Gaussian multiplicative forecast noise (Jadhav et al., 26 Aug 2025). The auction clears greedily while

$t$4

and unmet demand is procured from the grid while unsold surplus is sold to the grid (Jadhav et al., 26 Aug 2025).

Empirically, the extension reports a persistent local-first energy split of about 54% P2P and 46% grid across all three case studies: 1,026 kWh P2P versus 874 kWh grid in Case 1, 2,260 kWh P2P versus 1,960 kWh grid in Case 2, and about 4,900 kWh P2P versus about 4,250 kWh grid transactions in Case 3 (Jadhav et al., 26 Aug 2025). Consumers’ average monthly costs in the scaled synthetic community are reported as $t$5310, and $325 for the three consumers, approximately 27–28% lower than grid-only baselines, while the real-world case reports consumer bill reductions of about 30% versus grid-only (Jadhav et al., 26 Aug 2025). Fairness metrics remain high, with FPP above 0.95, FBS near 0.90, and FTG improving into the $t$6–$t$7 range depending on case and training stage (Jadhav et al., 26 Aug 2025). The paper also reports seller-share entropy between 0.60 and 0.96 with mean about 0.78 and Jain’s fairness index above 0.90 for roughly two-thirds of hours in the scaled case (Jadhav et al., 26 Aug 2025).

5. Relation to broader fair-market RL and mechanism design

FairMarket-RL belongs to a broader family of attempts to embed fairness into market decision systems, but it occupies a distinctive technical niche. In dynamic pricing, fairness has been introduced by shaping reward with a fairness statistic such as a rotated Jain’s index over group-average prices (Maestre et al., 2018). In risk-based pricing markets, a regulator can instead learn an interpretable bracketed tax schedule that reshapes firms’ incentives and raises demand fairness while preserving profitability (Thibodeau et al., 30 May 2025). In marketplace mechanism design, the operator can optimize exchange fees against a weighted generalized entropy measure of participant profits, explicitly decomposing within-group and between-group inequity (Dwarakanath et al., 2022). In offline RL for high-stakes domains, fairness can be enforced at the feasibility layer through subgroup-specific calibration thresholds before policy learning rather than by end-to-end fairness regularization (Basu et al., 11 Sep 2025).

FairMarket-RL differs from those approaches in two ways. First, fairness is not defined exclusively by a closed-form constraint or regularizer embedded directly in the environment. Instead, it is evaluated ex post on realized market outcomes by an external critic that returns a compact score (Jadhav et al., 28 Jun 2025). Second, the framework is explicitly multi-agent and decentralized, so fairness shaping acts on strategic interaction rather than only on a single policy or regulator. This suggests a flexible but less formally characterized mechanism.

At the same time, the broader literature clarifies several conceptual tensions that also apply to FairMarket-RL. Work on balancing profit and fairness in pricing markets warns that parity-style fairness can be achieved by “leveling down,” increasing opt-out for some groups even as disparity shrinks (Thibodeau et al., 30 May 2025). Equitable marketplace mechanism design shows that weighting fairness toward one strategic intermediary can indirectly improve outcomes for other classes, but only through equilibrium effects that remain domain-dependent (Dwarakanath et al., 2022). Fair offline RL work emphasizes that fairness interventions grounded in calibration and observability remain sensitive to subgroup definition, small-sample issues, and distribution shift (Basu et al., 11 Sep 2025). These adjacent results suggest that FairMarket-RL’s fairness scores should be interpreted as governance levers rather than guarantees of universally desirable social outcomes.

6. Limitations, interpretation, and subsequent significance

Several limitations are explicit. In the original framework, the LLM critic returns FTB and FBS, but the paper does not provide closed-form formulas for those scores, nor the exact prompt text or JSON schema used to elicit them (Jadhav et al., 28 Jun 2025). Invalid LLM outputs are discarded rather than replaced with heuristic substitutes, which preserves signal purity but may reduce sample efficiency; the paper does not report how often that occurs (Jadhav et al., 28 Jun 2025). The experimental environment is strategically nontrivial but small, with only two sellers and one buyer in the core case study (Jadhav et al., 28 Jun 2025). The electricity extension adds realism through a continuous double auction, weather-modulated PV, and battery dynamics, yet still omits explicit distribution-network constraints in the formal environment and does not disclose network architecture or full PPO implementation details (Jadhav et al., 26 Aug 2025).

There are also normative and methodological ambiguities. Because the fairness critic is external and language-based, the framework is more interpretable at the level of named fairness channels than a black-box learned critic, but less transparent than a system whose fairness function is explicitly specified as a closed-form objective (Jadhav et al., 28 Jun 2025). This suggests a trade-off between semantic flexibility and formal auditability. The later extension mitigates this partly by defining the intended semantics of FTG, FBS, and FPP in economic terms and by showing robustness under $t$8 PV and $t$9 load perturbations, but it still does not fully formalize the critic (Jadhav et al., 26 Aug 2025).

A further limitation is that neither version establishes classical mechanism-design guarantees such as incentive compatibility, equilibrium uniqueness, or welfare-optimality. FairMarket-RL is best understood as a reward-shaping architecture for decentralized market learning rather than as a closed-form fair mechanism. By contrast, adjacent work on taxes and subsidies in Fisher markets derives fairness interventions from dual variables in a constrained equilibrium program (Peysakhovich et al., 2022), and work on regulatory RL in pricing markets anchors learned policy to a monotone linear prior via an $$s_t = [I_1(t), \dots, I_{N_S}(t), D_1(t), \dots, D_{N_B}(t), p_1(t), q_1(t), \dots, p_{N_S}(t), q_{N_S}(t), o_t],$$0 deviation penalty (Thibodeau et al., 30 May 2025). Those approaches offer stronger analytical structure. FairMarket-RL instead prioritizes modularity: agents, market rules, and fairness evaluator can be recombined without solving a new constrained optimization problem from scratch.

Its significance lies in showing that fairness shaping can be inserted into the training loop of decentralized market agents through a critic that consumes structured market ledgers rather than raw natural language dialogue (Jadhav et al., 28 Jun 2025). This extends the use of LLMs beyond text generation into normative evaluation for non-text MARL. The later electricity-market variant shows that the same design can scale from a minimal peer-to-peer trading game to a larger partially observable continuous double auction with storage and real-world household profiles (Jadhav et al., 26 Aug 2025). A plausible implication is that FairMarket-RL serves less as a single fixed algorithm than as a reusable architectural pattern: decentralized market agents optimized by RL, a structured ledger of realized outcomes, and an external fairness critic whose scores are ramped into reward shaping over training.