---
title: 'FairMarket-RL: Fairness-Aware Market Learning'
url: https://www.emergentmind.com/topics/fairmarket-rl
type: topic
---

# FairMarket-RL: Fairness-Aware Market Learning

Searching arXiv for the FairMarket-RL paper and closely related market-fairness RL work.
FairMarket-RL denotes a line of fairness-aware market-learning frameworks in which reinforcement-learning agents operate inside decentralized trading environments and receive additional fairness guidance from an external critic rather than relying only on profit or efficiency rewards. In its original formulation, FairMarket-RL is a hybrid LLM+MARL framework for peer-to-peer trading in which a Large Language Model evaluates completed trading episodes and returns fairness scores that are injected into the reward used by Independent Proximal Policy Optimization (IPPO) agents [2506.22708]. A later extension applies the same design logic to peer-to-peer electricity markets under a continuous double auction, partial observability, discrete price-quantity actions, and storage dynamics, again using LLM-guided fairness shaping to complement economic incentives rather than replace them [2508.18610]. Across both formulations, the central design pattern is to treat fairness as a training-time evaluative signal layered onto decentralized market learning, rather than as a fixed hard constraint or a purely post-hoc audit [2506.22708].

## 1. Origin, scope, and conceptual position

The original FairMarket-RL paper situates the framework in decentralized peer-to-peer markets, with the motivating case being microgrid trading among multiple sellers and buyers [2506.22708]. Its premise is that plain RL optimizes the reward it is given and therefore tends to learn profit-maximizing behavior that can worsen inequity when rewards are purely monetary, while handcrafted fairness penalties are brittle, manually tuned, and difficult to transfer across market conditions [2506.22708]. FairMarket-RL responds by inserting an instruction-tuned LLM into the reward loop as a fairness critic, so that the learned policy is still optimized by RL but receives episodic fairness assessments grounded in a structured summary of the realized market outcome [2506.22708].

This design places FairMarket-RL in a distinctive position relative to prior market-learning work. Earlier fairness-aware pricing systems typically encoded fairness directly in the reward, as in dynamic pricing with Jain’s index [1803.09967], or learned an interpretable regulatory schedule that shifted firms’ incentives through taxation [2506.00140]. Related work in equitable marketplace mechanism design optimized exchange fees under a weighted inequality measure of participant profits [2209.15418]. FairMarket-RL instead uses an external fairness critic that evaluates completed episodes and returns normalized fairness scores that are then folded back into MARL training [2506.22708]. This suggests a modular separation between economic interaction, fairness evaluation, and policy optimization.

A subsequent extension preserves the FairMarket-RL name while moving from a simple peer-to-peer trading protocol to a continuous double auction for residential electricity trading with rooftop-PV prosumers, pure consumers, a price-taking grid, and optional battery storage [2508.18610]. In that version, the fairness critic evaluates grid reliance, seller-side equity, and price-spread fairness after each hourly trading slot and shapes PPO rewards through ramped coefficients [2508.18610]. The extension broadens the framework from a proof-of-concept microgrid market to a more structured electricity-market environment.

## 2. Original 2025 formulation: finite-horizon turn-based market game

In the original paper, the market is modeled as a finite-horizon, turn-based game
\[
G = \left(S,\{A_S^{(i)}\}_{i=1}^{N_S}, \{A_B^{(j)}\}_{j=1}^{N_B}, T, R \right),
\]
with seller agents \(S_1,\dots,S_{N_S}\), buyer agents \(B_1,\dots,B_{N_B}\), a deterministic transition function, and terminal rewards for all agents [2506.22708]. The global state at time \(t\) is
\[
s_t = [I_1(t), \dots, I_{N_S}(t), D_1(t), \dots, D_{N_B}(t), p_1(t), q_1(t), \dots, p_{N_S}(t), q_{N_S}(t), o_t],
\]
where seller inventories, buyer residual demands, current seller offers, and a stage indicator jointly define the market configuration [2506.22708].

The observation structure is asymmetric. A seller observes its own inventory and all buyers’ demands but not previous sellers’ offers, whereas a buyer observes the full state when its turn arrives [2506.22708]. Sellers therefore move under incomplete information about rival quoting, while buyers react to the posted market. Seller \(i\) chooses a price-quantity offer
\[
(p_i, q_i) \in \{1,\dots,10\} \times \{0,\dots,I_i(t)\}
\]
subject to \(q_i \le I_i(t)\), and buyer \(j\) chooses an allocation vector
\[
b^{(j)}(t) = \left(b_1^{(j)}(t), \dots, b_{N_S}^{(j)}(t)\right)
\]
subject to
\[
\sum_i b_i^{(j)} \le D_j(t), \qquad b_i^{(j)} \le q_i
\]
[2506.22708]. After all buyers act, inventories and residual demands update deterministically:
\[
I_i(t+1) = I_i(t) - \sum_j b_i^{(j)}, \qquad D_j(t+1) = D_j(t) - \sum_i b_i^{(j)}.
\]
Each episode consists of a single trading round unfolding in seller and then buyer stages, followed by reward computation and fairness scoring [2506.22708].

The game is mixed-motive rather than purely cooperative or purely competitive. Sellers compete on price, profits, and market share; buyers prefer lower cost and fulfilled demand; and all agents share a penalty tied to unmet demand [2506.22708]. This structure is important because FairMarket-RL is not merely learning a static fair allocation rule. It is learning decentralized strategic behavior inside a market whose native incentives do not automatically align with equitable outcomes.

## 3. Fairness critic, fairness metrics, and reward shaping

The defining feature of FairMarket-RL is the LLM-guided reward shaping loop. After each episode, the full outcome ledger is serialized into a deterministic prompt, sent to the LLM fairness critic \(f_{\text{LLM}}\), and mapped to fairness scores that are added to the agents’ rewards [2506.22708]. The original framework uses two fairness signals: Fairness-To-Buyer (FTB) and Fairness-Between-Sellers (FBS), both in \([0,1]\) [2506.22708].

FTB is buyer-indexed and is intended to reward outcomes that satisfy buyer demand, keep effective prices reasonable, reduce buyer shortfalls, and avoid overcharging [2506.22708]. FBS is a global score reflecting fairness among sellers, including balanced seller margins, profit parity, anti-monopoly behavior, and penalties for lopsided sales shares or excessive markups [2506.22708]. The paper is explicit that it does not provide closed-form mathematical definitions of FTB and FBS; they are operational outputs of the LLM critic computed from episode summaries rather than hand-written formulas in the paper text [2506.22708].

The raw seller reward is
\[
r_{i,\text{raw}}^{[S_i]} = (p_i - c)\cdot \sum_j b_i^{(j)} - \alpha \cdot D_{\text{unsat}} - \beta \cdot I_{i,\text{unsold}},
\]
where \(D_{\text{unsat}}\) is total unmet demand and \(I_{i,\text{unsold}}\) is seller \(i\)'s unsold inventory [2506.22708]. The raw buyer reward is
\[
r_{j,\text{raw}}^{[B_j]} = - \sum_i p_i b_i^{(j)} - \alpha \cdot D_{\text{unsat}}.
\]
The fairness-shaped rewards are
\[
R^{[S_i]} = r_{i,\text{raw}}^{[S_i]} + \lambda_{\text{buy}(t)} \cdot W_B \cdot \frac{\sum_j \mathrm{FTB}_j}{N_B} + \lambda_{\text{peer}(t)} \cdot W_P \cdot \mathrm{FBS}\cdot \Sigma_i
\]
for sellers and
\[
R^{[B_j]} = r_{j,\text{raw}}^{[B_j]} + \lambda_{\text{buy}(t)} \cdot W_B \cdot \mathrm{FTB}_j
\]
for buyers, where \(\Sigma_i\) denotes seller \(i\)’s share of total units sold [2506.22708]. Thus buyer fairness enters as an average seller-side bonus and an individual buyer-side bonus, while peer fairness among sellers is distributed in proportion to realized sales share.

The fairness coefficients are staged. \(\lambda_{\text{buy}(t)}\) ramps from 0 to 1 during the first 20% of training, while \(\lambda_{\text{peer}(t)}\) ramps from 30% to 80% of training [2506.22708]. The rationale is to let agents first learn basic profitable market functioning, then buyer fairness, and only later seller peer fairness. This suggests a curriculum-like approach in which fairness shaping is delayed until economic competence is established.

The later electricity-market extension uses the same architectural idea but with three different fairness scores: Fairness-to-Grid (FTG), Fairness-Between-Sellers (FBS), and Fairness-of-Pricing (FPP), again normalized to \([0,1]\) and produced by an LLM from the post-slot trade ledger [2508.18610]. FTG measures grid reliance, FBS measures equitable sharing of trade opportunity among sellers, and FPP measures price-spread fairness; the paper gives their economic semantics but not explicit algebraic formulas [2508.18610]. The prosumer reward becomes
\[
R_{i,t} = \Pi_{i,t} + \lambda_{\text{grid}(e)}\beta_{\text{grid}}\text{FTG}_t + \lambda_{\text{price}(e)}\beta_{\text{price}}\text{FPP}_t + \lambda_{\text{peer}(e)}\beta_{\text{peer}} \left( \frac{q^{\text{sold}}_{i,t}}{\sum_{k=1}^{m} q^{\text{sold}}_{k,t}} \right) \text{FBS}_t,
\]
where \(\Pi_{i,t}\) is raw prosumer profit and the \(\lambda\)-coefficients follow a piecewise linear ramp schedule over episodes [2508.18610].

## 4. Learning architecture and empirical findings

The original FairMarket-RL uses Independent Proximal Policy Optimization. Each agent has its own actor and critic, and the framework is described as fully decentralized, with complexity scaling as \(O(N_S + N_B)\) [2506.22708]. The paper reports a training horizon of 20,000 episodes and discount factor \(\gamma=0.95\), but does not provide PPO clip ratio, learning rate, architecture widths, or other optimization details in the text provided [2506.22708]. The key ablation is a No-LLM baseline obtained by setting the fairness-shaping coefficients to zero [2506.22708].

In the reported two-seller/one-buyer microgrid case study, the framework achieves over the final 2,000 episodes: 92.1% of episodes with full demand met, average FTB of 0.88, average FBS of 0.87, seller margins of 24–26%, and maximum seller sales share of 57% [2506.22708]. The paper sets success criteria of at least 90% total demand met, average FTB and FBS at least 0.80, seller margins in 20–30%, and no seller above 60% total sales share, so these outcomes satisfy the stated targets [2506.22708]. Training dynamics show that when buyer fairness shaping begins around episode 2,000, sellers reduce prices and buyer return improves, and when peer fairness shaping saturates around episode 4,500, seller reward curves converge toward each other [2506.22708].

The no-LLM ablation is the strongest causal evidence for the critic’s contribution. With fairness shaping disabled, fairness metrics plateau around 0.35–0.40, demand fulfillment drops to about 70%, and Seller 2 outperforms Seller 1 by roughly 35% in average profit [2506.22708]. This supports the interpretation that the fairness critic changes not only scalar reward values but the emergent market equilibrium.

The electricity-market extension moves from episodic single-round trading to hourly trading in a continuous double auction over horizons of 30 or 90 days, using PPO with fairness-shaped rewards [2508.18610]. The environment models true loads and PV outputs as
\[
L_{a,t} = L_a \cdot P_{\text{load}(h_t)} \cdot \eta_{a,t}, \qquad
G_{a,t} = O_a \cdot G_a \cdot K_{d_t}\cdot P_{\text{pv}(h_t)} \cdot \eta_{a,t},
\]
with battery state evolution
\[
B_{a,t+1} = B_{a,t} + \eta_c \, q^{\text{ch}}_{a,t} - \left(\frac{1}{\eta_d}\right) q^{\text{dis}}_{a,t},
\]
and local observations
\[
s_i^t = \left[ \hat L_{i,t} - \hat G_{i,t}, \hat L_{i,t}, \hat G_{i,t}, S_i\cdot B_{i,t}, h_t, K_{d_t}, d_t \right]
\]
under 5% Gaussian multiplicative forecast noise [2508.18610]. The auction clears greedily while
\[
P_{\text{buy}} \ge P_{\text{sell}},
\]
and unmet demand is procured from the grid while unsold surplus is sold to the grid [2508.18610].

Empirically, the extension reports a persistent local-first energy split of about 54% P2P and 46% grid across all three case studies: 1,026 kWh P2P versus 874 kWh grid in Case 1, 2,260 kWh P2P versus 1,960 kWh grid in Case 2, and about 4,900 kWh P2P versus about 4,250 kWh grid transactions in Case 3 [2508.18610]. Consumers’ average monthly costs in the scaled synthetic community are reported as \$295, \$310, and \$325 for the three consumers, approximately 27–28% lower than grid-only baselines, while the real-world case reports consumer bill reductions of about 30% versus grid-only [2508.18610]. Fairness metrics remain high, with FPP above 0.95, FBS near 0.90, and FTG improving into the \(0.72\)–\(0.85\) range depending on case and training stage [2508.18610]. The paper also reports seller-share entropy between 0.60 and 0.96 with mean about 0.78 and Jain’s fairness index above 0.90 for roughly two-thirds of hours in the scaled case [2508.18610].

## 5. Relation to broader fair-market RL and mechanism design

FairMarket-RL belongs to a broader family of attempts to embed fairness into market decision systems, but it occupies a distinctive technical niche. In dynamic pricing, fairness has been introduced by shaping reward with a fairness statistic such as a rotated Jain’s index over group-average prices [1803.09967]. In risk-based pricing markets, a regulator can instead learn an interpretable bracketed tax schedule that reshapes firms’ incentives and raises demand fairness while preserving profitability [2506.00140]. In marketplace mechanism design, the operator can optimize exchange fees against a weighted generalized entropy measure of participant profits, explicitly decomposing within-group and between-group inequity [2209.15418]. In offline RL for high-stakes domains, fairness can be enforced at the feasibility layer through subgroup-specific calibration thresholds before policy learning rather than by end-to-end fairness regularization [2509.09655].

FairMarket-RL differs from those approaches in two ways. First, fairness is not defined exclusively by a closed-form constraint or regularizer embedded directly in the environment. Instead, it is evaluated ex post on realized market outcomes by an external critic that returns a compact score [2506.22708]. Second, the framework is explicitly multi-agent and decentralized, so fairness shaping acts on strategic interaction rather than only on a single policy or regulator. This suggests a flexible but less formally characterized mechanism.

At the same time, the broader literature clarifies several conceptual tensions that also apply to FairMarket-RL. Work on balancing profit and fairness in pricing markets warns that parity-style fairness can be achieved by “leveling down,” increasing opt-out for some groups even as disparity shrinks [2506.00140]. Equitable marketplace mechanism design shows that weighting fairness toward one strategic intermediary can indirectly improve outcomes for other classes, but only through equilibrium effects that remain domain-dependent [2209.15418]. Fair offline RL work emphasizes that fairness interventions grounded in calibration and observability remain sensitive to subgroup definition, small-sample issues, and distribution shift [2509.09655]. These adjacent results suggest that FairMarket-RL’s fairness scores should be interpreted as governance levers rather than guarantees of universally desirable social outcomes.

## 6. Limitations, interpretation, and subsequent significance

Several limitations are explicit. In the original framework, the LLM critic returns FTB and FBS, but the paper does not provide closed-form formulas for those scores, nor the exact prompt text or JSON schema used to elicit them [2506.22708]. Invalid LLM outputs are discarded rather than replaced with heuristic substitutes, which preserves signal purity but may reduce sample efficiency; the paper does not report how often that occurs [2506.22708]. The experimental environment is strategically nontrivial but small, with only two sellers and one buyer in the core case study [2506.22708]. The electricity extension adds realism through a continuous double auction, weather-modulated PV, and battery dynamics, yet still omits explicit distribution-network constraints in the formal environment and does not disclose network architecture or full PPO implementation details [2508.18610].

There are also normative and methodological ambiguities. Because the fairness critic is external and language-based, the framework is more interpretable at the level of named fairness channels than a black-box learned critic, but less transparent than a system whose fairness function is explicitly specified as a closed-form objective [2506.22708]. This suggests a trade-off between semantic flexibility and formal auditability. The later extension mitigates this partly by defining the intended semantics of FTG, FBS, and FPP in economic terms and by showing robustness under \(\pm 20\%\) PV and \(\pm 10\%\) load perturbations, but it still does not fully formalize the critic [2508.18610].

A further limitation is that neither version establishes classical mechanism-design guarantees such as incentive compatibility, equilibrium uniqueness, or welfare-optimality. FairMarket-RL is best understood as a reward-shaping architecture for decentralized market learning rather than as a closed-form fair mechanism. By contrast, adjacent work on taxes and subsidies in Fisher markets derives fairness interventions from dual variables in a constrained equilibrium program [2210.02586], and work on regulatory RL in pricing markets anchors learned policy to a monotone linear prior via an \(\ell_1\) deviation penalty [2506.00140]. Those approaches offer stronger analytical structure. FairMarket-RL instead prioritizes modularity: agents, market rules, and fairness evaluator can be recombined without solving a new constrained optimization problem from scratch.

Its significance lies in showing that fairness shaping can be inserted into the training loop of decentralized market agents through a critic that consumes structured market ledgers rather than raw natural language dialogue [2506.22708]. This extends the use of language models beyond text generation into normative evaluation for non-text MARL. The later electricity-market variant shows that the same design can scale from a minimal peer-to-peer trading game to a larger partially observable continuous double auction with storage and real-world household profiles [2508.18610]. A plausible implication is that FairMarket-RL serves less as a single fixed algorithm than as a reusable architectural pattern: decentralized market agents optimized by RL, a structured ledger of realized outcomes, and an external fairness critic whose scores are ramped into reward shaping over training.

Source: https://www.emergentmind.com/topics/fairmarket-rl