---
title: Entropy-Regularized Fairness Metric
url: https://www.emergentmind.com/topics/entropy-regularized-fairness-metric
type: topic
---

# Entropy-Regularized Fairness Metric

Searching arXiv for the cited papers to ground the article in current arXiv metadata.
“Entropy-regularized fairness metric” denotes several distinct constructions rather than a single canonical object. In the cited literature, entropy appears as a direct fairness functional, as a regularizer on auxiliary weight variables, as an inequality index inside constrained empirical risk minimization, and as a centered discrepancy between group distributions. In one line of work on TU-cooperative games, fairness is a worst-case Rényi-divergence gap over core allocations relative to a benchmark solution concept [1208.0283]. In fairness-aware secure ISAC, the phrase refers more specifically to an entropy-regularized Jain-fairness formulation over user weights \(\boldsymbol{\mu}\), where fairness is assessed on weighted legitimate-user SINRs rather than on entropy itself [2507.11224]. In fair empirical risk minimization, generalized entropy is used as a hard fairness constraint on classifier-induced benefits [2202.11966]. Related work on entropy-regularized optimal transport provides a statistical foundation for Sinkhorn-based group-discrepancy measures, although it does not discuss fairness explicitly [1711.08947]. A further network-science formulation defines fairness as the complement of normalized Shannon entropy over QoS-transformed all-pairs accessibility scores [2508.06898]. Taken together, these formulations show that the term is best understood as a family of entropy-centered fairness notions whose mathematical role depends on the optimization domain and the object being normalized.

## 1. Conceptual scope and recurring design pattern

A recurring pattern across the literature is: choose an object that can be interpreted as a probability distribution or normalized weight vector, apply an entropy or entropy-related functional, and use the result to quantify dispersion, inequality, or deviation from a fairness benchmark. In the cooperative-game formulation, any nonnegative vector \(X\) is viewed as a probability distribution by normalization, and fairness is evaluated on normalized core imputations relative to a reference distribution such as the uniform benchmark \(U\) or the normalized Shapley value \(Sh\) [1208.0283]. In secure ISAC, entropy is computed on the simplex-constrained user-weight vector \(\boldsymbol{\mu}\), while the Jain-style fairness term is applied to the weighted SINR vector \(\{\mu_k\rho_k^L\}_{k=1}^K\) [2507.11224]. In generalized-entropy fair ERM, the underlying normalized object is the distribution of nonnegative benefits \(b_i\) across individuals [2202.11966]. In the network-imbalance construction, normalized pairwise QoS scores \(p_{u,v}\) are fed into Shannon entropy [2508.06898].

This diversity is accompanied by a terminological caution. Several of the cited works explicitly do **not** use “entropy-regularized” in the modern sense of a primary objective of the form “main term plus entropy penalty.” The TU-cooperative-game paper instead defines an entropy-based fairness evaluation metric and studies its worst-case value over feasible core allocations [1208.0283]. The network-imbalance paper likewise defines an entropy-derived fairness/imbalance functional rather than an optimization regularizer [2508.06898]. By contrast, the secure ISAC formulation does place an entropy term inside a composite objective, but that entropy acts on the user-weight allocation rather than directly on rates, secrecy, or beamforming powers [2507.11224]. This suggests that the phrase is structurally descriptive rather than semantically uniform.

## 2. Rényi-divergence worst-case fairness in TU-cooperative games

In the TU-cooperative-game setting, the game is \(\Gamma=(N,v)\), with \(N\) the player set and \(v:2^N\to\mathbb{R}\) the coalition value function. Fairness is assessed over core imputations, and the central construction is a pessimistic fairness index: choose a benchmark solution concept \(q\), compare each feasible allocation \(x\in\mathrm{core}(\Gamma)\cap\mathbb{Z}^N\) to \(q(\Gamma)\) via Rényi divergence, and then maximize this discrepancy over the feasible set [1208.0283]. The main definition is
\[
\mathrm{Fair}_\lambda(\Gamma,q)=\sup\left\{D_\lambda(x\|q(\Gamma)):x\in \mathrm{core}(\Gamma)\cap \mathbb{Z}^N\right\},
\]
where, for distributions \(P=(p_i)\), \(Q=(q_i)\), and \(\lambda>0\),
\[
D_\lambda(P\|Q)=\frac{1}{\lambda-1}\log_2\left(\sum_i p_i^\lambda q_i^{1-\lambda}\right).
\]
The Rényi entropy of order \(\lambda\) is
\[
H_\lambda(X)=\frac{1}{1-\lambda}\log_2\left(\sum_i p_i^\lambda\right),
\]
with the Shannon limit
\[
H(X)=H_1(X)=-\sum_i p_i\log_2 p_i.
\]

The paper treats rationality as core membership and discusses several benchmarks: the uniform distribution \(U\) as a strict egalitarian reference, the normalized Shapley value as a marginalist benchmark, and the egalitarian solution of Dutta–Ray / Arin–Iñarra as a possible reference concept [1208.0283]. When the benchmark is uniform,
\[
D_\lambda(P\|U)=\log_2|N|-H_\lambda(P),
\]
so worst-case fairness against \(U\) is equivalent to minimum-entropy core allocation. This is the clearest entropy-based interpretation in the paper:
\[
\mathrm{Fair}_\lambda(\Gamma,U)=\log_2|N|-\inf_{x\in\mathrm{core}(\Gamma)\cap\mathbb{Z}^N}H_\lambda(x).
\]
The abstract states that this parametric family is related to the Cowell–Kuga generalized entropy indices in welfare economics, and the paper explicitly frames the construction as a fairness analogue of price of anarchy: benchmark a designated fair solution, optimize over rational outcomes, and quantify the worst deterioration [1208.0283].

The paper also gives a bridge between entropy and divergence for nonuniform benchmarks. For a distribution \(R=(r_i)\), define
\[
\nu(R)=\log_2\frac{r_{\max}}{r_{\min}}.
\]
Lemma 3 states
\[
H_\lambda(Q)-H_\lambda(P)-\nu(R)\le D_\lambda(P\|R)-D_\lambda(Q\|R)\le H_\lambda(Q)-H_\lambda(P)+\nu(R).
\]
When \(R\) is uniform, \(\nu(R)=0\), so entropy differences and divergence differences coincide exactly [1208.0283]. That identity is central to the paper’s interpretation of fairness as entropy deficit relative to complete equality.

Computationally, the paper proves NP-completeness for deciding whether \(\mathrm{Fair}_\lambda(\Gamma,U)\ge\eta\) for induced subgraph games with nonnegative weights, for any \(\lambda>0\) [1208.0283]. To approximate the worst-case value, it analyzes the ReverseGreedy algorithm and derives additive guarantees in terms of packing constants \(\alpha(\Gamma)\) and \(\beta(\Gamma)\). For induced subgraph games, a structural lemma yields \(\alpha(\Gamma)=\beta(\Gamma)=1\), making the additive constants explicit. The paper also studies an alternative algorithm based on biased orientations and reports a three-player example in which strictly egalitarian worst-case fairness exceeds marginalist worst-case fairness relative to the Shapley benchmark [1208.0283].

## 3. Entropy-regularized Jain-fairness in secure ISAC

The secure ISAC formulation is narrower and more literal in its use of entropy regularization. The baseline problem optimizes beamformers \(\{\boldsymbol w_k\}\) and artificial noise \(\boldsymbol n\) to maximize secrecy-oriented performance under communication, sensing, and power constraints. The fairness issue identified by the authors is that conventional QoS constraints of the form \(\rho_k^L\ge\gamma_k\) tend to be “just satisfied,” allowing the optimizer to exploit stronger channels while weaker users remain marginalized [2507.11224]. Fairness is therefore defined across users’ legitimate communication quality, measured through the legitimate SINRs \(\rho_k^L\).

The fairness score itself is Jain-style rather than entropic:
\[
F_{\mathrm{SINR}(\boldsymbol{\mu})}\triangleq
\frac{\left[\sum_{k=1}^{K}\mu_k\cdot \rho_k^L\right]^2}
{K\sum_{k=1}^{K}(\mu_k\cdot \rho_k^L)^2},
\]
with
\[
\frac{1}{K}\le F_{\mathrm{SINR}(\boldsymbol{\mu})}\le 1,
\]
and equality \(F_{\mathrm{SINR}}=1\) iff the effective SINRs are equal across users [2507.11224]. The entropy term enters through the composite outer objective
\[
\mathcal{L}(\boldsymbol{\mu};\chi_t)=
(1-\chi_t)G\sum_{k=1}^{K}\mu_k\log_2(1+\rho_k^L)
+\chi_t F_{\mathrm{SINR}(\boldsymbol{\mu})}
-\nu H(\boldsymbol{\mu}),
\]
where
\[
H(\boldsymbol{\mu})=-\sum_{k=1}^K \mu_k\log\mu_k,
\qquad
\boldsymbol{\mu}\in\mathbb{R}_+^K,\quad \sum_{k=1}^{K}\mu_k=1.
\]
The paper states that \(\nu>0\) “encourages diversity in the weight allocation,” but the formula is exactly \(-\nu H(\boldsymbol{\mu})\), so the prose and the algebraic sign are somewhat in tension; reimplementation must preserve the printed objective [2507.11224].

A central technical point is that entropy is **not** taken over rates or SINRs. Entropy acts on the simplex-normalized weight vector \(\boldsymbol{\mu}\), whereas fairness acts on the weighted SINR vector \(\mu_k\rho_k^L\) [2507.11224]. The legitimate-user SINR is
\[
\rho_k^L=
\frac{\boldsymbol{h}_k \boldsymbol{W}_k \boldsymbol{h}_k^{\text{H}}\|\boldsymbol{s}_k\|^2}
{\sum_{i\ne k}\boldsymbol{h}_k\boldsymbol{W}_i\boldsymbol{h}_k^{\text{H}}\|\boldsymbol{s}_i\|^2
+\boldsymbol{h}_k\boldsymbol{R}_{n_{\text{eff}}}\boldsymbol{h}_k^{\text{H}}
+\sigma_{z_k}^2},
\]
and for PSK symbols \(\|\boldsymbol s_k\|^2=1\). A normalization constant
\[
G\triangleq \left(\max_{\boldsymbol{\mu}\in\Delta}\sum_{k=1}^{K}\mu_k\log_2(1+\rho_k^L)\right)^{-1}
\]
rescales the throughput term to the unit interval so that it can be combined with the fairness score [2507.11224].

Fairness enters the optimization at the outer weight-update level, not directly in the inner beamforming/AN subproblem. The thresholded fairness requirement is
\[
F_{\mathrm{SINR}(\boldsymbol{\mu})}\ge \xi_F,
\]
and the penalty reformulation is
\[
\mathcal{L}_{\text{penalized}}(\boldsymbol{\mu};\chi_t)
=
\mathcal{L}(\boldsymbol{\mu};\chi_t)
-\lambda\left(\max(0,\xi_F-F_{\mathrm{SINR}(\boldsymbol{\mu})})\right)^2.
\]
The paper identifies three fairness-related coefficients: the tradeoff parameter \(\chi\in[0,1]\), the entropy coefficient \(\nu>0\), and the penalty coefficient \(\lambda>0\), together with the fairness threshold \(\xi_F\in(1/K,1]\) [2507.11224]. The edge cases are explicit: for \(\chi=0\), all weight is assigned to the user with highest SINR; for \(\chi=1\), fairness is maximized by equalizing effective weighted SINRs, yielding
\[
\mu_k^\star=\frac{c}{\rho_k^L},
\qquad
c=\left(\sum_{k=1}^K\frac{1}{\rho_k^L}\right)^{-1}.
\]

Algorithmically, the fairness term is handled in an outer simplex-constrained gradient update, while the inner weighted sum-rate beamforming and AN problems are solved via an accelerated quadratic transform with a non-homogeneous bound [2507.11224]. The paper gives the gradient
\[
\begin{aligned}
\nabla_{\mu_k}\mathcal{L}
&=
(1-\chi_t)G\log_2(1+\rho_k^L) \\
&\quad+
\chi_t\left[
\frac{2(\boldsymbol{\mu}^T\boldsymbol r)r_k}{K\|\boldsymbol{\mu}\circ\boldsymbol r\|^2}
-\frac{2(\boldsymbol{\mu}^T\boldsymbol r)^2r_k^2}{K\|\boldsymbol{\mu}\circ\boldsymbol r\|^4}
\right]
-\nu(1+\log \mu_k),
\end{aligned}
\]
followed by simplex projection and trust-region interpolation [2507.11224]. The paper reports simulation values \(\nu=0.01\) and \(\xi_F=0.5\), and claims improvements in average secrecy rate, average data rate, and beam gain, but does **not** isolate fairness-specific ablations such as a sweep over \(\nu\) or a direct comparison with and without entropy regularization [2507.11224].

## 4. Generalized entropy as a fairness constraint in empirical risk minimization

In fair ERM, generalized entropy is not an auxiliary smoothing term but the fairness criterion itself. For a nonnegative benefit vector \(b=(b_1,\ldots,b_n)\) with mean \(\mu=\frac{1}{n}\sum_{i=1}^n b_i\), the generalized entropy index is
\[
I_{\alpha}(b;n)=
\begin{cases}
\frac{1}{n}\sum_{i=1}^n \ln \frac{\mu}{b_i}, & \alpha=0,\\[6pt]
\frac{1}{n}\sum_{i=1}^n \frac{b_i}{\mu}\ln \frac{b_i}{\mu}, & \alpha=1,\\[8pt]
\frac{1}{n\alpha(\alpha-1)}\sum_{i=1}^n
\left[\left(\frac{b_i}{\mu}\right)^\alpha-1\right], & \alpha\neq 0,1.
\end{cases}
\]
The paper notes that \(I_1\) is the Theil index and states that “the more positive \(\alpha\), the more sensitive \(I_\alpha(b;n)\) to the inequalities of high income distribution” [2202.11966].

The fairness application depends on a benefit mapping from prediction outcomes to nonnegative values. Extending the mapping of Speicher et al., the paper uses
\[
b_h(x)=a(h(x)-y)+c,
\qquad c>a>0,\qquad c-a\ge 1.
\]
This yields
\[
\text{correct prediction}: b_h(x)=c,\qquad
\text{false positive}: b_h(x)=c+a,\qquad
\text{false negative}: b_h(x)=c-a.
\]
Fairness is then inequality in the classifier-induced benefit vector \(b_h\) [2202.11966]. The main constrained learning problem is
\[
\min_{h\in H} R_S(h)
\quad\text{subject to}\quad
I_\alpha(b_h;n)\le \gamma,
\]
where
\[
R_S(h)=\frac{1}{n}\sum_{i=1}^n \mathds{1}\{h(x_i)\neq y_i\}.
\]
The paper solves a randomized version over distributions \(D\in\Delta\mathcal H\), with Lagrangian
\[
L(D,\lambda)=R_S(D)+\lambda(I_\alpha(D;n)-\gamma),
\]
but presents generalized entropy primarily as a hard fairness constraint rather than as a standalone regularizer-only objective [2202.11966].

A key reason generalized entropy is attractive in this context is additive decomposability. For a partition into groups \(g=1,\dots,G\),
\[
I(b^1,\ldots,b^G;n)=\sum_{g=1}^G w_g^G(\mu,n)I(b^g;n_g)+V,
\]
where
\[
w_g^G(\mu,n)=\frac{n_g}{n}\left(\frac{\mu_g}{\mu}\right)^\alpha,
\]
and \(V\) is the between-group term with explicit formulas for \(\alpha=0\), \(\alpha=1\), and \(\alpha\neq 0,1\) [2202.11966]. This decomposition supports the paper’s claim that generalized entropy can simultaneously represent individual fairness and group fairness. The paper further proves that if a classifier satisfies equal prediction, then the between-group term \(V=0\), while also emphasizing that controlling generalized entropy does **not** generally control equalized odds or equal opportunity [2202.11966].

The theoretical contribution is a set of fairness generalization bounds. Extending generalized entropy to the population distribution,
\[
I_{\alpha}(b,X,P_x)=\int_X f_\alpha\left(\frac{b(x)}{E[b(x)]}\right)\,dP_x,
\]
the paper proves that if \(b(x)\) is bounded, then the extended index satisfies additive decomposability and the Pigou–Dalton transfer principle for all \(\alpha\in[0,\infty)\) [2202.11966]. It then gives explicit \(O(n^{-1/2})\) bounds for
\[
|I_\alpha(h,P)-I_\alpha(b_h;n)|
\]
with constants depending on \(\alpha\), \(a\), and \(r=c/a\), and a sharper result that additionally depends on VC dimension \(d_H\) and empirical risk \(R_S(h)\) [2202.11966]. Practically, the paper draws two lessons: smaller \(\alpha\) is often preferable, and larger \(r\) improves fairness generalization bounds, although excessively large \(r\) may flatten the unfairness signal.

## 5. Entropy-regularized optimal transport as a fairness discrepancy foundation

The optimal-transport paper does not discuss fairness explicitly, but it provides a rigorous finite-space framework for building an entropy-regularized discrepancy between group distributions. On a finite metric space \(X=\{x_1,\ldots,x_N\}\), with \(a,b\in\Sigma_N\), transport polytope
\[
U(a,b)=\{T\in\mathbb{R}_+^{N\times N}\mid T\mathbf 1_N=a,\ T^T\mathbf 1_N=b\},
\]
and cost matrix \(C\), the entropy-regularized transport cost is
\[
W_{p,\varepsilon}^p(a,b)=
\min_{T\in U(a,b)}
\langle T,C\rangle+\varepsilon H(T|a\otimes b),
\]
where
\[
H(T|a\otimes b)=\sum_{i,j}\log\left(\frac{t_{ij}}{a_i b_j}\right)t_{ij}.
\]
The centered version, called the Sinkhorn loss in the paper, is
\[
\overline W_{p,\varepsilon}^p(a,b)=
W_{p,\varepsilon}^p(a,b)
-\frac12\Big(W_{p,\varepsilon}^p(a,a)+W_{p,\varepsilon}^p(b,b)\Big).
\]
The paper proves that
\[
\overline W_{p,\varepsilon}^p(a,b)\ge 0,\qquad
\overline W_{p,\varepsilon}^p(a,b)=0\iff a=b,
\]
and that \(\overline W_{p,\varepsilon}^p(a,b)\to W_p^p(a,b)\) as \(\varepsilon\to 0\) [1711.08947].

The fairness-oriented interpretation, which the paper itself does not claim normatively, is that group-conditional empirical distributions can be compared through the centered Sinkhorn loss:
\[
\mathcal F_\varepsilon(\mu_A,\mu_B)=\overline W_{p,\varepsilon}^p(\mu_A,\mu_B).
\]
This provides a full-distribution discrepancy rather than a scalar parity gap, and the cost matrix \(C\) encodes the geometry of the outcome space [1711.08947]. The paper’s principal value for such use is statistical. It establishes differentiability of the regularized OT cost,
\[
\nabla W_{p,\varepsilon}^p(a,b)(h_1,h_2)=\langle u_\varepsilon,h_1\rangle+\langle v_\varepsilon,h_2\rangle,
\]
where \((u_\varepsilon,v_\varepsilon)\) is an optimal dual pair, and derives CLTs for both the raw regularized OT cost and the centered Sinkhorn loss in one-sample and two-sample settings [1711.08947].

The null behavior of the centered loss is especially important if it is used as a fairness-audit statistic for equality of group distributions. At equality, the gradient vanishes,
\[
\nabla \overline W_{p,\varepsilon}^p(a,a)=0,
\]
so the null asymptotics are second order: \(n\,\overline W_{p,\varepsilon}^p(\hat a_n,a)\) and \(\frac{nm}{n+m}\overline W_{p,\varepsilon}^p(\hat a_n,\hat b_m)\) converge to chi-square mixtures rather than Gaussian limits [1711.08947]. The paper also proposes bootstrap procedures, noting that the ordinary bootstrap is valid under the alternative but fails under first-order degeneracy, where a corrected Babu bootstrap is required in practice. This makes the paper a statistical foundation for entropy-regularized OT discrepancy measurement on discretized fairness problems rather than a fairness framework by itself.

## 6. Entropy-derived imbalance and cross-framework issues

The network-imbalance framework provides a different entropy-based fairness construction. For an undirected graph \(G=(V,E)\), the shortest-path hop count is
\[
h(u,v)=
\begin{cases}
\min\{\text{length}(P_{uv})\}, & \text{if reachable},\\
+\infty, & \text{otherwise},
\end{cases}
\qquad d(u,v)=h(u,v).
\]
This raw QoS variable is mapped through a tunable sigmoid
\[
w(u,v)=
\begin{cases}
\frac{1}{1+\exp[a(d(u,v)-h_0)]}, & d(u,v)<+\infty,\\
0, & d(u,v)=+\infty,
\end{cases}
\]
with threshold \(h_0>0\) and steepness \(a>0\). Summing over ordered node pairs gives
\[
W=\sum_{x\ne y} w(x,y),
\qquad
p_{u,v}=\frac{w(u,v)}{W}.
\]
Shannon entropy is then
\[
H=-\sum_{u\ne v} p_{u,v}\log_2(p_{u,v}),
\qquad
Q=\frac{H}{\log_2(N(N-1))},
\qquad
I=1-Q.
\]
The paper states \(I\in[0,1]\), defines \(I=1\) when \(W=0\), and interprets low imbalance as high functional fairness in the sense of uniform end-to-end connection experiences [2508.06898].

This formulation is explicitly **not** an entropy-regularized optimization objective. Entropy is the aggregation mechanism, and unfairness is the complement of normalized entropy [2508.06898]. The paper proves boundedness, continuity and differentiability with respect to the parameters \(a\) and \(h_0\), and characterizes \(I(G)=0\) for connected graphs with finite nonzero \(a\) as equivalent to equality of all shortest-path lengths across ordered node pairs; for nontrivial connected graphs, this yields the complete graph \(K_N\) as the unique topology with zero imbalance [2508.06898]. It also proves that imbalance is not monotonic under edge addition or deletion, providing a counterexample on \(C_8\), and argues that low imbalance may arise either from topological symmetry or from extreme path compression in structurally heterogeneous networks such as BA scale-free graphs [2508.06898].

Across the cited literature, three recurrent issues emerge. First, entropy-based fairness is benchmark-sensitive or representation-sensitive: the fairness value depends on the chosen benchmark distribution \(q(\Gamma)\) in TU games [1208.0283], on the benefit encoding \(b_h(x)=a(h(x)-y)+c\) in fair ERM [2202.11966], on the cost matrix \(C\) and regularization level \(\varepsilon\) in OT [1711.08947], on the simplex weights \(\boldsymbol{\mu}\) and fairness threshold \(\xi_F\) in secure ISAC [2507.11224], and on the QoS parameters \((a,h_0)\) in network imbalance [2508.06898]. Second, “entropy-regularized” and “entropy-based” should not be conflated. The cooperative-game and network papers are entropy-based but not regularized in the conventional Lagrangian sense [1208.0283; 2508.06898], whereas the ISAC paper is more accurately described as an entropy-regularized Jain-fairness formulation [2507.11224]. Third, these entropy-centered notions are not interchangeable with standard parity constraints. The fair-ERM paper is explicit that controlling generalized entropy has no specific impact on equalized odds or equal opportunity [2202.11966], and the OT paper likewise does not claim that distributional discrepancy alone resolves the normative content of fairness [1711.08947].

In that sense, an entropy-regularized fairness metric is best understood not as a single metric but as a methodological family. Its common feature is the use of entropy, Rényi divergence, generalized entropy, or relative entropy to quantify concentration, inequality, or discrepancy after an application-specific normalization step. Its differences arise from what is normalized, where entropy is inserted in the optimization pipeline, and whether fairness is treated as worst-case deviation, outer-loop regularization, hard constraint, or centered distributional discrepancy [1208.0283; 2507.11224; 2202.11966; 1711.08947; 2508.06898].

Source: https://www.emergentmind.com/topics/entropy-regularized-fairness-metric