On Information Theoretic Fairness With A Bounded Point-Wise Statistical Parity Constraint: An Information Geometric Approach

Abstract: In this paper, we study an information-theoretic problem of designing a fair representation under a bounded point-wise statistical (demographic) parity constraint. More specifically, an agent uses some useful data (database) $X$ to solve a task $T$. Since both $X$ and $T$ are correlated with some latent sensitive attribute or secret $S$, the agent designs a representation $Y$ that satisfies a bounded point-wise statistical parity, that is, such that for all realizations of the representation $y\in\cal Y$, we have $χ^2(P_{S|y};P_S)\leq ε$. In contrast to our previous work, here we use the point-wise measure instead of a bounded mutual information, and we assume that the agent has no direct access to $S$ and $T$; hence, the Markov chains $S - X - Y$ and $T - X - Y$ hold. In this work, we design $Y$ that maximizes the mutual information $I(Y;T)$ about the task while satisfying a bounded compression rate constraint, that is, ensuring that $I(Y;X) \leq r$. Finally, $Y$ satisfies the point-wise bounded statistical parity constraint $χ^2(P_{S|y};P_S)\leq ε$. When $ε$ is small, concepts from information geometry allow us to locally approximate the KL-divergence and mutual information. To design the representation $Y$, we utilize this approximation and show that the main complex fairness design problem can be rewritten as a quadratic optimization problem that has simple closed-form solution under certain constraints. For the cases where the closed-form solution is not obtained we obtain lower bounds with low computational complexity. Here, we provide simple fairness designs with low complexity which are based on finding the maximum singular value and singular vector of a matrix. Finally, in a numerical example we compare our obtained results with the optimal solution.