Group Conditional Unbiased Logistic Regression

• GCULR is a method that enforces group fairness by minimizing misclassification disparities across sensitive groups using logistic regression.
• It employs a dynamic Bayesian framework with EKF filtering and Monte Carlo sampling, alongside a convex MRF approach with group-norm regularization.
• Empirical evaluations on synthetic and real-world data demonstrate GCULR’s ability to maintain fairness with minimal loss in predictive accuracy.

Group Conditional Unbiased Logistic Regression (GCULR) encompasses a class of constrained logistic regression methodologies designed to yield predictions that are unbiased with respect to predefined group variables. GCULR is principally motivated by mitigating disparate misclassification, particularly in settings where sensitive covariates (such as race or gender) induce differences in error rates. Implementations of GCULR include a fully Bayesian sequential approach for online classification tracking with fairness guarantees (Short et al., 2020) as well as a convex group-norm-regularized estimator for high-dimensional graphical model recovery (Wu et al., 2018).

1. Model Specification and Core Principle

GCULR explicitly enforces group-level fairness constraints in the learning process of logistic regression models, ensuring that prediction error rates (e.g., false positive/negative rates) are approximately equal across sensitive groups.

In the dynamic Bayesian framework (Short et al., 2020), the logistic regression model is formulated as: $$y_i\mid x_i,\;\beta_i\;\sim\;\mathrm{Bernoulli}\bigl(p_i\bigr),\quad p_i = \sigma\bigl(x_i^T\beta_i\bigr),\quad \sigma(u)=1/(1+e^{-u}),$$ where each parameter vector $\beta_i$ evolves according to a state-space random walk: $$\beta_i = \beta_{i-1} + w_i,\qquad w_i\sim\mathcal{N}(0,Q).$$

Additionally, for each group $z\in\{0,1\}$, the conditional feature distribution $x|z$ is modeled as $\mathcal{N}(\mu_{z,i},\Sigma_{z,i})$ with a Normal-inverse-Wishart conjugate prior, permitting online updating.

Alternately, in the Markov random field (MRF)/structured prediction context (Wu et al., 2018), GCULR consists of solving, for each node and every pair of states $(\alpha,\beta)$, a group-norm-regularized logistic regression: $$\hat w\in\argmin_{w\in\mathbb{R}^{n\times k}}\left\{ \frac{1}{|S|} \sum_{m\in S} \ln(1 + e^{-y^{(m)}\langle w, x^{(m)} \rangle}) \right\}\quad \text{s.t.}\quad \|w\|_{2,1} \le 2\lambda\sqrt{k}.$$ Here, $\|w\|_{2,1}$ denotes the sum of the $\ell_2$ norms of row groups, ensuring group-sparse solutions matched to the graphical structure.

2. Fairness Constraint Formulation

GCULR enforces fairness by constraining group-conditional misclassification disparities. In the dynamic Bayesian setting, the group-conditional false-negative and false-positive rates are defined as: $$\mathrm{FNR}_i(z) = \frac{\int_{x\,:\,\hat{p}(x)<\tau} \hat{p}_\mathrm{true}(x) \mathcal{D}_z(x)dx} {\int \hat{p}_\mathrm{true}(x) \mathcal{D}_z(x) dx},$$

$$\mathrm{FPR}_i(z) = \frac{\int_{x\,:\,\hat{p}(x)>\tau} [1-\hat{p}_\mathrm{true}(x)] \mathcal{D}_z(x)dx} {\int [1-\hat{p}_\mathrm{true}(x)] \mathcal{D}_z(x) dx},$$

with the scalar bias metric

$$\Delta_i = \sqrt{[\mathrm{FPR}_i(1)-\mathrm{FPR}_i(0)]^2 + [\mathrm{FNR}_i(1)-\mathrm{FNR}_i(0)]^2}.$$

GCULR imposes the hard constraint $\Delta_i<\epsilon$ at every update step. After updating posteriors, candidate parameter draws are Monte Carlo sampled and only those satisfying both the fairness constraint $\Delta_{ij}<\epsilon$ and minimum relative accuracy $\ge\alpha$ are retained. The resulting fair-constrained posterior is defined by the empirical mean and covariance of accepted samples.

In the MRF structure learning context, “unbiasedness” refers to an underlying distribution property ensuring every conditional entry is bounded below by a function of model width and alphabet size, a requirement critical for identifiability and generalization (Wu et al., 2018).

3. Algorithmic Workflow and Computational Methods

Bayesian Online Tracking (Dynamic GCULR)

The tracking algorithm involves:

• Bayesian filtering of logistic regression parameters via the Extended Kalman Filter (EKF), using the linearized log-likelihood update equations:

$$C_i^{-1}=C_{i\mid i-1}^{-1}+h_i\,h_i^T,\qquad \bar{\beta}_i=\bar{\beta}_{i\mid i-1}-C_i f_i,$$

where

$$f_i = (-1)^{y_i} x_i \sigma\big((-1)^{y_i}x_i^T \bar{\beta}_{i\mid i-1}\big),\quad h_i = x_i \sigma(x_i^T\bar{\beta}_{i\mid i-1}) [1-\sigma(x_i^T\bar{\beta}_{i\mid i-1})]^{1/2}.$$

• Monte Carlo estimation of group-conditional errors using samples from the predicted covariate distributions for each group.
• Iterative rejection sampling of posterior parameter draws to satisfy fairness and accuracy constraints, followed by propagation of constrained and unconstrained posteriors.

Convex Group-Norm Optimization (MRF GCULR)

• For each variable and state pair, one-hot encoding is performed on the features of subsetted samples.
• The core convex problem involves penalization by an $\ell_{2,1}$ group norm, solved using first-order mirror descent with a special distance-generating function; this ensures efficient optimization at scale.
• For $n$ variables with alphabet size $k$, the total complexity is $\tilde O(n^2)$ for fixed problem parameters, representing a significant computational improvement over prior art.

4. Statistical Guarantees and Theoretical Properties

No formal theorem is stated for the Bayesian online algorithm, but standard EKF and random-walk regularity assumptions yield:

• Consistency: parameter mean converges in probability to the true value for data generated by a logistic model.
• Fairness: enforcement of $\Delta_i<\epsilon$ guarantees bounded disparate misclassification at all times.
• Accuracy bound: the minimum group accuracy remains at least fraction $\alpha$ of the unconstrained solution (Short et al., 2020).

In the discrete graphical modeling framework:

• The key codependence between prediction risk and parameter error is quantified via population risk and Kullback–Leibler divergence. Specifically, with $$N=\tilde O(\lambda^2 k^4 e^{14\lambda}/\epsilon^4 \ln(nk/\rho))$$ samples, the estimator satisfies $$\max_{i<j,a,b}|W_{ij}(a,b)-\hat W_{ij}(a,b)| \leq \epsilon$$ with high probability, enabling exact structure recovery when $\epsilon<\eta/2$ (Wu et al., 2018).
• Unbiasedness is formally associated with the property that for all $a\in[k]$, $\Pr[Z_i=a|Z_{-i}]=\geq e^{-2\lambda}/k$, guaranteeing nondegeneracy in the conditioning structure.

5. Implementation and Hyperparameter Selection

Key hyperparameters for GCULR include:

Parameter	Role in GCULR (Short et al., 2020)	Recommendation
$Q$	Process noise covariance for $\beta_i$	Set based on system dynamics
$\epsilon$	Bias tolerance for $\Delta_i$	Chosen to balance fairness/accuracy
$\alpha$	Relative accuracy threshold	Typically $[0.65,0.85]$
$M_x$	Monte Carlo samples per group	Sufficient for integral accuracy
$M_\Theta$	Posterior samples for rejection	Large for tight constraint
$(m_{z,0},\lambda_{z,0},\Phi_{z,0},\nu_{z,0})$	Normal-inverse-Wishart prior	Large $\lambda$ for tracking

All model updates, rejection sampling, and posterior propagation steps are explicitly stated in Algorithm 1 in (Short et al., 2020). For the high-dimensional case, pseudocode follows the outlined samples, encoding, mirror-descent, and thresholding workflow (Wu et al., 2018).

6. Empirical Evaluation and Comparative Analysis

Static Synthetic Data

GCULR, applied to data from two Gaussian clusters, reduced group FPR (from $(0.62,0.23)$ to $(0.48,0.48)$) and FNR (from $(0.11,0.46)$ to $(0.28,0.28)$), with an overall accuracy drop from approximately $0.68$ to $0.62$. The Zafar et al. baseline yielded less balance: FPR $(0.53,0.51)$, FNR $(0.22,0.27)$ (Short et al., 2020).

Dynamic Synthetic Data

When group means swap over time, ordinary logistic regression exhibits fluctuating instantaneous bias while GCULR maintains $\Delta_i<0.05$ at every point, adapting to the evolving fairness boundary in real time.

ProPublica COMPAS Evaluation

On 5,278 criminal justice records, unconstrained GCULR achieved an accuracy of $\sim 0.75$ but with a disparity $\Delta>0.3$. Imposing GCULR with $\epsilon=0.05, \alpha=0.65$ yields disparity $\Delta<0.05$ and accuracy $\sim 0.68$. Competing methods (Zafar baseline) cannot achieve comparable (tight) fairness without severe accuracy trade-offs or trivial classification (Short et al., 2020).

MRF Graphical Model Recovery

In experiments with grid graphs, for $k=4,6$, GCULR consistently recovered true structure using fewer samples than the online Sparsitron, attributable to superior $\tilde O(k^4)$ sample complexity versus $\tilde O(k^5)$ of earlier approaches (Wu et al., 2018).

7. Extensions and Notable Properties

GCULR with group norm constraints ($\ell_{2,1}$) generalizes binary, $\ell_1$-constrained regression (Ising models) to $k$-ary alphabets. The group norm approach yields statistically and computationally preferable rates, particularly for high-dimensional problems. GCULR permits finite-sample, high-probability performance guarantees, efficient optimization, and certified fairness in dynamic and stationary regimes. In both Bayesian tracking and graphical model settings, the approach is robust to nonstationarities and provides posterior uncertainty quantification at each update (Short et al., 2020, Wu et al., 2018).