FAIR-SUB Framework Overview

• FAIR-SUB Framework is a comprehensive set of methods for subgroup-adaptive modeling and fairness-aware analysis across regression, classification, sensitivity analysis, and federated learning.
• It employs techniques such as weighted sample losses, alternating optimization, and dynamic submodel allocation to balance predictive performance with fairness for diverse subgroups.
• Empirical validations demonstrate significant error reduction for under-represented groups and improved fairness metrics without compromising overall model accuracy and interpretability.

The FAIR-SUB framework encompasses a spectrum of approaches that pursue subgroup-adaptive modeling or fairness-aware analysis in machine learning, spanning linear regression, classification, federated learning, and sensitivity analysis domains. The unifying objective is to enhance predictive validity and utility for under-represented, protected, or high-variance subgroups without sacrificing overall interpretability, computational tractability, or fairness guarantees. Major instantiations include Functionally Adaptive Interaction Regularization (regression context), subdata selection for fair classification, sensitivity-based subgroup analysis in vision models, and dynamic submodel allocation in federated learning. The following sections synthesize the technical details, methodological innovations, and empirical validations from these primary FAIR-SUB instantiations.

1. Subgroup-Adaptive Linear Regression via Functionally Adaptive Interaction Regularization

The regression-centric FAIR-SUB framework, as detailed in "Maximizing Predictive Performance for Small Subgroups: Functionally Adaptive Interaction Regularization (FAIR)" (Smolyak et al., 2024), addresses the challenge of maximizing performance for all population subgroups, particularly small or under-represented ones, while upholding interpretability and model tractability.

Mathematical Model

The framework fits a full linear interaction model to data $\{(X_i, y_i, G_i)\}_{i=1}^n$ where $y_i \in \mathbb{R}$ is the response, $X_i \in \mathbb{R}^p$ the covariate vector, and $G_i \in \{1, ..., K\}$ the group indicator. The prediction for each sample is given by: $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$ where $\beta \in \mathbb{R}^p$ captures global effects, and $\gamma_g \in \mathbb{R}^p$ encodes group-specific interaction slopes. Optionally, group-specific intercept shifts can be subsumed in $\gamma_g$ through a constant covariate.

Weighted-Sample Loss

The loss function weights each sample proportional to the inverse of its group size ($w_{G_i}=1/n_{G_i}$), thereby mitigating the dominance of large groups: $$L(\beta, \gamma) = \sum_{i=1}^n w_{G_i}\,(y_i - \hat y_i)^2.$$ This reweighting ensures that small but clinically relevant subgroups are adequately represented in parameter learning.

Group-wise Regularization

Independent penalties are imposed on the global and group-specific parameters: $y_i \in \mathbb{R}$0 Hyperparameters $y_i \in \mathbb{R}$1 are selected by cross-validation, emphasizing error reduction for the smallest subgroup, frequently adopting a composite subgroup-weighted MSE criterion.

Optimization and Algorithmic Strategy

The objective is solved efficiently via block-wise coordinate descent or proximal-gradient algorithms, leveraging the separability of the penalty structure. Each parameter block $y_i \in \mathbb{R}$2 and the global $y_i \in \mathbb{R}$3 are updated alternately via closed-form ridge (or soft-thresholding for Lasso variants) steps, aligning with the computational paradigms in glmnet-type software.

Interpretability and Practical Deployment

Parameter interpretability is central: $y_i \in \mathbb{R}$4 represents a baseline effect, while $y_i \in \mathbb{R}$5 quantifies deviation for group $y_i \in \mathbb{R}$6. Deployment protocols monitor per-group calibration, residuals, and MSE/MAE parity; adapting to emergent subgroups involves simply introducing new $y_i \in \mathbb{R}$7 vectors.

Empirical Evidence

In both controlled (sparse, heterogeneous 2-group) and real-world (UCI Diabetes 130-US hospitals) datasets, FAIR-SUB outperforms pooled, separate, and joint-Lasso baselines, reducing small-group MSE by 10–40%. Computationally, glmnet-compatible implementations yield a 10–20$y_i \in \mathbb{R}$8 speedup over specialized methods and scale to thousands of predictors (Smolyak et al., 2024).

2. Subdata Selection for Fair Classification

The classification-oriented FAIR-SUB approach, articulated in "Unbiased Subdata Selection for Fair Classification: A Unified Framework and Scalable Algorithms" (Ye et al., 2020), targets joint optimization of accuracy and group-fairness metrics through alternating subdata selection and classifier retraining.

Unified Objective

The foundational optimization objective integrates classification risk and explicit group-fairness penalties: $y_i \in \mathbb{R}$9 with $X_i \in \mathbb{R}^p$0 capturing measures such as demographic parity or equal opportunity in terms of outcome distributions across protected groups.

Mixed-Integer Convex Formulation

For linear SVMs (extendable to other models), the framework formulates a mixed-integer convex program (MICP) with binary variables $X_i \in \mathbb{R}^p$1 denoting correctly classified points, fairness-penalized objectives, and McCormick envelope relaxations to convexify bilinear terms. This enables exact optimization for moderate-sized instances.

Iterative Refining Strategy (IRS)

Large-scale application is achieved via the IRS, an alternating minimization scheme:

1. Fix classifier parameters $X_i \in \mathbb{R}^p$2, optimize $X_i \in \mathbb{R}^p$3 via strongly polynomial subdata-selector.
2. Fix $X_i \in \mathbb{R}^p$4, refit classifier (SVM, logistic regression, or black-box learner) on the selected unbiased subset. The alternation converges to a stationary point, with explicit approximation guarantees,

$X_i \in \mathbb{R}^p$5

where $X_i \in \mathbb{R}^p$6 is the global optimum and $X_i \in \mathbb{R}^p$7 the symmetric difference.

Extensions and Applications

FAIR-SUB generalizes to multiclass SVMs, logistic regression, kernel methods, black-box learners, and unbalanced data (via $X_i \in \mathbb{R}^p$8-based penalties). For each, alternate minimization and unbiased subdata-selection yield tractable, fairness-enhanced solutions.

Empirical Validation

Benchmarks on COMPAS, UCI credit, wine-quality, abalone, and medical-image datasets demonstrate strict fairness improvement (demographic parity and equal opportunity violation reduction to near zero), often with $X_i \in \mathbb{R}^p$9 accuracy loss, and runtime scalability superior to other in-processing and post-processing fair-learning algorithms (Ye et al., 2020).

3. Subgroup-Aware Sensitivity Analysis for Fairness

"Fair SA: Sensitivity Analysis for Fairness in Face Recognition" (Joshi et al., 2022) introduces a robust FAIR-SUB sensitivity analysis framework for subgroup-conditional robustness evaluation under controlled data perturbations, such as in face recognition.

Analytical Framework

• Input: $G_i \in \{1, ..., K\}$0 (input samples), $G_i \in \{1, ..., K\}$1 (subgroups), $G_i \in \{1, ..., K\}$2 (perturbation types), $G_i \in \{1, ..., K\}$3 (perturbation strengths).
• Performance Metric: $G_i \in \{1, ..., K\}$4 computes model (e.g., verification) performance on perturbed $G_i \in \{1, ..., K\}$5 at level $G_i \in \{1, ..., K\}$6.
• Group-Level Robustness: $G_i \in \{1, ..., K\}$7.
• Targeted Robustness Thresholds: $G_i \in \{1, ..., K\}$8 for fixed threshold $G_i \in \{1, ..., K\}$9.
• Fairness Metrics: Tolerance-gap $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$0; Area under curve (AUC) disparity $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$1.

AUC Matrix Visualization

A $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$2 matrix $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$3 is constructed, with signed per-subgroup AUC under each perturbation, visualized using diverging colormaps to accentuate subgroup-perturbation fairness differentials.

Workflow and Application

1. For each perturbation/strength, compute outputs and subgroup-wise robustness curves.
2. Derive $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$4 and AUCs; populate and visualize $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$5.
3. Extract attribute- and perturbation-wise fairness profiles.

Empirical Insights

Evaluations on CelebA (40 attributes; nine perturbations) revealed systematic robustness disadvantages borne by certain demographic subgroups (e.g., older or pale-skin faces under blur/exposure), with AUC matrices providing interpretable, attribute-specific bias localization (Joshi et al., 2022).

Generalization and Limitations

The framework extends beyond face recognition, applicable to any domain with controlled perturbations and well-defined subgroups (e.g., medical imaging, detection under weather). The selection of fair thresholds ($$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$6), grid granularities $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$7, and subgroup sampling adequacy critically influence metric robustness.

4. Submodel Allocation for Fairness in Federated Learning

"FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning" (Wang et al., 2024) presents a FAIR-SUB instantiation for federated optimization, introducing a submodel allocation protocol subject to bounded collaborative fairness (BCF).

Bounded Collaborative Fairness (BCF)

BCF formalizes the fairness constraint as

$$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$8

where $$\hat y_i = \beta_0 + X_i^\top \beta + X_i^\top \gamma_{G_i},$$9 is a client's solo accuracy and $\beta \in \mathbb{R}^p$0 post-federated accuracy. Fairness is measured by correlation $\beta \in \mathbb{R}^p$1.

Submodel Allocation Module

• Neuron Importance: Taylor-based one-shot estimates $\beta \in \mathbb{R}^p$2, normalized.
• Reputation Conversion: $\beta \in \mathbb{R}^p$3, driving neuron allocation size per client.
• Submodel Construction: Each client receives a personalized binary mask selecting its top $\beta \in \mathbb{R}^p$4 of neurons from the global model.

Dynamic Aggregation

Models are aggregated via frequency-weighted averaging over neuron support, ensuring equitable treatment for low-frequency (less-selected) neurons and preserving model diversity.

Theoretical Properties

BCF is guaranteed in post-training model allocations, with higher-contributing clients provably obtaining strictly better models. With standard smoothness/convexity assumptions, the framework's convergence rate is $\beta \in \mathbb{R}^p$5 in expected loss.

Experimental Results

Across CIFAR-10, SVHN, Fashion-MNIST (with power-law, Dirichlet, and class-imbalance heterogeneity), FedSAC achieves fairness-correlations $\beta \in \mathbb{R}^p$699% and matches or surpasses prior methods in accuracy. Communication is reduced by restricting each client to a subset of parameters (20–80% savings) (Wang et al., 2024).

5. Comparative Summary and Theoretical Significance

The FAIR-SUB framework, instantiated across regression, classification, sensitivity analysis, and federated optimization, provides systematic methodologies for adaptively controlling subgroup-specific inference quality, fairness, and interpretability. The approaches leverage principled optimization—block-wise regularization, mixed-integer programming, group-conditional weighting, and mask-based aggregation—to attain subgroup equity without degrading overall performance or tractability.

A plausible implication is that such frameworks will be instrumental for regulatory- and policy-driven analytical tasks where subgroup equity is not only desirable but mandated (e.g., clinical predictive modeling, credit scoring, large-scale federated analytics).

FAIR-SUB Variant	Primary Domain	Key Technique
Regression (FAIR)	Linear regression	Full interaction + group-wise penalty
Classification	Fair binary/multiclass	Alternating unbiased subdata selection
Sensitivity Analysis	Vision, robustness	Subgroup-conditioned AUC matrices
Federated Learning (FedSAC)	Federated optimization	Dynamic submodel allocation, mask-weighted averaging

The adoption of FAIR-SUB methodologies enables nuanced, context-aware mitigation of subgroup disparities, combining statistical rigor with practical interpretability. Extensions to kernel and black-box learners further enhance applicability in modern, high-dimensional settings.