Fair Directly Alternate Sampling (FDAS)

• FDAS is a framework that uses graph-based label propagation to correct noisy labels while promoting robust accuracy and fairness without explicit group constraints.
• It integrates prediction confidence, graph smoothness via curvature measures, and demographic parity incentives to optimize label correction and improve performance.
• Applications include fair classification and ASR rescoring, with empirical results showing reduced error rates and fairness gaps across datasets like CelebA and Waterbirds.

Fair Directly Alternate Sampling (FDAS) refers to a set of techniques, primarily graph-based label propagation frameworks, that target fair label correction under noisy supervision. FDAS, as studied in recent literature, leverages graph structures among examples to propagate label information or prediction confidence in a manner that promotes both accuracy and robust fairness metrics, often without explicit group regularization or costly retraining of the model from scratch. This paradigm is central to advances in fair classification, robust ASR hypothesis rescoring, and end-to-end learning systems dealing with subgroup fairness under label noise.

1. Graph-Based Label Propagation Foundations

Graph-based label propagation is a core mechanism by which FDAS achieves robust and fair label correction. Given a dataset with potentially noisy labels and sensitive group annotations, instances are embedded in a metric space (feature or latent), and a graph $G$ is constructed where the nodes represent individual samples and edges capture pairwise similarity, commonly via $k$-nearest neighbors (kNN) or affinity measures tied to data modality.

The label propagation objective typically minimizes a trade-off of label fidelity and graph-based smoothness. For example, in ASR rescoring, the optimal soft label matrix $F^*$ is given by:

$$F^* = \arg\min_F \|F - Y^{(0)}\|_F^2 + \lambda\, \mathrm{tr}(F^\top L_{sym} F)$$

with $L_{sym} = I - \Delta^{-1/2} W \Delta^{-1/2}$, where $W$ is the affinity matrix and $\Delta$ is the degree matrix. Iterative solution is provided via

$F^{(t+1)} = \alpha S F^{(t)} + (1-\alpha) Y^{(0)}$

with $S = \Delta^{-1/2} W \Delta^{-1/2}$ and hyperparameter $\alpha$ tuned on validation data (Tankasala et al., 2023).

In the context of classification with noisy labels, label-spreading is applied using majority voting or smoothed convex combinations in the graph neighborhood, with a single or a few hard/voting passes shown sufficient for robust results (Stromberg et al., 2024).

2. Fairness Optimization without Explicit Group Regularization

A distinctive aspect of FDAS is that fairness improvement frequently emerges purely from the graph construction and propagation protocol, rather than through explicit group-aware constraints or reweighting in the objective. In ASR rescoring, for instance, utterances from underrepresented accent groups are automatically linked via thresholded acoustic similarity, allowing hypothesis confidence to diffuse preferentially to linguistically similar samples. This implicitly diminishes majoritarian bias by leveraging the structure of accent similarity and hypothesis pooling (Tankasala et al., 2023).

In general classification, demographic parity is addressed via explicit incentives in the label correction scoring function, but the method does not require group membership at every stage: fairness metrics such as the demographic parity ratio are optimized by simulating individual flips and scoring their impact, keeping the modification localized to label states rather than model or sample reweighting (Sulaiman et al., 18 Jun 2025).

3. Integrating Prediction Confidence, Graph Curvature, and Demographic Parity

Modern FDAS approaches, exemplified by GFLC, combine three major components in their label correction strategies:

• Prediction confidence: Scores derived from a preliminary probabilistic classifier, quantifying margin or uncertainty outside predefined thresholds.
• Graph smoothness/curvature: Local Laplacian penalties are computed after optimizing the affinity graph via Ricci-flow, where edge weights are iteratively modulated using Forman—Ricci curvature. Edges that represent bottleneck or weak connections lose weight, while contracting edges strengthen, concentrating propagation within coherent clusters (Sulaiman et al., 18 Jun 2025).
• Demographic parity incentive: Each candidate label flip is scored for its improvement in parity, defined as the ratio

$k$0

and the score function aggregates confidence, smoothness violation, and parity gain for global optimization under sparsity constraints.

The algorithmic approach consists of sorting candidate samples by correction score, honoring flip budget via tolerance parameters, and updating labels accordingly. This direct, non-convex optimization via sorting exhibits computational advantages and empirical robustness (Sulaiman et al., 18 Jun 2025).

4. Empirical Results and Impact on Group Fairness

Empirical evaluation of FDAS-based graph propagation methods consistently demonstrates substantial improvements in both prediction robustness and fairness metrics under label imbalance or noise. In ASR rescoring, the word error rate (WER) and sentence error rate (SER) decrease markedly post-label propagation within clusters of acoustically overlapped utterances. The fairness gap, computed as the difference in WER between the worst and best performing accent groups, contracts significantly (e.g., from 6.59% to 2.92% on the VCTK dataset), indicating disproportionately larger gains for underrepresented accents (Tankasala et al., 2023).

In fair classification under synthetic group-dependent label noise, GFLC achieves higher AUC than prior flip-based correction baselines (e.g., Fair-OBNC), with demographic parity ratios maintained near 1.0 and equalized-odds gaps kept below 0.05 even at 20% injected group-dependent noise. Label-spreading as a preprocessing plug-in for two-stage retraining methods, as seen in kNN-RAD, provides state-of-the-art worst-group accuracy (WGA) on several datasets, oftentimes matching or outperforming oracle group-aware upweighting or downweighting (Stromberg et al., 2024).

Dataset	Method	WGA (WGN) at 20% label noise
CelebA	RAD	0%
CelebA	kNN-RAD	82.29% ± 1.25
Waterbirds	RAD	50.74% ± 2.23
Waterbirds	kNN-RAD	89.93% ± 1.10

These results underline the viability of FDAS as a label-noise-robust and fairness-preserving label correction scheme across modalities and learning tasks.

5. Applications Across Modalities and Task Settings

The generality of FDAS is reflected in its deployment across diverse domains:

• Automatic Speech Recognition (ASR): Graph-based label propagation across overlapping utterances mitigates majoritarian bias, recovers low-frequency word hypotheses, and enhances accent fairness within ASR output (Tankasala et al., 2023).
• Fair Classification with Label Noise: GFLC and kNN-label-spreading correct label noise for large tabular data (e.g., bank fraud detection with age as a sensitive group) and for vision/natural language datasets with spurious domain-label correlations (e.g., CMNIST, Waterbirds, CelebA, CivilComments) (Sulaiman et al., 18 Jun 2025, Stromberg et al., 2024).
• Last-Layer Retraining Paradigm: Within retraining pipelines such as RAD or SELF, FDAS serves as a drop-in preprocessing step, enhancing worst-group accuracy without large computational overhead (Stromberg et al., 2024).

A plausible implication is that FDAS methods are modality-agnostic and do not depend on semantic group annotation, provided an effective embedding space is available for graph construction.

6. Limitations and Opportunities for Extension

Current FDAS frameworks operate primarily in an offline or batch paradigm, requiring global graph construction and iterative propagation or flip-scoring routines. The performance is sensitive to accurate affinity estimation and clustering, especially in large, heterogeneous datasets. While fairness improvements often emerge naturally, some variants do not incorporate explicit fairness constraints, which leaves room for further guarantees via group-aware regularization, dynamic adaptation to streaming contexts, or integration of semantic/LM features into the graphs (Tankasala et al., 2023, Sulaiman et al., 18 Jun 2025).

Future work could address scalability to streaming data, theoretical guarantees under non-i.i.d. settings, and the development of adaptive thresholds or higher-order fairness metrics tailored to specific group structures.

7. Relationship to Contemporary Techniques and Theoretical Guarantees

FDAS can be seen as a midpoint between purely group-agnostic noise correction (e.g., majority-vote label spreading) and fully group-regularized, constraint-based retraining. Theoretical analysis (e.g., error bounds for $k$1NN under symmetric label noise) motivates the use of majority-vote propagation steps on graphs derived from learned or input feature spaces, with higher $k$2 required for higher noise rates (Stromberg et al., 2024). Graph curvature optimization, as in GFLC, introduces geometric regularization, distinguishing FDAS from conventional Laplacian-based smoothness penalization (Sulaiman et al., 18 Jun 2025).

FDAS is thus both theoretically grounded and empirically validated for fair and robust label correction in modern machine learning pipelines.