Adaptive Bias Integrated Learning (ABIL)

• Adaptive Bias Integrated Learning (ABIL) is a method that learns multiple bias-specific representations via dedicated encoders to expose known shortcut features in image classification.
• It employs a sample-adaptive fusion mechanism using cosine similarity to weight bias features and integrate them with the main backbone representation.
• By clarifying shortcut directions, ABIL prepares the input for a subsequent gradient suppression stage to reduce reliance on biases and improve classification robustness.

Searching arXiv for papers explicitly using or closely matching “Adaptive Bias Integrated Learning (ABIL)” and nearby usages of the acronym. Adaptive Bias Integrated Learning (ABIL) is a stage-1 representation-learning method introduced within the two-stage framework Generalized Multi-Bias Mitigation (GMBM) for image classification under multiple known shortcut attributes. In the formulation of "Multi Attribute Bias Mitigation via Representation Learning" (Dwivedi et al., 3 Sep 2025), ABIL deliberately identifies the influence of known shortcuts by training one encoder per bias attribute and integrating those bias-specific representations with the main backbone through sample-adaptive fusion. The stated purpose is not to suppress bias immediately, but to make shortcut channels explicit, disentangled, and sample-relevant so that a subsequent stage, Gradient Suppression Fine-Tuning, can prune those directions from the backbone. The same acronym has also been used for a distinct method, "ABductive Imitation Learning," in long-horizon neuro-symbolic planning (Shao et al., 2024), so the meaning of ABIL is context-dependent.

1. Definition and problem formulation

In (Dwivedi et al., 3 Sep 2025), the learning setting is a supervised image-classification problem with multiple annotated shortcut variables. The dataset is defined as

$$\mathcal{D}=\left\{\left(x^{(i)}, y^{(i)}, b_1^{(i)}, \dots, b_k^{(i)}\right)\right\}_{i=1}^n,$$

where $x^{(i)}$ is the input image, $y^{(i)} \in \{1,\dots,N\}$ is the target label, and $b_j^{(i)}$ is the annotation for the $j$-th bias attribute (Dwivedi et al., 3 Sep 2025). The backbone produces a penultimate-layer feature

$h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$

and the objective is to learn a classifier

$g:\mathbb{R}^d \to \{1,\dots,N\}$

that predicts $y$ accurately without relying on any of the bias attributes (Dwivedi et al., 3 Sep 2025).

The paper defines a bias attribute $b_j$ as spurious when the target can be inferred from it approximately, that is,

$H(Y \mid B_j)\approx 0,$

with conditional entropy

$x^{(i)}$0

(Dwivedi et al., 3 Sep 2025). The motivating claim is that one-bias-at-a-time mitigation is insufficient because, in a multi-bias setting, removing one shortcut can simply shift reliance to another. The paper describes this as a "Whac-A-Mole" phenomenon and argues that debiasing must explicitly handle multiple shortcut channels simultaneously (Dwivedi et al., 3 Sep 2025).

A central supervision assumption is that ABIL uses target labels and known bias-attribute labels during training. The framework is therefore label-light only in the sense that group or bias labels are required during training but not at test time (Dwivedi et al., 3 Sep 2025).

2. Position within GMBM

ABIL is the first stage of GMBM, followed by Gradient-Suppression Fine-Tuning. The paper characterizes ABIL as the bias-identification and representation-structuring stage: it learns a main backbone representation, trains one encoder per known bias attribute, integrates those bias features with the main feature, and forces the classifier to solve the task while these shortcut cues are explicitly present (Dwivedi et al., 3 Sep 2025).

This construction has a specific rationale. If shortcut information remains diffusely entangled inside the backbone, later suppression may be ineffective. By training dedicated encoders to recognize each known shortcut and then integrating them into the classifier input, ABIL makes shortcut directions explicit and separable (Dwivedi et al., 3 Sep 2025). A plausible implication is that ABIL is less a direct debiasing operator than a preparatory representation-learning stage that organizes the geometry needed for later suppression.

The paper describes the overall two-stage logic succinctly: ABIL first learns where the shortcuts live, and Gradient-Suppression Fine-Tuning then stops the classifier from moving along those directions (Dwivedi et al., 3 Sep 2025).

3. Architecture and adaptive fusion mechanism

ABIL consists of five components: a main backbone $x^{(i)}$1, a separate encoder $x^{(i)}$2 for each bias attribute $x^{(i)}$3, a bias-classifier head $x^{(i)}$4 for each attribute-specific encoder, an adaptive fusion mechanism, and a final task classifier $x^{(i)}$5 (Dwivedi et al., 3 Sep 2025).

For sample $x^{(i)}$6, the main backbone feature is

$x^{(i)}$7

and the $x^{(i)}$8-th bias feature is

$x^{(i)}$9

The adaptive weight assigned to each bias branch is computed from cosine similarity: $y^{(i)} \in \{1,\dots,N\}$0 followed by a softmax across the $y^{(i)} \in \{1,\dots,N\}$1 bias attributes: $y^{(i)} \in \{1,\dots,N\}$2 The integrated feature is then formed additively: $y^{(i)} \in \{1,\dots,N\}$3 The task prediction is

$y^{(i)} \in \{1,\dots,N\}$4

(Dwivedi et al., 3 Sep 2025).

This is a residual additive fusion rather than a concatenation or hard routing scheme. The paper explicitly presents the mechanism as adaptive because the contribution of each bias feature varies across attributes and across samples according to feature-conditioned cosine-softmax weights (Dwivedi et al., 3 Sep 2025). The intended interpretation is that larger $y^{(i)} \in \{1,\dots,N\}$5 indicates that the corresponding spurious cue is more aligned with the backbone’s current representation for that sample.

The following table summarizes the stage-1 modules described in (Dwivedi et al., 3 Sep 2025).

Component	Symbol	Role
Main backbone	$y^{(i)} \in \{1,\dots,N\}$6	Produces main feature $y^{(i)} \in \{1,\dots,N\}$7
Bias encoder $y^{(i)} \in \{1,\dots,N\}$8	$y^{(i)} \in \{1,\dots,N\}$9	Produces bias feature $b_j^{(i)}$0
Bias head $b_j^{(i)}$1	$b_j^{(i)}$2	Predicts bias label for attribute $b_j^{(i)}$3
Fusion weights	$b_j^{(i)}$4	Sample-adaptive cosine-softmax weighting
Task classifier	$b_j^{(i)}$5	Predicts target label from $b_j^{(i)}$6

The architectural requirement scales with the number of known bias attributes: one main backbone plus $b_j^{(i)}$7 separate bias encoders (Dwivedi et al., 3 Sep 2025). The paper gives concrete examples: FB-CMNIST uses two encoders for foreground color and background color; CelebA uses two encoders for Wearing_Lipstick and Heavy_Makeup; COCO uses two encoders for two object-bias categories (Dwivedi et al., 3 Sep 2025).

4. Objective function and training dynamics

The ABIL subsection presents the main classification loss on the fused feature as

$b_j^{(i)}$8

while the algorithm box adds auxiliary bias-prediction losses for the bias branches: $b_j^{(i)}$9 Because the notation overloads $j$0, the intended meaning is that each bias head predicts the corresponding bias label from the corresponding bias feature (Dwivedi et al., 3 Sep 2025). The joint ABIL objective is

$j$1

with reported default weight

$j$2

(Dwivedi et al., 3 Sep 2025).

The stage-1 training loop described in the algorithm is:

1. compute backbone features $j$3,
2. compute bias features $j$4,
3. compute attention weights $j$5,
4. fuse features $j$6,
5. compute main classification loss $j$7,
6. compute auxiliary bias loss $j$8,
7. update $j$9 with respect to $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$0 (Dwivedi et al., 3 Sep 2025).

A key point is that the bias encoders are not pretrained separately. They are jointly trained with the backbone and classifier during ABIL (Dwivedi et al., 3 Sep 2025). The paper identifies two mechanisms by which ABIL makes shortcut capture explicit: auxiliary bias supervision forces each branch to represent one shortcut, and fusion into the classifier input exposes the final classifier to a shortcut-enhanced representation (Dwivedi et al., 3 Sep 2025).

What ABIL does not include is also important. The stage itself does not define adversarial gradient reversal, orthogonality constraints, or explicit projection-based suppression. Those belong to the second stage or are absent (Dwivedi et al., 3 Sep 2025).

5. Relationship to Gradient-Suppression Fine-Tuning

ABIL is designed to prepare the inputs used by the second stage. After stage 1, the model has backbone features $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$1 and explicit bias-encoder features $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$2. Stage 2 constructs orthogonal residual bias directions

$h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$3

and penalizes the component of the cross-entropy gradient along these directions via

$h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$4

(Dwivedi et al., 3 Sep 2025).

This suggests that the purpose of ABIL is to learn bias representations that are meaningful enough for those orthogonal residuals $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$5 to act as usable shortcut directions. A plausible implication is that stage 2 depends critically on stage 1 not merely for extra supervision, but for the existence of identifiable per-bias vectors in the representation space.

At inference, the paper states that the bias encoders are not used. The final deployed model retains only a single compact debiased backbone and classifier; the auxiliary bias modules are training-time scaffolding (Dwivedi et al., 3 Sep 2025).

6. Empirical setting, implementation, and scope

The paper reports the following implementation details for the full GMBM pipeline, including ABIL stage 1: FB-CMNIST uses a 7-layer CNN, while CelebA and COCO use ResNet-18; Adam is the optimizer; the main learning rate is $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$6, the fine-tuning learning rate is $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$7, and the batch size is $h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$8 (Dwivedi et al., 3 Sep 2025). A hyperparameter table gives

$h^{(i)} = f_{\mathrm{pen}(x^{(i)}) \in \mathbb{R}^d,$9

for ABIL epochs, suppression epochs, bias-loss weight, gradient-penalty weight, and embedding width, respectively (Dwivedi et al., 3 Sep 2025). The prose also gives longer dataset-level schedules, which the paper presents inconsistently.

The evaluated datasets are FB-CMNIST, CelebA, and COCO (Dwivedi et al., 3 Sep 2025). The reported full-pipeline outcomes include improved unbiased and bias-conflicting accuracy and low Scaled Bias Amplification, including FB-CMNIST accuracies of $g:\mathbb{R}^d \to \{1,\dots,N\}$0, $g:\mathbb{R}^d \to \{1,\dots,N\}$1, and $g:\mathbb{R}^d \to \{1,\dots,N\}$2 at $g:\mathbb{R}^d \to \{1,\dots,N\}$3, respectively; CelebA Male classification unbiased/bias-conflicting results of $g:\mathbb{R}^d \to \{1,\dots,N\}$4 for Wearing_Lipstick and $g:\mathbb{R}^d \to \{1,\dots,N\}$5 for Heavy_Makeup; and COCO unbiased/bias-conflicting results of $g:\mathbb{R}^d \to \{1,\dots,N\}$6 and $g:\mathbb{R}^d \to \{1,\dots,N\}$7 for the two object-bias categories (Dwivedi et al., 3 Sep 2025).

However, the paper does not provide a clean standalone ablation isolating ABIL from Gradient-Suppression Fine-Tuning (Dwivedi et al., 3 Sep 2025). Consequently, strong quantitative claims about ABIL alone would exceed the evidence presented. This suggests that ABIL is best documented as a defined and motivated stage within GMBM rather than as an independently benchmarked method.

7. Terminological ambiguity and related usages

The acronym ABIL is not unique across the literature. In "Learning for Long-Horizon Planning via Neuro-Symbolic Abductive Imitation" (Shao et al., 2024), ABIL stands for ABductive Imitation Learning, a neuro-symbolic framework for long-horizon planning that combines abductive reasoning, sequential consistency, and operator-specific imitation policies. That use is unrelated to Adaptive Bias Integrated Learning except at the level of acronym overlap (Shao et al., 2024).

The broader research landscape also contains conceptually adjacent methods that integrate bias modeling into learning, even when they do not use the ABIL name. "Imbalanced Semi-supervised Learning with Bias Adaptive Classifier" introduces a bias adaptive classifier in which a learned residual "bias attractor" absorbs training bias through bilevel optimization (Wang et al., 2022). "BLADE: Bias-Linked Adaptive DEbiasing" adaptively refines synthetic bias-translated views based on per-sample bias susceptibility, without explicit bias labels (Arora et al., 5 Oct 2025). "BPL: Bias-adaptive Preference Distillation Learning for Recommender System" uses affinity-dependent dual distillation to adapt learning under recommendation bias (Kang et al., 17 Oct 2025). These methods share the design pattern that bias is explicitly modeled and structurally integrated into training rather than only penalized post hoc (Wang et al., 2022, Arora et al., 5 Oct 2025, Kang et al., 17 Oct 2025).

This suggests that Adaptive Bias Integrated Learning, in its explicit sense from (Dwivedi et al., 3 Sep 2025), belongs to a broader family of methods that treat bias as a learnable, representable component of the training pipeline.

8. Interpretation, limitations, and significance

ABIL’s distinctive claim is that known shortcut channels should first be learned well and only then suppressed (Dwivedi et al., 3 Sep 2025). This differs from debiasing methods that attempt immediate invariance or direct adversarial removal. The paper’s adaptive fusion

$g:\mathbb{R}^d \to \{1,\dots,N\}$8

embodies that principle: known shortcuts are not hidden from the classifier during stage 1, but injected back into the decision pathway with sample-dependent weighting (Dwivedi et al., 3 Sep 2025).

Several limitations are explicit. ABIL requires bias or group labels during training, which constrains applicability when such annotations are unavailable (Dwivedi et al., 3 Sep 2025). The approach also presumes that multiple relevant shortcuts are known in advance and can each be assigned a dedicated encoder. The paper further notes that some biases may remain difficult to disentangle when they are nearly inseparable from the target, as illustrated by the residual SBA on CelebA (Dwivedi et al., 3 Sep 2025).

A plausible implication is that ABIL is best suited to settings where shortcut variables are known, observable, and sufficiently structured to justify per-attribute branches. It is less naturally a method for unsupervised bias discovery than one for supervised multi-bias representation shaping.

Within that scope, its main significance is methodological. It formalizes a two-step view of debiasing: first expose known shortcut structure through adaptive integrated learning, then suppress the resulting directions in a compact final model (Dwivedi et al., 3 Sep 2025). In that sense, ABIL is less a complete end-state classifier than a principled first stage for multi-attribute bias mitigation.