Debiased Feature Augmentation (DeFA)

• DeFA is a feature-level debiasing framework that synthesizes diverse bias-conflicting features by disentangling intrinsic and bias attributes.
• It uses separate encoders and classifiers to isolate and swap latent features, reducing reliance on scarce bias-conflicting samples.
• Empirical results on benchmarks like Colored MNIST and Corrupted CIFAR-10 demonstrate that increased feature diversity enhances generalization.

Debiased Feature Augmentation (DeFA) most commonly denotes the feature-level debiasing framework introduced for image classification in "Learning Debiased Representation via Disentangled Feature Augmentation" (Lee et al., 2021). It addresses dataset bias by synthesizing diverse bias-conflicting samples in feature space rather than relying only on naturally occurring bias-conflicting examples, which are described as significantly scarce in biased datasets. The method learns disentangled representations of intrinsic attributes and bias attributes, then generates augmented features by swapping latent bias features while preserving class-relevant intrinsic features. In the supplied literature, the acronym "DeFA" is also reused for later feature-augmentation methods in other domains, including compositional zero-shot learning (Zhang et al., 16 Sep 2025); unless otherwise specified, however, DeFA usually refers to the 2021 debiasing method for biased visual classification (Lee et al., 2021).

1. Problem setting and motivating observation

DeFA is situated in the standard dataset-bias setting in which image classification models rely on peripheral attributes of data items that have strong correlation with a target variable, causing poor generalization on unbiased datasets (Lee et al., 2021). The framework is motivated by two observations stated explicitly in the source material. First, existing debiasing approaches often identify and emphasize samples with no such correlation, namely bias-conflicting samples, without defining the bias type in advance. Second, these bias-conflicting samples are significantly scarce in biased datasets, which limits the debiasing capability of such approaches (Lee et al., 2021).

The central empirical claim behind DeFA is that the diversity of bias-conflicting samples, not just their quantity, is crucial for debiasing as well as the generalization capability (Lee et al., 2021). This reframes the problem: rather than only reweighting or isolating rare bias-conflicting samples, DeFA seeks to synthesize diverse bias-conflicting features from the much larger pool of bias-aligned samples. A plausible implication is that DeFA shifts the bottleneck from sample selection to representation manipulation.

This perspective is consistent with the broader argument that data augmentation can alter the relative importance of various features, effectively making certain informative but hard to learn features more likely to be captured in the learning process, and that this effect is more pronounced for non-linear models, such as neural networks (Shen et al., 2022). In that sense, DeFA can be read as a targeted feature-manipulation strategy designed specifically for debiasing.

2. Disentangled representation learning

DeFA is built around a disentanglement module with two encoders and two linear classifiers (Lee et al., 2021). The encoder $E_i$ extracts intrinsic features $z_i$ from image $x$, and the encoder $E_b$ extracts bias features $z_b$ from the same image. The classifier $C_i$ predicts the target label from concatenated features, while $C_b$ predicts the bias label from concatenated features. The training protocol deliberately separates the two streams: gradient backpropagation is not shared between them, keeping bias and intrinsic representations independent (Lee et al., 2021).

The bias stream is trained to overfit to bias attributes using Generalized Cross-Entropy Loss (GCE), whereas the intrinsic stream uses standard Cross-Entropy Loss (CE) (Lee et al., 2021). DeFA further adopts the relative difficulty score

$$W(z) = \frac{CE(C_b(z), y)}{CE(C_i(z), y) + CE(C_b(z), y)},$$

where high $W(z)$ indicates that the sample is likely bias-conflicting because it is difficult for the bias-based classifier (Lee et al., 2021). These high-$W$ samples receive more emphasis when updating $z_i$0 and $z_i$1.

The overall disentanglement loss is

$z_i$2

with the additional constraint that $z_i$3's loss is not backpropagated to $z_i$4, and vice versa (Lee et al., 2021). This design is intended to separate class-defining information from spuriously correlated bias information while still using the biased stream to identify hard examples.

3. Feature augmentation by latent swapping

Once the two encoders are sufficiently disentangled, DeFA performs feature-level data augmentation by in-batch swapping (Lee et al., 2021). For each sample in a mini-batch, the intrinsic feature $z_i$5 is retained, while the bias feature $z_i$6 is randomly replaced with $z_i$7 from another sample. The resulting augmented feature is

$z_i$8

This operation is designed to break the label-bias correlation: the intrinsic part remains associated with the original class, but the bias part is taken from a different image (Lee et al., 2021). The source material states that the synthesized samples are diverse bias-conflicting features derived from the diversity of the majority bias-aligned samples. This suggests that DeFA treats the abundance of bias-aligned data not as a nuisance alone, but also as a reservoir from which diverse intrinsic attributes can be recombined into more informative training signals.

The augmentation objective is

$z_i$9

where $x$0 is the swapped sample's bias label (Lee et al., 2021). The full training objective is

$x$1

A critical implementation detail is scheduling. Swapping is introduced only after a certain point in training, when representations are already at least partially disentangled; early swapping injects noise and can harm learning (Lee et al., 2021). This is not a minor optimization detail but part of the method definition, because the usefulness of swapping depends on the quality of the learned factorization.

4. Empirical behavior and reported results

The reported experiments use Colored MNIST, Corrupted CIFAR-10, and BFFHQ (Lee et al., 2021). The paper first argues that models trained with diverse bias-conflicting samples, even in small numbers, outperform those with many but less diverse ones. That claim is central because it justifies feature synthesis as a debiasing mechanism rather than a mere data-expansion heuristic (Lee et al., 2021).

Against baseline methods, the reported scores are as follows on the stated biased settings: Vanilla achieves $x$2 on MNIST (1%), $x$3 on CIFAR-10 (1%), and $x$4 on BFFHQ (0.5%); LfF achieves $x$5, $x$6, and $x$7; ReBias achieves $x$8, $x$9, and $E_b$0; DeFA achieves $E_b$1, $E_b$2, and $E_b$3 (Lee et al., 2021). The source text summarizes these results by stating that DeFA routinely outperforms existing baselines, especially those not reliant on bias type specification, and that for real-world data BFFHQ it leads over all methods, including those with access to bias labels or specifically tailored architectures (Lee et al., 2021).

The ablation study further isolates the contribution of the main components. With disentanglement only, the reported scores are $E_b$4, $E_b$5, and $E_b$6; adding augmentation without scheduling yields $E_b$7, $E_b$8, and $E_b$9; adding scheduled augmentation yields $z_b$0, $z_b$1, and $z_b$2 (Lee et al., 2021). The accompanying interpretation is explicit: all components are crucial, especially scheduled augmentation.

The qualitative evidence is also part of the method’s empirical support. The source material reports that $z_b$3 and $z_b$4 cluster according to target and bias labels, respectively, in 2D t-SNE projections, and that reconstruction experiments show that swapping $z_b$5 changes the image’s color in MNIST or changes gender while preserving age in BFFHQ (Lee et al., 2021). These observations are used to support the claim that the representation is meaningfully disentangled rather than only operationally separated by the losses.

5. Relation to adjacent debiasing methods

DeFA belongs to a larger family of feature-space debiasing methods, but its mechanism is distinct. In contrast to approaches that require bias types or explicit bias labels, the source text characterizes DeFA as requiring no bias type labels or pre-defined bias type (Lee et al., 2021). In contrast to reweighting-only methods, it augments bias-conflicting sample diversity from the start by synthesizing new features (Lee et al., 2021).

A later image-classification method based on bias-contrastive pairs makes this contrast explicit. That work states that DeFA or disentangled feature augmentation increases the diversity of synthetic bias-conflicting data in feature space, but does not provide spatially explicit supervision or guidance regarding which features to focus on for debiasing (Park et al., 2024). The criticism is methodological rather than dismissive: DeFA addresses scarcity and diversity, whereas the later method addresses the separate question of where intrinsic features are located.

The acronym also appears in other domains with different meanings. In semantic segmentation, the supplied material summarizes a model-agnostic and stochastic training scheme, denoted there as "DropClass / DeFA," in which information related to a randomly sampled class is suppressed by a feature selection process in the feature space, reducing feature dependencies among classes and improving results especially for under-represented classes (Chu et al., 2021). In compositional zero-shot learning, a later method also called DeFA integrates a disentangle-and-reconstruct framework for feature augmentation with a debiasing strategy, using frequency-aware weighting to address long-tailed distributions of attributes, objects, and compositions (Zhang et al., 16 Sep 2025). These later usages indicate that "DeFA" has become an umbrella acronym for debiasing through feature augmentation, even though the concrete architectures and objectives differ substantially.

6. Conceptual significance and limitations

DeFA’s main conceptual contribution is to shift debiasing from sample mining toward feature synthesis. The source material describes this as a move from increasing the number of bias-conflicting samples, which is impractical in real-world rare settings, to increasing their diversity via synthetic augmentation in feature space (Lee et al., 2021). This is a precise claim about what the method changes in the debiasing pipeline.

A common misconception is to treat DeFA as ordinary data augmentation performed later in the pipeline. The 2021 method is more specific: it learns disentangled intrinsic and bias representations and then uses latent swapping to construct bias-conflicting features (Lee et al., 2021). Another misconception is that the method works simply because it adds more training instances. The motivating experiments instead emphasize that diversity of bias-conflicting samples is crucial, and the broader theory of augmentation as feature manipulation supports the view that augmentation changes which features are likely to be learned by the model rather than only enlarging the dataset (Shen et al., 2022).

The limitations described in the supplied material are also concrete. DeFA depends on sufficient disentanglement before swapping; early swapping injects noise and can harm learning (Lee et al., 2021). Moreover, later work argues that DeFA does not directly guide models where to focus for learning intrinsic features, because it lacks spatially explicit guidance (Park et al., 2024). This suggests that DeFA is strongest as a diversity-generation mechanism and weaker as an explicit localization mechanism.

Taken together, the supplied literature presents DeFA as a foundational feature-space debiasing strategy: it formalizes the synthesis of diverse bias-conflicting features from bias-aligned data, demonstrates competitive debiasing performance without bias type specification, and helped establish feature augmentation as a central instrument for robust representation learning under spurious correlation (Lee et al., 2021).