Label Flipping in Machine Learning

• Label flipping is the manipulation of class labels while keeping instance features unchanged, serving both malicious and constructive purposes.
• It is applied in adversarial attacks, data augmentation, fairness repair, and counterfactual evaluation, each with distinct operational objectives.
• Empirical studies reveal that even modest rates of label flipping can severely degrade model performance, prompting robust defense and detection strategies.

Label flipping denotes a change in class labels without a corresponding change in the underlying instance representation. In the canonical supervised-learning form, the training set is written as $D=\{(x_i,y_i)\}_{i=1}^N$, and a poisoned variant replaces some labels by incorrect values, producing $D'=\{(x_i,y_i')\}_{i=1}^N$ while leaving features unchanged (Nowroozi et al., 2024, Paudice et al., 2018). Across the literature, however, the same term is also used for constructive label transformation in data augmentation, fairness-oriented preprocessing, and counterfactual evaluation; its meaning is therefore context-dependent rather than uniform (Zhou et al., 2021, Zhang et al., 2022, Wang et al., 20 May 2025).

1. Core definitions and semantic scope

In adversarial machine learning, label flipping is a special case of data poisoning in which the attacker cannot change the input features $x$, but can change the labels $y$ of a subset of training points (Paudice et al., 2018). In the binary URL-detection formulation, with $y_i \in \{0,1\}$, the attacker flips benign labels $y_i=0$ to malicious $y_i'=1$ and malicious labels $y_i=1$ to benign $y_i'=0$, creating a poisoned training set $D'=\{(x_i,y_i')\}_{i=1}^N$ (Nowroozi et al., 2024). A common scalar parameter is the poisoning rate, written either as $D'=\{(x_i,y_i')\}_{i=1}^N$0 or $D'=\{(x_i,y_i')\}_{i=1}^N$1, denoting the fraction of training samples whose labels are altered (Nowroozi et al., 2024, Zhang et al., 2023).

The term does not always denote adversarial corruption. In FlipDA, “label flipping” means intentionally generating new training examples whose labels are different from the original example they are derived from, and then training on these new $D'=\{(x_i,y_i')\}_{i=1}^N$2 pairs (Zhou et al., 2021). In iFlipper, label flipping is a pre-processing technique for improving individual fairness by minimally changing labels subject to a bound on individual fairness violations (Zhang et al., 2022). In counterfactual data augmentation, label flipping is operationalized through the Label Flip Rate, $D'=\{(x_i,y_i')\}_{i=1}^N$3, where $D'=\{(x_i,y_i')\}_{i=1}^N$4 is the judge model’s label for the counterfactual and $D'=\{(x_i,y_i')\}_{i=1}^N$5 is the original ground-truth label (Wang et al., 20 May 2025). In adversarial computer vision, “label flipping” can also refer to the post-attack event $D'=\{(x_i,y_i')\}_{i=1}^N$6, which underlies fooling rate (Mopuri et al., 2020).

Context	Meaning of label flipping	Representative source
Data poisoning	Modify labels of existing training samples while leaving features unchanged	(Paudice et al., 2018)
Data augmentation	Generate new examples whose labels differ from the source example	(Zhou et al., 2021)
Fairness preprocessing	Minimally flip labels to reduce individual fairness violations	(Zhang et al., 2022)
Counterfactual evaluation	Measure whether a counterfactual’s label differs from the original label	(Wang et al., 20 May 2025)
Adversarial fooling	Count whether the model’s predicted label changes after perturbation	(Mopuri et al., 2020)

This multiplicity of meanings is a recurrent source of confusion. A precise account must therefore distinguish malicious label corruption, constructive label transformation, and evaluation-only notions of label change.

2. Poisoning formulations and empirical effects in centralized learning

The most explicit adversarial formulation is bilevel. For binary linear classification, the attacker chooses an indicator vector $D'=\{(x_i,y_i')\}_{i=1}^N$7 with $D'=\{(x_i,y_i')\}_{i=1}^N$8, where $D'=\{(x_i,y_i')\}_{i=1}^N$9 means that point $x$0 is flipped, and then maximizes validation loss after the learner retrains on the poisoned set $x$1 (Paudice et al., 2018). Because exact search over $x$2 possibilities is intractable, a greedy Label Flipping Attack iteratively flips the label whose change yields the largest validation loss after retraining (Paudice et al., 2018). In tabular learning, Falfa reformulates the adversary’s objective for cross-entropy classifiers as a linear program and shows that adversarial label-flipping attacks can camouflage a highly skewed dataset as an easily solvable classification problem, often misleading machine learning practitioners into lower defenses and miscalculations of potential risks (Chang et al., 2023).

Empirical effects are substantial even at modest budgets. In RF-based malicious URL detection, random LF at poisoning rates $x$3 preserves high training accuracy, typically $x$4, while yielding Attack Success Rate between $x$5 and $x$6; the clean RF baseline reaches $x$7 testing accuracy on all six URL datasets (Nowroozi et al., 2024). For linear classifiers on BreastCancer, MNIST 1 vs 7, and Spambase, flipping $x$8 of training labels increases average classification error by factors $x$9, $y$0, and $y$1, respectively (Paudice et al., 2018). In EEG-based emotion assessment with four classes, AdaBoost drops from about $y$2 accuracy at $y$3 poisoning to about $y$4 at $y$5 poisoning, while Random Forest drops from about $y$6 to about $y$7 under the same schedule (Zhang et al., 2023). In wearable human activity recognition, poisoning $y$8 of training labels reduces MLP accuracy from $y$9 to $y_i \in \{0,1\}$0, and at $y_i \in \{0,1\}$1 poisoning the same model reaches $y_i \in \{0,1\}$2 accuracy (Shahid et al., 2022).

Label flipping also appears in hybrid backdoor constructions. DirtyFlipping is a dirty label-flipping attack for audio models that attaches a trigger such as clapping to selected data patterns associated with the target class and flips labels to a dirty label; with about $y_i \in \{0,1\}$3 poisoned data it reports benign accuracy remaining high and attack success rate reaching $y_i \in \{0,1\}$4 across multiple architectures, including CNN, VGG16, LSTM, and several pre-trained audio transformers (Mengara, 2024). This suggests that the operational boundary between pure label poisoning and backdoor insertion is porous when triggers and label inversion are coupled.

3. Federated and distributed label flipping

In federated learning, label flipping is implemented locally. Each malicious client replaces its local dataset $y_i \in \{0,1\}$5 by $y_i \in \{0,1\}$6, then trains normally and uploads a poisoned model update to the server (Rahman et al., 26 Sep 2025). A common targeted rule chooses a source class $y_i \in \{0,1\}$7 and target class $y_i \in \{0,1\}$8, setting $y_i \in \{0,1\}$9 if $y_i=0$0 and $y_i=0$1 otherwise (Rahman et al., 26 Sep 2025, Jebreel et al., 2022). This is easy to perform because it does not require protocol violations or gradient fabrication; the server sees an apparently legitimate update (Rahman et al., 26 Sep 2025).

Several studies show that federated label flipping is strongly modulated by data heterogeneity and by how the attack budget is distributed. In distributed logistic regression, a budget of only $y_i=0$2 of labels at each training step can reduce the accuracy of the model by $y_i=0$3, and some models can perform worse than random guessing when up to $y_i=0$4 of labels are flipped (El-Kabid et al., 28 Feb 2025). For a fixed corrupted fraction $y_i=0$5, increasing write-access $y_i=0$6 generally yields stronger attacks than increasing the flipping budget $y_i=0$7 alone (El-Kabid et al., 28 Feb 2025). In a broader empirical study over MLR, SVC, MLP, CNN, Random Forest, XGBoost, and LSTM, models differ in their inherent robustness to the percentage of adversarial clients and the percentage of labels flipped by each adversarial client (Bhatnagar et al., 2024). MLR and SVC are comparatively stable across 10-client and 100-client settings, whereas Random Forest and XGBoost degrade strongly as the number of clients increases, even before attack (Bhatnagar et al., 2024).

Non-IID settings are particularly difficult. One study reports that existing FL methods perform similarly in IID settings but fail to maintain model robustness in Non-IID settings; with MNIST$y_i=0$8 under attack, Krum, Median, Trimmed Mean, and FLTrust yield global accuracies $y_i=0$9, $y_i'=1$0, $y_i'=1$1, and $y_i'=1$2, whereas HSCSFL reaches $y_i'=1$3 (Li et al., 2023). AntiFLipper addresses multi-class label-flipping attacks by shifting detection to the client side: under a constant attack with $y_i'=1$4 malicious clients, it reports $y_i'=1$5 accuracy and $y_i'=1$6 ms aggregation time on CIFAR-10 Non-IID, compared with $y_i'=1$7 and $y_i'=1$8 ms for FLAME (Rahman et al., 26 Sep 2025). A separate gradient-based defense shows that the contradicting objectives of attackers and honest peers on source-class examples are reflected in the gradients corresponding to the source and target neurons in the output layer, and reports effectiveness regardless of the data distribution or model dimensionality (Jebreel et al., 2022).

4. Detection, sanitization, and robust training defenses

The oldest recurring defense family is neighborhood-based label sanitization. In binary linear classification, a $y_i'=1$9-NN defense relabels a point when the mode of its $y_i=1$0 nearest neighbors has confidence at least $y_i=1$1; at $y_i=1$2 poisoning it keeps BreastCancer and Spambase accuracy close to the clean-data baseline and substantially mitigates the attack on MNIST 1 vs 7, although it causes a small performance drop when no attack is present (Paudice et al., 2018). The URL-defense variant first selects the best $y_i=1$3 on a trusted dataset, then performs an alarm-and-correction pass over the poisoned set by comparing each sample’s label with the mode of its $y_i=1$4 nearest neighbors; on six URL datasets and poisoning rates $y_i=1$5, detection accuracy reaches up to $y_i=1$6, post-defense RF training accuracy lies in the range $y_i=1$7, and ASR collapses to near $y_i=1$8 after retraining (Nowroozi et al., 2024). In wearable HAR, a KNN-based sanitization mechanism using a small trusted dataset recovers MLP accuracy to $y_i=1$9 at $y_i'=0$0 poisoning and $y_i'=0$1 at $y_i'=0$2 poisoning, compared with $y_i'=0$3 and $y_i'=0$4 without defense (Shahid et al., 2022).

Semi-supervised and clustering-based defenses appear when local label consistency is insufficient. In Android malware detection, a Silhouette Clustering-based Label Flipping Attack flips labels when the silhouette value is non-positive, and the proposed Label-based Semi-supervised Defense and clustering-based Semi-supervised Defense correct attacked labels by combining graph-based semi-supervised learning, CNN predictions, and clustering-quality criteria; using random forest feature selection and varying ratios of features can result in an improvement of up to $y_i'=0$5 accuracy when compared with the state-of-the-art method in the literature (Taheri et al., 2019). In federated learning, HSCSFL assigns each client an honest score equal to the dot product of its class-wise accuracy vector and the global risk vector, then aggregates only the top $y_i'=0$6 of clients; under MNIST$y_i'=0$7 and label flipping, it preserves attacked-class accuracy far better than Krum, Median, Trimmed Mean, or FLTrust (Li et al., 2023). AntiFLipper instead uses local evaluation feedback, trust inference, anomaly-guided filtering, and weighted aggregation; with evaluation on only $y_i'=0$8 of local data, the relative overhead is about $y_i'=0$9, and aggregation time remains close to FedAvg while robustness remains competitive or superior under both IID and Non-IID settings (Rahman et al., 26 Sep 2025).

A distinct defense logic exploits output-layer gradients rather than labels themselves. For targeted LF in FL, the gradients attached to source and target neurons are discriminative because honest and malicious peers optimize opposite objectives on source-class examples; dynamically extracting those gradients, clustering them, and filtering the suspicious cluster yields lower test error, higher overall accuracy, higher source-class accuracy, lower attack success rate, and higher stability of the source class accuracy than several robust aggregation baselines (Jebreel et al., 2022). Collectively, these results suggest that defenses against label flipping are most effective when they model feature-label inconsistency explicitly rather than assuming that poisoned examples are merely feature-space outliers.

5. Constructive and non-adversarial uses

A separate line of work treats label flipping as a useful operation rather than an attack. FlipDA argues that generating label-flipped data is more crucial to the performance than generating label-preserved data in few-shot natural language understanding (Zhou et al., 2021). The method first fine-tunes an initial classifier, then uses T5 with pattern-based cloze prompting to generate candidate texts, uses the classifier to select high-confidence candidates whose predicted label differs from the source label, and retrains on the union of original data, label-flipped data, and label-preserved data (Zhou et al., 2021). On ALBERT-xxlarge-v2 across eight SuperGLUE tasks, FlipDA improves the average score from $D'=\{(x_i,y_i')\}_{i=1}^N$0 to $D'=\{(x_i,y_i')\}_{i=1}^N$1 with MaxDrop $D'=\{(x_i,y_i')\}_{i=1}^N$2; on DeBERTa-v2-xxlarge it improves the average from $D'=\{(x_i,y_i')\}_{i=1}^N$3 to $D'=\{(x_i,y_i')\}_{i=1}^N$4 with MaxDrop $D'=\{(x_i,y_i')\}_{i=1}^N$5 (Zhou et al., 2021). The paper explicitly distinguishes this from adversarial label-flipping attacks: the text itself changes and the flipped label is validated by a classifier, so the new pair is intended to be semantically consistent (Zhou et al., 2021).

In fairness research, iFlipper formulates label flipping as constrained repair. Given binary labels $D'=\{(x_i,y_i')\}_{i=1}^N$6 and a similarity matrix $D'=\{(x_i,y_i')\}_{i=1}^N$7, it minimizes the number of flipped labels, $D'=\{(x_i,y_i')\}_{i=1}^N$8, subject to a total individual fairness violation constraint $D'=\{(x_i,y_i')\}_{i=1}^N$9 (Zhang et al., 2022). The underlying optimization is NP-hard, so the method converts the problem into an ILP, relaxes it to an LP, proves that the LP optimum can be transformed into a $D'=\{(x_i,y_i')\}_{i=1}^N$00-valued solution, then applies adaptive rounding and reverse greedy refinement while preserving feasibility (Zhang et al., 2022). Experiments on COMPAS, AdultCensus, Credit, and a large synthetic dataset show that iFlipper significantly outperforms pre-processing baselines in terms of individual fairness and accuracy on unseen test sets, and that it can be combined with in-processing techniques such as SenSR for further gains (Zhang et al., 2022).

These constructive uses materially alter the semantics of the term. Here label flipping is not corruption of unchanged instances but either the generation of semantically changed counterexamples or the optimization of labels to satisfy a fairness criterion. This suggests that the phrase should always be interpreted relative to the surrounding objective: attack, augmentation, or repair.

6. Evaluation metrics, counterfactuals, and recurring misconceptions

One recurring misconception is that label flipping is adequately captured by ordinary accuracy. Several attack studies show the opposite. In URL detection, training accuracy under $D'=\{(x_i,y_i')\}_{i=1}^N$01 poisoning remains $D'=\{(x_i,y_i')\}_{i=1}^N$02 while ASR lies between approximately $D'=\{(x_i,y_i')\}_{i=1}^N$03 and $D'=\{(x_i,y_i')\}_{i=1}^N$04 (Nowroozi et al., 2024). In federated settings, global accuracy can also conceal strong source-class degradation or attack-specific failures, which is why source-class accuracy, attack success rate, and class-wise risk vectors recur across defenses (Jebreel et al., 2022, Li et al., 2023). In adversarial image evaluation, fooling rate only checks whether the predicted label changes and ignores the severity of the change; FR@K, QI-Wup, and QI-Vis were proposed precisely because mere label flipping does not distinguish dog-to-dog from dog-to-vehicle confusions (Mopuri et al., 2020).

A second misconception is that all label noise is alike. BadLabel defines a label-noise type crafted from a label-flipping attack so that the loss values of clean and noisy labels become indistinguishable (Zhang et al., 2023). Under BadLabel on CIFAR-10 at $D'=\{(x_i,y_i')\}_{i=1}^N$05 corruption, DivideMix drops to $D'=\{(x_i,y_i')\}_{i=1}^N$06 best accuracy and ProMix to $D'=\{(x_i,y_i')\}_{i=1}^N$07, whereas the same methods stay above $D'=\{(x_i,y_i')\}_{i=1}^N$08 under $D'=\{(x_i,y_i')\}_{i=1}^N$09 symmetric noise (Zhang et al., 2023). The proposed robust LNL method perturbs labels adversarially at each epoch, then uses BayesGMM and semi-supervised learning to recover a mostly clean subset; ablations show that both label perturbation and filtering low-quality divisions are essential under BadLabel (Zhang et al., 2023). This indicates that the usual small-loss assumption is not a stable property of label noise, but a property of particular noise models.

A third misconception concerns counterfactual data augmentation. In this literature, label flipping is not the attack but the criterion for counterfactual validity: $D'=\{(x_i,y_i')\}_{i=1}^N$10 (Wang et al., 20 May 2025). Yet automatic evaluation is unstable: across two LLM-based generation methods, three datasets, five generator models, and fifteen judge models, independent, non-fine-tuned judges achieve the highest alignment ratio $D'=\{(x_i,y_i')\}_{i=1}^N$11, but even the best judges differ on average by $D'=\{(x_i,y_i')\}_{i=1}^N$12 percentage points in LFR from human evaluation (Wang et al., 20 May 2025). With a user study of $D'=\{(x_i,y_i')\}_{i=1}^N$13 and Cohen’s $D'=\{(x_i,y_i')\}_{i=1}^N$14, the paper concludes that a fully automated pipeline for CDA may be inadequate and requires human intervention (Wang et al., 20 May 2025).

Across these strands, label flipping is best understood as a family of operations on supervision rather than a single technique. In adversarial settings it is a poisoning mechanism that exploits the model’s trust in labels; in augmentation it is a way to generate informative counterexamples; in fairness it is a repair primitive; and in counterfactual evaluation it is a metric event. The shared technical core is the manipulation or detection of label-state transitions, but the surrounding objective determines whether the transition is harmful, beneficial, or merely diagnostic.