Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Rarity (2403.02363v1)

Published 4 Mar 2024 in cs.LG and cs.AI

Abstract: Real-world datasets commonly exhibit noisy labels and class imbalance, such as long-tailed distributions. While previous research addresses this issue by differentiating noisy and clean samples, reliance on information from predictions based on noisy long-tailed data introduces potential errors. To overcome the limitations of prior works, we introduce an effective two-stage approach by combining soft-label refurbishing with multi-expert ensemble learning. In the first stage of robust soft label refurbishing, we acquire unbiased features through contrastive learning, making preliminary predictions using a classifier trained with a carefully designed BAlanced Noise-tolerant Cross-entropy (BANC) loss. In the second stage, our label refurbishment method is applied to obtain soft labels for multi-expert ensemble learning, providing a principled solution to the long-tail noisy label problem. Experiments conducted across multiple benchmarks validate the superiority of our approach, Label Refurbishment considering Label Rarity (LR2), achieving remarkable accuracies of 94.19% and 77.05% on simulated noisy CIFAR-10 and CIFAR-100 long-tail datasets, as well as 77.74% and 81.40% on real-noise long-tail datasets, Food-101N and Animal-10N, surpassing existing state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. What is the effect of importance weighting in deep learning? In International conference on machine learning, pp.  872–881. PMLR, 2019.
  2. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 32, 2019.
  3. Heteroskedastic and imbalanced deep learning with adaptive regularization. arXiv preprint arXiv:2006.15766, 2020.
  4. Two wrongs don’t make a right: Combating confirmation bias in learning with label noise. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  14765–14773, 2023.
  5. Beyond class-conditional assumption: A primary attempt to combat instance-dependent label noise. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  11442–11450, 2021.
  6. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  15750–15758, 2021.
  7. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020.
  8. Remix: rebalanced mixup. In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pp.  95–110. Springer, 2020.
  9. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  9268–9277, 2019.
  10. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  11. No one left behind: Improving the worst categories in long-tailed learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  15804–15813, 2023.
  12. A multiple resampling method for learning from imbalanced data sets. Computational intelligence, 20(1):18–36, 2004.
  13. Contrastive learning improves model robustness under label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  2703–2708, 2021.
  14. Learning from class-imbalanced data: Review of methods and applications. Expert systems with applications, 73:220–239, 2017.
  15. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  17. Distilling virtual examples for long-tailed recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  235–244, 2021.
  18. Using trusted data to train deep networks on labels corrupted by severe noise. Advances in neural information processing systems, 31, 2018.
  19. Disentangling label distribution for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  6626–6636, 2021.
  20. Uncertainty-aware learning against label noise on imbalanced datasets. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.  6960–6969, 2022.
  21. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217, 2019.
  22. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  9676–9686, 2022.
  23. Learning multiple layers of features from tiny images. 2009.
  24. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
  25. Learning debiased representation via disentangled feature augmentation. Advances in Neural Information Processing Systems, 34:25123–25133, 2021.
  26. Cleannet: Transfer learning for scalable image classifier training with label noise. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  5447–5456, 2018.
  27. Learning to learn from noisy labeled data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5051–5059, 2019.
  28. Dividemix: Learning with noisy labels as semi-supervised learning. In ICLR 2020, 2020.
  29. Self-supervised learning is more robust to dataset imbalance. arXiv preprint arXiv:2110.05025, 2021.
  30. Label-noise learning with intrinsically long-tailed data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  1369–1378, 2023.
  31. Dimensionality-driven learning with noisy labels. In International Conference on Machine Learning, pp.  3355–3364. PMLR, 2018.
  32. Long-tail learning via logit adjustment. arXiv preprint arXiv:2007.07314, 2020.
  33. Influence-balanced loss for imbalanced visual classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  735–744, 2021.
  34. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  1944–1952, 2017.
  35. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06548, 2017.
  36. Balanced meta-softmax for long-tailed visual recognition. Advances in neural information processing systems, 33:4175–4186, 2020.
  37. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
  38. Meta-weight-net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, 32, 2019.
  39. Selfie: Refurbishing unclean samples for robust deep learning. In International Conference on Machine Learning, pp.  5907–5915. PMLR, 2019.
  40. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  41. Co-learning: Learning from noisy labels with self-supervision. In Proceedings of the 29th ACM International Conference on Multimedia, pp.  1405–1413, 2021.
  42. Joint optimization framework for learning with noisy labels. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  5552–5560, 2018.
  43. Local and global logit adjustments for long-tailed learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  11783–11792, 2023.
  44. Long-tailed recognition by routing diverse distribution-aware experts. arXiv preprint arXiv:2010.01809, 2020.
  45. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  322–330, 2019.
  46. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  13726–13735, 2020.
  47. Robust long-tailed learning under label noise. arXiv preprint arXiv:2108.11569, 2021.
  48. Robust early-learning: Hindering the memorization of noisy labels. In International conference on learning representations, 2020.
  49. Sample selection with uncertainty of losses for learning with noisy labels. arXiv preprint arXiv:2106.00445, 2021.
  50. Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  2691–2699, 2015.
  51. Jo-src: A contrastive approach for combating noisy labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5192–5201, 2021.
  52. Identifying hard noise in long-tailed sample distribution. In European Conference on Computer Vision, pp.  739–756. Springer, 2022.
  53. Fasa: Feature augmentation and sampling adaptation for long-tailed instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  3457–3466, 2021.
  54. When noisy labels meet long tail dilemmas: A representation calibration method. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  15890–15900, 2023a.
  55. Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023b.
  56. Mdcs: More diverse experts with consistency self-distillation for long-tailed recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  11597–11608, 2023.
  57. Improving calibration for long-tailed recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  16489–16498, 2021.
  58. Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  9719–9728, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now