Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical (2311.15502v3)

Published 27 Nov 2023 in cs.LG

Abstract: Complementary-label learning is a weakly supervised learning problem in which each training example is associated with one or multiple complementary labels indicating the classes to which it does not belong. Existing consistent approaches have relied on the uniform distribution assumption to model the generation of complementary labels, or on an ordinary-label training set to estimate the transition matrix in non-uniform cases. However, either condition may not be satisfied in real-world scenarios. In this paper, we propose a novel consistent approach that does not rely on these conditions. Inspired by the positive-unlabeled (PU) learning literature, we propose an unbiased risk estimator based on the Selected-Completely-at-Random assumption for complementary-label learning. We then introduce a risk-correction approach to address overfitting problems. Furthermore, we find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems when using the one-versus-rest strategy. Extensive experimental results on both synthetic and real-world benchmark datasets validate the superiority of our proposed approach over state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138–156, 2006.
  2. Semi-supervised novelty detection. Journal of Machine Learning Research, 11:2973–3009, 2010.
  3. Learning from similarity-confidence data. In Proceedings of the 38th International Conference on Machine Learning, pp.  1272–1282, 2021.
  4. A variational approach for learning from positive and unlabeled data. In Advances in Neural Information Processing Systems 33, pp.  14844–14854, 2020a.
  5. Negative sampling in semi-supervised learning. In Proceedings of the 37th International Conference on Machine Learning, pp.  1704–1714, 2020b.
  6. Unbiased risk estimators can mislead: A case study of learning with complementary labels. In Proceedings of the 37th International Conference on Machine Learning, pp.  1929–1938, 2020.
  7. Deep learning for classical Japanese literature. arXiv preprint arXiv:1812.01718, 2018.
  8. Risk bounds for positive-unlabeled learning under the selected at random assumption. Journal of Machine Learning Research, 24(107):1–31, 2023.
  9. GradPU: Positive-unlabeled learning via gradient penalty and positive upweighting. In Proceedings of the 37th AAAI Conference on Artificial Intelligence, pp.  7296–7303, 2023.
  10. Boosting semi-supervised learning with contrastive complementary labeling. Neural Networks, 170:417–426, 2024.
  11. Analysis of learning from positive and unlabeled data. In Advances in Neural Information Processing Systems 27, pp.  703–711, 2014.
  12. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.  213–220, 2008.
  13. Learning with multiple complementary labels. In Proceedings of the 37th International Conference on Machine Learning, pp.  3072–3081, 2020a.
  14. Provably consistent partial-label learning. In Advances in Neural Information Processing Systems 33, pp.  10948–10960, 2020b.
  15. Discriminative complementary-label learning with weighted loss. In Proceedings of the 38th International Conference on Machine Learning, pp.  3587–3597, 2021.
  16. Mixture proportion estimation and PU learning: A modern approach. In Advances in Neural Information Processing Systems 34, pp.  8532–8544, 2021.
  17. Robust loss functions under label noise for deep neural networks. In Proceedings of the 31st AAAI conference on artificial intelligence, pp.  1919–1925, 2017.
  18. Size-independent sample complexity of neural networks. In Proceedings of the 31st Conference On Learning Theory, pp.  297–299, 2018.
  19. Rethinking precision of pseudo label: Test-time adaptation via complementary learning. arXiv preprint arXiv:2301.06013, 2023.
  20. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp.  770–778, 2016.
  21. Active learning with partial feedback. In Proceedings of the 7th International Conference on Learning Representations, 2019.
  22. Densely connected convolutional networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp.  4700–4708, 2017.
  23. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, pp.  448–456, 2015.
  24. Learning from complementary labels. In Advances in Neural Information Processing Systems 30, pp.  5644–5654, 2017.
  25. Complementary-label learning for arbitrary losses and models. In Proceedings of the 36th International Conference on Machine Learning, pp.  2971–2980, 2019.
  26. Positive-unlabeled learning with label distribution alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15345–15363, 2023.
  27. NLNL: Negative learning for noisy labels. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, pp.  101–110, 2019.
  28. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015.
  29. Positive-unlabeled learning with non-negative risk estimator. In Advances in Neural Information Processing Systems 30, pp.  1674–1684, 2017.
  30. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
  31. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  32. Probability in Banach Spaces: Isoperimetry and Processes, volume 23. Springer Science & Business Media, 1991.
  33. Who is your right mixup partner in positive and unlabeled learning. In Proceedings of the 10th International Conference on Learning Representations, 2022.
  34. Reduction from complementary-label learning to probability estimates. In Proceedings of the 27th Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp.  469–481, 2023.
  35. Consistent complementary-label learning via order-preserving losses. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, pp.  8734–8748, 2023.
  36. Classification with noisy labels by importance reweighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(3):447–461, 2015.
  37. Decoupled weight decay regularization. In Proceedings of the 7th International Conference on Learning Representations, 2019.
  38. Mitigating overfitting in supervised classification from two unlabeled datasets: A consistent risk correction approach. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, pp.  1115–1125, 2020.
  39. Progressive identification of true labels for partial-label learning. In Proceedings of the 37th International Conference on Machine Learning, pp.  6500–6510, 2020.
  40. Rethinking safe semi-supervised learning: Transferring the open-set problem to a close-set one. In Proceedings of 2023 IEEE/CVF International Conference on Computer Vision, pp.  16370–16379, 2023.
  41. Foundations of Machine Learning. The MIT Press, 2012.
  42. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning, pp.  807–814, 2010.
  43. Theoretical comparisons of positive-unlabeled learning against positive-negative learning. In Advances in Neural Information Processing Systems 29, pp.  1199–1207, 2016.
  44. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp.  8026–8037, 2019.
  45. Decompositional generation process for instance-dependent partial label learning. In Proceedings of the 11th International Conference on Learning Representations, 2023.
  46. Mixture proportion estimation via kernel embeddings of distributions. In Proceedings of the 33nd International Conference on Machine Learning, pp.  2052–2060, 2016.
  47. Recurrent generative adversarial network for learning imbalanced medical image semantic segmentation. Multimedia Tools and Applications, 79(21-22):15329–15348, 2020.
  48. In defense of one-vs-all classification. Journal of Machine Learning Research, 5:101–141, 2004.
  49. Scott, C. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pp.  838–846, 2015.
  50. Classification with asymmetric label noise: Consistency and maximal denoising. In Proceedings of the 26th Annual Conference on Learning Theory, pp.  489–511, 2013.
  51. Learning from complementary labels via partial-output consistency regularization. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, pp.  3075–3081, 2021.
  52. CLCIFAR: CIFAR-derived benchmark datasets with human annotated complementary labels. arXiv preprint arXiv:2305.08295, 2023a.
  53. Binary classification with confidence difference. In Advances in Neural Information Processing Systems 36, 2023b.
  54. Beyond myopia: Learning from positive and unlabeled data through holistic predictive trends. In Advances in Neural Information Processing Systems 36, 2023.
  55. Mitigating memorization of noisy labels by clipping the model prediction. In Proceedings of the 40th International Conference on Machine Learning, pp.  36868–36886, 2023a.
  56. Class-imbalanced complementary-label learning via weighted loss. Neural Networks, 166:555–565, 2023b.
  57. An embarrassingly simple approach to semi-supervised few-shot learning. In Advances in Neural Information Processing Systems 35, pp.  14489–14500, 2022.
  58. Leveraged weighted loss for partial label learning. In Proceedings of the 38th International Conference on Machine Learning, pp.  11091–11100, 2021.
  59. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
  60. Progressive purification for instance-dependent partial label learning. In Proceedings of the 40th International Conference on Machine Learning, pp.  38551–38565, 2023.
  61. Rethinking class-prior estimation for positive-unlabeled learning. In Proceedings of the 10th International Conference on Learning Representations, 2022.
  62. Learning with biased complementary labels. In Proceedings of the 15th European Conference on Computer Vision, pp.  68–83, 2018.
  63. Exploiting class activation value for partial-label learning. In Proceedings of the 10th International Conference on Learning Representations, 2022.
  64. Zhang, T. Statistical analysis of some multi-category large margin classification methods. Journal of Machine Learning Research, 5:1225–1251, 2004.
  65. Learning from a complementary-label source domain: Theory and algorithms. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7667–7681, 2021.
  66. An unbiased risk estimator for learning with augmented classes. In Advances in Neural Information Processing Systems 33, pp.  10247–10258, 2020.
  67. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems 31, pp.  8792–8802, 2018.
  68. Dist-PU: Positive-unlabeled learning from a label distribution perspective. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  14441–14450, 2022.
  69. Adversarial training with complementary labels: On the benefit of gradually informative attacks. In Advances in Neural Information Processing Systems 35, pp.  23621–23633, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Wei Wang (1793 papers)
  2. Takashi Ishida (11 papers)
  3. Yu-Jie Zhang (38 papers)
  4. Gang Niu (125 papers)
  5. Masashi Sugiyama (286 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now