Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trends (2310.04078v1)

Published 6 Oct 2023 in cs.LG and cs.AI

Abstract: Learning binary classifiers from positive and unlabeled data (PUL) is vital in many real-world applications, especially when verifying negative examples is difficult. Despite the impressive empirical performance of recent PUL methods, challenges like accumulated errors and increased estimation bias persist due to the absence of negative labels. In this paper, we unveil an intriguing yet long-overlooked observation in PUL: \textit{resampling the positive data in each training iteration to ensure a balanced distribution between positive and unlabeled examples results in strong early-stage performance. Furthermore, predictive trends for positive and negative classes display distinctly different patterns.} Specifically, the scores (output probability) of unlabeled negative examples consistently decrease, while those of unlabeled positive examples show largely chaotic trends. Instead of focusing on classification within individual time frames, we innovatively adopt a holistic approach, interpreting the scores of each example as a temporal point process (TPP). This reformulates the core problem of PUL as recognizing trends in these scores. We then propose a novel TPP-inspired measure for trend detection and prove its asymptotic unbiasedness in predicting changes. Notably, our method accomplishes PUL without requiring additional parameter tuning or prior assumptions, offering an alternative perspective for tackling this problem. Extensive experiments verify the superiority of our method, particularly in a highly imbalanced real-world setting, where it achieves improvements of up to $11.3\%$ in key metrics. The code is available at \href{https://github.com/wxr99/HolisticPU}{https://github.com/wxr99/HolisticPU}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. A closer look at memorization in deep networks. In International conference on machine learning, pages 233–242. PMLR, 2017.
  2. J. Bekker and J. Davis. Learning from positive and unlabeled data: A survey. Machine Learning, 109(4):719–760, 2020.
  3. O. Catoni. Challenging the empirical mean and empirical variance: a deviation study. In Annales de l’IHP Probabilités et statistiques, volume 48, pages 1148–1185, 2012.
  4. S. Chaudhari and S. Shevade. Learning from positive and unlabelled examples using maximum margin clustering. In International Conference on Neural Information Processing, pages 465–473. Springer, 2012.
  5. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357, 2002.
  6. A variational approach for learning from positive and unlabeled data. Advances in Neural Information Processing Systems, 33:14844–14854, 2020.
  7. A generalized catoni’s m-estimator under finite α𝛼\alphaitalic_α-th moment assumption with α∈𝛼absent\alpha\initalic_α ∈(1, 2). Electronic Journal of Statistics, 15(2):5523–5544, 2021.
  8. Self-pu: Self boosted and calibrated positive-unlabeled training. In International Conference on Machine Learning, pages 1510–1519. PMLR, 2020.
  9. Class-prior estimation for learning from positive and unlabeled data. In Asian Conference on Machine Learning, pages 221–236. PMLR, 2016.
  10. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011.
  11. Risk bounds for positive-unlabeled learning under the selected at random assumption. Journal of Machine Learning Research, 24(107):1–31, 2023.
  12. Learning from positive and unlabeled examples. Theoretical Computer Science, 348(1):70–83, 2005.
  13. Convex formulation for learning from positive and unlabeled data. In International conference on machine learning, pages 1386–1394. PMLR, 2015.
  14. C. Elkan and K. Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 213–220, 2008.
  15. Mixture proportion estimation and pu learning: A modern approach. Advances in Neural Information Processing Systems, 34:8532–8544, 2021.
  16. Large-margin label-calibrated support vector machines for positive and unlabeled learning. IEEE transactions on neural networks and learning systems, 30(11):3471–3483, 2019.
  17. Centroid estimation with guaranteed efficiency: A general framework for weakly supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6):2841–2855, 2020.
  18. On positive-unlabeled classification in gan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8385–8393, 2020.
  19. K. H. Hamed. Trend detection in hydrologic data: the mann–kendall trend test under the scaling hypothesis. Journal of hydrology, 349(3-4):350–363, 2008.
  20. A modified mann-kendall trend test for autocorrelated data. Journal of hydrology, 204(1-4):182–196, 1998.
  21. Z. Hammoudeh and D. Lowd. Learning from positive and unlabeled data with arbitrary positive shift. Advances in Neural Information Processing Systems, 33:13088–13099, 2020.
  22. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
  23. Instance-dependent pu learning by bayesian optimal relabeling, 2018.
  24. Generative adversarial positive-unlabelled learning, 2017.
  25. Pu learning for matrix completion. In International conference on machine learning, pages 2445–2453. PMLR, 2015.
  26. Classification from positive, unlabeled and biased negative data. In International Conference on Machine Learning, pages 2820–2829. PMLR, 2019.
  27. Predictive adversarial learning from positive and unlabeled data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 7806–7814, 2021.
  28. O2u-net: A simple noisy label detection approach for deep neural networks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3326–3334, 2019.
  29. L. Junnan and S. C. Hoi. Dividemix: Learning with noisy labelsas semi-supervised learning. In ICLR. International Conference on Learning Representations (ICLR), 2020.
  30. Learning from positive and unlabeled data with a selection bias. In International conference on learning representations, 2019.
  31. Positive-unlabeled learning with non-negative risk estimator. Advances in neural information processing systems, 30, 2017.
  32. Learning multiple layers of features from tiny images, 2009.
  33. Who is your right mixup partner in positive and unlabeled learning. In International Conference on Learning Representations, 2022.
  34. Leave zero out: Towards a no-cross-validation approach for model selection, 2020.
  35. Learning to identify unexpected instances in the test set. In IJCAI, volume 7, pages 2802–2807, 2007.
  36. X.-L. Li and B. Liu. Learning from positive and unlabeled examples with different data distributions. In European conference on machine learning, pages 218–229. Springer, 2005.
  37. Early-learning regularization prevents memorization of noisy labels. Advances in neural information processing systems, 33:20331–20342, 2020.
  38. Pulns: Positive-unlabeled learning with effective negative sample selector. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8784–8792, 2021.
  39. Early stopping without a validation set. arXiv preprint arXiv:1703.09580, 2017.
  40. Learning from corrupted binary labels via class-probability estimation. In International conference on machine learning, pages 125–134. PMLR, 2015.
  41. D. Moore. Uncertainty. on the shoulders of giants: new approaches to numeracy. la steen, 1990.
  42. Deep generative positive-unlabeled learning under selection bias. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1155–1164, 2020.
  43. Learning with confident examples: Rank pruning for robust classification with noisy labels. In Conference on Uncertainty in Artificial Intelligence, 2017.
  44. Mixture proportion estimation via kernel embeddings of distributions. In International conference on machine learning, pages 2052–2060. PMLR, 2016.
  45. Positive unlabeled learning for deceptive reviews detection. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 488–498, 2014.
  46. Positive and unlabeled learning via loss decomposition and centroid estimation. In IJCAI, pages 2689–2695, 2018.
  47. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 33:596–608, 2020.
  48. Positive-unlabeled learning from imbalanced data. In IJCAI, pages 2995–3001, 2021.
  49. A novel perspective for positive-unlabeled learning via noisy labels, 2021.
  50. V. Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
  51. Asymmetric loss for positive-unlabeled learning. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2021.
  52. Mixpul: Consistency-based augmentation for positive and unlabeled learning, 2020.
  53. Positive-unlabeled learning using random forests via recursive greedy risk minimization. In Advances in Neural Information Processing Systems, 2022.
  54. Sample selection with uncertainty of losses for learning with noisy labels. arXiv preprint arXiv:2106.00445, 2021.
  55. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
  56. Positive-unlabeled learning for disease gene identification. Bioinformatics, 28(20):2640–2647, 2012.
  57. Pebl: positive example based learning for web page classification using svm. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 239–248, 2002.
  58. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34, 2021.
  59. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107–115, 2021.
  60. Positive and unlabeled learning with label disambiguation. In IJCAI, pages 4250–4256, 2019.
  61. Dist-pu: Positive-unlabeled learning from a label distribution perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14461–14470, 2022.
  62. Learning with local and global consistency. Advances in neural information processing systems, 16, 2003.
  63. Step: Out-of-distribution detection in the presence of limited in-distribution labeled data. Advances in Neural Information Processing Systems, 34:29168–29180, 2021.
  64. Z.-H. Zhou. A brief introduction to weakly supervised learning. National science review, 5(1):44–53, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Xinrui Wang (21 papers)
  2. Wenhai Wan (4 papers)
  3. Chuanxin Geng (1 paper)
  4. Songcan Chen (74 papers)
  5. Shaoyuan Li (33 papers)
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now