Prompt-Driven Feature Diffusion for Open-World Semi-Supervised Learning (2404.11795v1)
Abstract: In this paper, we present a novel approach termed Prompt-Driven Feature Diffusion (PDFD) within a semi-supervised learning framework for Open World Semi-Supervised Learning (OW-SSL). At its core, PDFD deploys an efficient feature-level diffusion model with the guidance of class-specific prompts to support discriminative feature representation learning and feature generation, tackling the challenge of the non-availability of labeled data for unseen classes in OW-SSL. In particular, PDFD utilizes class prototypes as prompts in the diffusion model, leveraging their class-discriminative and semantic generalization ability to condition and guide the diffusion process across all the seen and unseen classes. Furthermore, PDFD incorporates a class-conditional adversarial loss for diffusion model training, ensuring that the features generated via the diffusion process can be discriminatively aligned with the class-conditional features of the real data. Additionally, the class prototypes of the unseen classes are computed using only unlabeled instances with confident predictions within a semi-supervised learning framework. We conduct extensive experiments to evaluate the proposed PDFD. The empirical results show PDFD exhibits remarkable performance enhancements over many state-of-the-art existing methods.
- Towards open world recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2015.
- Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
- ReMixMatch: Semi-supervised learning with distribution matching and augmentation anchoring. In International Conference on Learning Representations (ICLR), 2020.
- Open-world semi-supervised learning. In International Conference on Learning Representations (ICLR), 2022.
- A simple framework for contrastive learning of visual representations. In International conference on machine learning (ICML). PMLR, 2020.
- Text-to-image diffusion models are zero-shot classifiers. In ICLR 2023 Workshop on Multimodal Representation Learning: Perks and Pitfalls, 2023.
- Imagenet: A large-scale hierarchical image database. In conference on computer vision and pattern recognition (CVPR), 2009.
- Protodiff: Learning to learn prototypical networks by task-guided diffusion. arXiv preprint arXiv:2306.14770, 2023.
- A unified objective for novel class discovery. In International Conference on Computer Vision (ICCV), 2021.
- Generative adversarial networks. Advances in Neural Information Processing Systems (NeurIPS), 2014.
- Semi-supervised learning by entropy minimization. Advances in neural information processing systems (NeurIPS), 2004.
- Safe deep semi-supervised learning for unseen-class unlabeled data. In International Conference on Machine Learning (ICML), 2020.
- Robust semi-supervised learning when not all classes have labels. Advances in Neural Information Processing Systems (NeurIPS), 2022.
- Automatically discovering and learning new visual categories with ranking statistics. In International Conference on Learning Representations (ICLR), 2019.
- Learning to discover novel visual categories via deep transfer clustering. In International Conference on Computer Vision (ICCV), 2019.
- Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), 2020.
- Diffusion-ss3d: Diffusion model for semi-supervised 3d object detection. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023.
- Learning to cluster in order to transfer across domains and tasks. In International Conference on Learning Representations (ICLR), 2018.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- Learning multiple layers of features from tiny images. Technical report, 2009.
- Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 1955.
- Temporal ensembling for semi-supervised learning. In International Conference on Learning Representations (ICLR), 2017.
- Your diffusion model is secretly a zero-shot classifier. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
- Calvin Luo. Understanding diffusion models: A unified perspective. arXiv preprint arXiv:2208.11970, 2022.
- Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
- Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.
- High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- Imagenet large scale visual recognition challenge. International journal of computer vision, 2015.
- Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), 2015.
- Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
- Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021.
- Conditional gaussian distribution learning for open set recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
- Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 2023.
- Diffusion models and semi-supervised learners benefit mutually with few labels. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023.
- Xiaojin Jerry Zhu. Semi-supervised learning literature survey. 2005.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.