Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Transfer and Alignment Network for Generalized Category Discovery (2312.16467v1)

Published 27 Dec 2023 in cs.CL and cs.LG

Abstract: Generalized Category Discovery is a crucial real-world task. Despite the improved performance on known categories, current methods perform poorly on novel categories. We attribute the poor performance to two reasons: biased knowledge transfer between labeled and unlabeled data and noisy representation learning on the unlabeled data. To mitigate these two issues, we propose a Transfer and Alignment Network (TAN), which incorporates two knowledge transfer mechanisms to calibrate the biased knowledge and two feature alignment mechanisms to learn discriminative features. Specifically, we model different categories with prototypes and transfer the prototypes in labeled data to correct model bias towards known categories. On the one hand, we pull instances with known categories in unlabeled data closer to these prototypes to form more compact clusters and avoid boundary overlap between known and novel categories. On the other hand, we use these prototypes to calibrate noisy prototypes estimated from unlabeled data based on category similarities, which allows for more accurate estimation of prototypes for novel categories that can be used as reliable learning targets later. After knowledge transfer, we further propose two feature alignment mechanisms to acquire both instance- and category-level knowledge from unlabeled data by aligning instance features with both augmented features and the calibrated prototypes, which can boost model performance on both known and novel categories with less noise. Experiments on three benchmark datasets show that our model outperforms SOTA methods, especially on novel categories. Theoretical analysis is provided for an in-depth understanding of our model in general. Our code and data are available at https://github.com/Lackel/TAN.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 1314–1323.
  2. New User Intent Discovery with Robust Pseudo Label Training and Source Domain Joint-training. IEEE Intelligent Systems.
  3. DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discovery. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 12292–12302.
  4. Generalized category discovery with decoupled prototypical network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 12527–12535.
  5. A theory of learning from different domains. Machine learning, 79: 151–175.
  6. Open-world semi-supervised learning. arXiv preprint arXiv:2102.03526.
  7. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), 132–149.
  8. Efficient intent detection with dual sentence encoders. arXiv preprint arXiv:2003.04807.
  9. Transfer Knowledge from Head to Tail: Uncertainty Calibration under Long-tailed Distribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19978–19987.
  10. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597–1607. PMLR.
  11. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  12. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821.
  13. Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. arXiv preprint arXiv:2001.01526.
  14. Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern recognition, 10(2): 105–112.
  15. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 8401–8409.
  16. Open-world Semi-supervised Generalized Relation Discovery Aligned in a Real-world Setting. arXiv preprint arXiv:2305.13533.
  17. Kuhn, H. W. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly, 2: 83–97.
  18. An evaluation dataset for intent classification and out-of-scope prediction. arXiv preprint arXiv:1909.02027.
  19. Prototypical contrastive learning of unsupervised representations. arXiv preprint arXiv:2005.04966.
  20. Discovering new intents via constrained deep adaptive clustering with cluster refinement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 8360–8367.
  21. Coarse-to-fine few-shot learning for named entity recognition. In Findings of the Association for Computational Linguistics: ACL 2023, 4115–4129.
  22. MacQueen, J.; et al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, 281–297. Oakland, CA, USA.
  23. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  24. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2239–2247.
  25. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 1532–1543.
  26. Ovanet: One-vs-all network for universal domain adaptation. In Proceedings of the ieee/cvf international conference on computer vision, 9000–9009.
  27. A Diffusion Weighted Graph Framework for New Intent Discovery. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 8033–8042.
  28. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30.
  29. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7492–7501.
  30. A Simple Parametric Classification Baseline for Generalized Category Discovery. arXiv preprint arXiv:2211.11727.
  31. HuggingFace’s Transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  32. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, 478–487. PMLR.
  33. Short text clustering via convolutional neural networks. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, 62–69.
  34. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In international conference on machine learning, 3861–3870. PMLR.
  35. Self-Labeling Framework for Novel Category Discovery over Domains. In Proceedings of the AAAI Conference on Artificial Intelligence.
  36. Discovering New Intents with Deep Aligned Clustering. In Proceedings of the AAAI Conference on Artificial Intelligence.
  37. Few-shot intent detection via contrastive pre-training and fine-tuning. arXiv preprint arXiv:2109.06349.
  38. New intent discovery with pre-training and contrastive learning. arXiv preprint arXiv:2205.12914.
  39. Neighborhood contrastive learning for novel class discovery. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10867–10875.
  40. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1): 43–76.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - Lackel/TAN: AAAI 2024 paper "Transfer and Alignment Network for Generalized Category Discovery" (7 stars)