Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Simple Semantic-Aided Few-Shot Learning (2311.18649v3)

Published 30 Nov 2023 in cs.CV

Abstract: Learning from a limited amount of data, namely Few-Shot Learning, stands out as a challenging computer vision task. Several works exploit semantics and design complicated semantic fusion mechanisms to compensate for rare representative features within restricted data. However, relying on naive semantics such as class names introduces biases due to their brevity, while acquiring extensive semantics from external knowledge takes a huge time and effort. This limitation severely constrains the potential of semantics in Few-Shot Learning. In this paper, we design an automatic way called Semantic Evolution to generate high-quality semantics. The incorporation of high-quality semantics alleviates the need for complex network structures and learning algorithms used in previous works. Hence, we employ a simple two-layer network termed Semantic Alignment Network to transform semantics and visual features into robust class prototypes with rich discriminative features for few-shot classification. The experimental results show our framework outperforms all previous methods on six benchmarks, demonstrating a simple network with high-quality semantics can beat intricate multi-modal modules on few-shot classification tasks. Code is available at https://github.com/zhangdoudou123/SemFew.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Learning to learn by gradient descent by gradient descent. In NeurIPS, 2016.
  2. Improved few-shot visual classification. In CVPR, pages 14493–14502, 2020.
  3. Irving Biederman. Recognition-by-components: a theory of human image understanding. Psychol Rev, 94(2):115, 1987.
  4. Language models are few-shot learners. NeurIPS, 33:1877–1901, 2020.
  5. Semantic prompt for few-shot image recognition. In CVPR, pages 23581–23591, 2023.
  6. Meta-baseline: Exploring simple meta-learning for few-shot learning. In ICCV, pages 9062–9071, 2021.
  7. Frequency guidance matters in few-shot learning. In ICCV, pages 11814–11824, 2023.
  8. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. Ieee, 2009.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  10. Crosstransformers: spatially-aware few-shot transfer. In NeurIPS, pages 21981–21993, 2020.
  11. Self-promoted supervision for few-shot transformer. In ECCV, pages 329–347. Springer, 2022.
  12. Meta-learning of neural architectures for few-shot learning. In CVPR, pages 12365–12375, 2020.
  13. Model-agnostic meta-learning for fast adaptation of deep networks. In Int. Conf. Mach. Learn., pages 1126–1135. PMLR, 2017.
  14. Styleadv: Meta style adversarial training for cross-domain few-shot learning. In CVPR, pages 24575–24584, 2023.
  15. Boosting few-shot visual learning with self-supervision. In ICCV, pages 8059–8068, 2019.
  16. Class-aware patch embedding adaptation for few-shot image classification. In ICCV, pages 18905–18915, 2023.
  17. Algorithm as 136: A k-means clustering algorithm. J R Stat Soc Ser C Appl Stat, 28(1):100–108, 1979.
  18. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  19. Rethinking generalization in few-shot classification. NeurIPS, 35:3582–3595, 2022.
  20. Cross attention network for few-shot classification. In NeurIPS, 2019.
  21. Adversarial feature augmentation for cross-domain few-shot classification. In ECCV, pages 20–37. Springer, 2022.
  22. Task agnostic meta-learning for few-shot learning. In CVPR, pages 11719–11727, 2019.
  23. Model-agnostic boundary-adversarial sampling for test-time generalization in few-shot learning. In ECCV, pages 599–617. Springer, 2020.
  24. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  25. Siamese neural networks for one-shot image recognition. In ICML Deep Learn. workshop. Lille, 2015.
  26. Learning multiple layers of features from tiny images. 2009.
  27. Meta-learning with differentiable convex optimization. In CVPR, pages 10657–10665, 2019.
  28. Boosting few-shot learning with adaptive margin loss. In CVPR, pages 12576–12584, 2020.
  29. Finding task-relevant features for few-shot learning by category traversal. In CVPR, pages 1–10, 2019.
  30. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, 2021.
  31. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, page 3. Atlanta, Georgia, USA, 2013.
  32. Charting the right manifold: Manifold mixup for few-shot learning. In IEEE Win. Conf. Appl. Comput. Vis., pages 2218–2227, 2020.
  33. George A Miller. Wordnet: a lexical database for english. Commun. ACM, 38(11):39–41, 1995.
  34. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
  35. A simple neural attentive meta-learner. arXiv preprint arXiv:1707.03141, 2017.
  36. OpenAI. Chatgpt. https://chat.openai.com, 2023.
  37. Tadam: Task dependent adaptive metric for improved few-shot learning. In NeurIPS, 2018.
  38. Few-shot image recognition with knowledge transfer. In ICCV, pages 441–449, 2019.
  39. Learning transferable visual models from natural language supervision. In Int. Conf. Mach. Learn., pages 8748–8763. PMLR, 2021.
  40. Optimization as a model for few-shot learning. In ICLR, 2017.
  41. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676, 2018.
  42. Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960, 2018.
  43. Few-shot learning with graph neural networks. In ICLR, 2018.
  44. Generalized zero-and few-shot learning via aligned variational autoencoders. In CVPR, pages 8247–8255, 2019.
  45. Hongyang Gao Siyuan Sun. Meta-adam: A meta-learned adaptive optimizer with momentum for few-shot learning. In NeurIPS, 2023.
  46. Prototypical networks for few-shot learning. In NeurIPS, 2017.
  47. Learning structured output representation using deep conditional generative models. In NeurIPS, 2015.
  48. Learning to compare: Relation network for few-shot learning. In CVPR, pages 1199–1208, 2018.
  49. Rethinking few-shot image classification: a good embedding is all you need? In ECCV, pages 266–282. Springer, 2020.
  50. Learning compositional representations for few-shot recognition. In ICCV, pages 6372–6381, 2019.
  51. Cross-domain few-shot classification via learned feature-wise transformation. In ICLR, 2020.
  52. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. J Mach Learn Res, 9(11), 2008.
  53. Matching networks for one shot learning. In NeurIPS, 2016.
  54. The caltech-ucsd birds-200-2011 dataset. 2011.
  55. Cross-domain few-shot classification via adversarial task augmentation. In IJCAI, pages 1075–1081. International Joint Conferences on Artificial Intelligence Organization, 2021.
  56. Tafe-net: Task-aware feature embeddings for low shot learning. In CVPR, pages 1831–1840, 2019.
  57. Adaptive cross-modal few-shot learning. In NeurIPS, 2019.
  58. Generating representative samples for few-shot classification. In CVPR, pages 9003–9013, 2022.
  59. Aligning visual prototypes with bert embeddings for few-shot learning. In ICMR, pages 367–375, 2021.
  60. Few-shot learning via embedding adaptation with set-to-set functions. In CVPR, pages 8808–8817, 2020.
  61. Prototype completion with primitive knowledge for few-shot learning. In CVPR, pages 3754–3762, 2021.
  62. Deepemd: Differentiable earth mover’s distance for few-shot learning. IEEE TPAMI, 2022.
  63. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452–1464, 2017.
  64. Revisiting prototypical network for cross domain few-shot learning. In CVPR, pages 20061–20070, 2023.
  65. Fast context adaptation via meta-learning. In Int. Conf. Mach. Learn., pages 7693–7702. PMLR, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Hai Zhang (69 papers)
  2. Junzhe Xu (2 papers)
  3. Shanlin Jiang (1 paper)
  4. Zhenan He (5 papers)
Citations (3)
View on Semantic Scholar
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - zhangdoudou123/SemFew: [CVPR2024] Simple Semantic-Aided Few-Shot Learning (32 stars)