Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Evolving Semantic Prototype Improves Generative Zero-Shot Learning (2306.06931v1)

Published 12 Jun 2023 in cs.LG and cs.CV

Abstract: In zero-shot learning (ZSL), generative methods synthesize class-related sample features based on predefined semantic prototypes. They advance the ZSL performance by synthesizing unseen class sample features for better training the classifier. We observe that each class's predefined semantic prototype (also referred to as semantic embedding or condition) does not accurately match its real semantic prototype. So the synthesized visual sample features do not faithfully represent the real sample features, limiting the classifier training and existing ZSL performance. In this paper, we formulate this mismatch phenomenon as the visual-semantic domain shift problem. We propose a dynamic semantic prototype evolving (DSP) method to align the empirically predefined semantic prototypes and the real prototypes for class-related feature synthesis. The alignment is learned by refining sample features and semantic prototypes in a unified framework and making the synthesized visual sample features approach real sample features. After alignment, synthesized sample features from unseen classes are closer to the real sample features and benefit DSP to improve existing generative ZSL methods by 8.5\%, 8.0\%, and 9.7\% on the standard CUB, SUN AWA2 datasets, the significant performance improvement indicates that evolving semantic prototype explores a virgin field in ZSL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Evaluation of output embeddings for fine-grained image classification. In CVPR, pp.  2927–2936, 2015.
  2. Label-embedding for image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38:1425–1438, 2016.
  3. Generalized zero-shot learning via synthesized examples. In CVPR, pp.  4281–4289, 2018.
  4. Free: Feature refinement for generalized zero-shot learning. In ICCV, 2021a.
  5. Hsva: Hierarchical semantic-visual adaptation for zero-shot learning. In NeurIPS, 2021b.
  6. Transzero: Attribute-guided transformer for zero-shot learning. In AAAI, 2022a.
  7. Gndan: Graph navigated dual attention network for zero-shot learning. IEEE transactions on neural networks and learning systems, 2022b.
  8. Msdn: Mutually semantic distillation network for zero-shot learning. In CVPR, 2022c.
  9. Transzero++: Cross attribute-guided transformer for zero-shot learning. IEEE transactions on pattern analysis and machine intelligence, 2022d.
  10. Adaptive and generative zero-shot learning. In ICLR, 2021.
  11. Multi-modal cycle-consistent generalized zero-shot learning. In ECCV, 2018.
  12. Transductive multi-view zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37:2332–2345, 2015.
  13. Generative adversarial nets. In NeurIPS, 2014.
  14. Deep residual learning for image recognition. In CVPR, pp.  770–778, 2016.
  15. Semantic compression embedding for generative zero-shot learning. In IJCAI, 2022.
  16. Fine-grained generalized zero-shot learning via dense attribute-based attention. In CVPR, pp.  4482–4492, 2020a.
  17. Compositional zero-shot learning via fine-grained dense feature composition. In NeurIPS, 2020b.
  18. Deep unbiased embedding transfer for zero-shot learning. IEEE Transactions on Image Processing, 29:1958–1971, 2020.
  19. Auto-encoding variational bayes. In ICLR, 2014.
  20. Kong, X. En-compactness: Self-distillation embedding & contrastive generation for generalized zero-shot learning.
  21. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579–2605, 2008.
  22. I2dformer: Learning image to document attention for zero-shot image classification. In NeurIPS, 2022.
  23. Latent embedding feedback and discriminative features for zero-shot classification. In ECCV, 2020.
  24. Automated flower classification over a large number of classes. In ICVGIP, pp.  722–729, 2008.
  25. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR, pp.  2751–2758, 2012.
  26. A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence, 2022.
  27. Generalized zero- and few-shot learning via aligned variational autoencoders. In CVPR, pp.  8239–8247, 2019.
  28. Invertible zero-shot recognition flows. In ECCV, 2020.
  29. Class normalization for (continual)? generalized zero-shot learning. In ICLR, 2021.
  30. Leveraging seen and unseen semantic relationships for generative zero-shot learning. In ECCV, 2020.
  31. Transductive zero-shot learning with visual structure constraint. In NeurIPS, 2019.
  32. Dual progressive prototype network for generalized zero-shot learning. In NeurIPS, 2021.
  33. Caltech-ucsd birds 200. Technical Report CNS-TR-2010-001, Caltech,, 2010.
  34. Feature generating networks for zero-shot learning. In CVPR, pp.  5542–5551, 2018.
  35. Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:2251–2265, 2019a.
  36. F-vaegan-d2: A feature generating framework for any-shot learning. In CVPR, pp.  10267–10276, 2019b.
  37. Attentive region embedding network for zero-shot learning. In CVPR, pp.  9376–9385, 2019.
  38. Attribute prototype network for zero-shot learning. In NeurIPS, 2020.
  39. Matrix tri-factorization with manifold regularizations for zero-shot learning. In CVPR, pp.  2007–2016, 2017.
  40. Zeronas: Differentiable generative adversarial networks search for zero-shot learning. IEEE transactions on pattern analysis and machine intelligence, 2021.
  41. Stacked semantics-guided attention model for fine-grained zero-shot learning. In NeurIPS, 2018.
  42. Counterfactual zero-shot and open-set visual recognition. In CVPR, 2021.
  43. Co-representation network for generalized zero-shot learning. In ICML, 2019.
  44. Triple verification network for generalized zero-shot learning. IEEE Transactions on Image Processing, 28:506–517, 2019.
  45. Semantic-guided multi-attention localization for zero-shot learning. In NeurIPS, 2019.
  46. Closed-form sample probing for learning generative models in zero-shot learning. In ICLR, 2022.
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now