2000 character limit reached
When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications (2005.11700v2)
Published 24 May 2020 in cs.CL
Abstract: Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method, is successfully employed in NLP applications including few-shot text classification and multi-domain low-resource language generation. Many impacting factors, including data quantity, similarity among tasks, and the balance between general LLM and task-specific adaptation, can affect the performance of MAML in NLP, but few works have thoroughly studied them. In this paper, we conduct an empirical study to investigate these impacting factors and conclude when MAML works the best based on the experimental results.
- 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- 2016. Learning to learn by gradient descent by gradient descent. In NIPS, pages 3981–3989.
- 2018. Metareg: Towards domain generalization using meta-regularization. Advances in neural information processing systems, 31.
- 2020. Few-shot text classification with distributional signatures. ICLR.
- 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXiv:2104.08758.
- 2017. Model-agnostic meta-learning for fast adaptation of deep networks. ICML, pages 1126–1135.
- 2018. Transfer learning for style-specific text generation. In Proceedings of 32nd Conference on Neural Information Processing Systems, NIPS, volume 2018.
- 2018. Meta-learning for low-resource neural machine translation. EMNLP.
- 2018. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In EMNLP, pages 4803–4809.
- 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In ICIWWW, pages 507–517.
- 2022. Unsupervised neural stylistic text generation using transfer learning and adapters. arXiv preprint arXiv:2210.03264.
- 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
- 2019. Personalizing dialogue agents via meta-learning. In ACL, pages 5454–5459.
- 2019. Meta-learning for low-resource natural language generation in task-oriented dialogue systems. In IJCAI, pages 3151–3157.
- 2019. Model-agnostic meta-learning for relation classification with limited supervision. In ACL, pages 5873–5879.
- OpenAI. 2022. Chatgpt: Optimizing language models for dialogue. Technical blog.
- 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, pages 311–318.
- 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543.
- 2019. Domain adaptive dialog generation via meta learning. In ACL, pages 2639–2649.
- 2017. Optimization as a model for few-shot learning.
- 2010. Software framework for topic modelling with large corpora. In LREC. Citeseer.
- 2016. Meta-learning with memory-augmented neural networks. In ICML, pages 1842–1850.
- 2021. Few-shot text generation with natural language instructions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 390–402.
- 2015. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742.
- 2020. Learning to customize model structures for few-shot dialogue generation tasks. In ACL, pages 5832–5841.
- 2017. Attention is all you need. In NIPS, pages 5998–6008.
- 2002. A perspective view and survey of meta-learning. Artificial intelligence review, 18:77–95.
- 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In ACL, pages 2204–2213.
- 2022. Improving meta-learning for low-resource text classification and generation via memory imitation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 583–595.
- Zequn Liu (14 papers)
- Ruiyi Zhang (98 papers)
- Yiping Song (14 papers)
- Wei Ju (46 papers)
- Ming Zhang (313 papers)