Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Margin Discrepancy-based Adversarial Training for Multi-Domain Text Classification (2403.00888v1)

Published 1 Mar 2024 in cs.CL and cs.LG

Abstract: Multi-domain text classification (MDTC) endeavors to harness available resources from correlated domains to enhance the classification accuracy of the target domain. Presently, most MDTC approaches that embrace adversarial training and the shared-private paradigm exhibit cutting-edge performance. Unfortunately, these methods face a non-negligible challenge: the absence of theoretical guarantees in the design of MDTC algorithms. The dearth of theoretical underpinning poses a substantial impediment to the advancement of MDTC algorithms. To tackle this problem, we first provide a theoretical analysis of MDTC by decomposing the MDTC task into multiple domain adaptation tasks. We incorporate the margin discrepancy as the measure of domain divergence and establish a new generalization bound based on Rademacher complexity. Subsequently, we propose a margin discrepancy-based adversarial training (MDAT) approach for MDTC, in accordance with our theoretical analysis. To validate the efficacy of the proposed MDAT method, we conduct empirical studies on two MDTC benchmarks. The experimental results demonstrate that our MDAT approach surpasses state-of-the-art baselines on both datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Dual adversarial co-learning for multi-domain text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6438–6445, 2020.
  2. Multi-domain sentiment classification. In Proceedings of ACL-08: HLT, Short Papers, pages 257–260, 2008.
  3. A survey on data augmentation in large model era. arXiv preprint arXiv:2401.15422, 2024.
  4. Adversarial multi-task learning for text classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1–10, 2017.
  5. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
  6. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167, 2008.
  7. Collaborative multi-domain sentiment classification. In 2015 IEEE International Conference on Data Mining, pages 459–468. IEEE, 2015.
  8. Multinomial adversarial networks for multi-domain text classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1226–1240, 2018.
  9. Generative adversarial nets. Advances in neural information processing systems, 27:2672–2680, 2014.
  10. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1):2096–2030, 2016.
  11. Domain separation networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 343–351, 2016.
  12. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning, pages 7404–7413. PMLR, 2019.
  13. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
  14. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2272–2281, 2017.
  15. Domain adaptation on the statistical manifold. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2481–2488, 2014.
  16. Unsupervised domain adaptation with residual transfer networks. Advances in neural information processing systems, 29, 2016.
  17. A brief review of domain adaptation. Advances in data science and information engineering, pages 877–894, 2021.
  18. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3723–3732, 2018.
  19. Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 912–921, 2015.
  20. Foundations of machine learning. MIT press, 2018.
  21. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19:137, 2007.
  22. A theory of learning from different domains. Machine learning, 79(1-2):151–175, 2010.
  23. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430, 2009.
  24. Yi Lin. A note on margin-based loss functions in classification. Statistics & probability letters, 68(1):73–82, 2004.
  25. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1):1–50, 2002.
  26. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 440–447, 2007.
  27. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  28. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
  29. Vector analysis versus vector calculus. Springer Science & Business Media, 2012.
  30. Michel Talagrand. Upper and lower bounds for stochastic processes: modern methods and classical problems, volume 60. Springer Science & Business Media, 2014.
  31. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pages 55–60, 2014.
  32. Marginalized denoising autoencoders for domain adaptation. In Proceedings of the 29th International Coference on International Conference on Machine Learning, pages 1627–1634, 2012.
  33. Multiple source domain adaptation with adversarial training of neural networks. arXiv preprint arXiv:1705.09684, 2017.
  34. Maximum batch frobenius norm for multi-domain text classification. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3763–3767. IEEE, 2022.
  35. Co-regularized adversarial learning for multi-domain text classification. In International Conference on Artificial Intelligence and Statistics, pages 6690–6701. PMLR, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now