Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Perturbation-Based Two-Stage Multi-Domain Active Learning (2306.10700v1)

Published 19 Jun 2023 in cs.LG

Abstract: In multi-domain learning (MDL) scenarios, high labeling effort is required due to the complexity of collecting data from various domains. Active Learning (AL) presents an encouraging solution to this issue by annotating a smaller number of highly informative instances, thereby reducing the labeling effort. Previous research has relied on conventional AL strategies for MDL scenarios, which underutilize the domain-shared information of each instance during the selection procedure. To mitigate this issue, we propose a novel perturbation-based two-stage multi-domain active learning (P2S-MDAL) method incorporated into the well-regarded ASP-MTL model. Specifically, P2S-MDAL involves allocating budgets for domains and establishing regions for diversity selection, which are further used to select the most cross-domain influential samples in each region. A perturbation metric has been introduced to evaluate the robustness of the shared feature extractor of the model, facilitating the identification of potentially cross-domain influential samples. Experiments are conducted on three real-world datasets, encompassing both texts and images. The superior performance over conventional AL strategies shows the effectiveness of the proposed strategy. Additionally, an ablation study has been carried out to demonstrate the validity of each component. Finally, we outline several intriguing potential directions for future MDAL research, thus catalyzing the field's advancement.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (22)
  1. Deep batch active learning by diverse, uncertain gradient lower bounds. In Proceedings of the 8th International Conference on Learning Representations. Addis Ababa, Ethiopia.
  2. Domain Separation Networks. In Proceedings of the Annual Conference on Neural Information Processing Systems 2016. Barcelona, Spain, 343–351.
  3. Marginalized denoising autoencoders for domain adaptation. (June 2012).
  4. Xilun Chen and Claire Cardie. 2018. Multinomial adversarial networks for multi-domain text classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, LA, 1226–1240.
  5. Imre Csiszár. 1975. I-divergence geometry of probability distributions and minimization problems. The annals of probability (1975), 146–158.
  6. Saliency Attack: Towards Imperceptible Black-Box Adversarial Attack. ACM Transactions on Intelligent Systems and Technology 14, 3, Article 45 (2023).
  7. Mark Dredze and Koby Crammer. 2008. Online methods for multi-domain learning and adaptation. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Honolulu, HI, 689–697.
  8. Melanie Ducoffe and Frédéric Precioso. 2018. Adversarial Active Learning for Deep Networks: a Margin Based Approach. arXiv preprint arXiv:1802.09841 (2018).
  9. Multi-Domain Active Learning: Literature Review and Comparative Study. IEEE Transactions on Emerging Topics in Computational Intelligence (2022).
  10. Multi-class active learning for image classification. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Miami, FL, 2372–2379.
  11. Deeper, Broader and Artier Domain Generalization. In Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy, 5543–5551.
  12. Multi-domain active learning for text classification. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Beijing, China, 1086–1094.
  13. Adversarial Multi-task Learning for Text Classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 1–10.
  14. Columbia object image library (coil-20). Technical Report CUCS-005-96 (1996).
  15. Ozan Sener and Silvio Savarese. 2018. Active Learning for Convolutional Neural Networks: A Core-Set Approach. In Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada.
  16. Burr Settles and Mark Craven. 2008. An analysis of active learning strategies for sequence labeling tasks. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. ACL, Honolulu, HI, 1070–1079.
  17. Investigating Multi-source Active Learning for Natural Language Inference. arXiv preprint arXiv:2302.06976 (2023).
  18. Multi-domain Active Learning for Semi-supervised Anomaly Detection. ECML/PKDD 2022 published proceedings (2022).
  19. Conditional Adversarial Networks for Multi-Domain Text Classification. arXiv preprint arXiv:2102.10176 (2021).
  20. Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks. In 25th Annual Network and Distributed System Security Symposium, Vol. 325. San Diego, USA, 2116–2123.
  21. A Comparative Survey: Benchmarking for Pool-based Active Learning. In Proceedings of the 30th International Joint Conference on Artificial Intelligenc. Virtual Event / Montreal, Canada.
  22. Active discriminative text representation learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, CA, 3386–3392.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now