Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 100 tok/s
Gemini 2.5 Pro 58 tok/s Pro
GPT-5 Medium 29 tok/s
GPT-5 High 29 tok/s Pro
GPT-4o 103 tok/s
GPT OSS 120B 480 tok/s Pro
Kimi K2 215 tok/s Pro
2000 character limit reached

TACIT: A Target-Agnostic Feature Disentanglement Framework for Cross-Domain Text Classification (2312.17263v1)

Published 25 Dec 2023 in cs.CL

Abstract: Cross-domain text classification aims to transfer models from label-rich source domains to label-poor target domains, giving it a wide range of practical applications. Many approaches promote cross-domain generalization by capturing domain-invariant features. However, these methods rely on unlabeled samples provided by the target domains, which renders the model ineffective when the target domain is agnostic. Furthermore, the models are easily disturbed by shortcut learning in the source domain, which also hinders the improvement of domain generalization ability. To solve the aforementioned issues, this paper proposes TACIT, a target domain agnostic feature disentanglement framework which adaptively decouples robust and unrobust features by Variational Auto-Encoders. Additionally, to encourage the separation of unrobust features from robust features, we design a feature distillation task that compels unrobust features to approximate the output of the teacher. The teacher model is trained with a few easy samples that are easy to carry potential unknown shortcuts. Experimental results verify that our framework achieves comparable results to state-of-the-art baselines while utilizing only source domain data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Generating Sentences from Disentangled Syntactic and Semantic Spaces. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, 6008–6019.
  2. PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models. Trans. Assoc. Comput. Linguistics, 8: 504–521.
  3. Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics.
  4. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2172–2180.
  5. Learning Disentangled Textual Representations via Statistical Measures of Similarity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, 2614–2630.
  6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, 4171–4186.
  7. Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, 4019–4028.
  8. Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, 915–929.
  9. A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification. CoRR, abs/2304.09820.
  10. Fotopoulos, S. B. 2007. All of Nonparametric Statistics. Technometrics, 49(1): 103.
  11. Domain-Adversarial Training of Neural Networks. J. Mach. Learn. Res., 17: 59:1–59:35.
  12. Shortcut learning in deep neural networks. Nat. Mach. Intell., 2(11): 665–673.
  13. Deberta: decoding-Enhanced Bert with Disentangled Attention. In 9th International Conference on Learning Representations, ICLR 2021.
  14. Continual Learning for Text Classification with Information Disentanglement Based Regularization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, 2736–2746.
  15. Disentangled Representation Learning for Non-Parallel Text Style Transfer. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, 424–434.
  16. UDALM: Unsupervised Domain Adaptation through Language Modeling. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, 2579–2590.
  17. On the characterisation of the normal population by the independence of the sample mean and the sample variance. Journal of the Mathematical Society of Japan, 1(2): 111–115.
  18. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014.
  19. Why Machine Reading Comprehension Models Learn Shortcuts? In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, volume ACL/IJCNLP 2021 of Findings of ACL, 989–1002.
  20. Cross-Domain Sentiment Classification using Semantic Representation. In Findings of the Association for Computational Linguistics: EMNLP 2022, 289–299.
  21. Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), 5852–5859.
  22. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs/1907.11692.
  23. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019.
  24. Mere Contrastive Learning for Cross-Domain Sentiment Analysis. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, 7099–7111.
  25. MASKER: Masked Keyword Regularization for Reliable Text Classification. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, 13578–13586.
  26. Cross-Domain Sentiment Classification with Target Domain Specific Information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, 2505–2513.
  27. A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, 2870–2883.
  28. Adversarial Category Alignment Network for Cross-domain Sentiment Classification. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, 2496–2508.
  29. Neural Unsupervised Domain Adaptation in NLP - A Survey. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, 6838–6855.
  30. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108.
  31. Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, 8717–8729.
  32. Towards Debiasing NLU Models from Unknown Biases. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, 7597–7610.
  33. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86): 2579–2605.
  34. Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation. CoRR, abs/2205.14141.
  35. Adversarial Soft Prompt Tuning for Cross-Domain Sentiment Analysis. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL, May 22-27, 2022, 2438–2447.
  36. Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, 7386–7399.
  37. Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, 236–246.
  38. OPT: Open Pre-trained Transformer Language Models. CoRR, abs/2205.01068.
  39. SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, 568–579.
  40. Pivot Based Language Modeling for Improved Neural Domain Adaptation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, 1241–1251.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now