Cordyceps@LT-EDI: Patching Language-Specific Homophobia/Transphobia Classifiers with a Multilingual Understanding (2309.13561v1)
Abstract: Detecting transphobia, homophobia, and various other forms of hate speech is difficult. Signals can vary depending on factors such as language, culture, geographical region, and the particular online platform. Here, we present a joint multilingual (M-L) and language-specific (L-S) approach to homophobia and transphobic hate speech detection (HSD). M-L models are needed to catch words, phrases, and concepts that are less common or missing in a particular language and subsequently overlooked by L-S models. Nonetheless, L-S models are better situated to understand the cultural and linguistic context of the users who typically write in a particular language. Here we construct a simple and successful way to merge the M-L and L-S approaches through simple weight interpolation in such a way that is interpretable and data-driven. We demonstrate our system on task A of the 'Shared Task on Homophobia/Transphobia Detection in social media comments' dataset for homophobia and transphobic HSD. Our system achieves the best results in three of five languages and achieves a 0.997 macro average F1-score on Malayalam texts.
- Deep learning models for multilingual hate speech detection. ArXiv, abs/2004.06465.
- Code-switching patterns can be an effective route to improve performance of downstream nlp applications: A case study of humour, sarcasm and hate speech detection. In Annual Meeting of the Association for Computational Linguistics.
- Xlm-t: Multilingual language models in twitter for sentiment analysis and beyond. In International Conference on Language Resources and Evaluation.
- Social media use and health and well-being of lesbian, gay, bisexual, transgender, and queer youth: Systematic review. Journal of Medical Internet Research, 24.
- Universal dependency parsing for hindi-english code-switching. In North American Chapter of the Association for Computational Linguistics.
- Hate or non-hate: Translation based hate speech identification in code-mixed hinglish data set. 2021 IEEE International Conference on Big Data (Big Data), pages 2470–2475.
- How can we detect homophobia and transphobia? experiments in a multilingual code-mixed setting for social media governance. International Journal of Information Management Data Insights, 2(2):100119.
- Fusing finetuned models for better pretraining. ArXiv, abs/2204.03044.
- Curriculum design for code-switching: Experiments with language identification and language modeling with deep neural networks. In ICON.
- Bernice: A multilingual pre-trained encoder for Twitter. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6191–6205, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cold fusion: Collaborative descent for distributed multitask finetuning. ArXiv, abs/2212.01378.
- Leveraging transformers for hate speech detection in conversational code-mixed tweets. In Fire.
- Knowledge is a region in weight space for fine-tuned language models. ArXiv, abs/2302.04863.
- Robert Henderson and Eric McCready. 2017. How dogwhistles work. In JSAI-isAI Workshops.
- Muhammad Okky Ibrohim and Indra Budi. 2019. Translated vs non-translated method for multilingual hate speech identification in twitter. International Journal on Advanced Science, Engineering and Information Technology.
- Patching open-vocabulary models by interpolating weights. ArXiv, abs/2208.05592.
- Md Saroar Jahan and Mourad Oussalah. 2021. A systematic review of hate speech automatic detection using natural language processing. ArXiv, abs/2106.00742.
- Mapping transgender policies in the us 2017-2021: The role of geography and implications for health equity. Health & place, 80:102985.
- Ai4bharat-indicnlp corpus: Monolingual corpora and word embeddings for indic languages. ArXiv, abs/2005.00085.
- Detecting the hate code on social media. In International Conference on Web and Social Media.
- From dogwhistles to bullhorns: Unveiling coded rhetoric with language models. ArXiv, abs/2305.17174.
- What the [mask]? making sense of language-specific bert models. ArXiv, abs/2003.02912.
- Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language detection: A hybrid approach with deep learning and a multilingual lexicon. In Annual Meeting of the Association for Computational Linguistics.
- Investigating cross-lingual training for offensive language detection. PeerJ Computer Science, 7.
- Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55:477 – 523.
- Structural transphobia is associated with psychological distress and suicidality in a large national sample of transgender adults. Social Psychiatry and Psychiatric Epidemiology, pages 1 – 10.
- mluke: The power of entity representations in multilingual pretrained language models. In Annual Meeting of the Association for Computational Linguistics.
- Multilingual hatecheck: Functional tests for multilingual hate speech detection models. ArXiv, abs/2206.09917.
- On negative interference in multilingual models: Findings and a meta-learning treatment. In Conference on Empirical Methods in Natural Language Processing.
- Robust fine-tuning of zero-shot models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7949–7961.
- mt5: A massively multilingual pre-trained text-to-text transformer. In North American Chapter of the Association for Computational Linguistics.