Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Active learning in annotating micro-blogs dealing with e-reputation (1706.05349v4)

Published 16 Jun 2017 in cs.SI and cs.CL

Abstract: Elections unleash strong political views on Twitter, but what do people really think about politics? Opinion and trend mining on micro blogs dealing with politics has recently attracted researchers in several fields including Information Retrieval and Machine Learning (ML). Since the performance of ML and NLP approaches are limited by the amount and quality of data available, one promising alternative for some tasks is the automatic propagation of expert annotations. This paper intends to develop a so-called active learning process for automatically annotating French language tweets that deal with the image (i.e., representation, web reputation) of politicians. Our main focus is on the methodology followed to build an original annotated dataset expressing opinion from two French politicians over time. We therefore review state of the art NLP-based ML algorithms to automatically annotate tweets using a manual initiation step as bootstrap. This paper focuses on key issues about active learning while building a large annotated data set from noise. This will be introduced by human annotators, abundance of data and the label distribution across data and entities. In turn, we show that Twitter characteristics such as the author's name or hashtags can be considered as the bearing point to not only improve automatic systems for Opinion Mining (OM) and Topic Classification but also to reduce noise in human annotations. However, a later thorough analysis shows that reducing noise might induce the loss of crucial information.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Overview of replab 2013: Evaluating online reputation monitoring systems. In Information Access Evaluation. Multilinguality, Multimodality, and Visualization, pp.  333–352. Springer.
  2. Artstein R., Poesio M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics 34(4), 555–596.
  3. Batista L. B., Ratte S. (2012). A multi-classifier system for sentiment analysis and opinion mining. In Proc. of the 2012 International Conference on Advances in Social Networks Analysis and Mining, pp.  96–100. IEEE Computer Society.
  4. Blum A., Mitchell T. (1998). Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pp.  92–100. ACM.
  5. Boyadjian J. (2014). Twitter, un nouveau «baromètre de l’opinion publique»? Participations 8, 55–74.
  6. Brun C., Roux C. (2014). Decomposing hashtags to improve tweet polarity classification [in french]. In Proceedings of TALN 2014 (Volume 2: Short Papers), pp. 473–478. Association pour le Traitement Automatique des Langues.
  7. Burton S., Soboleva A. (2011). Interactive or reactive? : marketing with Twitter. Journal of Consumer Marketing 28(7), 491–499.
  8. Orma: A semi-automatic tool for online reputation monitoring in twitter. In Advances in Information Retrieval, pp.  742–745. Springer.
  9. Improving generalization with active learning. Machine learning 15(2), 201–221.
  10. Lia@replab 2013. In CLEF.
  11. NLP-based classifiers to generalize experts assessments in e-reputation. In International Conference of the Cross-Language Evaluation Forum for European Languages Experimental IR meets Multilinguality, Multimodality, and Interaction, pp.  340–351. Springer.
  12. Dagan I., Engelson S. P. (1995). Committee-based sampling for training probabilistic classifiers. In Proceedings of the Twelfth International Conference on Machine Learning, pp.  150–157. The Morgan Kaufmann series in machine learning,(San Francisco, CA, USA).
  13. Bootstrapping spoken dialog systems with data reuse. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, Cambridge, MA, April.
  14. Gerlitz C., Rieder B. (2013). Mining one percent of twitter: Collections, baselines, sampling. M/C Journal 16(2).
  15. Deriving marketing intelligence from online discussion. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pp.  419–428. ACM.
  16. Hendricks C., Schill D. (2014). Presidential Campaigning and Social Media: An Analysis of the 2012 Campaign. Oxford University Press.
  17. Hoffman T. (2008). Online reputation management is hot—but is it ethical. Computerworld, February, 1–4.
  18. Hu M., Liu B. (2004). Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’04, pp.  168–177. ACM.
  19. Twitter power: Tweets as electronic word of mouth. Journal of the American society for information science and technology 60(11), 2169–2188.
  20. Why the Pirate Party won the German election of 2009 or the trouble with predictions: A response to Tumasjan, A., Sprenger, to, Sander, PG, & Welpe, in ’predicting elections with Twitter: What 140 characters reveal about political sentiment’. Social Science Computer Review 30(2), 229–234.
  21. Taking sides: User classification for informal online political discourse. Internet Research 18(2), 177–190.
  22. Koehn P., Knight K. (2003). Empirical methods for compound splitting. In Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics - Volume 1, EACL ’03, Stroudsburg, PA, USA, pp.  187–193. Association for Computational Linguistics.
  23. Kohen J. (1960). A coefficient of agreement for nominal scale. Educ Psychol Meas 20, 37–46.
  24. Lamontagne L., Abi-Zeid I. (2006). Combining multiple similarity metrics using a multicriteria approach. In Advances in Case-Based Reasoning, pp.  415–428. Springer.
  25. Incorporating reviewer and product information for review rating prediction. In IJCAI, Volume 11, pp.  1820–1825.
  26. Liere R., Tadepalli P. (1997). Active learning with committees for text categorization. In AAAI/IAAI, pp.  591–596.
  27. Malaga R. A. (2001). Web-based reputation management systems: Problems and suggested solutions.  1(4), 403–417.
  28. McCallumzy A. K., Nigamy K. (1998). Employing em and pool-based active learning for text classification. In Proc. International Conference on Machine Learning (ICML), pp.  359–367.
  29. How (not) to predict elections. In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third Inernational Conference on Social Computing (SocialCom), 2011 IEEE Third International Conference on, pp.  165–171. IEEE.
  30. From tweets to polls: Linking text sentiment to public opinion time series. In International AAAI Conference on Weblogs and Social Media.
  31. Twitter as a corpus for sentiment analysis and opinion mining. In N. C. C. Chair), K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias (Eds.), Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA).
  32. The politics of comments: predicting political orientation of news stories with commenters’ sentiment patterns. In Proceedings of the ACM 2011 conference on Computer supported cooperative work, pp.  113–122. ACM.
  33. Pla F., Hurtado L.-F. (2014). Political tendency identification in twitter using sentiment analysis techniques. In Proc. of COLING.
  34. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado, pp.  486–495. Association for Computational Linguistics.
  35. Robertson S. (2004). Understanding inverse document frequency: on theoretical arguments for idf. Journal of documentation 60(5), 503–520.
  36. Corpus annotation through crowdsourcing: Towards best practice guidelines. In Proc. LREC.
  37. Salton G., Buckley C. (1988). Term-weighting approaches in automatic text retrieval. Information processing & management 24(5), 513–523.
  38. Settles B. (2012). Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 6(1), 1–114.
  39. Monitoring the twitter sentiment during the bulgarian elections. In Data Science and Advanced Analytics (DSAA), 2015. 36678 2015. IEEE International Conference on, pp.  1–10. IEEE.
  40. Smeaton A. F. (1999). Using NLP or NLP resources for information retrieval tasks. In Natural language information retrieval, pp.  99–111. Springer.
  41. Sparck Jones K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of documentation 28(1), 11–21.
  42. Active learning for entity filtering in microblog streams. In SIGIR 2015: 38th international ACM SIGIR conference on Research and development in information retrieval.
  43. Opinion detection as a topic classification problem. In É. Gaussier and F. Yvon (Eds.), Textual Information Access: Statistical Models, pp.  337–368. Wiley-ISTE. URL: http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1848213220.html.
  44. Tsoumakas G., Katakis I. (2006). Multi-label classification: An overview. International Journal of Data Warehousing and Mining: Concepts, Methodologies, Tools, and Applications 3, 64–74.
  45. Investigating the image of entities in social media: Dataset design and first results. In Proceedings of Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland, pp.  818–822.
  46. Tass-workshop on sentiment analysis at sepln. Comité Editorial 50, 37–44.
  47. Walter T. P., Back A. (2013). A text mining approach to evaluate submissions to crowdsourcing contests. In System Sciences (HICSS), 2013 46th Hawaii International Conference on, pp.  3109–3118. IEEE.
  48. Perspectives on crowdsourcing annotations for natural language processing. Language resources and evaluation 47(1), 9–31.
  49. A system for real-time twitter sentiment analysis of 2012 u.s. presidential election cycle. In Proceedings of the ACL 2012 System Demonstrations, ACL ’12, pp.  115–120. Association for Computational Linguistics.
  50. Incorporating diversity and density in active learning for relevance feedback. Springer.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now