RIP Twitter API: A eulogy to its vast research contributions (2404.07340v1)
Abstract: Since 2006, Twitter's Application Programming Interface (API) has been a treasure trove of high-quality data for researchers studying everything from the spread of misinformation, to social psychology and emergency management. However, in the spring of 2023, Twitter (now called X) began changing $42,000/month for its Enterprise access level, an essential death knell for researcher use. Lacking sufficient funds to pay this monthly fee, academics are now scrambling to continue their research without this important data source. This study collects and tabulates the number of studies, number of citations, dates, major disciplines, and major topic areas of studies that used Twitter data between 2006 and 2023. While we cannot know for certain what will be lost now that Twitter data is cost prohibitive, we can illustrate its research value during the time it was available. A search of 8 databases and 3 related APIs found that since 2006, a total of 27,453 studies have been published in 7,432 publication venues, with 1,303,142 citations, across 14 disciplines. Major disciplines include: computational social science, engineering, data science, social media studies, public health, and medicine. Major topics include: information dissemination, assessing the credibility of tweets, strategies for conducting data research, detecting and analyzing major events, and studying human behavior. Twitter data studies have increased every year since 2006, but following Twitter's decision to begin charging for data in the spring of 2023, the number of studies published in 2023 decreased by 13% compared to 2022. We assume that much of the data used for studies published in 2023 were collected prior to Twitter's shutdown, and thus the number of new studies are likely to decline further in subsequent years.
- The spread of true and false news online. Science [Internet]. 2018;359(6380):1146–51. Available from: http://dx.doi.org/10.1126/science.aap9559
- Golder SA, Macy MW. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science [Internet]. 2011;333(6051):1878–81. Available from: http://dx.doi.org/10.1126/science.1202775
- Using social media to enhance emergency situation awareness. IEEE Intell Syst [Internet]. 2012;27(6):52–9. Available from: http://dx.doi.org/10.1109/mis.2012.6
- Getting started with the Twitter API [Internet]. Twitter. [cited 2024 Feb 24]. Available from: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api
- Developer Agreement and Policy [Internet]. Twitter. [cited 2024 Feb 24]. Available from: https://developer.twitter.com/en/developer-terms/agreement-and-policy
- Ledford H. Researchers scramble as Twitter plans to end free data access. Nature [Internet]. 2023;614(7949):602–3. Available from: http://dx.doi.org/10.1038/d41586-023-00460-z
- Hutchinson A. Twitter continues its efforts to fight spam and misuse with new API restrictions [Internet]. Social Media Today. 2018 [cited 2024 Feb 24]. Available from: https://www.socialmediatoday.com/news/twitter-continues-its-efforts-to-fight-spam-and-misuse-with-new-api-restric/528533/
- Sebastian KS. The EU’s Digital Services Act and its impact on online platforms [Internet]. EU Law Working Papers [Internet]. 2024;85. Available from: https://law.stanford.edu/publications/no-85-the-eus-digital-services-act-and-its-impact-on-online-platforms/
- A systematic review of predicting elections based on social media data: Research challenges and future directions. IEEE Trans Comput Soc Syst [Internet]. 2021;8(4):819–43. Available from: http://dx.doi.org/10.1109/tcss.2021.3063660
- Social media and attitudes towards a COVID-19 vaccination: A systematic review of the literature. EClinicalMedicine [Internet]. 2022;48(101454):101454. Available from: http://dx.doi.org/10.1016/j.eclinm.2022.101454
- Alqurashi T. Arabic Sentiment Analysis for Twitter Data: A Systematic Literature Review. Engineering. Technology & Applied Science Research [Internet]. 2023;13(2):10292–300. Available from: https://etasr.com/index.php/ETASR/article/view/5662
- Twitter and research: A systematic literature review through text mining. IEEE Access [Internet]. 2020;8:67698–717. Available from: http://dx.doi.org/10.1109/access.2020.2983656
- Yu J, Muñoz-Justicia J. A bibliometric overview of Twitter-related studies indexed in Web of science. Future Internet [Internet]. 2020;12(5):91. Available from: http://dx.doi.org/10.3390/fi12050091
- Understanding COVID-19 response by twitter users: A text analysis approach. Heliyon [Internet]. 2022;8(8):e09994. Available from: http://dx.doi.org/10.1016/j.heliyon.2022.e09994
- Why we twitter: Understanding microblogging usage and communities. Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis. [Internet]. 2007 Aug 12; New York, USA: p. 56-65.
- Tromble R. Where have all the data gone? A critical reflection on academic digital research in the post-API age. Soc Media Soc [Internet]. 2021;7(1):205630512198892. Available from: http://dx.doi.org/10.1177/2056305121988929
- Robinson B. X Corp lawsuits target data scraping. The National Law Review. 2023 Aug; XIV(1). Available from: https://www.natlawreview.com/article/x-corp-lawsuits-target-data-scraping