Tweets as impact indicators: Examining the implications of automated bot accounts on Twitter (1410.4139v1)

Abstract: This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific papers deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. We present preliminary findings that automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose differentiating between different levels of engagement from tweeting only bibliographic information to discussing or commenting on the content of a paper.

• The paper finds that at least 9% of tweets for arXiv papers are generated by automated bot accounts, which can artificially inflate impact metrics.
• The paper categorizes Twitter accounts into platform feeds, topic feeds, and selective accounts to distinguish between automated and human interactions.
• The paper advocates for incorporating account-type and engagement distinctions into altmetric frameworks to enhance research evaluation accuracy.

Analysis of Automated "Bot" Accounts and Their Influence on Altmetrics

The paper entitled "Tweets as Impact Indicators: Examining the Implications of Automated 'Bot' Accounts on Twitter" provides a detailed investigation into the impact of automated Twitter accounts on the altmetric landscape, particularly focusing on their interaction with scholarly documents on the arXiv preprint repository. This research scrutinizes the role these bots play in distributing links to scientific papers and assesses the implications of their behaviors on using tweet counts as altmetric indicators.

Overview of Findings

The researchers present evidence that a significant number of tweets referencing arXiv papers are generated by automated accounts, or "bots." These bots differ from typical social bots noted for mimicking human behavior; they specifically target scholarly documents without human mediation or discretion. The paper categorizes Twitter accounts promoting arXiv papers into three distinct types: platform feeds, topic feeds, and selective/qualitative accounts. While platform and topic feeds function automatically, selective accounts involve a degree of human judgment in tweeting content.

Numerical Results and Analytical Insights

The paper identified that automated accounts—both platform and topic feeds—represent at least 9% of tweets in their analysis of 44,163 arXiv submissions. The bot accounts reported tweeting on average between 2,032 and 2,510 tweets per account, conveying substantial dissemination capability compared to human-managed accounts with considerably fewer tweets.

In addition to their distribution volume, bots demonstrated unique behavioral patterns; for instance, they had limited followee networks, unlike typical social bots that engage in back-and-forth following schemes to garner attention. Out of the accounts analyzed, platform feeds displayed the least human-like attributes with some acquiring Bot or Not? scores as low as 26%, yet they often went undetected by automated bot identification tools.

Implications for Altmetrics and Research Evaluation

The findings underscore the complications introduced by bots in using raw tweet counts as indicators of research impact. As altmetrics begin gaining traction in academic CVs and funding assessments, the presence of non-discriminatory bot-driven tweets necessitates consideration for enhancing accuracy and validity. The research suggests incorporating distinguishers of account types and engagement levels into future altmetric evaluation frameworks to mitigate inflation or bias caused by bot-driven activities.

Theoretical Considerations and Future Directions

Theoretically, the investigation expands the understanding of how digital metrics interact with automated systems, illustrating potential biases in data commonly used for assessing scholarly influence. Future research is encouraged to explore the breadth of automated influence across other platforms and to refine detection mechanisms that distinguish between genuine and automated interactions to safeguard the credibility of altmetric systems.

The authors propose a nuanced classification system to differentiate human, cyborg, and bot tweets, considering both origin and engagement level with the article content. This advancement offers a pathway to more accurately interpret the social metrics associated with scholarly communications.

In conclusion, this paper situates itself at the intersection of digital scholarship and data ethics, advocating for comprehensive analytical strategies to monitor and adjust altmetric systems in light of automated account influences. It invites further dialogue and methodological refinement to navigate the evolving dynamics of researcher visibility through new media channels.