Visual Themes and Sentiment on Social Networks To Aid First Responders During Crisis Events

Abstract: Online Social Networks explode with activity whenever a crisis event takes place. Most content generated as part of this activity is a mixture of text and images, and is particularly useful for first responders to identify popular topics of interest and gauge the pulse and sentiment of citizens. While multiple researchers have used text to identify, analyze and measure themes and public sentiment during such events, little work has explored visual themes floating on networks in the form of images, and the sentiment inspired by them. Given the potential of visual content for influencing users' thoughts and emotions, we perform a large scale analysis to compare popular themes and sentiment across images and textual content posted on Facebook during the terror attacks that took place in Paris in 2015. Using state-of-the-art image summarization techniques, we discovered multiple visual themes which were popular in images, but were not identifiable through text. We uncovered instances of misinformation and false flag (conspiracy) theories among popular image themes, which were not prominent in user generated textual content, and can be of particular inter- est to first responders. Our analysis also revealed that while textual content posted after the attacks reflected negative sentiment, images inspired positive sentiment. To the best of our knowledge, this is the first large scale study of images posted on social networks during a crisis event.