Geotagging One Hundred Million Twitter Accounts with Total Variation Minimization (1404.7152v2)

Abstract: Geographically annotated social media is extremely valuable for modern information retrieval. However, when researchers can only access publicly-visible data, one quickly finds that social media users rarely publish location information. In this work, we provide a method which can geolocate the overwhelming majority of active Twitter users, independent of their location sharing preferences, using only publicly-visible Twitter data. Our method infers an unknown user's location by examining their friend's locations. We frame the geotagging problem as an optimization over a social network with a total variation-based objective and provide a scalable and distributed algorithm for its solution. Furthermore, we show how a robust estimate of the geographic dispersion of each user's ego network can be used as a per-user accuracy measure which is effective at removing outlying errors. Leave-many-out evaluation shows that our method is able to infer location for 101,846,236 Twitter users at a median error of 6.38 km, allowing us to geotag over 80\% of public tweets.

• The paper introduces a novel convex optimization method using total variation minimization to infer user locations from social network structures.
• It achieves high accuracy by leveraging per-user dispersion measures, reporting a median geolocation error of 6.38 km and 89.7% city-level precision.
• The distributed implementation on Apache Spark scales to geotag over 100 million Twitter users, enabling significant applications in geospatial analysis.

Geotagging Twitter Accounts Using Total Variation Minimization

The paper "Geotagging One Hundred Million Twitter Accounts with Total Variation Minimization" by Ryan Compton, David Jurgens, and David Allen introduces a scalable approach to geolocate a vast majority of Twitter users utilizing publicly accessible Twitter data. The primary objective of the described method is to infer user locations based solely on their online social structures—specifically the locations of friends as reflected in Twitter connections—independent of the often sparse publicly visible location information provided voluntarily by users.

Methodological Overview

The authors present the geolocation problem within a social network as an optimization task framed by a total variation-based objective function. They propose a distributed solution leveraging a convex global optimization technique—specifically parallel coordinate descent—for inferring user locations. This approach calculates the estimated location for a user by minimizing the total variation of locations within their ego network, where the ego network comprises the user and their connected friends.

Numerical Results

The paper reports the geolocation of approximately 101 million Twitter users, achieving a median error of 6.38 kilometers. This demonstrates the algorithm's robustness, allowing for geotagging of over 80% of public tweets with substantial accuracy. The evaluation is conducted using a leave-many-out method, revealing city-level accuracy (under 10 km) for 89.7% of the users tested.

Key Contributions

1. Global Convex Optimization: The paper illustrates that social network-based geotagging can be effectively modeled as a globally-defined convex optimization problem—a novel approach compared to node-wise heuristics traditionally used.
2. Per-User Accuracy Estimation: The authors develop an innovative accuracy measure based on the geographic dispersion within a user's ego network. By implementing restrictions on dispersion values, the algorithm effectively filters out errors due to dispersed social graphs.
3. Scale and Implementation: Utilizing Apache Spark for distributed computing, the authors implement the algorithm, demonstrating the scalability required to handle data from over 110 million users—surpassing previous literature both in coverage and accuracy.

Implications and Future Directions

The practical implications of this research are significant, with potential applications ranging from business-oriented geotagging services to public health monitoring and sociolinguistic research. As the algorithm focuses on static geolocation, further exploration can be directed towards addressing within-city motion or dynamic modeling of user trajectories. The reliance on social network connections for geolocation, rather than self-reported profiles or GPS data, provides a distinct advantage in both coverage and linguistic agnosticism, offering a robust tool for geospatial analysis across diverse locales.

Future research might explore integration with machine learning models to refine location predictions further or the application of similar optimization techniques in other types of network data beyond social platforms. Additionally, investigating the stability of inferred locations over time and user activity could yield insights into temporal dynamics in social media geotagging.

In summary, the paper provides a comprehensive methodology for high-accuracy, large-scale user location inference on social media platforms, posing significant contributions to social network analysis and geospatial data processing using innovative optimization techniques.