Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US (1702.06683v2)

Published 22 Feb 2017 in cs.CV

Abstract: The United States spends more than $1B each year on initiatives such as the American Community Survey (ACS), a labor-intensive door-to-door study that measures statistics relating to race, gender, education, occupation, unemployment, and other demographic factors. Although a comprehensive source of data, the lag between demographic changes and their appearance in the ACS can exceed half a decade. As digital imagery becomes ubiquitous and machine vision techniques improve, automated data analysis may provide a cheaper and faster alternative. Here, we present a method that determines socioeconomic trends from 50 million images of street scenes, gathered in 200 American cities by Google Street View cars. Using deep learning-based computer vision techniques, we determined the make, model, and year of all motor vehicles encountered in particular neighborhoods. Data from this census of motor vehicles, which enumerated 22M automobiles in total (8% of all automobiles in the US), was used to accurately estimate income, race, education, and voting patterns, with single-precinct resolution. (The average US precinct contains approximately 1000 people.) The resulting associations are surprisingly simple and powerful. For instance, if the number of sedans encountered during a 15-minute drive through a city is higher than the number of pickup trucks, the city is likely to vote for a Democrat during the next Presidential election (88% chance); otherwise, it is likely to vote Republican (82%). Our results suggest that automated systems for monitoring demographic trends may effectively complement labor-intensive approaches, with the potential to detect trends with fine spatial resolution, in close to real time.

View on arXiv

Authors (7)

Timnit Gebru (15 papers)
Jonathan Krause (14 papers)
Yilun Wang (39 papers)
Duyun Chen (2 papers)
Jia Deng (93 papers)
Erez Lieberman Aiden (2 papers)
Li Fei-Fei (199 papers)

Citations (394)

View on Semantic Scholar

Summary

Estimating Demographic Makeup of the US Using Deep Learning and Google Street View

The paper presents a compelling paper that leverages deep learning techniques and digital imagery to estimate demographic factors across the United States by analyzing vehicular images captured by Google Street View. This research aims to provide a viable alternative to traditional, labor-intensive surveys by offering near-real-time analyses with enhanced spatial resolution.

Methodology

The paper relies on a dataset comprising 50 million images across 200 cities in the United States. These images are processed for vehicle identification using computer vision algorithms grounded in deep learning, specifically Convolutional Neural Networks (CNNs). The algorithm categorizes vehicles identified in the images into one of 2,657 classes, which are defined by make, model, and year. The resulting vehicular data assists in extrapolating demographic statistics such as income, race, education levels, and political preferences, benchmarked at the resolution of a precinct, which on average, includes approximately 1,000 people.

To determine demographic associations, the researchers trained regression models using demographic data from the American Community Survey (ACS) and voting patterns, correlating these with the vehicle characteristics observed in selected training set cities. This allows for the generation of demographic estimates across test cities without utilizing local survey data.

Results and Implications

This approach yielded strong correlations with existing ACS data, achieving correlations as strong as r=0.87 for median income and r=0.87 for the percentage of Asian residents in cities within the paper. Furthermore, the paper found fascinating patterns in voter preferences; for instance, precincts with higher occurrences of sedans generally leaned Democratic, whereas those with more pickup trucks favored Republican candidates. The reliability of these insights was underscored by an 82% accuracy in predicting cities' candidate support based solely on vehicular data, demonstrating that vehicle data could effectively infer political lean from visual street-based assessments.

Impacts and Future Directions

This research highlights the practicality and effectiveness of using automated computer vision methods for socioeconomic monitoring and trend prediction. The implications extend to real-time updates and provide insights into demographic transitions that could potentially inform policy-making with a higher temporal and spatial granularity compared to existing practices. The paper also posits that expanding the classification dataset and integrating alternative imagery or social data sources could significantly enhance demographic inference capabilities.

However, the work does not neglect the responsible use of public data; it emphasizes the ethical necessity to safeguard privacy while exploiting public street-level imagery. Future developments could further harness emerging data streams from self-driving vehicle sensors, expanding the comprehensiveness and applicability of this methodology.

Bibliography

Gebru, T., Krause, J., Wang, Y., Chen, D., Deng, J., Lieberman Aiden, E., & Fei-Fei, L. (n.d.). Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US.

PDF Markdown

Related Papers

Find Related Papers