Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Urban Visual Intelligence: Studying Cities with AI and Street-level Imagery (2301.00580v2)

Published 2 Jan 2023 in cs.CV and cs.CY

Abstract: The visual dimension of cities has been a fundamental subject in urban studies, since the pioneering work of scholars such as Sitte, Lynch, Arnheim, and Jacobs. Several decades later, big data and AI are revolutionizing how people move, sense, and interact with cities. This paper reviews the literature on the appearance and function of cities to illustrate how visual information has been used to understand them. A conceptual framework, Urban Visual Intelligence, is introduced to systematically elaborate on how new image data sources and AI techniques are reshaping the way researchers perceive and measure cities, enabling the study of the physical environment and its interactions with socioeconomic environments at various scales. The paper argues that these new approaches enable researchers to revisit the classic urban theories and themes, and potentially help cities create environments that are more in line with human behaviors and aspirations in the digital age.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (159)
  1. Valuing iconic design: Frank Lloyd Wright architecture in Oak Park, Illinois. Housing studies, 27(8):1079–1099.
  2. Google street view: Capturing the world at street level. Computer, 43(6):32–38.
  3. Livable streets, protected neighborhoods. University of California Press.
  4. Arnheim, R. (1965). Art and visual perception: A psychology of the creative eye. University of California Press.
  5. Perceived environmental aesthetics and convenience and company are associated with walking for exercise among australian adults. Preventive Medicine, 33(5):434–440.
  6. Batty, M. (2021). Defining urban science. In Shi, W., Goodchild, M. F., Batty, M., Kwan, M.-p., and Zhang, A., editors, Urban Informatics. Springer, Singapore. The Urban Book Series.
  7. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences, 117(48):30071–30078.
  8. Street view imagery in urban analytics and GIS: A review. Landscape and Urban Planning, 215:104217.
  9. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
  10. Estimating pedestrian volume using street view images: A large-scale validation test. Computers, Environment and Urban Systems, 81:101481.
  11. A survey on automatic image annotation. Applied Intelligence, 50:3412–3428.
  12. Panoramic street-level imagery in data-driven urban research: A comprehensive global review of applications, techniques, and practical considerations. ISPRS International Journal of Geo-Information, 10(7):471.
  13. Neighbourhood drivability: environmental and individual characteristics associated with car use across Europe. International journal of behavioral nutrition and physical activity, 17(8):1–11.
  14. What makes Paris look like Paris? ACM Transactions on Graphics, 31(4):01053876.
  15. What urban cameras reveal about the city: The work of the Senseable City Lab. In Shi, W., Goodchild, M. F., Batty, M., Kwan, M.-P., and Zhang, A., editors, Urban Informatics. Springer, Singapore. The Urban Book Series.
  16. Deep learning the city: Quantifying urban perception at a global scale. In European Conference on Computer Vision, pages 196–212. Springer.
  17. Viewing obesogenic advertising in children’s neighbourhoods using Google Street View. Geographical Research, 57(1):84–97.
  18. Rhythm of transit stations-uncovering the activity-travel dynamics of transit-oriented development in the US. IEEE Transactions on Intelligent Transportation Systems.
  19. Neighbourhood typology based on virtual audit of environmental obesogenic characteristics. Obesity Reviews, 17:19–30.
  20. A computational approach to ‘The Image of the City’. Cities, 89:14–25.
  21. The modifiable areal unit problem in multivariate statistical analysis. Environment and planning A, 23(7):1025–1044.
  22. Freestone, R. (2011). Reconciling beauty and utility in early city planning: the contribution of john nolen. Journal of Urban History, 37(2):256–277.
  23. A data-synthesis-driven method for detecting and extracting vague cognitive regions. International Journal of Geographical Information Science, 31(6):1245–1271.
  24. Using deep learning and google street view to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences, 114(50):13108–13113.
  25. Gehl, J. (1971). Life between buildings: using public space. Danish Architectural Press.
  26. Estimating city-level travel patterns using street imagery: A case study of using Google Street View in Britain. PloS one, 13(5):e0196521.
  27. Goodchild, M. F. (2007). Citizens as sensors: The world of volunteered geography. GeoJournal, 69(4):211–221.
  28. Goodchild, M. F. (2011). Formalizing Place in Geographic Information Systems. In Burton, L. M., Matthews, S. A., Leung, M., Kemp, S. P., and Takeuchi, D. T., editors, Communities, Neighborhoods, and Health: Expanding the Boundaries of Place, pages 21–33. Springer New York, New York, NY.
  29. Introduction: Forum on reproducibility and replicability in geography. Annals of the American Association of Geographers, 111(5):1271–1274.
  30. Replication across space and time must be weak in the social and environmental sciences. Proceedings of the National Academy of Sciences, 118(35).
  31. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, pages 2672–2680.
  32. Assessing walking and cycling environments in the streets of madrid: comparing on-field and virtual audits. Journal of Urban Health, 92(5):923–939.
  33. Predicting bicycling and walking traffic using street view imagery and destination data. Transportation research part D: transport and environment, 90:102651.
  34. Association of street greenery and physical activity in older adults: A novel study using pedestrian-centered photographs. Urban Forestry & Urban Greening, 55:126789.
  35. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, pages 2980–2988.
  36. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778.
  37. Urban neighbourhood environment assessment based on street view image processing: A review of research trends. Environmental Challenges, 4:100090.
  38. Using deep learning to examine street view green and blue spaces and their associations with geriatric depression in Beijing, China. Environment International, 126:107–117.
  39. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82–97.
  40. Did safe cycling infrastructure still matter during a COVID-19 lockdown? Sustainability, 12(20):8672.
  41. Classification and mapping of urban canyon geometry using google street view images and deep multitask learning. Building and Environment, 167:106424.
  42. Investigation of clusters and injuries in pedestrian crashes using GIS in Changsha, China. Safety science, 127:104710.
  43. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2261–2269.
  44. The image of the city on social media: A comparative study using “big data” and “small data” methods in the tri-city region in Poland. Landscape and Urban Planning, 206:103977.
  45. Understanding cities with machine eyes: A review of deep computer vision in urban analytics. Cities, 96:102481.
  46. Urban-i: From urban scenes to mapping slums, transport modes, and pedestrians in cities using deep learning and computer vision. Environment and Planning B: Urban Analytics and City Science, 48(1):76–93.
  47. Deep mapping gentrification in a large Canadian city using deep learning and Google Street View. PLoS One, 14(3):e0212814.
  48. Google Street View assessment of environmental safety features at the scene of pedestrian automobile injury. Journal of trauma and acute care surgery, 87(1):82–86.
  49. Toward an urban design manifesto. Journal of the American Planning Association, 53(1):112–120.
  50. Jacobs, J. (2011). ”The Uses of Sidewalks: Safety”: from The Death and Life of Great American Cities (1961). In The City Reader, pages 137–141. Routledge.
  51. A review of the health benefits of greenness. Current Epidemiology Reports, 2(2):131–142.
  52. Six GIScience ideas that must die. AGILE: GIScience Series, 3:1–8.
  53. Valuing curb appeal. The Journal of Real Estate Finance and Economics, 60(1):111–133.
  54. Injury outcome in crashes with guardrail end terminals. Traffic Injury Prevention, 16(sup2):S103–S108.
  55. Building instance classification using street view images. ISPRS Journal of Photogrammetry and Remote Sensing, 145:44–59.
  56. Extracting human emotions at different places based on facial expressions and spatial clustering analysis. Transactions in GIS, 23(3):450–480.
  57. A review of urban physical environment sensing using street view imagery in public health studies. Annals of GIS, 26(3):261–275.
  58. Human settlement value assessment from a place perspective: Considering human dynamics and perceptions in house price modeling. Cities, 118:103333.
  59. Understanding house price appreciation using multi-source big geo-data and machine learning. Land Use Policy, 111:104919.
  60. The experience of nature: A psychological perspective. CUP Archive.
  61. Reproducibility and replicability: opportunities and challenges for geospatial research. International Journal of Geographical Information Science, 35(3):427–445.
  62. Health and the built environment in United States cities: Measuring associations using Google Street View-derived indicators of the built environment. BMC public health, 20(1):1–10.
  63. Looking beyond the visible scene. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3710–3717.
  64. Kim, A. M. (2015). Sidewalk city: remapping public space in Ho Chi Minh City. University of Chicago Press.
  65. Places for play: Understanding human perception of playability in cities using street view images and deep learning. Computers, Environment and Urban Systems, 90:101693.
  66. Standardized green view index and quantification of different metrics of urban green vegetation. Sustainability, 12(18):7434.
  67. An examination of the intersection environment associated with perceived crash risk among school-aged children: using street-level imagery and computer vision. Accident Analysis & Prevention, 146:105716.
  68. Evaluating street view exposure measures of visible green space for health research. Journal of Exposure Science & Environmental Epidemiology, 29(4):447.
  69. Geocoding of trees from street addresses and street-level images. ISPRS Journal of Photogrammetry and Remote Sensing, 162:125–136.
  70. Take a look around: using street view and satellite images to estimate house prices. ACM Transactions on Intelligent Systems and Technology (TIST), 10(5):1–19.
  71. Geographical variation in cardiovascular disease, risk factors, and their control in older women: British women’s heart and health study. Journal of Epidemiology & Community Health, 57(2):134–140.
  72. Deep learning. Nature, 521(7553):436–444.
  73. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. The Lancet, 380(9838):219–229.
  74. Are perceptions of the local environment related to neighbourhood satisfaction and mental health in adults? Preventive Medicine, 47(3):273–278.
  75. Matching study areas using google street view: A new application for an emerging technology. Evaluation and program planning, 53:72–79.
  76. Spatial, temporal, and socioeconomic patterns in the use of Twitter and Flickr. cartography and geographic information science, 40(2):61–77.
  77. Tobler’s first law in GeoAI: A spatially explicit deep learning model for terrain feature detection under weak supervision. Annals of the American Association of Geographers, pages 1–19.
  78. Mapping the spatial distribution of shade provision of street trees in Boston using Google Street View panoramas. Urban Forestry & Urban Greening, 31:109–119.
  79. Investigating the association between streetscapes and human walking activities using Google Street View and human trajectory data. Transactions in GIS, 22(4):1029–1044.
  80. Does the visibility of greenery increase perceived safety in urban areas? Evidence from the Place Pulse 1.0 dataset. ISPRS International Journal of Geo-Information, 4(3):1166–1183.
  81. Estimating building age from Google Street View images using deep learning. In 10th International Conference on Geographic Information Science.
  82. C-IMAGE: city cognitive mapping through geo-tagged photos. GeoJournal, 81(6):817–861.
  83. Ssd: Single shot multibox detector. In European Conference on Computer Vision, pages 21–37. Springer.
  84. Urban land uses and traffic ‘source-sink areas’: Evidence from GPS-enabled taxi data in Shanghai. Landscape and Urban Planning, 106(1):73–87.
  85. Towards feasibility of photovoltaic road for urban traffic-solar energy estimation using street view image. Journal of Cleaner Production, 228:303–318.
  86. The effect of street-level greenery on walking behavior: Evidence from hong kong. Social Science & Medicine, 208:41–49.
  87. Associations between overhead-view and eye-level urban greenness and cycling behaviors. Cities, 88:10–18.
  88. Lynch, K. (1960). The image of the city. MIT press.
  89. Lynch, K. (1984). Good city form. MIT press.
  90. Typeface reveals spatial economic patterns. Scientific Reports, 9(1):15946.
  91. Mackintosh, P. G. (2005). ‘the development of higher urban life’and the geographic imagination: beauty, art, and moral environmentalism in toronto, 1900–1920. Journal of Historical Geography, 31(4):688–722.
  92. An approach to environmental psychology. the MIT Press.
  93. Milgram, S. (1970). The experience of living in cities: A psychological analysis. Psychology and the problems of society.
  94. Desirable streets: Using deviations in pedestrian trajectories to measure the value of the built environment. Computers, Environment and Urban Systems, page 101563.
  95. Learning to navigate in cities without a map. arXiv preprint arXiv:1804.00168.
  96. Effect of exposure to natural environment on health inequalities: an observational population study. The Lancet, 372(9650):1655–1660.
  97. Development and validation of a Google Street View pedestrian safety audit tool. Epidemiology, 31(2):301.
  98. Morison, B. (2002). On location: Aristotle’s concept of place. Oxford University Press on Demand.
  99. Mulford, L. S. (1899). Plate design. Arts Education Policy Review, 1(6):126.
  100. Computer vision uncovers predictors of physical urban change. Proceedings of the National Academy of Sciences, 114(29):7571–7576.
  101. Nasar, J. L. (1994). Urban design aesthetics: The evaluative qualities of building exteriors. Environment and behavior, 26(3):377–401.
  102. Nasar, J. L. (1998). The evaluative image of the city. Sage Publications.
  103. Landscapes of fear and stress. Environment and behavior, 29(3):291–323.
  104. Exploring generalization in deep learning. arXiv preprint arXiv:1706.08947.
  105. Neighbourhood looking glass: 360º automated characterisation of the built environment for neighbourhood effects research. J Epidemiol Community Health, 72(3):260–266.
  106. Exploring the vertical dimension of street view image based on deep learning: a case study on lowest floor elevation estimation. International Journal of Geographical Information Science, 35(12):1–26.
  107. A deep image of the city: Generative urban-design visualization. Challenge, 7:30.
  108. Learning high-level judgments of urban perception. In European Conference on Computer Vision, pages 494–510. Springer.
  109. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2751–2758. IEEE.
  110. Maintaining research traditions on place: Diversity of thought and scientific progress. Journal of environmental psychology, 25(4):361–380.
  111. Optimising measurement of health-related characteristics of the built environment: comparing data collected by foot-based street audits, virtual street audits and routine secondary data sources. Health & Place, 43:75–84.
  112. Places in information science. Journal of the Association for Information Science and Technology, 70(11):1173–1182.
  113. Pushkarev, B. S. (1976). Urban Space for Pedestrians: A Quantitative Approach. MIT Press.
  114. Measuring social functions of city regions from large-scale taxi behaviors. In IEEE International Conference on Pervasive Computing and Communications Workshops, pages 384–388.
  115. A graph convolutional network model for evaluating potential congestion spots based on local urban built environments. Transactions in GIS, 24(5):1382–1401.
  116. Subjective or objective measures of street environment, which are more effective in explaining housing prices? Landscape and Urban Planning, 221:104358.
  117. Dataset shift in machine learning. MIT Press.
  118. Rapoport, A. (1990). The meaning of the built environment: A nonverbal communication approach. University of Arizona Press.
  119. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems, pages 91–99.
  120. Enhancing photorealism enhancement. arXiv:2105.04619.
  121. Mismatch between perceived and objectively measured environmental obesogenic features in european neighbourhoods. Obesity Reviews, 17:31–41.
  122. LabelMe: a database and web-based tool for image annotation. International journal of computer vision, 77(1-3):157–173.
  123. Systematic review of the use of google street view in health research: major themes, strengths, weaknesses and possibilities for future research. Health & Place, 52:240–246.
  124. Crowdsourcing architectural beauty: Online photo frequency predicts building aesthetic ratings. PloS One, 13(7):e0194369.
  125. The collaborative image of the city: mapping the inequality of urban perception. PLoS One, 8(7):e68400.
  126. Seeing disorder: Neighborhood stigma and the social construction of “broken windows”. Social psychology quarterly, 67(4):319–342.
  127. Green streets- quantifying and mapping urban trees with street-level imagery and computer vision. Landscape and Urban Planning, 165:93–101.
  128. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484–489.
  129. Sitte, C. (1889). City Building According to Artistic Principles. Columbia University Press New York.
  130. Utility data annotation with Amazon Mechanical Turk. In IEEE computer society conference on computer vision and pattern recognition workshops, pages 1–8. IEEE.
  131. Multimodal deep learning from satellite and street-level imagery for measuring income, overcrowding, and environmental deprivation in urban areas. Remote Sensing of Environment, 257:112339.
  132. Measuring social, environmental and health inequalities using deep learning and street imagery. Scientific Reports, 9(1):6229.
  133. Understanding building energy efficiency with administrative and emerging urban big data by deep learning in glasgow. Energy and Buildings, 273:112331.
  134. Understanding architecture age and style through deep learning. Cities, 128:103787.
  135. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112.
  136. Demonstrating the effect of exposure to nature on happy facial expressions via flickr data: Advantages of non-intrusive social network data analyses and geoinformatics methodologies. Journal of Environmental Psychology, 58:93–100.
  137. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1–9.
  138. Urban residential environments and senior citizens’ longevity in megacity areas: the importance of walkable green spaces. Journal of Epidemiology & Community Health, 56(12):913–918.
  139. Beyond relativism: Reclaiming the search for good city form. Journal of Planning Education and Research, 22(1):36–49.
  140. Recognizing traffic black spots from street view images using environment-aware image processing and neural network. IEEE Access, 8:121469–121478.
  141. Tuan, Y.-F. (1977). Space and place: The perspective of experience. University of Minnesota Press.
  142. Tuan, Y.-F. (1979). Landscapes of fear. University of Minnesota Press.
  143. Ulrich, R. S. (1984). View through a window may influence recovery from surgery. Science, 224(4647):420–421.
  144. Google walkability: a new tool for local planning and public health research? Journal of Physical Activity and Health, 9(5):689–697.
  145. Differences in physical environmental characteristics between adolescents’ actual and shortest cycling routes: a study using a Google Street View-based audit. International journal of health geographics, 17(1):1–15.
  146. Comparing the normalized difference vegetation index with the Google Street View measure of vegetation to assess associations between greenness, walkability, recreational physical activity, and health in Ottawa, Canada. International Journal of Environmental Research and Public Health, 15(8):1719.
  147. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence.
  148. Deep visual domain adaptation: A survey. Neurocomputing, 312:135–153.
  149. Urban greenery and mental wellbeing in adults: Cross-sectional mediation analyses on multiple pathways across different greenery measures. Environmental research, 176:108535.
  150. The relationship between visual enclosure for neighbourhood street walkability and elders’ mental health in China: Using street view images. Journal of Transport & Health, 13:90–102.
  151. Quantifying legibility of indoor spaces using deep convolutional neural networks: Case studies in train stations. Building and Environment, 160:106099.
  152. Warntz, W. (1989). Newton, the newtonians, and the geographia generalis varenii. Annals of the Association of American Geographers, 79(2):165–191.
  153. Whyte, W. H. (1980). The social life of small urban spaces. Public Spaces.
  154. Wilson, W. H. (1964). city beautiful movement in Kansas City. University of Missouri Press.
  155. Wohlwill, J. F. (1976). Environmental aesthetics: The environment as a source of affect. In Human behavior and environment, pages 37–86. Springer.
  156. GANmapper: geographical data translation. International Journal of Geographical Information Science, pages 1–29.
  157. Characterizing tourism destination image using photos’ visual content. ISPRS International Journal of Geo-Information, 9(12):730.
  158. Architectural style classification using multinomial latent logistic regression. In European Conference on Computer Vision, pages 600–615. Springer.
  159. The financial impact of street-level greenery on New York commercial buildings. Available at SSRN 3714858.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now