Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Self-supervised learning unveils change in urban housing from street-level images (2309.11354v2)

Published 20 Sep 2023 in cs.CV and cs.LG

Abstract: Cities around the world face a critical shortage of affordable and decent housing. Despite its critical importance for policy, our ability to effectively monitor and track progress in urban housing is limited. Deep learning-based computer vision methods applied to street-level images have been successful in the measurement of socioeconomic and environmental inequalities but did not fully utilize temporal images to track urban change as time-varying labels are often unavailable. We used self-supervised methods to measure change in London using 15 million street images taken between 2008 and 2021. Our novel adaptation of Barlow Twins, Street2Vec, embeds urban structure while being invariant to seasonal and daily changes without manual annotations. It outperformed generic embeddings, successfully identified point-level change in London's housing supply from street-level images, and distinguished between major and minor change. This capability can provide timely information for urban planning and policy decisions toward more liveable, equitable, and sustainable cities.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pages 12310–12320. PMLR, 2021.
  2. Steffen Wetzstein. The global urban housing affordability crisis. Urban studies, 54(14):3159–3177, 2017.
  3. Albert Saiz. The global housing affordability crisis: Policy options and strategies. MIT Center for Real Estate Research Paper, 01(23), 2023.
  4. Victoria Masterson. What has caused the global housing crisis - and how can we fix it? World Economic Forum, 2022.
  5. Jenny Schuetz. Fixer-upper: How to repair America’s broken housing systems. Brookings Institution Press, 2022.
  6. Six transformations to achieve the sustainable development goals. Nature sustainability, 2(9):805–814, 2019.
  7. Is housing unaffordable? why isn’t it more affordable? Journal of Economic Perspectives, 18(1):191–214, 2004.
  8. Mary Shaw. Housing and public health. Annu. Rev. Public Health, 25:397–418, 2004.
  9. The effect of house price on fertility: Evidence from hong kong. Economic Inquiry, 48(3):635–650, 2010.
  10. Stefanos Chen. Despite deep rent cuts, essential workers are mostly left out. The New York Times, 2021.
  11. The Guardian. Early career essential workers unable to afford house prices anywhere in sydney and melbourne. The Guardian, 2023.
  12. The Guardian. Covid frontline workers priced out of homeowning in 98britain. The Guardian, 2021.
  13. New york officials failed to address the housing crisis. now what? The New York Times, 2023.
  14. A new urban narrative for sustainable development. Nature Sustainability, 6(2):115–117, 2023.
  15. Victoria Masterson. Are new housing policy reforms working? we need better research to find out. The Brookings Institution, 2022.
  16. Citizen science and the united nations sustainable development goals. Nature Sustainability, 2(10):922–930, 2019.
  17. Big data and big cities: The promises and limitations of improved measures of urban life. Economic Inquiry, 56(1):114–137, 2018.
  18. Street view imagery in urban analytics and gis: A review. Landscape and Urban Planning, 215:104217, 2021.
  19. Urban form and composition of street canyons: A human-centric big data and deep learning approach. Landscape and Urban Planning, 183:122–132, 2019.
  20. Using deep learning and google street view to estimate the demographic makeup of neighborhoods across the united states. Proceedings of the National Academy of Sciences, 114(50):13108–13113, 2017.
  21. Measuring social, environmental and health inequalities using deep learning and street imagery. Scientific reports, 9(1):6229, 2019.
  22. Multimodal deep learning from satellite and street-level imagery for measuring income, overcrowding, and environmental deprivation in urban areas. Remote Sensing of Environment, 257:112339, 2021.
  23. Green streets- quantifying and mapping urban trees with street-level imagery and computer vision. Landscape and Urban Planning, 165:93–101, 2017.
  24. Google street view shows promise for virtual street tree surveys. Urban Forestry & Urban Greening, 21:11–15, 2017.
  25. Assessing street-level urban greenery using google street view and a modified green view index. Urban Forestry & Urban Greening, 14(3):675–685, 2015.
  26. How green are the streets? an analysis for central areas of chinese cities using tencent street view. PloS one, 12(2):e0171110, 2017.
  27. Treepedia 2.0: applying deep learning for large-scale quantification of urban tree cover. In 2018 IEEE International Congress on Big Data (BigData Congress), pages 49–56. IEEE, 2018.
  28. Cataloging public objects using aerial and street-level images - urban trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6014–6023, June 2016.
  29. Take a look around: using street view and satellite images to estimate house prices. ACM Transactions on Intelligent Systems and Technology (TIST), 10(5):1–19, 2019.
  30. The impact of single-container malt liquor sales restrictions on urban crime. Journal of Urban Health, 94:289–300, 2017.
  31. Using street view imagery to predict street-level particulate air pollution. Environmental Science & Technology, 55(4):2695–2704, 2021.
  32. What you see is what you breathe? estimating air pollution spatial variation using street-level imagery. Remote Sensing, 14(14):3429, 2022.
  33. A picture tells a thousand… exposures: opportunities and challenges of deep learning image analyses in exposure science and environmental epidemiology. Environment international, 122:3–10, 2019.
  34. High-resolution air pollution mapping with google street view cars: exploiting big data. Environmental science & technology, 51(12):6999–7008, 2017.
  35. Streetscore-predicting the perceived safety of one million streetscapes. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 779–785, 2014.
  36. Computer vision uncovers predictors of physical urban change. Proceedings of the National Academy of Sciences, 114(29):7571–7576, 2017.
  37. Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning models. Journal of Exposure Science & Environmental Epidemiology, 32(6):892–899, 2022.
  38. Places for play: Understanding human perception of playability in cities using street view images and deep learning. Computers, Environment and Urban Systems, 90:101693, 2021.
  39. Where are the people? counting people in millions of street-level images to explore associations between people’s urban density and urban characteristics. Computers, Environment and Urban Systems, 102:101971, 2023.
  40. ‘big data’for pedestrian volume: Exploring the use of google street view images for pedestrian counts. Applied Geography, 63:337–345, 2015.
  41. The effect of street-level greenery on walking behavior: Evidence from hong kong. Social Science & Medicine, 208:41–49, 2018.
  42. Predicting walking-to-work using street-level imagery and deep learning in seven canadian cities. Scientific reports, 12(1):18380, 2022.
  43. Towards automatic assessment of perceived walkability. In Computational Science and Its Applications–ICCSA 2018: 18th International Conference, Melbourne, VIC, Australia, July 2–5, 2018, Proceedings, Part III 18, pages 351–365. Springer, 2018.
  44. Objective scoring of streetscape walkability related to leisure walking: Statistical modeling approach with semantic segmentation of google street view images. Health & Place, 66:102428, 2020.
  45. Development and validation of a google street view pedestrian safety audit tool. Epidemiology (Cambridge, Mass.), 31(2):301, 2020.
  46. Combining crowdsourcing and google street view to identify street-level accessibility problems. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 631–640, 2013.
  47. Estimating city-level travel patterns using street imagery: A case study of using google street view in britain. PloS one, 13(5):e0196521, 2018.
  48. Social sensing from street-level imagery: A case study in learning spatio-temporal urban mobility patterns. ISPRS journal of photogrammetry and remote sensing, 153:48–58, 2019.
  49. Urban-i: From urban scenes to mapping slums, transport modes, and pedestrians in cities using deep learning and computer vision. Environment and Planning B: Urban Analytics and City Science, 48(1):76–93, 2021.
  50. Predicting bicycling and walking traffic using street view imagery and destination data. Transportation research part D: transport and environment, 90:102651, 2021.
  51. An examination of the spatial coverage and temporal variability of google street view (gsv) images in small-and medium-sized cities: A people-based approach. Computers, Environment and Urban Systems, 102:101956, 2023.
  52. Google street view: Capturing the world at street level. Computer, 43(6):32–38, 2010.
  53. The mapillary vistas dataset for semantic understanding of street scenes. In Proceedings of the IEEE international conference on computer vision, pages 4990–4999, 2017.
  54. A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210, 2023.
  55. An unsupervised approach to geographical knowledge discovery using street level and street network images. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, pages 56–65, 2019.
  56. Urban2vec: Incorporating street view imagery and pois for multi-modal urban neighborhood embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1013–1020, 2020.
  57. Tile2vec: Unsupervised representation learning for spatially distributed data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3967–3974, 2019.
  58. Geography-aware self-supervised learning. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 10161–10170. IEEE, 2021.
  59. Seasonal contrast: Unsupervised pre-training from uncurated remote sensing data. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 9394–9403. IEEE, 2021.
  60. Change-aware sampling and contrastive learning for satellite images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5261–5270, 2023.
  61. Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis. ISPRS journal of photogrammetry and remote sensing, 107:50–63, 2015.
  62. Change detection techniques for remote sensing applications: A survey. Earth Science Informatics, 12:143–160, 2019.
  63. Self-supervised representation learning for remote sensing image change detection based on temporal prediction. Remote Sensing, 12(11):1868, 2020.
  64. Self-supervised multisensor change detection. IEEE Trans. Geosci. Remote. Sens., 60:1–10, 2022.
  65. Self-supervised change detection in multiview remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60:1–12, 2022.
  66. Self-supervised learning for high-resolution remote sensing images change detection with variational information bottleneck. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.
  67. Knowledge-infused contrastive learning for urban imagery-based socioeconomic prediction. In Proceedings of the ACM Web Conference 2023, pages 4150–4160, 2023.
  68. Using open data and open-source software to develop spatial indicators of urban design and transport features for achieving healthy and sustainable cities. The Lancet Global Health, 10(6):e907–e918, 2022.
  69. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  70. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  71. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
  72. Karl Pearson F.R.S. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559–572, 1901.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now