3D Reconstruction from public webcams

Abstract: We investigate the possibility of 3D scene reconstruction from two or more overlapping webcam streams. A large, and growing, number of webcams observe places of interest and are publicly accessible. The question naturally arises: can we make use of this free data source for 3D computer vision? It turns out that the task to reconstruct scene structure from webcam streams is very different from standard structure-from-motion (SfM), and conventional SfM pipelines fail. In the webcam setting there are very few views of the same scene, in most cases only the minimum of two. These viewpoints often have large baselines and/or scale differences, their overlap is rather limited, and besides unknown internal and external calibration also their temporal synchronisation is unknown. On the other hand, they record rather large fields of view continuously over long time spans, so that they regularly observe dynamic objects moving through the scene. We show how to leverage recent advances in several areas of computer vision to adapt SfM reconstruction to this particular scenario and reconstruct the unknown camera poses, the 3D scene structure, and the 3D trajectories of dynamic objects.