Enhancing Worldwide Image Geolocation by Ensembling Satellite-Based Ground-Level Attribute Predictors

Abstract: We examine the challenge of estimating the location of a single ground-level image in the absence of GPS or other location metadata. Currently, geolocation systems are evaluated by measuring the Great Circle Distance between the predicted location and ground truth. Because this measurement only uses a single point, it cannot assess the distribution of predictions by geolocation systems. Evaluation of a distribution of potential locations (areas) is required when there are follow-on procedures to further narrow down or verify the location. This is especially important in poorly-sampled regions e.g. rural and wilderness areas. In this paper, we introduce a novel metric, Recall vs Area (RvA), which measures the accuracy of estimated distributions of locations. RvA treats image geolocation results similarly to document retrieval, measuring recall as a function of area: For a ranked list of (possibly discontiguous) predicted regions, we measure the area required for accumulated regions to contain the ground truth coordinate. This produces a curve similar to a precision-recall curve, where "precision" is replaced by square kilometers area, enabling evaluation for different downstream search area budgets. Following from this view of the problem, we then examine an ensembling approach to global-scale image geolocation, which incorporates information from multiple sources, and can readily incorporate multiple models, attribute predictors, and data sources. We study its effectiveness by combining the geolocation models GeoEstimation and the current state-of-the-art, GeoCLIP, with attribute predictors based on Oak Ridge National Laboratory LandScan and European Space Agency Climate Change Initiative Land Cover. We find significant improvements in image geolocation for areas that are under-represented in the training set, particularly non-urban areas, on both Im2GPS3k and Street View images.