Image-Based Localization for Augmented Reality Applications

Jerry Zhang, Aaron Hallquist, Eric Liang, Eric Tzeng, and Avideh Zakhor

Image based localization is an important problem in many civilian and military applications. The basic idea is to match a user generated query image against a database of geo-tagged images with known 6 degrees of freedom poses. Once a correct match is retrieved, we use the correspondence between the two images to project data onto the query image in a pixel accurate fashion. In this project, we develop an approach to large scale image retrieval for user localization in urban environment by taking advantage of coarse position estimates available, e.g. via GPS or cell tower triangulation, on many mobile devices today. By partitioning the large image database for a region into a number of overlapping geographic cells each with its own prebuilt search and retrieval structure, we overcome the performance degradation faced by many city-scale retrieval systems as a result of increased database size. Once a correct image match is made, the point to point correspondence between query and retrieved image is used to compute a homography transformation which can be used to transfer pixel accurate tag information onto the query image. We demonstrate retrieval results over a ~12,000 image database covering a 1 km2 area of downtown Berkeley and illustrate tag transfer results over the same dataset.


Sample Results

Sample results from our tagging/retrieval system on images taken in downtown Berkeley.

Query SetSceneCamera/PhoneOrientationFocal LengthCountDownload
1 1Nikon D40x Landscape Fixed 100zip
2 2Nikon D40x Landscape Fixed 65 zip
3 3Nikon D40x Landscape Varying84 zip
4 4HTC Droid Incredible Portrait Fixed 112zip
5h 1HTC Droid Incredible Landscape Fixed 100zip
5v 1HTC Droid Incredible Portrait Fixed 100zip


Download Software


Related Publications

  • J. Zhang, A. Hallquist, A. Zakhor, "Location-Based Image Retrieval for Urban Environments," accepted to ICIP 2011, Brussels, Belgium, September 11-14, 2011. [Adobe PDF]


Related Talks

  • Berkeley EECS Annual Research Symposium - "Augmenting Reality Via Client/Cloud Platforms", by Avideh Zakhor, February 17, 2011. [Video] (begins around 1:16, lasts 30 minutes)


Related Work