Jerry Zhang, Aaron Hallquist, Eric Liang, Eric Tzeng, and Avideh Zakhor
Image based localization is an important problem in many civilian and military applications. The basic idea is to match a user generated query image against a database of geo-tagged images with known 6 degrees of freedom poses. Once a correct match is retrieved, we use the correspondence between the two images to project data onto the query image in a pixel accurate fashion. In this project, we develop an approach to large scale image retrieval for user localization in urban environment by taking advantage of coarse position estimates available, e.g. via GPS or cell tower triangulation, on many mobile devices today. By partitioning the large image database for a region into a number of overlapping geographic cells each with its own prebuilt search and retrieval structure, we overcome the performance degradation faced by many city-scale retrieval systems as a result of increased database size. Once a correct image match is made, the point to point correspondence between query and retrieved image is used to compute a homography transformation which can be used to transfer pixel accurate tag information onto the query image. We demonstrate retrieval results over a ~12,000 image database covering a 1 km2 area of downtown Berkeley and illustrate tag transfer results over the same dataset.
Sample Results
Sample results from our tagging/retrieval system on images taken in downtown Berkeley.
| Query Set | Scene | Camera/Phone | Orientation | Focal Length | Count | Download |
|---|---|---|---|---|---|---|
| 1 | 1 | Nikon D40x | Landscape | Fixed | 100 | zip |
| 2 | 2 | Nikon D40x | Landscape | Fixed | 65 | zip |
| 3 | 3 | Nikon D40x | Landscape | Varying | 84 | zip |
| 4 | 4 | HTC Droid Incredible | Portrait | Fixed | 112 | zip |
| 5h | 1 | HTC Droid Incredible | Landscape | Fixed | 100 | zip |
| 5v | 1 | HTC Droid Incredible | Portrait | Fixed | 100 | zip |
Download Software
Related Publications
- J. Zhang, A. Hallquist, A. Zakhor, "Location-Based Image Retrieval for Urban Environments," accepted to ICIP 2011, Brussels, Belgium, September 11-14, 2011. [Adobe PDF]
Related Talks
- Berkeley EECS Annual Research Symposium - "Augmenting Reality Via Client/Cloud Platforms", by Avideh Zakhor, February 17, 2011. [Video] (begins around 1:16, lasts 30 minutes)
Related Work
- Automated 3D Modeling of Building Interiors
- Fast automated outdoor modeing of urban environments
- 3D modeling using airborne LiDAR and EO sensors