Aerial Geo-Localization

1. Generating data with Google Earth Studio

Generating Aerial Imagery with Google Earth Studio.

Sample Image.

3D tracking data generated by Google Earth Engine, providing camera ground truth location. Camera FOV here was originally set to the default 20 degrees, but should be set to a higher value, which would allow for larger coverage at a lower altitude (which makes more sense for a drone).

Some image pre-processing methods.

2. Relative Localization with Visual Odometry

ORB feature extraction.

Find distinct features in each image which can then be compared and matched with features in next images.

Matching keypoints accross consecutive frames.

3. Absolute Localization from Satellite Imagery

Download satellite imagery through the Copernicus browser.

Idea: assuming we approximately know the size (in meters) of the area covered by the field of view of the onboard camera, divide the Sentinel-2 satellite data into small tiles, for each of which we’ll run template matching with the aerial image.

Satellite data cut into small tiles.

Sample cut tiles.

Results: correct solution appears within top 12 candidates, but not the best based on the template matching score.

Idea: Use ORB keypoints to find the correct satellite tile.

Testing the top 20 candidates with ORB feature matching, we end up with the correct solution.

ORB matching features after RANSAC.

Comparision of metrics with other candidatese.

Location of solution tile on full satellite imag.

Note: why not just skip the template matching and test ORB feature matching across all cut satellite tiles (template matching is sensitive to scale and changes in rotation, whereas ORB is rotation-invariant and more robust to scale changes).

Comparision of metrics with other candidatese.

Location of solution tile on full satellite imag.

Note: we see here that this is likely not a robust solutions, as other incorrect solutions are not far behind in terms of the number of ORB keypont inliers. We could easily imagine considering an image frame with much less distinguishing features than the considered aerial frame here (fully mountainous, worst case mostly featureless image). It should also be tested how robust this approach is for changes in heading, altered terrain in parts of the area, altered lighting conditions, altered color due to seasons, … The full satellite imagery here is also relatively small, although we can assume in the case of drones that we can already have a reasonably close initial guess on gps location. The approach tested here is by no means the best, but is a fun project to test out different computer vision concepts.