Geotagging Images from a Telemetry Log

IGIS Tech Notes describe workflows and techniques or using geospatial science and technologies in research and extension. They are works in progress, and we welcome feedback and comments below.

Summary

This tech note describes how to geotag photos taken from a camera on a drone, when the camera lacks an integrated GPS module (such as the GoPro). This can be done using the telemetry log from the drone, provided you can link the telemetry log and the images using the timestamp.

Intro

If the sensor (camera) on your drone does not have an integrated GPS unit (i.e., GoPro cameras), then the images will not be geotagged. This creates a problem because while Pix4D and other stitching programs (e.g., Agisoft PhotoScan) can usually work with non-geotagged images, the results may not be great. You’ll get far better (and faster) results if you geotag the images before stitching them using the drone’s telemetry log.

You can use the drone telemetry log to geotag images. The telemetry log records the location of the drone during the flight at a high temporal frequency. There are utilities that can insert these locations into the header of the JPG files. For this to work however, the camera timestamps must be ‘synced’ to match the GPS time that the drone uses. This should be simple, but it isn’t. For one thing, digital cameras like the GoPro Hero4 have far less accurate clocks than the super-accurate atomic clocks used in GPS chips. You have to either sync the clocks to be the same, or find a way to determine the difference between camera time and GPS time, which you can then use to correct the image timestamps.

Sample Workflow: 3DR Solo & GoPro

There is more than one way to geotag images using the telemetry log from a drone. The following technique^[1] can been used to geotag images from a Hero4 GoPro RGB camera onboard a 3DR Solo.

Overview

Before the flight:
1. Set the GoPro camera time to match GPS time within a minute or less.
2. Take a picture of ‘GPS Time’ displayed on a PC or tablet to get the offset between the GPS time and the timestamp saved in the camera. This photo will have the GPS time in the image, as well as the camera time saved in the EXIF header of the JPG file. You can then calculate the offset between GPS time and camera time.
Fly the mission
Transfer the images from the camera to your computer
Download the telemetry log from the drone to your computer
Compute the time offset between the camera time and GPS time
Use the free command line tool EXIFTool to insert GPS coordinates into the images using the telemetry log.

Programs You’ll Need

WinSCP - an FTP program for Windows to get Solo telemetry logs (any FTP utility that supports UDP connections should work)
Mission Planner - Windows utility to convert tlog files to gpx files
ExifTool - awesome Windows command line tool. After you download and unzip the Exiftool, rename the "exiftool(-k).exe" to "exiftool.exe" for command-line use. Copy or move it to a directory on the path.
ExifToolGUI

Step 1. Set Camera Time

Tip: The GoPro internal clock is known to drift, particularly if it’s been turned off for a while. So it is recommended you set the GoPro time and compute the GPS time offset shortly before your flight.

Before the flight, set the time on the GoPro to match the time on your phone (phones generally have time automatically set using servers, and they tend to be accurate to within a few seconds. Setting the time on your GoPro can be done manually using the buttons on the camera, or by pairing the GoPro to your phone or tablet (in which case time is synced automatically). In this workflow, it only has to be accurate within a minute or so (because we’re going to get the time offset in seconds).

Step 2. Take a Picture of the GPS Time Displayed on Mission Planner or ClockSync

Option 1. On Android, download an app called 'ClockSync' to see how far off your phone time is from 'atomic time'). Open ClockSync on your tablet (or another app that displays atomic time), and display Atomic Time. Turn on the GoPro and take a picture of the tablet's screen. To get the tablet focused, have the GoPro away about 4-5 feet from the screen. You may want to preview the image to make sure the GPS time is visible.

Option 2. Download and install Mission Planner onto your Windows laptop. Open Mission Planner.

Turn the Solo and its controller on
Connect your Windows laptop to the Solo’s WiFi
Connect Mission Planner to the Solo
1. Select ‘UDP’ from the drop down box in the upper right
2. Select the ‘Connect’ button
3. Accept the default port
Once Mission Planner is connected to your drone, you should see the GPS time displayed in the main screen. The Drone’s GPS Time is displayed on the left pane (see Figure below).

Step 3. Fly the mission

Record (e.g., on a sheet of paper) the time when your flights start and end (i.e., when you turn the Solo on and off). This will help you identify the correct telemetry log file by matching the ‘file modified’ date with the time the flight ended.

Step 4. Get the photos from the camera

Download the files from the GoPro to your laptop by removing the memory card, using a USB cable, or the GoPro Connect app.

Step 5. Compute the GoPro-GPS Time offset

Find one of the GoPro photos you took of the ClockSync or Mission Planner screen displaying GPS time. Preview the photo to see the GPS time when the picture was taken. Use the ExifToolGUI to get the EXIF DateTimeOriginal tag. Manually calculate the time difference.

Example:
Image: GOPR5099.JPG (taken 20 minutes before the flight)
GPS Time displayed in the photo: 12:45:07
EXIF DateTimeOriginal Tag: 2017:04:19 12:44:48
Difference: camera is behind by 19 seconds

For reliability, you can compute the camera-GPS offset using multiple images before and after the flight. The offset should be the same or very similar.

Step 6. Get the log files off the Solo

Turn the controller and Solo on, and connect your laptop to the Solo’s Wifi
Launch WinSCP (or another FTP program that supports the UDP protocol)
On the Solo, navigate to the ‘/logs’ directory and look for files named solo.tlog.n, where n is a number. The Solo saves the last 10 flight logs. Download them all to your laptop.

Tip: if you make a lot of flights, you'll need to download the log files throughout the day (i.e., after every 8th flight) to prevent them from being overwritten. Remember the Solo starts a new flight log every time it is turned on.

Step 7. Find the log file for your flight(s)

The Solo saves the 10 most recent tlog files in its memory. solo.tlog is the latest, while solo.tlog.10 is the oldest.

Note that telemetry logs are created whenever you turn the Solo on, whether or not the Solo actually flew. A relatively large log file simply means the Solo was turned on for a while.

To help you find the right telemetry log, go to the images for your flight and look at the date modified for the last image from that flight. Then look for a telemetry log file (*.tlog) that was also saved shortly after that image (however long it took you to turn off the Solo after the flight).

To confirm you have the correct telemetry log, you can open it in Mission Planner and play it.

Step 8. Convert the telemetry log from tlog format to gpx format

The EXIFTool (which we’ll use next step) can not read tlog files, so we need to convert them to gpx (another more common format for telemetry logs). In Mission Planner, open the tlog file, select the file, then click the convert button.

Step 9a. Use the exiftool to add geostamps

The EXIFtool is a command line tool for viewing or manipulating the EXIF info (metadata) saved in the header of JPG and TIF files. Exiftool has arguments that can be used to insert GPS coordinates into JPG based on a telemetry log.

-geosync is the value to be added to the DateTimeOriginal values from the image file EXIF info. Use a negative value of seconds if the camera clock was ahead of GPS time. Otherwise use a positive offset.

## General syntax
exiftool -geotag track.gpx -geosync=+19 my_picts_dir

## Example
exiftool -geotag "C:\Pix4D\2017-04-19_ElkusRanch\solo.tlog.gpx" -geosync=+19 "C:\Pix4D\2017-04-19_ElkusRanch\Images\GoPro_Flight01_WaterTower\GeoStamped"

Your images should now have approximately correct geostamps, which will help your stitching software find adjacent images and find matching keypoints.

Acknowledgements

This workflow was adapted from Eric Compas, Director of the GIS Center at the University of Wisconsin-Whitewater.