Acquire GPS signals from rtl_sdr recording

This one took a few evenings of head-scratching to finally resolve.

I want to be able to record I/Q off an RTL-SDR stick and then post-process it with gnss-sdr. The rtl_sdr software by default dumps interleaved unsigned shorts - that is an (uchar)i_sample, then a (uchar)q_sample, then an (uchar)i_sample, so on and so forth.

gnss-sdr has a couple of different SignalSources and a couple of different item_types and I initially tried varying combinations of those hoping something would work out of the box. Part of my confusion was the existence of an Osmosdr_Signal_Source which uses the RTL-SDR stick and outputs gr_complex without any intermediate conversion. I figured there would be a way to use this SignalSource with raw RTL-SDR dumps but this is not the case. In any case, with the current gnss-sdr infrastructure one must convert the rtl_sdr dump into a file format that gnss-sdr will read.

There are two components to getting an rtl_sdr dump into gnss-sdr. The first is converting the input file from interleaved uchars to gr_complex. The second is properly configuring your input file.

Converting rtl_sdr interleaved uchar into gr_complex

This is fairly well documented in this post at the Amateur Radio stackexchange. There are two methods proposed, one is writing up a flowgraph in GNURadio to convert the file. The second is a short C program posted by Dr. Paul Brewer which performs the exact same operation in 45 lines of code (with comments!) I opted for the C code. Once you compile the program, the process looks like this:

1. Acquire dump using "rtl_sdr /path/to/mydump.bin -s 2.048e6 -f 1575.420e6 &"
2. Convert file using "rtlsdr-to-gqrx /path/to/mydump.bin /path/to/complex.bin"

A third option is to write a radio receiver in GNURadio which uses the RTL-SDR stick to directly write a file in the appropriate format. I will go through this in a future post, as it is a good introduction into gnuradio using only two blocks.

Creating an input file

You can use the "gnss-sdr_GPS_L1_rtlsdr_realtime.conf" as a template and make the following changes:

1. SignalSource.implementation=File_Signal_Source
2. SignalSource.filename=/path/to/complex.bin
3. Change every sampling_frequency to the sampling frequency used in the initial acquisition
4. Change the InputFilter.IF to your IF

Once you make these changes you should be able to decode the file.

Bonus: Have your cake and eat it too!

You can skip the whole mess by modifying two line of code in gnss-sdr_GPS_L1_rtlsdr_realtime.conf which will let you dump the raw I/Q in complex format to file while in parallel running the tracking loop and getting a GPS solution.

To enable this,

1. Set SignalSource.dump=true
2. Set SignalSource.dump_filename=/path/to/dump.bin

Now you can track in realtime and generate a data file to post-process simultaneously!

As always any questions, comments or suggestions please leave a message.

Project Background

A few months ago Paul Breed of Unreasonable Rocket posted on Twitter asking for help from getting GNSS-SDR running on an Intel Compute stick...

I responded and Paul provided an Intel Compute Stick, RTL-SDR stick with a 1PPM TXCO and the bias tee mod and an active antenna. It was my job, then, to make it work in parallel with Paul's investigateions.

The initial concept is simply to record the GPS L1 frequency through flight and post-process position-velocity-altitude after the fact. This would let Paul work out kinks in the hardware ad give us data to play with to tune GNSS-SDR's tracking loops to work in the high acceleration environment. 

Eventually, it would be useful to do this in real time on the rocket. 

So the path forward looks something like this:

Dark blue steps are complete. 

Help is welcome - I'm learning GNSS-SDR but if you know GNSS-SDR and can provide a few answers via email I'd love to hear from you!

Acquiring GPS signals with an RTLSDR dongle using gnss-sdr

Once you've installed gnss-sdr and tested your installation with prerecorded data, the next step is to try and get a live GPS solution. The following is based off this node at gnss-sdr.org and this paper. The paper is a really good mix of theory and implementation and I strongly recommend reading it.

The first thing you need to do is calibrate the front end of your RTL-SDR device. The calibration procedure will do the following

1. Download an ephemeris via SUPL. This provides your computer with advance knowledge of the GPS satellite locations, allowing you to get a quick fix.
2. Search for GPS satellites
3. Calculate the doppler shift assuming your RTL-SDR stick is perfect (trust me, it's not!)
4. Using the predicted doppler shift and some fancy math (see the paper, section 5), calculate
1. The actual sampling frequency
2. The actual IF bias

Before running the calibration you need to update the front-end-cal.conf. Make a copy of the version installed (typically to /usr/local/share/gnss-sdr/conf/front-end-cal.conf). On the first couple of lines you will see a number of GPS lat/lon/alt locations commented out using a semicolon. You need to provide your best estimate at lat/lon/alt, bearing in mind altitude is in meters. The easiest way to do this is use a tool like the aptly named mapcoordinates.net. The other change I had to make was to change both SUPL servers to supl.google.com in place of supl.nokia.com - for some reason the nokia servers were not working. Once complete, you can run the front end calibration like so 

sudo front-end-cal --config_file=/path/to/my/front-end-cal.conf

Your output should look something like this

linux; GNU C++ version 4.9.1; Boost_105500; UHD_003.007.003-0-unknown
Initializing... Please wait.
Logging will be done at /tmp
Use front-end-cal --log_dir=/path/to/log to change that.
Trying to read ephemeris from SUPL server...
SUPL: Trying to read GPS ephemeris from SUPL server...
SUPL: Received Ephemeris for GPS SV 1
SUPL: Received Ephemeris for GPS SV 2
SUPL: Received Ephemeris for GPS SV 3
SUPL: Received Ephemeris for GPS SV 5
SUPL: Received Ephemeris for GPS SV 6
SUPL: Received Ephemeris for GPS SV 7
SUPL: Received Ephemeris for GPS SV 8
SUPL: Received Ephemeris for GPS SV 9
SUPL: Received Ephemeris for GPS SV 10
SUPL: Received Ephemeris for GPS SV 11
SUPL: Received Ephemeris for GPS SV 12
SUPL: Received Ephemeris for GPS SV 13
SUPL: Received Ephemeris for GPS SV 14
SUPL: Received Ephemeris for GPS SV 15
SUPL: Received Ephemeris for GPS SV 16
SUPL: Received Ephemeris for GPS SV 17
SUPL: Received Ephemeris for GPS SV 18
SUPL: Received Ephemeris for GPS SV 19
SUPL: Received Ephemeris for GPS SV 20
SUPL: Received Ephemeris for GPS SV 21
SUPL: Received Ephemeris for GPS SV 23
SUPL: Received Ephemeris for GPS SV 24
SUPL: Received Ephemeris for GPS SV 25
SUPL: Received Ephemeris for GPS SV 26
SUPL: Received Ephemeris for GPS SV 27
SUPL: Received Ephemeris for GPS SV 28
SUPL: Received Ephemeris for GPS SV 29
SUPL: Received Ephemeris for GPS SV 30
SUPL: Received Ephemeris for GPS SV 31
SUPL: Received Ephemeris for GPS SV 32
SUPL: Trying to read Acquisition assistance from SUPL server...
Actual RX Rate: 2000000.000000 [SPS]...
Actual RX Freq: 1575420000.000000 [Hz]...
PLL Frequency tune error 0.000000 [Hz]...Actual RX Gain: 40.200000 dB...
Front-end RAW samples captured
Using Volk machine: sse4_2_64_orc
Searching for GPS Satellites in L1 band...
[ . . . . 5 . 7 8 9 . . . . . . . . . 19 . . . 23 . . . 27 28 . 30 . . ]
Total signal acquisition run time 2.73478 [seconds]
Reference Time:
GPS Week: 851
GPS TOW: 276680 22134.400000
~ UTC: Tue Dec 15 20:51:21 2015
Current TOW obtained from SUPL assistance = 276680
Doppler analysis results:
SV ID Measured [Hz] Predicted [Hz]
5 3000.00 871.77
7 2875.00 524.39
8 2500.00 170.24
9 1125.00 -1220.17
19 6000.00 3734.82
23 -750.00 -3032.99
27 687.50 -1703.53
28 4375.00 2306.86
30 3750.00 1621.38
Parameters estimation for Elonics E4000 Front-End:
Sampling frequency =1999997.27 [Hz]
IF bias present in baseband=2151.87 [Hz]
Reference oscillator error =-1.37 [ppm]
Corrected Doppler vs. Predicted
SV ID Corrected [Hz] Predicted [Hz]
5 848.13 871.77
7 723.13 524.39
8 348.13 170.24
9 -1026.87 -1220.17
19 3848.13 3734.82
23 -2901.87 -3032.99
27 -1464.37 -1703.53
28 2223.13 2306.86
30 1598.13 1621.38GNSS-SDR Front-end calibration program ended.

The important outputs are highlighted above, the sampling frequency and the IF bias. The sampling frequency is nominally 2 MHz but deviation of a few Hz are normal. Baseband is defined as 0 Hz, so when the signal is shifted to baseband if the signal is not exactly centered at 0 Hz this bias is calculated as an IF bias. 

To receive GPS, then, we can copy the gnss-sdr_GPS_L1_rtlsdr_realtime.conf file (typically located in /usr/local/share/gnss-sdr/conf) to another directory and make a few edits. Handily enough there are comments in the file like this

NSS-SDR WITH RTLSDR DONGLES USER MUST SET THE CALIBRATED SAMPLE RATE HERE
; i.e. using front-end-cal as reported here:http://www.cttc.es/publication/turning-a-television-into-a-gnss-receiver/

at each place we need to make an edit. The four edits you need to make are

GNSS-SDR.internal_fs_hz=(your sampling frequency)
SignalSource.sampling_frequency=(your sampling frequency)
InputFilter.sampling_frequency=(your sampling frequency)
InputFilter.IF=(your IF bias)

Once complete, you can run the receiver 

gnss-sdr ---config_file=/path/to/your/gnss-sdr_GPS_L1_rtlsdr_realtime.conf

It will likely take a minute or two to get a fix as a certain amount of ephemeris and almanac data must be downloaded from the satellites. You also must have at least four satellites in view. Here is example output once a fix was found:

Using the KML file generated by gnss-sdr I was able to plot the locations generated in Google Earth

The red dots are individual position fixes. The blue arrow points to the skylight where my GPS antenna is mounted. The road is a two-lane road with no shoulder and the boat in the backyard is 20' long. There is a bias which may be due to the fact that the roof occludes satellites to the east (the four satellites I picked up were, at the time, overhead and eastward). I want to move the antenna up to a (plastic) roof vent in the attic to get a better view of the sky.

Testing gnss-sdr with prerecorded data

Under the documentation section of gnss-sdr.org there is a post entitled "First positioning fix using Galileo" where they document getting a position fix using (at the time) all four Galileo satellites (there are now ten in orbit). They provide not only the position fix but the raw I/Q data which can be fed into gnss-sdr to replicate the fix. This makes a good first test of a gnss-sdr installation.

The page is not incredibly verbose so I will add some verbosity here. First off download and decompress the data from the link provided on the page. Inside the folder you will find the raw data file (.dat) along with a configuration file (.conf) USRP recording output log (.log) and output from their gnss-sdr run (.txt), Google Earth output (.kml) and NMEA messages (.nmea) 

Unfortunately the provided configuration file doesn't work with the current gnss-sdr. However the default configuration file provided with gnss-sdr works just fine. So do the following

1. Copy gnss-sdr.conf from the installation location (typically /usr/local/share/gnss-sdr/conf/gnss-sdr.conf) to the directory containing the raw data file you downloaded
2. Update the line SignalSource.filename to the full path filename of the data file
3. Set GNSS-SDR.SUPL_read_gps_assistance_xml=false

Once you made those two tweaks you can execute the example by running

gnss-sdr --config_file=/path/to/my/gnss-sdr.conf

You should see similar but not identical output to the output provided with the data file. Once the run completes you should have a number of new files. The KML and NMEA files were already described, but you will also see some files named GSDR350w40.15N and GSDR350w40.15O. The file ending in "N" is the navigation solution and the file ending in "O" is the observables. Basically, the navigation solution is your 'fix' and the observables are what information you received from the satellite to generate that fix. Lastly, the gps_ephemeris.xml is an ephemeris which is a description of the satellites' orbit in space.

Setting up gnss-sdr in Kali Linux

Here's instructions on how to set up gnss-sdr in Kali Linux virtual machine. Kali Linux is

... an open source project that is maintained and funded by Offensive Security, a provider of world-class information security training and penetration testing services. In addition to Kali Linux, Offensive Security also maintains the Exploit Database and the free online course, Metasploit Unleashed.

Most importantly, it comes with a modern version of GNURadio and up to date versions of nearly all the dependencies, making getting gnss-sdr installed quite easy.

Installation Instructions

1. Download the Kali Linux VM, or install Kali Linux from ISO.

2. Create a user account and home directory

3. Update apt-get

sudo apt-get update

4. Install cmake

sudo apt-get install cmake

5. Install gnss-sdr dependancies

sudo apt-get install libboost-all-dev gnuradio-dev liblapack-dev libgnutls-openssl-dev libgoogle-glog-dev liblog4cpp5 liblog4cpp5-dev liblog4cpp-doc

6. Install Doxygen

sudo apt-get install doxygen doxygen-gui graphviz

7. Install gnss-sdr (instructions derived from http://gnss-sdr.org/node/45)

git clone https://github.com/gnss-sdr/gnss-sdr
cd gnss-sdr/build $ cmake -DENABLE_OSMOSDR=ON ../
make
sudo make install
make doc
make pdfmanual
make doc-clean

The -DENABLE_OSMOSDR=ON is required to build the rtlsdr drivers!

You should now have a working copy of gnss-sdr; to test it out type gnss-sdr into your console and you should see something like this:

There is an error message because no configuration file was specified, and the default configuration file doesn't point to a valid signal source on my file system (and likely yours, either). No worries we'll fix that later. The message generated here indicates a good installation.

First proof-of-concept: A GPS Fix in Windows using an RTL-SDR stick

Using a $20 RTL-SDR stick with 1ppm TXCO and a simple mod to power an active GPS antenna, it is possible to download and decode GPS signals in real time.

Software

GNSS-SDRLIB:  https://github.com/taroz/GNSS-SDRLIB

RTKLIB: http://www.rtklib.com/

Both packages are Open Source, with a default build target is Windows.

Both packages are Open Source (GPL2 and BSD 2-clause, respectively). The default build target for both packages is Windows, although RTKLIB has been compiled under Linux.

Please note that GNSS-SDRLIB is not to be confused with GNSS-SDR!

Resources

Useful presentation by the author of GNSS-SDRGUI for a summer school course. Also check the manuals included with GNSS-SDRLIB and RTKLIB.

Step-By-Step Implementation

1. Ensure RTL-SDR stick is working in Windows. If your driver is not working, try using the Zadig driver installation method outlined here.
2. Install GNSS-SDRLIB and RTKLIB to any convenient directory
3. Open GNSS-SDRGUI and select the following options
1. Input Type: RTL-SDR
2. [x] RTCM MSM, Port 9999
3. Change "output interval" dropdown to 10 hz
4. [x] Plot Tracking
5. [x] All GPS, GLOSNASS, Galileo satellites 
6. (optional) enter approximate lat/lon into MISC and click the "..." button to get  current satellite locations in relation to your location.
4. Click "Start", a number of command consoles will open then close for each satellite being tracked.
5. Click "M" for log
6. Now, open RTKNAVI
1. Click on the "I" button
1. check "rover", type TCP Client, format RTCM3
2. click OPT button and set address to "localhost" and port to "9999"
3. click OK
4. Click OK
2. Click on the "start" button
3. Within a few seconds you should see satellites in the Rover:Base SNR pane
4. Once a solution exists it will update lat/lon in the left pane
5. Click "Plot" to generate a plot of the random walk of lat/lon over time

This is what your GNSS-SDRGUI should look like:

This is what your RTKNAVI input should look like:

If everything is working you should get a GPS solution, as shown in this video

Next Actions

Try using GNSS-SDR and/or GNURadio to decode the GNSS signal; this would provide native, Linux compatible headless execution and a cursory Google suggests this has been done successfully with the RTL-SDR stick. 

First Post

This blog is designed to document as I learn about using software-defined radios to decode GPS signals. I plan on providing references to existing projects and papers ("theory") along with my actual implementations and experiences ("practice") which any properly motivated individual could replicate for their own purposes.

For most users consumer level GPS devices are sufficient for most applications, but there are niche applications where consumer level hardware fails to satisfy and professional or unrestricted hardware are unattainable.

There is also the educational aspect. I've had my ham radio license for almost 10 years and didn't use it much beyond the first year, but the recent development of ubiquitous software-defined radios and fast, cheap computers makes learning about radios accessible in a way that involves compilers instead of solder. Nothing against solder! Working with GPS in particular also brings to light satellite communications, orbital mechanics, Doppler shifts, and other fun physical phenomenon. My children are not quite old enough to write code or solder properly, but talk to them about listening to satellites and you can just see the awe on their faces, when you explain basic triangulation and they start drawing their own triangulation intersection circles, there's a level of practicality they can connect with, which I can then develop into a curiosity about radios, and code, and solder.