Real-Time Maps With a Raspberry Pi, Golang, and HERE XYZ

By now, you’ve probably seen a few of my tutorials related to Raspberry Pi and location data. I’m a big fan of these small Internet of Things (IoT) devices and have written tutorials around WLAN positioning with Golang and GPS positioning with Node.js.

I wanted to continue down the Golang route and do a tutorial around GPS positioning and storing that data in HERE XYZ to be viewed in real-time. In other words, have a Raspberry Pi collect GPS data with Golang, push it to HERE XYZ, and view it on some web client in real-time by querying the data in HERE XYZ.

We’re going to see how to accomplish all of this and if everything goes smooth, we might end up with something like the following:

The above image is a result of what I got when I drove around with the Raspberry Pi in my car.

The Requirements

There are a few requirements that must be met to be successful with this tutorial. Variations could probably be used, but I can only confirm what’s worked for me.

In terms of hardware, I’m using the following:

• Raspberry Pi Zero W
• NEO 6M GPS
• Active External Antenna
• U.FL Adapter for External Antenna

You could probably get away with using any of the Raspberry Pi models that have WiFi. If you have an LTE module for your Raspberry Pi, even better, because I was just tethering to my phone in the car for access to the internet.

The GPS module is the same module that I referenced in my previous tutorial where it was directly connected to my computer. You’ll need an antenna for it; otherwise, it could take days to get a fix on the satellites.

In terms of software and services, you’ll need the following:

• Golang installed on your host development computer
• A free HERE account

You’ll need a HERE account to configure HERE XYZ for storing your location data. Since Golang compiles to native binaries, you just need it on your development computer. If you try to develop on your Raspberry Pi, the build process might be slow due to the weaker hardware specifications.

Collecting GPS Data on the Raspberry Pi With Golang

If you’ve read my previous tutorial titled, Reverse Geocoding NEO 6M GPS Positions with Golang and a Serial UART Connection, some of this will look familiar. It doesn’t hurt to have a refresher though.

Before you create a new project, execute the following commands:

go get github.com/paulmach/go.geojson
go get github.com/adrianmo/go-nmea
go get github.com/jacobsa/go-serial/serial

The above commands will get our necessary packages to save us a lot of development time. The serial package will allow us to access the serial connection between Raspberry Pi and GPS module, the go-nmea package will allow us to parse the raw GPS data into something we can understand, and the go.geojson package will allow us to create GeoJSON formatted data to be sent to HERE XYZ.

With the necessary packages available, create a main.go file somewhere within your $GOPATH and add the following boilerplate code:

package main

import (
    "bufio"
    "bytes"
    "flag"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "net/url"
    geojson "github.com/paulmach/go.geojson"
    "github.com/adrianmo/go-nmea"
    "github.com/jacobsa/go-serial/serial"
)

type HereDev struct {
    Token   string `json:"token"`
    SpaceId string `json:"space_id"`
}

func NewGeoJSON(latitude float64, longitude float64) ([]byte, error) {}
func (here *HereDev) PushToXYZ(data []byte) ([]byte, error) {}
func main() {}

Because we plan to access the HERE XYZ API and that API requires a space id and token, we’re creating a data structure to hold that information. We’ll be creating a NewGeoJSON function to create GeoJSON formatted data and a PushToXYZ function to push our GeoJSON data to HERE XYZ. However, we’ll worry about that in a bit.

Let’s take a look at our main function:

func main() {
    options := serial.OpenOptions{
        PortName:        "/dev/ttyS0",
        BaudRate:        9600,
        DataBits:        8,
        StopBits:        1,
        MinimumReadSize: 4,
    }
    serialPort, err := serial.Open(options)
    if err != nil {
        log.Fatalf("serial.Open: %v", err)
    }
    defer serialPort.Close()
    reader := bufio.NewReader(serialPort)
    scanner := bufio.NewScanner(reader)
    for scanner.Scan() {
        fmt.Println(scanner.Text())
    }
}

The above code is a good starting point. What we’re saying is to open the /dev/ttyS0 serial port on the Raspberry Pi and use a baud rate of 9600. For any data that comes in, we’re going to access it with a Scanner and print it out. We’re using a Scanner because we want complete lines of data, not bits and pieces.

So what does the Raspberry Pi and NEO 6M wire configuration look like? Mine looks like the following:

In case you’re new to Raspberry Pi and the GPIO pin configurations, or it is difficult to see in the image, my wire configuration is like the following:

• GPS VCC to RPI 5V (Red)
• GPS GND to RPI GND (Black)
• GPS TX to RPI RXD (Green)
• GPS RX to RPI TXD (White)

If you don’t have a pin template from Adafruit or similar, get one because it is a worthy investment since the pins are not labeled on the hardware itself.

Just having connected the GPS module and the Raspberry Pi isn’t good enough. The serial port still needs to be enabled through the software settings of the Raspberry Pi. You can use the raspi-config tool in Raspbian to do this. If you’re unfamiliar, please check out my previous tutorial titled, Interacting with a NEO 6M GPS Module using Golang and a Raspberry Pi Zero W, which includes images and thorough details.

Now that we’re collecting data from the serial connection, we need to parse it. This can be easily accomplished with the NMEA library that we had installed. Let’s update the main function to look like the following:

func main() {
    options := serial.OpenOptions{
        PortName:        "/dev/ttyS0",
        BaudRate:        9600,
        DataBits:        8,
        StopBits:        1,
        MinimumReadSize: 4,
    }
    serialPort, err := serial.Open(options)
    if err != nil {
        log.Fatalf("serial.Open: %v", err)
    }
    defer serialPort.Close()
    reader := bufio.NewReader(serialPort)
    scanner := bufio.NewScanner(reader)
    for scanner.Scan() {
        s, err := nmea.Parse(scanner.Text())
        if err == nil {
            if s.DataType() == nmea.TypeGGA {
                data := s.(nmea.GGA)
                if data.Latitude != 0 && data.Longitude != 0 {
                    // Print out position data...
                }
            }
        }
    }
}

We’re taking the text from the Scanner and we’re parsing it. My GPS unit provides GPGGA formatted strings, but yours might differ. Figure out what yours returns and check for it before you try to print out latitude and longitude information. You’ll want to make sure you’re parsing it correctly.

Sending GeoJSON Data to HERE XYZ With Golang

We have latitude and longitude information, but we can’t really send that directly to HERE XYZ. Instead, we need to create a GeoJSON compliant object to be sent instead.

Within your main.go file, create the following function:

func NewGeoJSON(latitude float64, longitude float64) ([]byte, error) {
    featureCollection := geojson.NewFeatureCollection()
    feature := geojson.NewPointFeature([]float64{longitude, latitude})
    featureCollection.AddFeature(feature)
    return featureCollection.MarshalJSON()
}

The above function will take a latitude and longitude coordinate and create a new feature collection from it. If we wanted to get a little more extravagant, we could create tags for our data and other properties, but we won’t for this example.

After we parse our data, we can now make use of the NewGeoJSON function like so:

for scanner.Scan() {
    s, err := nmea.Parse(scanner.Text())
    if err == nil {
        if s.DataType() == nmea.TypeGGA {
            data := s.(nmea.GGA)
            if data.Latitude != 0 && data.Longitude != 0 {
                gjson, err := NewGeoJSON(data.Latitude, data.Longitude)
                if err != nil {
                    log.Fatal(err)
                }
                fmt.Println(string(gjson))
            }
        }
    }
}

I actually wrote a similar tutorial titled, How to Work with GeoJSON Data in Golang for HERE XYZ, if you want to get more information on working with GeoJSON data in Golang.

This is where we start to get involved with HERE XYZ. Our data is ready to go, so we need to be able to send it from our Raspberry Pi as it is collected and store it on HERE XYZ. We’re going to use a mixture of the XYZ CLI and web portals for this. This should be done on your host computer, not the Raspberry Pi.

Install the XYZ CLI by executing the following:

npm install -g @here/cli

After installing the CLI, your HERE account needs to be registered to it. This can be done by executing the following:

here configure account

Make sure to provide your credentials when prompted. This information will be stored securely and locally on your computer.

With the CLI configured, a new XYZ project needs to be created. To do this, execute the following:

here xyz create --title "raspberry-pi-project" --message "data from the raspberry pi"

The above command will create a new XYZ space that can be accessed from the CLI or from the web dashboards. Make note of the space id that is returned because it is necessary within the Golang code as well as our interactive map code.

We’re also going to need an access token for our space. Open the token dashboard and choose to generate a new token for your space.

Make sure you take note of the token because it too will be used with the space id in the Go code and the interactive map code. At any point, if you wish to view the data you’ve uploaded, you can visit the Data Hub in XYZ Studio.

Now that we have a space id and an access token, we can continue our project development.

Add the following to the main.go file:

func (here *HereDev) PushToXYZ(data []byte) ([]byte, error) {
    endpoint, _ := url.Parse("https://xyz.api.here.com/hub/spaces/" + here.SpaceId + "/features")
    request, _ := http.NewRequest("PUT", endpoint.String(), bytes.NewBuffer(data))
    request.Header.Set("Content-Type", "application/geo+json")
    request.Header.Set("Authorization", "Bearer "+here.Token)
    client := &http.Client{}
    response, err := client.Do(request)
    if err != nil {
        return nil, err
    } else {
        return ioutil.ReadAll(response.Body)
    }
}

The above PushToXYZ function will take our GeoJSON data and do an HTTP request against the HERE XYZ API. The response isn’t too useful to us for this example, but we’ll return it anyways.

For more information on making HTTP requests with Golang, check out my previous tutorial titled, Consume RESTful API Endpoints within a Golang Application.

Go back into the main function and update it to the following:

func main() {
    options := serial.OpenOptions{
        PortName:        "/dev/ttyS0",
        BaudRate:        9600,
        DataBits:        8,
        StopBits:        1,
        MinimumReadSize: 4,
    }
    serialPort, err := serial.Open(options)
    if err != nil {
        log.Fatalf("serial.Open: %v", err)
    }
    defer serialPort.Close()
    reader := bufio.NewReader(serialPort)
    scanner := bufio.NewScanner(reader)
    here := HereDev{Token: "ACCESS_TOKEN_HERE", SpaceId: "ACCESS_TOKEN_HERE"}
    for scanner.Scan() {
        s, err := nmea.Parse(scanner.Text())
        if err == nil {
            if s.DataType() == nmea.TypeGGA {
                data := s.(nmea.GGA)
                if data.Latitude != 0 && data.Longitude != 0 {
                    gjson, err := NewGeoJSON(data.Latitude, data.Longitude)
                    if err != nil {
                        log.Fatal(err)
                    }
                    here.PushToXYZ(gjson)
                }
            }
        }
    }
}

In the above code, we are initializing our here variable and we are making use of it after we’ve generated our GeoJSON data. This Golang example has one job and it is to push data to XYZ.

To build our application for use on a Raspberry Pi, we can run the following:

GOOS=linux GOARCH=arm GOARM=5 go build

The above command will build a binary meant for ARM, which is what the Raspberry Pi uses. For more on cross-compiling, check out a previous tutorial I wrote.

Collect GPS Data at Boot

Our application should work if we transfer it to a Raspberry Pi. However, we might want to make it so it auto-starts when we turn on the Raspberry Pi. To do this, we need to create a service.

Add the following to a tracking.service file in the /etc/systemd/system/path of your Raspberry Pi:

[Unit]
Description=GPS-Project
Documentation=
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/gps-project

[Service]
WorkingDirectory=/usr/local/

User=root
Group=staff

ExecStart=/usr/local/bin/gps-project

Restart=always

LimitNOFILE=65536

TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

The above service assumes a few things. It assumes that your binary is called gps-project and it exists in the /usr/local/bin path. It also assumes the owner is root and the group is staff. Finally, it has to have execute permissions. Feel free to adjust the above service as you feel necessary.

With the binary and service in place, execute the following:

systemctl enable tracking.service

Now, the binary should run in the background every time the Raspberry Pi starts. This means that if you’re like me, you can hop in your car, plug the Raspberry Pi into a USB port, and have it start collecting data on its own without any host computer involved.

Displaying Route Data From HERE XYZ on a Map in Real-Time

At this point in time, we should have a Raspberry Pi sending GPS data to HERE XYZ. Now, we need a way to visualize that data on a map, preferably in near real-time.

We need to create a separate project. Create an index.html file with the following boilerplate HTML markup:

<html>
    <head>
        <link rel="stylesheet" href="https://unpkg.com/leaflet@1.4.0/dist/leaflet.css" />
    </head>
    <body style="margin: 0">
        <div id="map" style="width: 100vw; height: 100vh"></div>
        <script src="https://unpkg.com/leaflet@1.4.0/dist/leaflet.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.18.0/axios.min.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/geojson/0.5.0/geojson.min.js"></script>
        <script>
            const start = async () => {
                const tiles = "https://1.base.maps.api.here.com/maptile/2.1/basetile/newest/normal.day/{z}/{x}/{y}/512/png8?app_id={appId}&app_code={appCode}";
                const map = new L.Map("map", {
                    center: [37, -121],
                    zoom: 11,
                    layers: [L.tileLayer(tiles, { appId: "HERE_APP_ID", appCode: "HERE_APP_CODE" })]
                });
                var polyline, sourceMarker, destinationMarker;
                setInterval(async () => {}, 5000);
            }
            start();
        </script>
    </body>
</html>

The above code will use Leaflet.js and the HERE Map Tile API. To protect my privacy, I chose to use basetiles which include no street label information. You can pick whatever works best for you.

Make sure to replace the app id and app code values with those of your own.

So far, our code will just show a map on the screen. Now, we want to draw our path of travel and center on it. We’re even going to update the map automatically to look for new changes.

Let’s take a look at our setInterval usage:

setInterval(async () => {
    const response = await axios({
        method: "GET",
        url: "https://xyz.api.here.com/hub/spaces/" + "XYZ_SPACE_ID" + "/search",
        params: {
            access_token: "XYZ_TOKEN"
        }
    });
    map.eachLayer(layer => {
        if(!layer._url) {
            layer.remove();
        }
    });
    const polylineData = [];
    response.data.features.forEach(feature => {
        let position = feature.geometry.coordinates;
        polylineData.push({ lat: position[1], lng: position[0] });
    });
    polyline = new L.Polyline(polylineData, { weight: 5 });
    sourceMarker = new L.Marker(polylineData[0]);
    destinationMarker = new L.Marker(polylineData[polylineData.length - 1]);
    polyline.addTo(map);
    sourceMarker.addTo(map);
    destinationMarker.addTo(map);
    const bounds = new L.LatLngBounds(polylineData);
    map.fitBounds(bounds);
}, 5000);

The above code says that every 5000ms, we will make an HTTP request to the HERE XYZ API. Instead of adding data we are querying it. After doing our HTTP request, we clear the map. Remember we’re repeating this so we don’t want old lines and markers to appear every 5000ms.

After clearing the map, we look at the GeoJSON that comes back and draw new polylines and markers. We’re pretty much just connecting each of the points that are coming back in our XYZ query.

Conclusion

You just saw quite a few things in this tutorial. We saw how to collect GPS position data on a Raspberry Pi, parse it, and upload it to HERE XYZ. In my circumstance, I had my Raspberry Pi Zero W in my car and it was publishing my position data in real-time as I drove around. We then saw how to query HERE XYZ for our data and display it on a map. If we wanted to, someone could view the map and see us driving around in real-time.

This tutorial was an upgrade to my previous tutorial titled, Tracking a Raspberry Pi with WLAN and Golang, then Displaying the Results with HERE XYZ, which used WLAN instead of GPS. We also made use of the HERE XYZ API instead of manually processing our data.