GNU Code Coverage on Embedded Targets

Code coverage IN SPACE! Well, FOR space. Here's how to ensure code coverage testing on embedded devices using free tools with the confidence to go into space.

Erich Styger

Jun. 20, 17 · Tutorial

Likes (3)

Comment

Save

9.7K Views

For a research project, we are going to send a satellite with an embedded ARM Cortex microcontroller into space early next year. Naturally, it has to work the first time. As part of all the ESA paperwork, we have to prove that we tested the hardware and software thoroughly. One piece of the that is to collect and give test coverage evidence. And there is no need for expensive tools: Free-of-charge Eclipse and GNU tools can do the job for a space mission

Eclipse with Coverage Views

A while back (see Code Coverage for Embedded Target with Eclipse, gcc, and gcov) I wrote tutorials about how to use GNU Coverage tools (gcov) with Eclipse for development on embedded ARM Cortex-M devices. Since then, new Eclipse and ARM toolchain versions have been released. As I recently received many questions and requests on how to make it work with the MCUXpresso IDE, here we go!

This tutorial is about how to collect coverage information using GNU gcov with the MCUXpresso IDE (Eclipse Neon with GNU Tools for ARM Embedded 5 toolchain). It describes what has to be added to projects to enable coverage, what compiler and linker settings have to be used and how to retrieve coverage information with semihosting.

The article does *not* go into details how coverage works. For this, please check the links at the end of this article, and especially https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/.

The sources of the example project used in this tutorial can be found on GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Preconditions

This article assumes you have all the needed tools for using GNU coverage installed. You need

Eclipse IDE: I’m using MCUXpresso IDE v10.0.0 (build 344), which is Eclipse Neon-based.
GNU Toolchain for ARM: I’m using the GNU Tools for ARM Embedded 5 – Q3 which is installed with the MCUXpresso IDE, or use the one from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm.
GCov Eclipse plugins: see “Adding GNU Coverage Tools to Eclipse”.
Debug probe and debug connection capable of doing file I/O semihosting: This means that the application can open, read, and write files on the host through the debug connection. In this article, I’m using the Segger J-Link debug connection.
A functional and working project. I recommend you start playing with an empty or very simple project. In this tutorial, I’m using a project created by the ‘new project wizard’ in the IDE for the NXP FRDM-K64F board.

I’m using the MCUXpresso IDE and its included toolchain in this tutorial, but I describe things in a generic way so it should be applicable to any other configuration you have.

Semihosting With File I/O

Because the approach presented in this article depends on semihosting file I/O, I have found that not every library/debug connection configuration works for me.

With the ARM gcc 5 Q3-2015 version, I was able to use semihosting file I/O with the newlib-nano, but not with the newlib library. I have not tried the gcc 6 version yet. The RedLib library in MCUXpresso IDE cannot be used, as it does not include the necessary gcov libraries.

At the time of writing this article, I have tried the following version of the P&E Eclipse GDB plugin, but I was not able to get semihosting file I/O working, but a future version might be able to support it:

GNU ARM PEMicro Interface Debugging Support 3.0.3.201706082119 com.pemicro.debug.gdbjtag.pne.feature.feature.group P&E Microcomputer Systems Inc.

What I have used successfully in this article with semihosting I/O is the following Segger J-Link version, both with the embedded OpenSDA (as on the FRDM-K64F board) and the external J-Link Pro/EDU:

SEGGER J-Link GDB Server V6.16b

What supports semihosting with file I/O is as well the LPC-Link2/CMSIS-DAP debug probe:

LPC-Link2 debugging FRDM-K64F

Project

I’m using a SDK v2.2 project with MCUXpresso IDE:

Coverage Example Project in MCUXpresso IDE

This is a simple project created with the wizard. I have pushed the project to GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

In addition to the changes below, I usually create a dedicated build configuration (e.g. named ‘Coverage’) so I can switch between coverage and non-coverage mode. For more on how to use build configurations, see Build Configurations in Eclipse.

To record and store coverage information, I have created support files for it. Add the following two files to your project:

gcov_support.h
gcov_support.c

Coverage support files added

It provides the following:

GCOV_DO_COVERAGE: macro/setting to turn on/off coverage.
gcov_check(): function to check if semihosting file I/O is working.
gcov_write(): function to write the coverage data.
gcov_init(): function to initialize the coverage data.

The latest version is inside the GitHub project and the files are pasted below.

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/
 
#ifndef GCOV_SUPPORT_H_
#define GCOV_SUPPORT_H_
 
#define GCOV_DO_COVERAGE               (1)
  /*<! 1: to enable coverage; 0: to disable it */
 
/*!
 * \brief Test function to verify file I/O needed for gcov information generation.
 * \return 1 if file I/O does work, 0 otherwise
 */
int gcov_check(void);
 
/*!
 * \brief Flush and write the coverage information collected so far
 */
void gcov_write(void);
 
/*!
 * \brief Initialize the coverage information/constructors. Need to call this at the start of main().
 */
void gcov_init(void);
 
#endif /* GCOV_SUPPORT_H_ */

Below is the implementation file:

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/
 
#include <gcov_support.h>
#include <stdint.h>
#include <stdio.h>
 
void __gcov_flush(void); /* internal gcov function to write data */
 
int gcov_check(void) {
#if GCOV_DO_COVERAGE
  FILE *file = NULL;
 
  file = fopen ("c:\\tmp\\test.txt", "w");
  if (file!=NULL) {
    fputs("hello world\r\n", file);
    (void)fwrite("hello\r\n", sizeof("hello\r\n")-1, 1, file);
    fclose(file);
    return 1; /* ok */
  }
  return 0; /* failed */
#else
  return 1; /* ok */
#endif
}
 
void gcov_write(void) {
#if GCOV_DO_COVERAGE
  __gcov_flush();
#endif
}
 
/* call the coverage initializers if not done by startup code */
void gcov_init(void) {
#if GCOV_DO_COVERAGE
  void (**p)(void);
  extern uint32_t __init_array_start, __init_array_end; /* linker defined symbols, array of function pointers */
  uint32_t beg = (uint32_t)&__init_array_start;
  uint32_t end = (uint32_t)&__init_array_end;
 
  while(beg<end) {
    p = (void(**)(void))beg; /* get function pointer */
    (*p)(); /* call constructor */
    beg += sizeof(p); /* next pointer */
  }
#endif /* GCOV_DO_COVERAGE */
}

Enabling Coverage in the Application

The following shows the needed modifications in the application to enable coverage:

Enabling Coverage in Application

Include the header file:#include “gcov_support.h”
Initialize coverage information, best right after main():gcov_init();
Optional: check if file I/O is working properly:if (!gcov_check()) { /* file I/O is not working */}
Anytime in the application, dump the data with: gcov_write();

Linker: Symbols for gcov_init()

In gcov_init(), I have to initialize the data and constructors for the gcov library. Each source file instrumented for coverage needs to be initialized properly. Because the NXP SDK startup code does not initialize these constructors, I have to call gcov_init() from my application. gcov_init() expects two special symbols for the start and end of the constructors, see ‘Coverage Constructors’ in Code Coverage for Embedded Target with Eclipse, gcc and gcov for details.

The default linker script files in MCUXpresso IDE do not generate the needed symbols. The easiest way add this is to use the concept of Linker Script Templates (see menu Help > Help Contents > MCUXpresso IDE User Guide > Memory Configuration and LinkerScripts).

Inside the project root folder, create a folder named ‘linkscripts‘:

Linkscripts Folder

Inside that folder, create a file named ‘main_text.ldt‘ with the following content:

 /************************************************
 * start of main_text.ldt: *
 ************************************************/
 *(.text*)
 /* added in template for gcov: */
 PROVIDE_HIDDEN (__init_array_start = .);
 KEEP (*(SORT(.init_array.*)))
 KEEP (*(.init_array*))
 PROVIDE_HIDDEN (__init_array_end = .);
 /************************************************ 
 * end of main_text.ldt *
 ************************************************/

This will create the two extra symbols __init_array_start and __init_array_end, which are needed by gcov_init().

If not using managed linker scripts, then follow this article how to add it to the linker script file.

Linker: Semihosting

The coverage library will use normal file I/O (fopen(), fwrite(), etc) to write the information to the host using semihosting.

Usually, I don’t recommend using semihosting unless you really know what you are doing. In the case of coverage and profiling, it is actually very useful. Semihosting means that calling things like printf() or fopen() will trap/halt the target to start communication with the debugger on the host. So it is rather slow, plus if there is no debugger attached, the application on the board might stall! So don’t use semihosting if no active debug session is going on!

In the linker settings, choose the semihosting library:

Semihosting in Linker Settings

The newlib (semihost) library did not work for me. The Redlib semihosting libraries cannot be used as they do not include the GNU coverage library.

Linker: ––coverage Flag

The tell the linker to link with the necessary coverage libraries, add the following flag to the linker settings:

--coverage

Linker Coverage Flag

Source Files ––coverage Flag

Add the following compiler flag to each source file you want to have coverage for:

--coverage

Coverage enabled for main.c

Right-click on a file or folder to set special options for it. See as well Icon and Label Decorators in Eclipse.

I recommend only enabling coverage only for the files needed:

RAM: coverage needs counters in RAM
Time: writing coverage might need several seconds for each file

Coverage information is combined at the end. I can run coverage for one part of the application and then for the other.

Debugger Settings

Make sure your debug (launch) configuration has semihosting enabled:

Segger with Enabled Semihosting

For the LinkServer/CMSIS-DAP connection the setting is here:

LinkServer Semihosting Support

Build

Now build the application. For every coverage instrumented source file, it will generate *.gcno files:

Generated gcno files

The files are generated in the same folder as the object files.

Debug

Start a normal debug session. Upon executing the call to gcov_write(), it will write the *.gcda files:

Coverage information file generated with semihosting

Inspecting Coverage

Double click on any of the coverage files, and it opens a dialog where I have to specify the binary file used:

Open Coverage File

This opens a gcov view in Eclipse:

Coverage Overview

With double-clicking on a file/function, I get the detail information, including how many times a line has been executed:

Coverage Source View

Troubleshooting

As always, things might not work out the first time. Here are a few tips:

Make sure that you use the same version of gcov and gcc. Check if you have any other GNU tools in your system path.
If the *.gcda files do not show up in the Eclipse Project Explorer view, do a view refresh.
File I/O and other semihosting functions need a lot of stack. Increase the application and/or task stack size.
Writing *.gcda files can be very slow with semihosting, depending on file size. Start instrumenting with just one file. Consider using a different debug interface/probe. For me, the Segger J-Link is three times faster with semihosting file I/O compared to CMSIS-DAP.
Check if gcov_check() can open a file, check the file created. If file I/O fails, verify that you are actually using a semihosting library. Make sure you are not overwriting the file I/O functions (e.g. do not use a custom _write() function which is part e.g. in the Segger RTT library).
Perform a ‘clean’ or delete the output (usually ‘Debug’) folder and do a clean build.

Summary

Using the Eclipse IDE for code coverage with GNU gcov is a great addition. It helps me showing progress with my automated testing, and I can easily see which code pieces are not touched by my testing yet.

Using semihosting with file I/O is not the fastest way to save the coverage information to the host, but is a workable solution. I started on an experimental project to port the gcov library so I can use it to save the data to a memory device on the target system or send it to the host using any other connection method (serial, RTT, USB CDC, …). Let me know if you are interested in such a thing and I’ll see if I could write a tutorial about this.

The example project of this tutorial can be downloaded from GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Related

Trending