Measuring Code Coverage by Protractor End-to-End Tests

Was just setting up new JavaScript project based on Grunt. I scaffolded the project template by Yeoman with usage of angular-fullstack generator. I decided to try MEAN stack without MongoDB for my new project (DB isn't needed).

Next step was integrating Require.JS and configuring measurement of code coverage on client and server by Instanbul. When was this all done I was wondering if it is possible to measure code coverage by Protractor end-to-end testing.

After quick search I found that Ryan Bridges recently released grunt-protractor-coverage plugin. Interesting coincidence. So I decided to try it and can confirm that it's working fine with mentioned stack. Configuration was smooth and Ryan fixed small issue very promptly. It's based on grunt-protractor-runner plugin.

I created separate Grunt configuration file just for this purpose not to mess around with normal build. I had also problems to run 'makeReport' task of grunt-istanbul plugin for two different directories (Mocha server side code coverage measurement is using same task).

So here is the Grunt flow. First we need to copy non JavaScript files into target directory. It needs to be in separate directory because JavaScript files will need to be instrumented.

copy: {
  coverageE2E: {
    files: [{
      expand: true,
      dot: true,
      cwd: '<%= dirs.app %>',
      dest: '<%= dirs.instrumentedE2E %>/app',
      src: [
        '*.{ico,png,txt}',
        '.htaccess',
        'bower_components/**/*',
        'images/**/*',
        'fonts/**/*',
        'views/**/*',
        'styles/**/*',
      ]
    }]
  },
},

Next step is instrumentation of the code. It is needed for gathering coverage stats. Each line is decorated by special instructions that helps during measurement. Pay attention to fact that we are instrumenting server and client side code. Instrumented code is placed into target directory represented by placeholder <%= dirs.instrumentedE2E %>.

instrument: {
  files: ['server/**/*.js', 'app/scripts/**/*.js'],
  options: {
  lazy: true,
  basePath: '<%= dirs.instrumentedE2E %>/'
  }
},

Next we start Express from target directory.

express: {
  options: {
    port: process.env.PORT || 9000
  },
  coverageE2E: {
    options: {
      script: '<%= dirs.instrumentedE2E %>/lib/server.js',
      debug: true
    }
  },
},

And the protractor_coverage task of grunt-protractor-coverage plugin. Configuration should be the same as for grunt-protractor-runner plugin.

protractor_coverage: {
  options: {
    configFile: 'test/protractor/protractorConf.js', // Default config file
    keepAlive: true, // If false, the grunt process stops when the test fails.
    noColor: false, // If true, protractor will not use colors in its output.
    coverageDir: '<%= dirs.instrumentedE2E %>',
    args: {}
  },
  chrome: {
    options: {
      args: {
        baseUrl: 'http://localhost:3000/',
        // Arguments passed to the command
        'browser': 'chrome'
      }
    }
  },
},

Last step is generation  of coverage report.

makeReport: {
  src: '<%= dirs.instrumentedE2E %>/*.json',
  options: {
    type: 'html',
    dir: '<%= dirs.coverageE2E %>/reports',
    print: 'detail'
  }
},

Finally, this is grunt task gathering all steps.

grunt.registerTask('default', [
  'clean:coverageE2E',
  'copy:coverageE2E',
  'instrument',
  'express:coverageE2E',
  'protractor_coverage:chrome',
  'makeReport'
]);

EDIT: Notice that following Github link was changed to branch, because project structure was significantly changed: 

Source code for this project can be found on GitHub.  To run this Grunt configuration file:

grunt --gruntfile Gruntfile-e2e.js