How to Test Gradle Plugins
In this article, I share my experience of creating functional tests for a custom Gradle plugin and how to configure the plugin project.
Join the DZone community and get the full member experience.
Join For FreeIn this article, I share my experience of creating functional tests for a custom Gradle plugin and how to configure the plugin project to collect code coverage metrics from tests.
In the previous article, I described how to build a custom Gradle plugin. Here, we will continue to work with it. Before we start, I’d recommend recapping things in the previous article to get a better understanding of where we started.
0. Clone the Project
git clone -b chapter-1 https://github.com/SurpSG/code-lines-counter-gradle-plugin.git
1. Configuration
Create a separate source set that will contain functional tests. Start with the creation of a simple directory, src/funcTests/kotlin
.
After creation, the directory looks like a usual folder, and it's not recognized as a code source by the IDE.
It's time to explain to Gradle that this directory will contain a code or in other words make it a "sourceSet."
Open the 'build.gradle' file in the root of the project, add a new source set configuration, and reimport the project:
x
sourceSets {
functionalTest { // declares a new sourceset
// specifies code source dir
kotlin.srcDir file('src/funcTests/kotlin')
// specifies resource dir
resources.srcDir file('src/funcTests/resources')
// specifies dependencies to compile test classes
compileClasspath += sourceSets.main.output + configurations.testRuntimeClasspath
// specifies dependencies to run tests
runtimeClasspath += output + compileClasspath
}
}
Create a Gradle task that will run functional tests and add a Kotlin std lib dependency for the test configuration:
xxxxxxxxxx
task functionalTest(type: Test) {
description = 'Runs the functional tests.'
group = 'verification'
testClassesDirs = sourceSets.functionalTest.output.classesDirs
classpath = sourceSets.functionalTest.runtimeClasspath
}
check.dependsOn functionalTest
2. Tests Creation
You can use any testing framework you like. In this example, I use JUnit 4.12. Create com.github.CodeLinesPluginTest
Kotlin class:
xxxxxxxxxx
class CodeLinesPluginTest {
// creates temp directory for a gradle project <-------- (1)
@get:Rule
var testProjectDir = TemporaryFolder()
private lateinit var buildFile: File
private lateinit var gradleRunner: GradleRunner
@Before
fun setup() {
// creates empty build.gradle file in the test gradle project <-------- (2)
buildFile = testProjectDir.newFile("build.gradle")
// creates and configures gradle runner <-------- (3)
gradleRunner = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(testProjectDir.root)
.withTestKitDir(testProjectDir.newFolder())
}
@Test
fun `check test setup`() {
// runs `tasks` gradle task <-------- (4)
val result = gradleRunner
.withArguments("tasks")
.build()
println(result.output)
}
}
It is a simple functional test in the example above. The test creates a Gradle project and runs the "tasks" Gradle task. Let's explore the test step by step:
- Declare the rule that cares about temporary directory creation. The directory is used as the project root.
- Create an empty build.gradle file in the project's root directory.
- Create a Gradle Runner that will help us to set up/build/run a test Gradle project.
- Execute the Gradle task, `tasks`. The Gradle Runner returns the execution result that is used for assertions.
Run the test and observe that basic Gradle tasks are printed to the console. We've just checked that our configuration is correct.
Apply the "code-lines" plugin:
xxxxxxxxxx
@Before
fun setup() {
buildFile = testProjectDir.newFile("build.gradle")
// add common configuration for all tests in this class
buildFile.appendText("""
plugins {
id 'java' // `code-lines` plugin is dependent on `java` plugin
id 'com.github.code-lines'
}
""".trimIndent())
...
}
Then, update the test:
xxxxxxxxxx
@Test
fun `codeLines task should print '0' when there is no source code`() {
val result = gradleRunner
.withArguments("codeLines") // <------- (1)
.build()
assertEquals(SUCCESS, result.task(":codeLines")!!.outcome) // <------- (2)
assertTrue(result.output.contains("Total lines: 0")) // <------- (3)
}
Now, the test does the following steps:
- Invokes the codeLines task.
- Verifies codeLines' execution status.
- Verifies the output contains an expected message. The result is "Total lines: 0" because the test project doesn't have any code.
It's the simplest happy path test for the plugin. Let's add another one:
- Add a Java class to verify the plugin counts lines properly.
- Apply the non-default plugin's configuration.
Create a simple Java class by location code-lines-counter-gradle-plugin/src/funcTests/resources/TestClass.java
xxxxxxxxxx
public class TestClass {
public static void main(String[] args) {
System.out.println("Hello world");
}
}
Add a new test:
xxxxxxxxxx
@Test
fun `codeLines task should print 'Total lines 6'`() {
// creates folders in the temp project
val testClassLocation: File = testProjectDir.newFolder("src", "main", "java").resolve("TestClass.java")
CodeLinesPluginTest::class.java.classLoader
.getResource("TestClass.java")!!.file.let(::File)
.copyTo(testClassLocation) // copies test file from resources to test project
val result = gradleRunner
.withArguments("codeLines")
.build()
assertEquals(SUCCESS, result.task(":codeLines")!!.outcome)
assertTrue(result.output.contains("Total lines: 6"))
}
Add one more test to check the plugin's configuration:
xxxxxxxxxx
@Test
fun `codeLines should skip blank lines`() {
val testClassLocation: File = testProjectDir.newFolder("src", "main", "java").resolve("TestClass.java")
CodeLinesPluginTest::class.java.classLoader
.getResource("TestClass.java")!!.file.let(::File)
.copyTo(testClassLocation)
// apply `code-lines` plugin configuration
buildFile.appendText("""
codeLinesStat {
sourceFilters.skipBlankLines = true
}
""".trimIndent())
val result = gradleRunner
.withArguments("codeLines")
.build()
assertEquals(SUCCESS, result.task(":codeLines")!!.outcome)
assertTrue(result.output.contains("Total lines: 5"))
}
3. Code Coverage
Apply the Jacoco plugin. Open build.gradle and update it with:
plugins {
id 'jacoco'
}
jacocoTestReport {
reports.html.enabled = true
executionData.setFrom fileTree(buildDir).include("/jacoco/*.exec") // <------ (1)
}
jacoco {
toolVersion = '0.8.7' // <----- (2)
}
- Specify what coverage data files to analyze. As we have two separate test source sets, we need to tell Jacoco about it.
- Set Jacoco version. I prefer the latest version, especially if I'm using Kotlin.
At this step, we should add some extra configurations. Tests executed with the TestKit are run in daemon JVM. That's why we need to tell daemon JVM to use the Jacoco Java agent.
Luckily, we can use Jacoco-gradle-testkit-plugin that helps us here:
plugins {
id "pl.droidsonroids.jacoco.testkit" version "1.0.8"
}
functionalTest.dependsOn generateJacocoTestKitProperties
jacocoTestKit {
applyTo("functionalTestRuntimeOnly", tasks.named("functionalTest"))
}
Update the GradleRunner configuration in the CodeLinesPluginTest
class:
gradleRunner = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(testProjectDir.root)
.withTestKitDir(testProjectDir.newFolder())
.apply {
// gradle testkit jacoco support
javaClass.classLoader.getResourceAsStream("testkit-gradle.properties")?.use { inputStream ->
File(projectDir, "gradle.properties").outputStream().use { outputStream ->
inputStream.copyTo(outputStream)
}
}
}
Now, we can run tests and check coverage:
xxxxxxxxxx
gradlew check jacocoTestReport
Open {PROJECT-ROOT}/build/reports/jacoco/test/html/index.html
in your favorite browser:
There’s one more plugin I recommend to use for your project. DiffCoverage builds a coverage report based on new or modified code. It may help to keep a high percentage of code coverage.
Full Working Example
Follow by the link or clone the project:
xxxxxxxxxx
git clone -b chapter-2-testing https://github.com/SurpSG/code-lines-counter-gradle-plugin.git
Conclusions
Functional tests are important because they show you if your plugin works correctly. You can check your plugin, but don't overdo the creation of tests because they are very poor in terms of performance. I prefer common happy path scenarios, like checking a plugin's configuration, plugin's task lifecycle, interaction with other plugins.
Code coverage tools help to detect untested code, so potentially, you can find and fix defects before your code is committed. Don't fully rely on such tools because they cannot guarantee that all corner cases are covered. They just show you if a code was invoked on tests run.
References
- Testing Gradle plugins (Gradle official guide).
- Code lines counter plugin (Github).
- Jacoco Gradle plugin.
- Diff coverage plugin.
Opinions expressed by DZone contributors are their own.
Comments