In this example we are going to see some of the capabilities from the maven surefire plugin.
Maven is a build automation tool used mainly for java projects from apache.
You can access to the maven surefire plugin here.
We are going to see some examples of the capabilities of the maven surefire plugin.
For this example we use the following technologies:
- MAC OSX
- Eclipse Mars.1
- Maven3
- JDK 1.8.0_65 64bits
- Maven surefire plugin 2.19
- Junit 4.12
1. Introduction
The maven surefire plugin is called implicitely by the maven lifecycle in the appropiate phase so it is a ‘special’ plugin. We don´t need to define it inside pom.xml it will be downloaded and executed when maven needs it.
Despite that, we can define it inside pom.xml in order to run our unit tests inside of src/test/java folder.
The maven surefire plugin has only one goal defined:
test: Allow us to run the unit tests of the application
There is no need to define that goal inside pom.xml, as we said before, maven will invoke that goal when the maven lifecycle have to run the unit tests.
The maven surefire plugin allow us to use several testing frameworks like
junit (3.8 or 4.x)testNGWith version under or equal to 5.11, you have to define thejdk15tag in the dependency declaration in mavenPOJO test
You can configure maven to ignore the unit tests as we are going to see below.
2. Example project
For this example, we are going to use a java project with maven nature that will be packaged as a jar file. Eclipse Mars comes with maven support out of the box, so you don´t have to install anything. Our project will look like this
At this point, we have an empty maven project. We are going to define the maven surefire plugin inside pom.xml in order to test the plugin capabilities.
We are going to use junit as test framework.
The project has one class called Calc that defines a method called add, the method will accept two Integers values and will return the add of those numbers as Integer.
The project has some unit tests for the Calc class.
Also, you can see the files that will generate surefire plugin after its execution under target/surefire-reports folder. Those files allow to other tools like sonarqube or eclipse to interpretate the tests result.
3. Running single/parallel test
The surefire plugin allow us to run parallel test, you can achieve it in several ways. One of those ways is to use fork, so the surefire plugin will spawn some JVM in order to run our test. You can see below a pom.xml using fork:
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.javacodegeeks.examples</groupId>
<artifactId>maven-surefire-plugin-example</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>Maven surefire :: example</name>
<url>http://maven.apache.org</url>
<properties>
<junit.version>4.12</junit.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<!-- configuration> <parallel>methods</parallel> <threadCount>10</threadCount>
</configuration -->
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m</argLine>
<systemPropertyVariables>
<databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
</systemPropertyVariables>
<workingDirectory>FORK_DIRECTORY_${surefire.forkNumber}</workingDirectory>
</configuration>
</plugin>
</plugins>
</build>
</project>
Here you can see the execution output:
fork output:
[INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven surefire :: example 1.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven-surefire-plugin-example --- [INFO] Deleting /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.19:test (default-test) @ maven-surefire-plugin-example --- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.javacodegeeks.CalcTest Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec - in com.javacodegeeks.CalcTest Results : Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ maven-surefire-plugin-example --- [INFO] Building jar: /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ maven-surefire-plugin-example --- [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/pom.xml to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.295 s [INFO] Finished at: 2015-12-04T19:06:13+01:00 [INFO] Final Memory: 18M/235M [INFO] ------------------------------------------------------------------------
You can see more details about using fork here.
For junit 4.7 onwards you can use the parallel tag in order to run parallel test, in this way surefire will spawn some threads in order to run our tests. This is a good approach to run slow tests that have a high concurrency. You can see below a pom.xml using parallel
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.javacodegeeks.examples</groupId>
<artifactId>maven-surefire-plugin-example</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>Maven surefire :: example</name>
<url>http://maven.apache.org</url>
<properties>
<junit.version>4.12</junit.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<parallel>methods</parallel>
<threadCount>10</threadCount>
</configuration>
<!-- configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m</argLine>
<systemPropertyVariables>
<databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
</systemPropertyVariables>
<workingDirectory>FORK_DIRECTORY_${surefire.forkNumber}</workingDirectory>
</configuration -->
</plugin>
</plugins>
</build>
</project>
Here you can see the execution output:
parallel output:
[INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven surefire :: example 1.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven-surefire-plugin-example --- [INFO] Deleting /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.19:test (default-test) @ maven-surefire-plugin-example --- [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/2.19/surefire-junit47-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/2.19/surefire-junit47-2.19.pom (7 KB at 5.1 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/2.19/common-junit48-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/2.19/common-junit48-2.19.pom (4 KB at 15.0 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit4/2.19/common-junit4-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit4/2.19/common-junit4-2.19.pom (3 KB at 8.2 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit3/2.19/common-junit3-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit3/2.19/common-junit3-2.19.pom (2 KB at 7.4 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/2.19/common-java5-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/2.19/common-java5-2.19.pom (3 KB at 13.3 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/0.9/maven-shared-utils-0.9.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/0.9/maven-shared-utils-0.9.pom (7 KB at 30.2 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/2.19/surefire-grouper-2.19.pom [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/2.19/surefire-grouper-2.19.pom (3 KB at 12.3 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/2.19/surefire-junit47-2.19.jar [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit4/2.19/common-junit4-2.19.jar [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/2.19/common-junit48-2.19.jar [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/2.19/common-java5-2.19.jar [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit3/2.19/common-junit3-2.19.jar [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/2.19/common-java5-2.19.jar (44 KB at 105.0 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/0.9/maven-shared-utils-0.9.jar [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit3/2.19/common-junit3-2.19.jar (12 KB at 19.9 KB/sec) [INFO] Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/2.19/surefire-grouper-2.19.jar [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/2.19/common-junit48-2.19.jar (22 KB at 29.9 KB/sec) [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit4/2.19/common-junit4-2.19.jar (25 KB at 26.9 KB/sec) [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/2.19/surefire-junit47-2.19.jar (150 KB at 117.7 KB/sec) [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/0.9/maven-shared-utils-0.9.jar (168 KB at 183.8 KB/sec) [INFO] Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/2.19/surefire-grouper-2.19.jar (38 KB at 38.4 KB/sec) ------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.javacodegeeks.CalcTest Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec - in com.javacodegeeks.CalcTest Results : Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ maven-surefire-plugin-example --- [INFO] Building jar: /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ maven-surefire-plugin-example --- [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/pom.xml to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.049 s [INFO] Finished at: 2015-12-04T18:48:16+01:00 [INFO] Final Memory: 21M/325M [INFO] ------------------------------------------------------------------------
You can obviously run the test sequentially as you can see in the following pom.xml
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.javacodegeeks.examples</groupId>
<artifactId>maven-surefire-plugin-example</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>Maven surefire :: example</name>
<url>http://maven.apache.org</url>
<properties>
<junit.version>4.12</junit.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<!-- configuration>
<parallel>methods</parallel>
<threadCount>10</threadCount>
</configuration>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m</argLine>
<systemPropertyVariables>
<databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
</systemPropertyVariables>
<workingDirectory>FORK_DIRECTORY_${surefire.forkNumber}</workingDirectory>
</configuration -->
</plugin>
</plugins>
</build>
</project>
Here you can see the execution output
sequential output:
[INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven surefire :: example 1.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven-surefire-plugin-example --- [INFO] Deleting /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ maven-surefire-plugin-example --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ maven-surefire-plugin-example --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.19:test (default-test) @ maven-surefire-plugin-example --- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.javacodegeeks.CalcTest Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec - in com.javacodegeeks.CalcTest Results : Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ maven-surefire-plugin-example --- [INFO] Building jar: /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ maven-surefire-plugin-example --- [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/target/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.jar [INFO] Installing /Users/fhernandez/Documents/workspaceJavaCodeGeeks/maven surefire plugin/pom.xml to /Users/fhernandez/.m2/repository/com/javacodegeeks/examples/maven-surefire-plugin-example/1.0.0-SNAPSHOT/maven-surefire-plugin-example-1.0.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.500 s [INFO] Finished at: 2015-12-04T18:49:17+01:00 [INFO] Final Memory: 18M/235M [INFO] ------------------------------------------------------------------------
As you can see, the sequential execution is faster (0.003 sec) than parallel execution (0.006 sec) and faster than fork execution (0.049 sec) too. This is caused by the overhead needed to spawn the desired threads (parallel case)/the desired JVM (fork case). The behaviour and nature of your test will determine the best approach to use, in the example case, sequential execution is best because we don´t have concurrency and the tests are really fast to execute.
4. Skip test
In some situations we can need to avoid tests execution, we can configure maven to do that in several ways:
4.1 pom.xml
We can configure the surefire plugin in order to avoid execute test in pom.xml with the following code
in pom.xml:
<configuration>
<skipTests>true</skipTests>
</configuration>
4.2 Command line
You can avoid the tests execution in command line with maven.test.skip instruction
in pom.xml:
mvn install -Dmaven.test.skip=true
5. Conclusions
As we have saw the maven surefire plugin offers some interesting capabilities which we can take advantage of in order to execute our tests, you can get more details in the link at the introduction of this example.
6. Download the eclipse project
Download
You can download the full source code of this example here: maven-surefire-plugin-example.zip
You can download the full source code of this example here: maven-surefire-plugin-example.zip
