In this post, we will take a look at TestNG annotations and how we can use them in unit tests for maximum benefit.
1. TestNG Annotations – Introduction
TestNG is a testing framework for the Java programming language created by Cedric Beust and inspired by JUnit and NUnit. The design goal of TestNG is to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use functionalities. The main features of TestNG include
- Annotations.
- Run your tests in arbitrarily big thread pools with various policies available
- Test that your code is multithread safe.
- Flexible test configuration.
- Support for data-driven testing (with @DataProvider).
- Support for parameters.
- Powerful execution model.
- Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc…).
- Embeds Bean Shell for further flexibility.
- Default JDK functions for runtime and logging (no dependencies).
- Dependent methods for application server testing.
In this post, we will deep dive into annotations and discover how it helps for testing workflows. Before that, we will look at the steps involved in creating a TestNG project. We will use Gradle as the build tool of choice. The section below discusses the build file
pom.xml
plugins {
id 'java'
}
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'org.testng', name: 'testng', version: '7.3.0'
}
test {
useTestNG() {
useDefaultListeners = true
}
testLogging {
events "PASSED", "FAILED", "SKIPPED"
}
}
- We have specified the repository for our dependencies as Maven central.
- In the dependencies section, we specify TestNG as the dependency.
- In the test task, we specify
useTestNGto indicate the test task must use TestNG to run the tests. - We also specify logging status for each test rather than the overall status.
2. Test related annotations
We will cover three annotations in this section – @Test, @BeforeClass and @AfterClass. To illustrate the idea we will look at testing a simple calculator class.
Calculator.java
public class Calculator {
public int add(int a, int b){
return a+b;
}
public int subtract(int a, int b){
return a-b;
}
}
- A simple class containing two operations – add and subtract
To test this class we will define a test class which asserts the working of these functions
CalculatorTest.java
import org.testng.Assert;
import org.testng.annotations.Test;
public class CalculatorTest {
@Test
public void addTest() {
Calculator calculator = new Calculator();
Assert.assertEquals(calculator.add(2,3),5);
}
@Test
public void subtractTest() {
Calculator calculator = new Calculator();
Assert.assertEquals(calculator.subtract(4,3),1);
}
}
- We have specified
Testannotation to indicate this is a test method - We are using
Assertto verify the expected result and actual result.
The @Test annotation can be applied to the class level as well. When applied all public methods inside the class are executed as test cases.
In the above example, we notice that we are initializing the Calculator class in every test. A better way to do that would be to use @BeforeClass annotation.
CalculatorTest.java
import org.testng.Assert;
import org.testng.annotations.Test;
public class CalculatorTest {
Calculator calculator;
@BeforeClass
public void setUp() {
calculator = new Calculator();
}
...
- We have initialized the
Calculatorclass insetUpmethod which runs once before any of the test methods in the current class starts running. - This ensures that we need not initialize the class during each test.
Complementary of @BeforeClass is @AfterClass. This is generally used for closing the resources(IO) which are used in tests. For the above example, a scenario could be releasing the instance of Calculator class. This might not be necessary for our case with JVM doing the work but it is illustrated below to give the flavor.
CalculatorTest.java
import org.testng.Assert;
import org.testng.annotations.Test;
public class CalculatorTest {
Calculator calculator;
@BeforeClass
public void setUp() {
System.out.println("initialize calculator");
calculator = new Calculator();
}
@AfterClass
public void tearDown() {
System.out.println("teardown calculator");
calculator = null;
}
...
Running this produces the following output
initialize calculator Gradle suite > Gradle test > com.jcg.testng.CalculatorTest > addTest PASSED Gradle suite > Gradle test > com.jcg.testng.CalculatorTest > subtractTest PASSED teardown calculator
In the above class, the class is initialized only once before any of the test is run. There might be cases where we want code to run for each test method. For this purpose there are annotations @BeforeMethod and @AfterMethod.
CalculatorTest.java
import org.testng.Assert;
import org.testng.annotations.Test;
public class CalculatorTest {
Calculator calculator;
@BeforeMethod
public void setUp() {
System.out.println("initialize calculator");
calculator = new Calculator();
}
@AfterMethod
public void tearDown() {
System.out.println("teardown calculator");
calculator = null;
}
...
The output below indicates the execution of the methods before each test method is being called.
initialize calculator Gradle suite > Gradle test > com.jcg.testng.CalculatorTest > addTest PASSED teardown calculator initialize calculator Gradle suite > Gradle test > com.jcg.testng.CalculatorTest > subtractTest PASSED teardown calculator
3. Test Group related annotations
In this section, we will explore annotations that will act when using a group of tests. We will start with @BeforeSuite and @AfterSuite annotations. A suite is represented by one XML file. It can contain one or more tests and is defined by the <suite> tag.
testng.xml
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1">
<test name="add">
<classes>
<class name="com.jcg.testng.AddTest"/>
</classes>
</test>
<test name="subtract">
<classes>
<class name="com.jcg.testng.SubtractTest"/>
</classes>
</test>
</suite>
This is an XML suite file which contains a suite of two tests containing AddTest and SubtractTest which are just broken down from the CalculatorTest class.
To execute something before the entire suite is executed i.e. initializing a heavy resource which would take time to initialize before each test class or method @BeforeSuite can be used.
AddTest.java
...
@BeforeSuite
public void setUpSuite() {
System.out.println("initialize before suite");
}
...
@AfterSuite
public void tearDown() {
System.out.println("after suite");
}
BeforeSuitecan be present in any of the test classes- It will be executed once before the entire suite is started
- It will be useful to initialize global variables which are needed by all executing tests(ObjectMother pattern)
initialize before suite initialize calculator initialize calculator after suite
- It first executes the
BeforeSuitemethod - It executes the two
BeforeClassmethods located in each test class - Finally, it executes the
AfterSuitemethod
We will explore one more grouping other than the suite. It is the basic grouping of a test. A test need not be condensed into a single test method or class. Test here refers to a group of test cases logically grouped to verify a particular behavior while a suite consists of many tests. The hierarchy in TestNG is Suite > Test > Test Class > Test Method.
To illustrate this scenario lets add another test to our application for the functionality multiply.
Calculator.java
public int multiply(int a, int b) {
return a * b;
}
This is a simple function created to multiply two numbers as part of the calculator functionality. To assert this, we are creating another Test class.
MultiplyTest.java
...
public class MultiplyTest {
Calculator calculator;
@BeforeClass
public void setUp() {
System.out.println("initialize calculator");
calculator = new Calculator();
}
@BeforeTest
public void beforeTest() {
System.out.println("Before Test");
}
@Test
public void multiplyTest() {
Assert.assertEquals(calculator.multiply(4, 3), 12);
}
@AfterTest
public void afterTest() {
System.out.println("After Test");
}
}
- This class is very similar to above tests.
- We have added
BeforeTestandAfterTestannotations to ensure these get executed before any test method runs in the test group. - To create a test group, refer the below testng.xml
testng.xml
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1">
<test name="addmul">
<classes>
<class name="com.jcg.testng.AddTest"/>
<class name="com.jcg.testng.MultiplyTest"/>
</classes>
</test>
<test name="subtract">
<classes>
<class name="com.jcg.testng.SubtractTest"/>
</classes>
</test>
</suite>
Here, We have added the MultiplyTest as part of the test containing AddTest. Now running the testng.xml produces the following result
initialize before suite Before Test initialize calculator initialize calculator After Test initialize calculator after suite
- We can see the
BeforeSuiteruns first followed by before test - Then
BeforeClassruns once for each test class - This is followed by the
AfterTest - The before class runs as part of the
SubtractTestclass - Finally, the
AfterSuitethe method runs printing the message to console
The last aspect we are going to look at is annotations catering to a group of tests. Let’s look at changing CalculatorTest to include groups.
CalculatorTest.java
@BeforeGroups({"addgrp"})
public void beforeGroup() {
System.out.println("Before Group");
}
@Test(groups = {"addgrp"})
public void addTest() {
Assert.assertEquals(calculator.add(2, 3), 5);
}
@Test(groups = {"subgrp"})
public void subtractTest() {
Assert.assertEquals(calculator.subtract(4, 3), 1);
}
@AfterGroups({"addgrp"})
public void afterGroup() {
System.out.println("After Group");
}
- We have added the
groupsattribute to test methodsaddTestandsubtractTestto indicate the groups to which the test belongs - The complements
BeforeGroupsandAfterGroupswere added to demonstrate the behaviour
testng.xml
<test name="calc_add">
<groups>
<run>
<include name="addgrp"/>
</run>
</groups>
<classes>
<class name="com.jcg.testng.CalculatorTest"/>
</classes>
</test>
<test name="calc_sub">
<groups>
<run>
<include name="subgrp"/>
</run>
</groups>
<classes>
<class name="com.jcg.testng.CalculatorTest"/>
</classes>
</test>
In the class definition, we just indicated the group for each test method. In the XML, we specify the groups for each test. We declare two additional tests within the same suite but under each test associate a specific group. We need to specify the classes associated with the test but also specify the group we want to include. groups also has the option of exclude using which we can exclude tests belonging to a group. Running this produces the following output
initialize before suite Before Test initialize calculator After Test initialize calculator Before Group After Group after suite
4. Annotation Attributes
In the previous section, we covered the basic annotations available as part of TestNG. We covered a few attributes available as part of the annotations. This section is about the attributes available for the annotations.
- alwaysRun – Applicable for all except BeforeGroups. When it is set to true, it will run irrespective of any failures.
- dependsOnGroups – This is used to indicate the test groups on which the annotated method depends on. If there is a failure in the group, this method is skipped.
- dependsOnMethods – Very similar to above except here it provides flexibility to specify methods than groups
- enabled – provides the capability to flexibility enable or disable annotated methods or classes
- inheritGroups – This annotation indicates that annotated method should inherit the groups from the test class
CalculatorAttributeTest.java
@Test
public class CalculatorAttributeTest {
Calculator calculator = new Calculator();
@Test
public void addTest() {
Assert.assertEquals(calculator.add(4, 3), 6);
}
@Test(dependsOnMethods = {"addTest"})
public void subtractTest() {
Assert.assertEquals(calculator.subtract(4, 3), 1);
}
@Test(enabled = false)
public void multiplyTest() {
Assert.assertEquals(calculator.multiply(4, 3), 12);
}
}
- Running the above as it displays the output 1 skipped and 1 failed. subtractTest method never runs as addTest method fails
- Now adding the attribute alwaysRun with the value true to subtractTest the method ensures that it is run even if
addTestfails. - Now
multiplyTestcan be enabled by setting theenabledattribute to true or by removing the attribute itself. - The changes above result in 2 successful tests and 1 failing test.
The example has been illustrated with only the Test annotation but it is very similar for other annotations.
5. Download the Source code
That was a TestNG Basic Annotations Tutorial.
You can download the full source code of this example here: TestNG Basic Annotations Tutorial
