TestNG

TestNG HTML and XML Reports Example

In this article, we will go through the TestNG HTML and XML reports.

We will be doing the following:

  1. Start with a TestNG project which we will later run to generate reports. We will be running a main suite containing a couple of child suites, this will create enough data for us to review the reports generated.
  2. Go through the TestNG report model.
  3. Examine TestNG provided default reports that are created by default in the directory ./test-output.
  4. In the end, implement a custom report using Reporter API.

Let’s start with the setup:

1. TestNG Report Project

The main motivation behind the design of our example is to come with an optimum view of the report. It consists of the following:

  1. Suites: A main suite that contains a couple of child suites.
  2. Tests: Most of them pass and some fail.
  3. Data-driven testing: One of the test uses @DataProvider.
  4. Groups: Some of the tests are grouped.
  5. Ignored Methods: One of the tests is disabled, so it will be ignored during the test run.
  6. Dynamic Test Name: We will use a factory instantiated test which implements ITest so that each test instance gets a dynamic name.

main-suite.xml is our main suite. It contains child suites suite1.xml and suite2.xml.

  1. suite1.xml
  2. suite2.xml

main-suite.xml has one test main-suite-test1 and a couple of test classes TestClass and TestClass1.

main-suite.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="main-suite" parallel="false">
	<suite-files>
		<suite-file path="./suite1.xml" />
		<suite-file path="./suite2.xml" />
	</suite-files>
    <test name="main-suite-test1">
		<classes>
		    <class name="com.javacodegeeks.testng.reports.TestClass" />
			<class name="com.javacodegeeks.testng.reports.TestClass1" />
		</classes>
	</test>
</suite>

Test methods of TestClass:

  1. a1() – will pass
  2. a2() – expects parameter param, will fail as we are not passing the value
  3. a3() – disabled so will show up in ignored methods

TestClass:

package com.javacodegeeks.testng.reports;

import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class TestClass {

	@Test
	public void a1() {
	}
	
	@Parameters("param")
	@Test
	public void a2(String param) {
		
	}	
	
	@Test(enabled=false)
	public void a3() {
	}
}

Test methods of TestClass1:

  1. t1() and t2(), both belong to group myGroup and are expected to fail.
  2. t3() belongs to group myGroup, will pass.
  3. t4() will pass.
  4. t5() expects a parameter and depends on a DataProvider called dp() for its parameter values.

TestClass1:

package com.javacodegeeks.testng.reports;

import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class TestClass1 {

	@Test(groups="myGroup")
	public void t1() {
		Assert.assertTrue(false);
	}
	
	@Test(groups="myGroup")
	public void t2() {
		Assert.assertTrue(false);
	}
	
	@Test(groups="myGroup")
	public void t3() {
	}
	
	@Test
	public void t4() {
	}
	
	@Test(dataProvider="dp")
	public void t5(String param) {
	}
	
	@DataProvider
	private Object[][] dp() {
		return new Object[][]{{"one"}, {"two"}};
	}
}

suite1.xml has a couple of tests:

  1. suite1-test1
  2. suite1-test2

Test suite1-test1 is made up of below classes:

  1. TestClass1
  2. TestClass2

And suite1-test2 is made up of:

  1. TestClass3
  2. FactoryInstantiatedTestClass

suite1.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="suite1" parallel="false">
	<test name="suite1-test1">
		<classes>
			<class name="com.javacodegeeks.testng.reports.TestClass1" />
			<class name="com.javacodegeeks.testng.reports.TestClass2" />
		</classes>
	</test>
	<test name="suite1-test2">
		<classes>
			<class name="com.javacodegeeks.testng.reports.TestClass3" />
			<class name="com.javacodegeeks.testng.reports.FactoryInstantiatedTestClass"/>
		</classes>
	</test>
</suite>

Now we will go through test methods of each class.

Test methods of TestClass2.

  1. Methods c1(), c2() and c3() are expected to pass.
  2. c4() is expected to fail

TestClass2:

package com.javacodegeeks.testng.reports;

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestClass2 {

	@Test
	public void c1() {
	}
	
	@Test
	public void c2() {
	}
	
	@Test
	public void c3() {
	}
	
	@Test
	public void c4() {
		Assert.assertTrue(false);
	}
}

Test methods of TestClass3.

  1. d1() and d2 are expected to pass.
  2. d1() belongs to group myGroup.

TestClass3:

package com.javacodegeeks.testng.reports;

import org.testng.annotations.Test;

public class TestClass3 {

	@Test(groups="myGroup")
	public void d1() {
	}
	
	@Test
	public void d2() {
	}	
}

Points to note about FactoryInstantiatedTestClass:

  1. FactoryInstantiatedTestClass acts as a test class as well as a factory class that instantiates itself using static method create().
  2. If you notice it implements ITest so that getTestName() can return a distinct name for each test instance.
  3. FactoryInstantiatedTestClass creates two test instances. Test method f() will fail for one of them.

FactoryInstantiatedTestClass:

package com.javacodegeeks.testng.reports;

import org.testng.Assert;
import org.testng.ITest;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class FactoryInstantiatedTestClass implements ITest {
	private String param;
	
	public FactoryInstantiatedTestClass(String param) {
		this.param = param;
	}
	
	public String getTestName() {
		return getClass().getSimpleName() + "-" + param;
	}
	
	@Factory
	public static Object[] create() {
		return new Object[]{new FactoryInstantiatedTestClass("TestNG"),
				new FactoryInstantiatedTestClass("Reports")};
	}
	
	@Test
	public void f() {
		if (param.equals("Reports")) {
			Assert.assertTrue(false);
		}
	}
}

Our final suite suite2.xml contains just one test suite2-test1 which in turn contains just one test class TestClass4.

suite2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="suite2" parallel="false">
	<test name="suite2-test1">
		<classes>
			<class name="com.javacodegeeks.testng.reports.TestClass4" />			
		</classes>
	</test>
</suite>

Test methods of TestClass4.

  1. e1() will pass
  2. e2() will also pass. Note that it calls Reporter.log(), this will print message passed-in to the HTML reports.

TestClass4:

package com.javacodegeeks.testng.reports;

import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestClass4 {

	@Test
	public void e1() {
	}
	
	@Test
	public void e2() {
		Reporter.log("Method name is e2");
	}	
}

2. Report Model

ISuite defines a Test Suite. It provides access to the full information on the result of a suite. It contains methods that are applicable at suite level. Below are some of the methods that will help you in building custom reports.

  1. getName() returns name of the suite.
  2. getInvokedMethods() returns a list of the methods that were invoked. Each item in the list is of type IInvokedMethod.
  3. getResults() returns a map of the test results, key being the test name and value an ISuiteResult object.
  4. ISuiteResult represents the result of a suite run. getTestContext() returns the testing context object ITestContext.
  5. ITestContext contains all the information for a given test run, for example, getPassedTests() returns result of passed tests in form of IResultMap object.
  6. IResultMap contains result of the tests run. For example, getAllResults() returns a set of ITestResult object.
  7. ITestResult describes the result of a test. getMethod() returns an ITestNGMethod object which is the test method this result represents.
  8. ITestNGMethod describes a TestNG annotated method. getMethodName() returns name of the method.

Below is diagram that shows the POJO model of the TestNG report.

TestNG Reporting Model
TestNG Reporting Model

3. TestNG Default Reports

TestNG comes with certain predefined listeners and by default they are added to the test execution. Whenever TestNG is run, HTML and XML reports are generated by default in the directory ./test-output.

For implementing a reporting class, the class has to implement an org.testng.IReporter interface. TestNG has its own reporter objects and these objects are called when the whole suite run ends. The object containing the information of the whole test run is passed on to the report implementations. The default implementations are:

  1. Main – This is the main HTML reporter for suites. You can access the main report by opening index.html from test-output directory.
  2. FailedReporter – This reporter is responsible for creating testng-failed.xml that contains just the test methods failed.
  3. XMLReporter – TestNG offers an XML reporter capturing TestNG specific information that is not available in JUnit reports, creates testng-results.xml.
  4. EmailableReporter2 – A reporter that generates one big HTML file emailable-report.html that’s easy to email to coworkers. If system property noEmailableReporter is set to some non-null value, the emailable report will not be generated.
  5. JUnitReportReporter – It creates JUnit compatible XML file
  6. SuiteHTMLReporter – This is the older version of HTML reporter for suites.

If we want only the custom reporters run then we make use of the flag –useDefaultListeners.

For example:

java org.testng.TestNG –useDefaultListeners false testng.xml

4. Main Report Layout

test-output directory contains an index.html file that is the entry point to the TestNG HTML report.

Main Report
Main Report

The main layout is composed of top pane, left pane and the details pane.

  1. Top Pane – Summary of test results.
  2. Left Pane – Suite based information. All the suites will be listed here.
  3. Details Pane – Once we click on a piece of information in the left pane, the details of it will be shown here.

TestNG Report Layout
TestNG Report Layout

4.1. Test Results Summary

The top-level report gives us a list of all the suites that were just run, along with an individual and compound total for each passed, failed, and skipped test.

Test Results Summary Panel
Test Results Summary Panel

4. 2. Suite Pane

In our example, we have three suites. You can see all of the three suites listed here:

  1. main-suite
  2. suite2
  3. suite1

Suite Panel
Suite Panel

Clicking on ‘All Suites’ will expand the suites.

Each suite has two sections.

  1. Info – Suite information
  2. Result – Test results

The information section will have the following details:

  1. testng.xml – Contents of testng.xml file.
  2. Tests run
  3. Groups Involved
  4. Test Timings
  5. Reporter Output – If the test logs a message by calling Reporte.log() then those messages can be seen here
  6. Any ignored methods, for example, a disabled method
  7. Chronological view – List of test methods executed in chronological order

Suite Components
Suite Components

4. 3. Suite Details Pane

The detail panel will show the details of whatever is selected on the left panel. By default, if there are any failures, the detail panel will show those failures.

Suite Details Panel
Suite Details Panel

Click on ‘Test’ to see the summary of the tests that suite has run for.

Tests Summary
Tests Summary

Click on ‘Groups’ to see the groups involved.

Groups Used
Groups Used

Click on ‘Times’ to see how long each case took.

Test Times
Test Times

Click on ‘Reporter Output’ to see the report log.

Reporter Log
Reporter Log

Click on ‘Ignored Methods’ to see the ignored methods if any.

Ignored Methods
Ignored Methods

If a data provider is used to inject parameter values, you can see them in the test method.

Test Parameter Values
Test Parameter Values

5. Emailable Report

A reporter that generates one big HTML file that’s easy to email to other team members. Open test-output/emailable-report.html to see the report.

Emailable report
Emailable report

The first section contains a summary of all the tests.

Emailable Report Suite Summary
Suite Summary

The next section contains details of each test run.

Emailable Main Suite Details
Main Suite Details

The last section contains details of each test method.

Test Details
Test Details

6. Old Suite Html Reporter

These are the old styled suite reports, you will find them in folder test-output/old.

Old Suite HTML Reports
Old Suite HTML Reports

The top-level report gives us a list of all the suites that were just run, along with an individual and compound total for each passed, failed, and skipped test. Click on ‘link’ to take a look at the testng.xml file used for each suite.

Suite Html Reporter
Suite Html Reporter

Clicking on the suite link brings up the main suite report.

The left pane shows a list of all the tests found in this suite. Clicking on any link in the left pane will show the details in the right pane.

Suite Details
Suite Details

7. TestNg.xml just for the failed Tests

testng-failed.xml file contains a subset of testng.xml to rerun just the failed tests.

testng-failed.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Failed suite [suite1]">
  <listeners>
    <listener class-name="com.javacodegeeks.testng.reports.Reporter"/>
  </listeners>
  <test name="suite1-test1(failed)">
    <classes>
      <class name="com.javacodegeeks.testng.reports.TestClass1">
        <methods>
          <include name="t1"/>
          <include name="t2"/>
        </methods>
      </class>
      <class name="com.javacodegeeks.testng.reports.TestClass2">
        <methods>
          <include name="c4"/>
        </methods>
      </class>
    </classes>
  </test>
  <test name="suite1-test2(failed)">
    <classes>
      <class name="com.javacodegeeks.testng.reports.FactoryInstantiatedTestClass">
        <methods>
          <include name="f"/>
        </methods>
      </class>
    </classes>
  </test>
</suite>

8. TestNG Results in XML

Right click on TestClass2 and run TestNG. This will produce the XML results just for TestClass2.

testng-results.xml is the TestNG report of test results in XML.

testng-results.xml:

<?xml version="1.0" encoding="UTF-8"?>
<testng-results skipped="0" failed="1" total="4" passed="3">
  <reporter-output>
  </reporter-output>
  <suite name="Default suite" duration-ms="20" started-at="2015-03-10T06:11:58Z" finished-at="2015-03-10T06:11:58Z">
    <groups>
    </groups>
    <test name="Default test" duration-ms="20" started-at="2015-03-10T06:11:58Z" finished-at="2015-03-10T06:11:58Z">
      <class name="com.javacodegeeks.testng.reports.TestClass2">
        <test-method status="PASS" signature="c2()[pri:0, instance:com.javacodegeeks.testng.reports.TestClass2@1c2c22f3]" name="c2" duration-ms="0" started-at="2015-03-10T11:41:58Z" finished-at="2015-03-10T11:41:58Z">
          <reporter-output>
          </reporter-output>
        </test-method> <!-- c2 -->
        <test-method status="PASS" signature="c3()[pri:0, instance:com.javacodegeeks.testng.reports.TestClass2@1c2c22f3]" name="c3" duration-ms="10" started-at="2015-03-10T11:41:58Z" finished-at="2015-03-10T11:41:58Z">
          <reporter-output>
          </reporter-output>
        </test-method> <!-- c3 -->
        <test-method status="PASS" signature="c1()[pri:0, instance:com.javacodegeeks.testng.reports.TestClass2@1c2c22f3]" name="c1" duration-ms="0" started-at="2015-03-10T11:41:58Z" finished-at="2015-03-10T11:41:58Z">
          <reporter-output>
          </reporter-output>
        </test-method> <!-- c1 -->
        <test-method status="FAIL" signature="c4()[pri:0, instance:com.javacodegeeks.testng.reports.TestClass2@1c2c22f3]" name="c4" duration-ms="0" started-at="2015-03-10T11:41:58Z" finished-at="2015-03-10T11:41:58Z">
          <exception class="java.lang.AssertionError">
            <message>
              <![CDATA[expected [true] but found [false]]]>
            </message>
            <full-stacktrace>
              <![CDATA

expected [true] but found [false] at org.testng.Assert.fail(Assert.java:94) at org.testng.Assert.failNotEquals(Assert.java:494) at org.testng.Assert.assertTrue(Assert.java:42) at org.testng.Assert.assertTrue(Assert.java:52) at com.javacodegeeks.testng.reports.TestClass2.c4(TestClass2.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291) at org.testng.SuiteRunner.run(SuiteRunner.java:240) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224) at org.testng.TestNG.runSuitesLocally(TestNG.java:1149) at org.testng.TestNG.run(TestNG.java:1057) at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111) at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204) at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175) ]]> 
            </full-stacktrace> 
          </exception> 
          <reporter-output> 
          </reporter-output> 
        </test-method>
      </class> 
    </test> 
  </suite> 
</testng-results>

9. JUnit XML Reports

TestNG by default generates the JUnit XML reports for any test execution.

TEST-com.javacodegeeks.testng.reports.TestClass4.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated by org.testng.reporters.JUnitReportReporter -->
<testsuite hostname="INMAA1-L1005" name="com.javacodegeeks.testng.reports.TestClass4" tests="2" failures="0" timestamp="8 Mar 2015 13:27:14 GMT" time="0.001" errors="0">
  <testcase name="e1" time="0.001" classname="com.javacodegeeks.testng.reports.TestClass4"/>
  <testcase name="e2" time="0.000" classname="com.javacodegeeks.testng.reports.TestClass4"/>
</testsuite> <!-- com.javacodegeeks.testng.reports.TestClass4 -->

We can use these XML report files as input for generation of a JUnit HTML report. Below is the ant build configuration XML file that uses Junit’s target junitreport to generate an HTML report for the test execution.
Once executed a JUnit HTML report will be generated in the configured directory ./test-output/junit-html-report.

build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="JUnit Report in HTML" default="junit-htm-report" basedir=".">
	<property name="junit-xml-reports-dir" value="./test-output/junitreports" />
	<property name="report-dir" value="./test-output/junit-html-report" />
	<target name="junit-htm-report">
		<!-- Delete and recreate the html report directories -->
		<delete dir="${report-dir}" failonerror="false" />
		<mkdir dir="${report-dir}" />
		<junitreport todir="${report-dir}">
			<fileset dir="${junit-xml-reports-dir}">
				<include name="**/*.xml" />
			</fileset>
			<report format="noframes" todir="${report-dir}" />
		</junitreport>
	</target>
</project>

Open file named junit-noframes.html to see the results.

JUnit Report in HTML
JUnit Report in HTML

10. Progressive Html Report for Individual Tests

TestHTMLReporter is a ITestListener reporter that generates HTML report for individual tests. There will be one folder for each suite with the same name as the suite’s name. For each test, there will be one HTML and XML report. Since it implements ITestListener, one can see the report progressing as the tests progress in their run.

suite1 contains two test ssuite1-test1 and suite1-test2 so you can see one set of HTML and XML files for each test. There is also one testng-failed.xml that contains the failed test methods.

HTML reporter for individual tests
HTML reporter for individual tests

Open suite1-test1.html to see the report.

The first section shows the summary of the test results.

Test Results Summary
Test Results Summary

In the second section, you see the details of failed tests.

Results from the failed tests
Results from the failed tests

In the final section, you will see the details of passed tests.

Result from passed tests
Result from passed tests

11. Custom Report

In this example, we will see the method of writing your custom reporter and attaching it to TestNG.
To write a custom reporter class, our extension class should implement the IReporter interface.

It implements the definition for the method generateReport of the IReporter interface. The method takes three arguments :

  1. The first is the XmlSuite object, which is the list suites mentioned in the testng XML being executed.
  2. The second one being suites which contains the suite information after the test execution; this object contains all the information about the packages, classes, test methods, and their test execution results.
  3. The third being the outputDirectory, which contains the information of the output folder path where the reports will be generated.

The custom report writes to an HTML file as well as prints in the console. Once custom report is run, it will generate HTML file test-output/custom-report.html.

Custom Report
Custom Report

Reporter:

package com.javacodegeeks.testng.reports;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.testng.IReporter;
import org.testng.IResultMap;
import org.testng.ISuite;
import org.testng.ISuiteResult;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.xml.XmlSuite;

public class CustomReporter implements IReporter {
	private PrintWriter mOut;

	public void generateReport(List xmlSuites, List suites,
			String outputDirectory) {
		new File(outputDirectory).mkdirs();
		try {
			mOut = new PrintWriter(new BufferedWriter(new FileWriter(new File(
					outputDirectory, "custom-report.html"))));
		} catch (IOException e) {
			System.out.println("Error in creating writer: " + e);
		}
		startHtml();
		print("Suites run: " + suites.size());
		for (ISuite suite : suites) {
			print("Suite>" + suite.getName());
			Map<String, ISuiteResult> suiteResults = suite.getResults();
			for (String testName : suiteResults.keySet()) {
				print("    Test>" + testName);
				ISuiteResult suiteResult = suiteResults.get(testName);
				ITestContext testContext = suiteResult.getTestContext();
				print("        Failed>" + testContext.getFailedTests().size());
				IResultMap failedResult = testContext.getFailedTests();
				Set testsFailed = failedResult.getAllResults();
				for (ITestResult testResult : testsFailed) {
					print("            " + testResult.getName());
					print("                " + testResult.getThrowable());
				}
				IResultMap passResult = testContext.getPassedTests();
				Set testsPassed = passResult.getAllResults();
				print("        Passed>" + testsPassed.size());
				for (ITestResult testResult : testsPassed) {
					print("            "
							+ testResult.getName()
							+ ">took "
							+ (testResult.getEndMillis() - testResult
									.getStartMillis()) + "ms");
				}
				IResultMap skippedResult = testContext.getSkippedTests();
				Set testsSkipped = skippedResult.getAllResults();
				print("        Skipped>" + testsSkipped.size());
				for (ITestResult testResult : testsSkipped) {
					print("            " + testResult.getName());
				}

			}
		}
		endHtml();
	    mOut.flush();
	    mOut.close();
	}

	private void print(String text) {
		System.out.println(text);
		mOut.println(text + "
");
	}

	private void startHtml() {
		mOut.println("");
		mOut.println("");
		mOut.println("TestNG Html Report Example");		
		mOut.println("");
		mOut.println("");
	}
	
	private void endHtml() {
		mOut.println("");
	}
}

The Reporter class is added as a listener to the test suite using the listeners and listener tag as deined in the testng.xml file.

main-suite.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="main-suite" parallel="false">
	<listeners>
		<listener class-name="com.javacodegeeks.testng.reports.CustomReporter" />
	</listeners>
	<suite-files>
		<suite-file path="./suite1.xml" />
		<suite-file path="./suite2.xml" />
	</suite-files>
    <test name="main-suite-test1">
		<classes>
		    <class name="com.javacodegeeks.testng.reports.TestClass" />
			<class name="com.javacodegeeks.testng.reports.TestClass3" />
		</classes>
	</test>
</suite>

Output:

Suites run: 3
Suite>main-suite
    Test>main-suite-test1
        Failed>3
            a2
                org.testng.TestNGException: 
Parameter 'param' is required by @Test on method a2 but has not been marked @Optional or defined
in C:\javacodegeeks_ws\testngReports\src\test\resources\main-suite.xml
            t1
                java.lang.AssertionError: expected [true] but found [false]
            t2
                java.lang.AssertionError: expected [true] but found [false]
        Passed>5
            a1>took 0ms
            t5>took 0ms
            t3>took 0ms
            t4>took 1ms
            t5>took 0ms
        Skipped>0
Suite>suite2
    Test>suite2-test1
        Failed>0
        Passed>2
            e2>took 0ms
            e1>took 0ms
        Skipped>0
Suite>suite1
    Test>suite1-test1
        Failed>3
            c4
                java.lang.AssertionError: expected [true] but found [false]
            t2
                java.lang.AssertionError: expected [true] but found [false]
            t1
                java.lang.AssertionError: expected [true] but found [false]
        Passed>7
            c3>took 0ms
            t5>took 0ms
            t5>took 0ms
            c2>took 0ms
            t3>took 0ms
            t4>took 0ms
            c1>took 0ms
        Skipped>0
    Test>suite1-test2
        Failed>1
            FactoryInstantiatedTestClass-Reports
                java.lang.AssertionError: expected [true] but found [false]
        Passed>3
            FactoryInstantiatedTestClass-TestNG>took 0ms
            d1>took 0ms
            d2>took 0ms
        Skipped>0

Download the Eclipse Project

This was an article about TestNG HTML and XML reports.

Download
You can download the full source code of this example here: testngReports.zip

Ram Mokkapaty

Ram holds a master's degree in Machine Design from IT B.H.U. His expertise lies in test driven development and re-factoring. He is passionate about open source technologies and actively blogs on various java and open-source technologies like spring. He works as a principal Engineer in the logistics domain.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Maha Fouad
Maha Fouad
5 years ago

Thanks a lot for the detailed explanation.

Shiva Adirala
5 years ago

Great post. Even I created a custom html report “testng-metrics” generated using testng listener

Highlights:

1. No code changes required in your automation scripts
2. Export test results (csv,pdf,excel)
3. Sort/Search results
4. Visualize test results
5. Can be shared without any content loss

Sample report – https://testng-metrics.netlify.com
Repo – https://github.com/adiralashiva8/testng-metrics/

Back to top button