Batch

Spring Batch Job Parameters Example

In this article we shall show you how you can pass job parameters to a spring batch job.

1. Introduction

A typical enterprise application requires a batch job that consists of one or more steps. Each step reads the input data, processes the input data, and writes the processed data to the configured output. In a batch job with more than one step, the output of a step is often used as an input of the next step.

Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management.

2. Basic concept

A Spring Batch job consists of the following components:

  1. A spring batch job combines one or more steps that belong logically together in a flow. All steps in a job are processed sequentially. A job has different job instances and these job instances differ on the basis of job parameters. JobParameters is a set of parameters used to start a batch job. JobParameters can be used for identification or even as reference data during the job run.They have reserved names, so to access them we can use Spring Expression Language. For example to access a property ‘abc’ on job parameters: we can access it using the syntax #{jobParameters[abc]}. A JobExecution has all the information about what happened during the job run.
  2. A Step represents an independent logical task.
  3. A JobLauncher represents a simple interface for launching a Job with a given set of JobParameters.
  4. An ItemReader reads the input data and provides the found items one by one.
  5. An ItemProcessor transforms the items into a form that is understood by the ItemWriter one item at a time.
  6. An ItemWriter writes an information of an item to the output one item at a time.

3. Sample Project

Let us see an example how we can pass and access JobParameters in a spring batch job.

3.1 Technologies used

I have used the following technologies to build this example.

  1. Spring sts 3.9
  2. JDK 1.8

In this project, I have a list of students in an XML format. I read it, I filter out the failed students i.e students with less than 30 marks, and I create a CSV file with all the passed students. I pass the name of the XML file and the CSV file using JobParameters to JobLauncher.

3.2 Project Structure

The following image shows the final project structure.

Fig.1 Project structure

3.3 Project Creation

This section shows how to create a Java-based Maven project in Spring Sts. In Spring Sts go to File-->New-->Maven Project.

Fig.2 Create maven project

In the New Maven Project window, enable ‘Use default workspace location‘ and ‘Create a simple project (skip archetype selection)‘ checkbox and click on the next button to proceed.

Fig.3 Project details

Enter the group details and the artifact id for the project as shown. The version number will be by default: 0.0.1-SNAPSHOT.

Fig.4 Archetype Selection

Click finish and a new maven project will be created.

3.4 Maven Dependencies

In the below xml file, I have defined the dependencies for spring batch, spring core and spring oxm.

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>SpringBatchJobParam</groupId>
  <artifactId>SpringBatchJobParam</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
        <!-- Spring Core -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>4.3.5.RELEASE</version>
        </dependency>
        
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-core</artifactId>
            <version>3.0.7.RELEASE</version>
        </dependency>
        
        		<!-- Spring XML to/back object -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-oxm</artifactId>
			<version>4.3.5.RELEASE</version>
		</dependency>
  </dependencies>
</project>

3.5 Java Class Creation

Let’s create the required Java files. Right-click on src/main/java folder, New -> Package.

Fig.5 Package creation

Enter the package name com.jcg.main in the popup window for package creation.

Fig.6 Package details

Following the above two steps for package creation, create two more packages com.jcg.model and com.jcg.processor.

Once the package is created let us create implementation classes as shown below. Right-click on the newly created package, New -> Class.

Fig.7 Java class creation

Enter classname as MainApplication. A new class will be created under the package com.jcg.main.

Fig.8 Java class details

Following the above steps for class creation, create a class for Student.java and PassedStudentReportProcessor.java under the package com.jcg.model and com.jcg.processor respectively.

Create a folder to hold the context and job related property files named config and an another folder xml to hold the xml file with a list of students as shown below.

Fig.9 Folder creation

Let us create the configuration files appcontext.xml and reportcard.xml for the project under config folder under src/main/resources.

Fig.10 File creation

Create an xml file StudentReport.xml with a list of all the students with their marks under folder xml under src/main/resources.

Below is the code for all the classes defined above.

In the below main class, I have passed the required JobParameters to the class JobLauncher alongwith the job.

MainApplication.java

package com.jcg.main;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class MainApplication {

	public static void main(String[] args) {


		ApplicationContext context = 
				new ClassPathXmlApplicationContext("config/reportcard.xml","config/appcontext.xml");
		
		JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
		Job job = (Job) context.getBean("studentReportJob");

		try {
			
			JobParametersBuilder jobBuilder= new JobParametersBuilder();
			jobBuilder.addString("passstudentfilename", "PassedStudents");
			jobBuilder.addString("marksfileName","StudentReport");
			JobParameters jobParameters =jobBuilder.toJobParameters();
			JobExecution execution = jobLauncher.run(job, jobParameters);
			System.out.println("Completion Status : " + execution.getStatus());

		} catch (Exception e) {
			e.printStackTrace();
		}

		System.out.println("Done");

	}
	
}

The below model class has the required xml tags for marshalling and unmarshalling the xml objects into java objects.

Student.java

package com.jcg.model;

import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement(name = "student")
public class Student {

		private int studentId;
		private String name;
		private int marks;

		@XmlAttribute(name = "studentId")
		public int getStudentId() {
			return studentId;
		}

		public void setStudentId(int studentId) {
			this.studentId = studentId;
		}

		@XmlElement(name = "name")
		public String getName() {
			return name;
		}
		
		public void setName(String name) {
			this.name = name;
		}

		@XmlElement(name = "marks")
		public int getMarks() {
			return marks;
		}

		public void setMarks(int marks) {
			this.marks = marks;
		}


}

In the below processor class, I am filtering all the students who have scored less than 30 marks.

PassedStudentReportProcessor.java

package com.jcg.processor;

import org.springframework.batch.item.ItemProcessor;

import com.jcg.model.Student;

public class PassedStudentReportProcessor implements ItemProcessor {

	public Student process(Student item) throws Exception {
		//filter object with marks< 30
		if(item.getMarks() <30){
			return null; // null = ignore this object
		}
		return item;
	}

	
	
}

The below xml file has definitions for JobRepository and TransactionManager.

appcontext.xml

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">

	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="transactionManager" ref="transactionManager" />
	</bean>
 	
	<bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
	 
	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
	</bean>

</beans>

The below xml file defines a job that reads an xml file, processes it and creates a csv file. Here, the xml filename and the csv filename are provided during runtime.

reportcard.xml

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:batch="http://www.springframework.org/schema/batch" xmlns:task="http://www.springframework.org/schema/task"
	xmlns:util="http://www.springframework.org/schema/util" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/batch
		http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
		http://www.springframework.org/schema/util 
		http://www.springframework.org/schema/util/spring-util-3.2.xsd">

	<batch:job id="studentReportJob">
		<batch:step id="step1">
			<batch:tasklet>
				<batch:chunk reader="xmlItemReader" writer="cvsFileItemWriter" processor="passedStudentReportProcessor"
					commit-interval="1">
				</batch:chunk>
			</batch:tasklet>
		</batch:step>
	</batch:job>

	<bean id="passedStudentReportProcessor" class="com.jcg.processor.PassedStudentReportProcessor" scope="step"/>

	<bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="step">
		<property name="fragmentRootElementName" value="student" />
		<property name="resource" value="classpath:xml/#{jobParameters['marksfileName']}.xml" />
		<property name="unmarshaller" ref="reportUnmarshaller" />
	</bean>

	<!-- Read and map values to object, via jaxb2 -->
	<bean id="reportUnmarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
		<property name="classesToBeBound">
			<list>
				<value>com.jcg.model.Student</value>
			</list>
		</property>
	</bean>

	<bean id="cvsFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">

		<!-- write to this csv file -->
		<property name="shouldDeleteIfExists" value="true" />
 		<property name="resource"
      value="file:csv/#{jobParameters['passstudentfilename']}.csv" />
		<property name="lineAggregator">
			<bean
				class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
				<property name="delimiter" value="," />
				<property name="fieldExtractor">
					<bean
						class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
						<property name="names" value="studentId,name,marks" />
					</bean>
				</property>
			</bean>
		</property>

	</bean>

</beans>

The below xml file has a list of students with their respective marks.

StudentReport.xml

<?xml version="1.0" encoding="UTF-8" ?>
<College>
    <student studentId="1001">
        <name>Gaurav</name>
        <marks>40</marks>
    </student>
    <student studentId="1002">
        <name>Garima</name>
        <marks>10</marks>
        <dob>26/7/1983</dob>
        <income>100,999</income>
    </student>
    <student studentId="1003">
        <name>Mahima</name>
        <marks>35</marks>
    </student>
</College>

3.6 Maven Build

Build the application with command as Run As-->Maven Build--> Provide goals as clean package .

Fig.11 Maven build

3.7 Run Java Application

Run the application using the command Run As--> Java Application.

3.8 Project Demo

When I run the application as a java application, a PassedStudents.csv is created under the folder csv under the project alongside src folder with contents as shown in the below image.

Fig.12 CSV output

The above image has all the students that have marks greater than 30.

4. Conclusion

That’s all for getting the developers started with spring batch job parameters. I hope this article served you well. Developers can download the sample application as an Eclipse project in the Downloads section.

5. Download the Source Code

This was an example on how you can pass JobParameters to spring batch job.

Download
You can download the full source code of this example here: Spring Batch Job Parameters Example

Garima Pandey

Garima is a software developer and have more than 7 years of experience in web application development using various java technologies with a passion to write quality code.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments
Back to top button