In this article, we will explain how to use a Spring Batch metadata table on the different schema of the same DB.
1. Spring Batch Metadata Table on Different Schema of the Same DB
This is an in-depth article related to the Spring Batch. Spring Batch Framework is an open-source library for batch processing. Batch Processing is the execution of a series of jobs. Spring Batch has classes and APIs to read/write resources, job creation, job metadata storage, job data persistence, transaction management, job processing statistics, job restart, and partitioning techniques to process high-volume of data. It has a job repository that takes care of scheduling and job interaction. A job consists of multiple steps. Each step has the sequence of reading job metadata and processing the input to write the output.
2. Spring Batch
2.1 Prerequisites
Java 8 or 9 is required on the Linux, windows, or Mac operating system. Maven 3.6.1 is required for building the spring and hibernate application.
2.2 Download
You can download Java 8 can be downloaded from the Oracle web site . Apache Maven 3.6.1 can be downloaded from Apache site. Spring framework latest releases are available from the spring website.
2.3 Setup
You can set the environment variables for JAVA_HOME and PATH. They can be set as shown below:
Environment Setup for Java
JAVA_HOME=”/jboss/jdk1.8.0_73″ export JAVA_HOME PATH=$JAVA_HOME/bin:$PATH export PATH
The environment variables for maven are set as below:
Environment Setup for Maven
JAVA_HOME=”/jboss/jdk1.8.0_73″ export M2_HOME=/users/bhagvan.kommadi/Desktop/apache-maven-3.6.1 export M2=$M2_HOME/bin export PATH=$M2:$PATH
2.4 Building the application
2.4.1 Spring
You can start building Spring applications using the Spring Boot framework. Spring Boot has a minimal configuration of Spring. Spring Boot has features related to security, tracing, application health management, and runtime support for web servers. Spring configuration is done through maven pom.xml. The XML configuration is shown below:
Spring Configuration
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.springframework</groupId>
<artifactId>spring-helloworld</artifactId>
<version>0.1.0</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.4.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<properties>
<java.version>1.8</java.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
You can create a HelloWorldController class as the web controller. The class is annotated using @RestController. Rest Controller is used to handle requests in Spring Model View Controller framework. Annotation @RequestMapping is used to annotate the index() method. The code for the HelloWorldController class is shown below:
HelloWorld Controller
package helloworld;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;
@RestController
public class HelloWorldController {
@RequestMapping("/")
public String index() {
return "Hello World\n";
}
}
HelloWorldApp is created as the Spring Boot web application. When the application starts, beans, and settings are wired up dynamically. They are applied to the application context. The code for HelloWorldApp class is shown below:
HelloWorld App
package helloworld;
import java.util.Arrays;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
@SpringBootApplication
public class HelloWorldApp {
public static void main(String[] args) {
ApplicationContext ctx = SpringApplication.run(HelloWorldApp.class, args);
System.out.println("Inspecting the beans");
String[] beans = ctx.getBeanDefinitionNames();
Arrays.sort(beans);
for (String name : beans) {
System.out.println("Bean Name" +name);
}
}
}
Maven is used for building the application. The command below builds the application.
Maven Build Command
mvn package
The output of the executed command is shown below.
The jar file spring-helloworld-0.1.0.jar is created. The following command is used for executing the jar file.
Run Command
java -jar target/spring-helloworld-0.1.0.jar
The output of the executed command is shown below.
Curl command is invoked on the command line for the execution of index method. The method returns a String “Hello World” text. @RestController aggregates the two annotations @Controller and @ResponseBody. This results in returning data. The ouput is shown as below.
2.5 Spring Batch Features
Spring Batch is a lightweight, comprehensive batch framework. Developers can use it to create batch applications. Batch applications are designed to handle the daily and end of the daily operations of the enterprise. It has features as mentioned below:
- Large Data Processing
- Logging
- Tracing
- Resource Management
- Job Processing
- Transaction management
- Chunk based processing
- Declarative I/O
- Start/Stop/Restart
- Retry/Skip
- Web-based administration interface
Let us start looking at creation of a Spring batch job.
2.5.1 Batch Input
Sample input csv which has batch transactions for first batch job is shown below:
Input CSV
username, user_id, transaction_date, transaction_amount david, 13234, 31/10/2018, 10000 tom, 25134, 3/12/2018, 12321 george, 12134, 2/02/2018, 23411
Sample input csv which has the batch transaction data for skip batch job and skip policy job is shown below:
Skip Input CSV
username, user_id, transaction_date, transaction_amount david, 13234, 31/10/2018, 10000 tom, 25134, 3/12/2018, 12321 george, 12134, 2/02/2018, 23411 , 25436, 3/10/2020, 300 mathew, 91876, 5/11/2019, -510 , 34215, 10/10/2018, 110
2.5.2 Batch Data Schema
Spring Batch Database schema sql of repository.sqlite is shown below:
Database schema sql
CREATE TABLE BATCH_JOB_EXECUTION ( JOB_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, VERSION INTEGER , JOB_INSTANCE_ID INTEGER NOT NULL, CREATE_TIME TIMESTAMP NOT NULL, START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , EXIT_CODE VARCHAR(100) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, JOB_CONFIGURATION_LOCATION VARCHAR(2500), constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) );
2.5.3 Batch Metadata Configuration
Spring Batch Metadata configuration is stored in a database schema which is for metadata. The metadata data source is configured in the spring.xml which is shown below:
Spring Configuration
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-4.2.xsd">
<!-- connect to SQLite database -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.sqlite.JDBC" />
<property name="url" value="jdbc:sqlite:repository.sqlite" />
<property name="username" value="" />
<property name="password" value="" />
</bean>
<!-- create job-meta tables automatically -->
<jdbc:initialize-database data-source="dataSource">
<jdbc:script
location="org/springframework/batch/core/schema-drop-sqlite.sql" />
<jdbc:script location="org/springframework/batch/core/schema-sqlite.sql" />
</jdbc:initialize-database>
<!-- stored job-meta in memory -->
<!-- <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" /> </bean> -->
<!-- stored job-meta in database -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseType" value="sqlite" />
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
</beans>
2.5.4 Maven pom.xml
Maven pom.xml is shown below:
Maven pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>spring-batch</artifactId>
<version>0.1-SNAPSHOT</version>
<name>spring-batch</name>
<packaging>jar</packaging>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.baeldung</groupId>
<artifactId>parent-boot-2</artifactId>
<version>0.0.1-SNAPSHOT</version>
<relativePath>../parent-boot-2</relativePath>
</parent>
<dependencies>
<!-- JAXB APIs & runtime no longer provided in JDK 11 -->
<!-- see http://openjdk.java.net/jeps/320 -->
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>${jaxb.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
<version>${jaxb.version}</version>
<scope>runtime</scope>
</dependency>
<!-- SQLite database driver -->
<dependency>
<groupId>org.xerial</groupId>
<artifactId>sqlite-jdbc</artifactId>
<version>${sqlite.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-oxm</artifactId>
<version>${spring.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>${spring.batch.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-test</artifactId>
<version>${spring.batch.version}</version>
</dependency>
<dependency>
<groupId>com.opencsv</groupId>
<artifactId>opencsv</artifactId>
<version>${opencsv.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<version>${awaitility.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<properties>
<spring.version>5.2.0.RELEASE</spring.version>
<spring.batch.version>4.2.0.RELEASE</spring.batch.version>
<sqlite.version>3.15.1</sqlite.version>
<opencsv.version>4.1</opencsv.version>
<jaxb.version>2.3.1</jaxb.version>
<awaitility.version>3.1.1</awaitility.version>
</properties>
</project>
2.5.3 Spring Batch Application
Spring batch Application class code is shown below.
Spring Batch Application
package org.javacodegeeks.batch;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
public class Application {
private static final Logger LOGGER = LoggerFactory.getLogger(Application.class);
public static void main(final String[] args) {
final AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(SpringConfig.class);
context.register(SpringBatchConfig.class);
context.refresh();
runJob(context, "firstBatchJob");
runJob(context, "skippingBatchJob");
runJob(context, "skipPolicyBatchJob");
}
private static void runJob(AnnotationConfigApplicationContext context, String batchJobName) {
final JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
final Job job = (Job) context.getBean(batchJobName);
LOGGER.info("Starting the batch job: {}", batchJobName);
try {
JobParameters jobParameters = new JobParametersBuilder().addString("jobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
final JobExecution execution = jobLauncher.run(job, jobParameters);
LOGGER.info("Job Status : {}", execution.getStatus());
} catch (final Exception e) {
e.printStackTrace();
LOGGER.error("Job failed {}", e.getMessage());
}
}
}
Model class BatchTransaction is created and the code is shown below:
package org.javacodegeeks.batch.model;
import org.javacodegeeks.batch.service.adapter.LocalDateTimeAdapter;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.time.LocalDateTime;
@SuppressWarnings("restriction")
@XmlRootElement(name = "transactionRecord")
public class BatchTransaction {
private String username;
private int userId;
private int age;
private String postCode;
private LocalDateTime transactionDate;
private double amount;
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
@XmlJavaTypeAdapter(LocalDateTimeAdapter.class)
public LocalDateTime getTransactionDate() {
return transactionDate;
}
public void setTransactionDate(LocalDateTime transactionDate) {
this.transactionDate = transactionDate;
}
public double getAmount() {
return amount;
}
public void setAmount(double amount) {
this.amount = amount;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public String getPostCode() {
return postCode;
}
public void setPostCode(String postCode) {
this.postCode = postCode;
}
@Override
public String toString() {
return "Transaction [username=" + username + ", userId=" + userId + ", age=" + age + ", postCode=" + postCode + ", transactionDate=" + transactionDate + ", amount=" + amount + "]";
}
}
Code can be executed using the below command.
Maven command
mvn package cd spring-batch mvn package
The output is shown below:
Output
apples-MacBook-Air:spring-batch bhagvan.kommadi$ mvn package [INFO] Scanning for projects... [INFO] gitflow-incremental-builder is disabled. [INFO] [INFO] ------------------------------------------- [INFO] Building spring-batch 0.1-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- directory-maven-plugin:0.3.1:directory-of (directories) @ spring-batch --- [INFO] Directory of com.baeldung:parent-modules set to: /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample [INFO] [INFO] --- maven-install-plugin:2.5.2:install-file (install-jar-lib) @ spring-batch --- [INFO] Installing /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/custom-pmd-0.0.1.jar to /Users/bhagvan.kommadi/.m2/repository/org/baeldung/pmd/custom-pmd/0.0.1/custom-pmd-0.0.1.jar [INFO] Installing /var/folders/cr/0y892lq14qv7r24yl0gh0_dm0000gp/T/mvninstall1760246420817513314.pom to /Users/bhagvan.kommadi/.m2/repository/org/baeldung/pmd/custom-pmd/0.0.1/custom-pmd-0.0.1.pom [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ spring-batch --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 11 resources [INFO] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ spring-batch --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] >>> maven-pmd-plugin:3.13.0:check (default) > :pmd @ spring-batch >>> [INFO] [INFO] --- maven-pmd-plugin:3.13.0:pmd (pmd) @ spring-batch --- [INFO] [INFO] <<< maven-pmd-plugin:3.13.0:check (default) < :pmd @ spring-batch <<< [INFO] [INFO] [INFO] --- maven-pmd-plugin:3.13.0:check (default) @ spring-batch --- [INFO] PMD version: 6.21.0 [INFO] [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ spring-batch --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 6 resources [INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ spring-batch --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ spring-batch --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.javacodegeeks.SpringContextTest 22:42:20,273 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml] 22:42:20,274 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy] 22:42:20,274 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/spring-batch/target/classes/logback.xml] 22:42:20,410 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set 22:42:20,417 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender] 22:42:20,428 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT] 22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead. 22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder. 22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details 22:42:20,499 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.taskletsvschunks] to DEBUG 22:42:20,499 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.taskletsvschunks] to false 22:42:20,499 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.taskletsvschunks] 22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.batchscheduler] to DEBUG 22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.batchscheduler] to false 22:42:20,500 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.batchscheduler] 22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.batchtesting] to DEBUG 22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.batchtesting] to false 22:42:20,500 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.batchtesting] 22:42:20,501 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to ERROR 22:42:20,501 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT] 22:42:20,501 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration. 22:42:20,502 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@78b729e6 - Registering current configuration as safe fallback point Processing...Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] Processing...Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] Processing...Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=, userId=25436, age=0, postCode=null, transactionDate=2020-10-03T00:00, amount=300.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=mathew, userId=91876, age=0, postCode=null, transactionDate=2019-11-05T00:00, amount=-510.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=, userId=34215, age=0, postCode=null, transactionDate=2018-10-10T00:00, amount=110.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] 22:42:25.026 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step skippingStep in job skippingBatchJob org.springframework.batch.core.step.skip.SkipLimitExceededException: Skip limit of '2' exceeded SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=, userId=25436, age=0, postCode=null, transactionDate=2020-10-03T00:00, amount=300.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=mathew, userId=91876, age=0, postCode=null, transactionDate=2019-11-05T00:00, amount=-510.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] SkippingItemProcessor: Batch Transaction [username=, userId=34215, age=0, postCode=null, transactionDate=2018-10-10T00:00, amount=110.0] SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0] SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0] SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0] 22:42:25.183 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step skipPolicyStep in job skipPolicyBatchJob org.springframework.retry.RetryException: Non-skippable exception in recoverer while processing; nested exception is org.javacodegeeks.batch.service.MissingUsernameException at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$2.recover(FaultTolerantChunkProcessor.java:289) [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.076 s - in org.javacodegeeks.SpringContextTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ spring-batch --- [INFO] Building jar: /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/spring-batch/target/spring-batch-0.1-SNAPSHOT.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 16.270 s [INFO] Finished at: 2020-10-26T22:42:26+05:30 [INFO] ------------------------------------------------------------------------ apples-MacBook-Air:spring-batch bhagvan.kommadi$
3. Download the Source Code
In this article, we explained how to use a Spring Batch metadata table on different schema of the same DB.
Download
You can download the full source code of this example here: Spring Batch Metadata Table on Different Schema of the Same DB
You can download the full source code of this example here: Spring Batch Metadata Table on Different Schema of the Same DB
