Batch

Spring Batch Metadata Table on Different Schema of the Same DB

In this article, we will explain how to use a Spring Batch metadata table on the different schema of the same DB.

1. Spring Batch Metadata Table on Different Schema of the Same DB

This is an in-depth article related to the Spring Batch. Spring Batch Framework is an open-source library for batch processing. Batch Processing is the execution of a series of jobs. Spring Batch has classes and APIs to read/write resources, job creation, job metadata storage, job data persistence, transaction management, job processing statistics, job restart, and partitioning techniques to process high-volume of data. It has a job repository that takes care of scheduling and job interaction. A job consists of multiple steps. Each step has the sequence of reading job metadata and processing the input to write the output.

2. Spring Batch

2.1 Prerequisites

Java 8 or 9 is required on the Linux, windows, or Mac operating system. Maven 3.6.1 is required for building the spring and hibernate application.

2.2 Download

You can download Java 8 can be downloaded from the Oracle web site . Apache Maven 3.6.1 can be downloaded from Apache site. Spring framework latest releases are available from the spring website.

2.3 Setup

You can set the environment variables for JAVA_HOME and PATH. They can be set as shown below:

Environment Setup for Java

JAVA_HOME=”/jboss/jdk1.8.0_73″
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH

The environment variables for maven are set as below:

Environment Setup for Maven

JAVA_HOME=”/jboss/jdk1.8.0_73″
export M2_HOME=/users/bhagvan.kommadi/Desktop/apache-maven-3.6.1
export M2=$M2_HOME/bin
export PATH=$M2:$PATH

2.4 Building the application

2.4.1 Spring

You can start building Spring applications using the Spring Boot framework. Spring Boot has a minimal configuration of Spring. Spring Boot has features related to security, tracing, application health management, and runtime support for web servers. Spring configuration is done through maven pom.xml. The XML configuration is shown below:

Spring Configuration

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
  
    <groupId>org.springframework</groupId>
    <artifactId>spring-helloworld</artifactId>
    <version>0.1.0</version>
  
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.4.RELEASE</version>
    </parent>
  
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
  
    <properties>
        <java.version>1.8</java.version>
    </properties>
  
  
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
  
</project>

You can create a HelloWorldController class as the web controller. The class is annotated using @RestController. Rest Controller is used to handle requests in Spring Model View Controller framework. Annotation @RequestMapping is used to annotate the index() method. The code for the HelloWorldController class is shown below:

HelloWorld Controller

package helloworld;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;
  
@RestController
public class HelloWorldController {
      
    @RequestMapping("/")
    public String index() {
        return "Hello World\n";
    }
      
}

HelloWorldApp is created as the Spring Boot web application. When the application starts, beans, ​and settings are wired up dynamically. They are applied to the application context. The code for HelloWorldApp class is shown below:

HelloWorld App

package helloworld;
import java.util.Arrays;
  
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
  
@SpringBootApplication
public class HelloWorldApp {
      
    public static void main(String[] args) {
        ApplicationContext ctx = SpringApplication.run(HelloWorldApp.class, args);
          
        System.out.println("Inspecting the beans");
          
        String[] beans = ctx.getBeanDefinitionNames();
        Arrays.sort(beans);
        for (String name : beans) {
            System.out.println("Bean Name" +name);
        }
    }
  
}

Maven is used for building the application. The command below builds the application.

Maven Build Command

mvn package

The output of the executed command is shown below.

spring batch metadata - Maven Build
Maven Build

The jar file spring-helloworld-0.1.0.jar is created. The following command is used for executing the jar file.

Run Command

java -jar target/spring-helloworld-0.1.0.jar

The output of the executed command is shown below.

spring batch metadata - Spring Beans
Spring Beans

Curl command is invoked on the command line for the execution of index method. The method returns a String “Hello World” text. @RestController aggregates the two annotations @Controller and @ResponseBody. This results in returning data. The ouput is shown as below.

spring batch metadata - Curl Command
Curl Command

2.5 Spring Batch Features

Spring Batch is a lightweight, comprehensive batch framework. Developers can use it to create batch applications. Batch applications are designed to handle the daily and end of the daily operations of the enterprise. It has features as mentioned below:

  • Large Data Processing
  • Logging
  • Tracing
  • Resource Management
  • Job Processing
  • Transaction management
  • Chunk based processing
  • Declarative I/O
  • Start/Stop/Restart
  • Retry/Skip
  • Web-based administration interface

Let us start looking at creation of a Spring batch job.

2.5.1 Batch Input

Sample input csv which has batch transactions for first batch job is shown below:

Input CSV

username, user_id, transaction_date, transaction_amount
david, 13234, 31/10/2018, 10000
tom, 25134, 3/12/2018, 12321
george, 12134, 2/02/2018, 23411

Sample input csv which has the batch transaction data for skip batch job and skip policy job is shown below:

Skip Input CSV

username, user_id, transaction_date, transaction_amount
david, 13234, 31/10/2018, 10000
tom, 25134, 3/12/2018, 12321
george, 12134, 2/02/2018, 23411
, 25436, 3/10/2020, 300
mathew, 91876, 5/11/2019, -510
, 34215, 10/10/2018, 110

2.5.2 Batch Data Schema

Spring Batch Database schema sql of repository.sqlite is shown below:

Database schema sql

CREATE TABLE BATCH_JOB_EXECUTION ( JOB_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, VERSION INTEGER , JOB_INSTANCE_ID INTEGER NOT NULL, CREATE_TIME TIMESTAMP NOT NULL, START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , EXIT_CODE VARCHAR(100) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, JOB_CONFIGURATION_LOCATION VARCHAR(2500), constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) );

2.5.3 Batch Metadata Configuration

Spring Batch Metadata configuration is stored in a database schema which is for metadata. The metadata data source is configured in the spring.xml which is shown below:

Spring Configuration

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
		http://www.springframework.org/schema/jdbc 
		http://www.springframework.org/schema/jdbc/spring-jdbc-4.2.xsd">

    <!-- connect to SQLite database -->
    <bean id="dataSource"
        class="org.springframework.jdbc.datasource.DriverManagerDataSource">
        <property name="driverClassName" value="org.sqlite.JDBC" />
        <property name="url" value="jdbc:sqlite:repository.sqlite" />
        <property name="username" value="" />
        <property name="password" value="" />
    </bean>

    <!-- create job-meta tables automatically -->
    <jdbc:initialize-database data-source="dataSource">
        <jdbc:script
            location="org/springframework/batch/core/schema-drop-sqlite.sql" />
        <jdbc:script location="org/springframework/batch/core/schema-sqlite.sql" />
    </jdbc:initialize-database>

    <!-- stored job-meta in memory -->
    <!-- <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> 
        <property name="transactionManager" ref="transactionManager" /> </bean> -->

    <!-- stored job-meta in database -->
    <bean id="jobRepository"
        class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="transactionManager" ref="transactionManager" />
        <property name="databaseType" value="sqlite" />
    </bean>

    <bean id="transactionManager"
        class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

    <bean id="jobLauncher"
        class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
    </bean>

</beans>

2.5.4 Maven pom.xml

Maven pom.xml is shown below:

Maven pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <artifactId>spring-batch</artifactId>
    <version>0.1-SNAPSHOT</version>
    <name>spring-batch</name>
    <packaging>jar</packaging>
    <url>http://maven.apache.org</url>

    <parent>
        <groupId>com.baeldung</groupId>
        <artifactId>parent-boot-2</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath>../parent-boot-2</relativePath>
    </parent>

    <dependencies>
        <!-- JAXB APIs & runtime no longer provided in JDK 11 -->
        <!-- see http://openjdk.java.net/jeps/320 -->
        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>${jaxb.version}</version>
            <scope>runtime</scope>
        </dependency>
        
        <dependency>
            <groupId>org.glassfish.jaxb</groupId>
            <artifactId>jaxb-runtime</artifactId>
            <version>${jaxb.version}</version>
            <scope>runtime</scope>
        </dependency>
        
        <!-- SQLite database driver -->
        <dependency>
            <groupId>org.xerial</groupId>
            <artifactId>sqlite-jdbc</artifactId>
            <version>${sqlite.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-oxm</artifactId>
            <version>${spring.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>commons-logging</artifactId>
                    <groupId>commons-logging</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
            <version>${spring.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-core</artifactId>
            <version>${spring.batch.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-test</artifactId>
            <version>${spring.batch.version}</version>
        </dependency>
        
        <dependency>
            <groupId>com.opencsv</groupId>
            <artifactId>opencsv</artifactId>
            <version>${opencsv.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>

        <dependency>
            <groupId>org.hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>org.awaitility</groupId>
            <artifactId>awaitility</artifactId>
            <version>${awaitility.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <properties>
        <spring.version>5.2.0.RELEASE</spring.version>
        <spring.batch.version>4.2.0.RELEASE</spring.batch.version>
        <sqlite.version>3.15.1</sqlite.version>
        <opencsv.version>4.1</opencsv.version>
        <jaxb.version>2.3.1</jaxb.version>
        <awaitility.version>3.1.1</awaitility.version>
    </properties>

</project>

2.5.3 Spring Batch Application

Spring batch Application class code is shown below.

Spring Batch Application

package org.javacodegeeks.batch;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;

public class Application {

    private static final Logger LOGGER = LoggerFactory.getLogger(Application.class);

    public static void main(final String[] args) {
   
        final AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
        context.register(SpringConfig.class);
        context.register(SpringBatchConfig.class);
       

        context.refresh();

   
        runJob(context, "firstBatchJob");
        runJob(context, "skippingBatchJob");
        runJob(context, "skipPolicyBatchJob");
   

    }

    private static void runJob(AnnotationConfigApplicationContext context, String batchJobName) {
        final JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
        final Job job = (Job) context.getBean(batchJobName);

        LOGGER.info("Starting the batch job: {}", batchJobName);
        try {
            
            JobParameters jobParameters = new JobParametersBuilder().addString("jobID", String.valueOf(System.currentTimeMillis()))
                .toJobParameters();
            final JobExecution execution = jobLauncher.run(job, jobParameters);
            LOGGER.info("Job Status : {}", execution.getStatus());
        } catch (final Exception e) {
            e.printStackTrace();
            LOGGER.error("Job failed {}", e.getMessage());
        }
    }
}

Model class BatchTransaction is created and the code is shown below:

package org.javacodegeeks.batch.model;

import org.javacodegeeks.batch.service.adapter.LocalDateTimeAdapter;

import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.time.LocalDateTime;

@SuppressWarnings("restriction")
@XmlRootElement(name = "transactionRecord")
public class BatchTransaction {
    private String username;
    private int userId;
    private int age;
    private String postCode;
    private LocalDateTime transactionDate;
    private double amount;


    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public int getUserId() {
        return userId;
    }

    public void setUserId(int userId) {
        this.userId = userId;
    }

    @XmlJavaTypeAdapter(LocalDateTimeAdapter.class)
    public LocalDateTime getTransactionDate() {
        return transactionDate;
    }

    public void setTransactionDate(LocalDateTime transactionDate) {
        this.transactionDate = transactionDate;
    }

    public double getAmount() {
        return amount;
    }

    public void setAmount(double amount) {
        this.amount = amount;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getPostCode() {
        return postCode;
    }

    public void setPostCode(String postCode) {
        this.postCode = postCode;
    }

    @Override
    public String toString() {
        return "Transaction [username=" + username + ", userId=" + userId + ", age=" + age + ", postCode=" + postCode + ", transactionDate=" + transactionDate + ", amount=" + amount + "]";
    }

}

Code can be executed using the below command.

Maven command

mvn package
cd spring-batch
mvn package

The output is shown below:

Output

apples-MacBook-Air:spring-batch bhagvan.kommadi$ mvn package
[INFO] Scanning for projects...
[INFO] gitflow-incremental-builder is disabled.
[INFO] 
[INFO] -------------------------------------------
[INFO] Building spring-batch 0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- directory-maven-plugin:0.3.1:directory-of (directories) @ spring-batch ---
[INFO] Directory of com.baeldung:parent-modules set to: /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install-file (install-jar-lib) @ spring-batch ---
[INFO] Installing /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/custom-pmd-0.0.1.jar to /Users/bhagvan.kommadi/.m2/repository/org/baeldung/pmd/custom-pmd/0.0.1/custom-pmd-0.0.1.jar
[INFO] Installing /var/folders/cr/0y892lq14qv7r24yl0gh0_dm0000gp/T/mvninstall1760246420817513314.pom to /Users/bhagvan.kommadi/.m2/repository/org/baeldung/pmd/custom-pmd/0.0.1/custom-pmd-0.0.1.pom
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ spring-batch ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ spring-batch ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] >>> maven-pmd-plugin:3.13.0:check (default) > :pmd @ spring-batch >>>
[INFO] 
[INFO] --- maven-pmd-plugin:3.13.0:pmd (pmd) @ spring-batch ---
[INFO] 
[INFO] <<< maven-pmd-plugin:3.13.0:check (default) < :pmd @ spring-batch <<<
[INFO] 
[INFO] 
[INFO] --- maven-pmd-plugin:3.13.0:check (default) @ spring-batch ---
[INFO] PMD version: 6.21.0
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ spring-batch ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 6 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ spring-batch ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ spring-batch ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.javacodegeeks.SpringContextTest
22:42:20,273 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
22:42:20,274 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
22:42:20,274 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/spring-batch/target/classes/logback.xml]
22:42:20,410 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
22:42:20,417 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
22:42:20,428 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead.
22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
22:42:20,496 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
22:42:20,499 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.taskletsvschunks] to DEBUG
22:42:20,499 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.taskletsvschunks] to false
22:42:20,499 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.taskletsvschunks]
22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.batchscheduler] to DEBUG
22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.batchscheduler] to false
22:42:20,500 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.batchscheduler]
22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.javacodegeeks.batchtesting] to DEBUG
22:42:20,500 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.javacodegeeks.batchtesting] to false
22:42:20,500 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.javacodegeeks.batchtesting]
22:42:20,501 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to ERROR
22:42:20,501 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
22:42:20,501 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
22:42:20,502 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@78b729e6 - Registering current configuration as safe fallback point

Processing...Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
Processing...Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
Processing...Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=, userId=25436, age=0, postCode=null, transactionDate=2020-10-03T00:00, amount=300.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=mathew, userId=91876, age=0, postCode=null, transactionDate=2019-11-05T00:00, amount=-510.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=, userId=34215, age=0, postCode=null, transactionDate=2018-10-10T00:00, amount=110.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
22:42:25.026 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step skippingStep in job skippingBatchJob
org.springframework.batch.core.step.skip.SkipLimitExceededException: Skip limit of '2' exceeded
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=, userId=25436, age=0, postCode=null, transactionDate=2020-10-03T00:00, amount=300.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=mathew, userId=91876, age=0, postCode=null, transactionDate=2019-11-05T00:00, amount=-510.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
SkippingItemProcessor: Batch Transaction [username=, userId=34215, age=0, postCode=null, transactionDate=2018-10-10T00:00, amount=110.0]
SkippingItemProcessor: Batch Transaction [username=david, userId=13234, age=0, postCode=null, transactionDate=2018-10-31T00:00, amount=10000.0]
SkippingItemProcessor: Batch Transaction [username=tom, userId=25134, age=0, postCode=null, transactionDate=2018-12-03T00:00, amount=12321.0]
SkippingItemProcessor: Batch Transaction [username=george, userId=12134, age=0, postCode=null, transactionDate=2018-02-02T00:00, amount=23411.0]
22:42:25.183 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step skipPolicyStep in job skipPolicyBatchJob
org.springframework.retry.RetryException: Non-skippable exception in recoverer while processing; nested exception is org.javacodegeeks.batch.service.MissingUsernameException
	at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$2.recover(FaultTolerantChunkProcessor.java:289)
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.076 s - in org.javacodegeeks.SpringContextTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ spring-batch ---
[INFO] Building jar: /Users/bhagvan.kommadi/Desktop/JavacodeGeeks/Code/springbatchexample/spring-batch/target/spring-batch-0.1-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.270 s
[INFO] Finished at: 2020-10-26T22:42:26+05:30
[INFO] ------------------------------------------------------------------------
apples-MacBook-Air:spring-batch bhagvan.kommadi$

3. Download the Source Code

In this article, we explained how to use a Spring Batch metadata table on different schema of the same DB.

Download
You can download the full source code of this example here: Spring Batch Metadata Table on Different Schema of the Same DB

Bhagvan Kommadi

Bhagvan Kommadi is the Founder of Architect Corner & has around 20 years’ experience in the industry, ranging from large scale enterprise development to helping incubate software product start-ups. He has done Masters in Industrial Systems Engineering at Georgia Institute of Technology (1997) and Bachelors in Aerospace Engineering from Indian Institute of Technology, Madras (1993). He is member of IFX forum,Oracle JCP and participant in Java Community Process. He founded Quantica Computacao, the first quantum computing startup in India. Markets and Markets have positioned Quantica Computacao in ‘Emerging Companies’ section of Quantum Computing quadrants. Bhagvan has engineered and developed simulators and tools in the area of quantum technology using IBM Q, Microsoft Q# and Google QScript. He has reviewed the Manning book titled : "Machine Learning with TensorFlow”. He is also the author of Packt Publishing book - "Hands-On Data Structures and Algorithms with Go".He is member of IFX forum,Oracle JCP and participant in Java Community Process. He is member of the MIT Technology Review Global Panel.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button