Home » Enterprise Java » hibernate » Hibernate Batch Processing Example

About Yatin

Yatin
The author is graduated in Electronics & Telecommunication. During his studies, he has been involved with a significant number of projects ranging from programming and software engineering to telecommunications analysis. He works as a technical lead in the information technology sector where he is primarily involved with projects based on Java/J2EE technologies platform and novel UI technologies.

Hibernate Batch Processing Example

Hibernate Batch processing is an easy way to add multiple statements into a batch and execute that batch by making a single round trip to the database. This tutorial shows how to create batch insert and batch update statements using JPA and Hibernate. JDBC offers support for Batching together SQL statements that can be represented as a single PreparedStatement.
 
 
 
 
 
 

1. Introduction

1.1 Hibernate

  • Object-Relational Mapping or ORM is the programming technique to map application domain model objects to the relational database tables
  • Hibernate is Java based ORM tool that provides a framework for mapping application domain objects to the relational database tables and vice versa. It provides reference implementation of Java Persistence API, that makes it a great choice as an ORM tool with benefits of loose coupling
  • A framework that provides option to map plain old Java objects to traditional database tables with the use of JPA annotations as well as XML based configuration
Fig. 1: Hibernate Overview

Fig. 1: Hibernate Overview

1.2 Hibernate Annotations

  • Hibernate annotations is the newest way to define mappings without the use of an XML file
  • Developers use annotations to provide metadata configuration along with the Java code. Thus, making the code easy to understand
  • XML provides the ability to change the configuration without building the project. Thus, annotations are less powerful than XML configuration and should only be used for table and column mappings
  • Annotations are preconfigured with sensible default values, which reduce the amount of coding required. For e.g. Class name defaults to Table name and field names defaults to column names

1.3 Hibernate Batch Processing

Let’s start by trying to understand the concept of Batch processing. It’s an automatic treatment of the non-interactive jobs. Non-interactive means that there is no human intervention as, for example, form filling or manual configuration for every treated task.

A good example of batch processing is the billing system of your mobile. Last day of every month you receive a billing with an amount to pay. The amount is calculated by adding the price of every call you made. The calculation is made automatically, at the end of every month. You don’t receive the invoice after every call.

Consider a situation when developers need to upload a large number of records into the database using Hibernate. Below is the code snippet to achieve this using Hibernate.

sessionObj.beginTransaction();

int totalRecords = 1000000;
// - - - - - - - - - - - - - - Hibernate/JPA Batch Insert Example - - - - - - - - - - - - //
for (int i = 0; i < totalRecords; i++) {
	Product product = new Product("Product " + i);
	sessionObj.save(product);				
}
System.out.println("\n.......Records Saved Successfully to The Database.......\n");
			
// Committing The Transactions to The Database
sessionObj.getTransaction().commit();

By default, Hibernate framework will cache all the persisted objects in the session-level cache and ultimately the application would fall over with an OutOfMemoryException.

...
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.util.jar.Attributes.read(Attributes.java:394)
...

Developers can resolve this problem if they are using batch processing with Hibernate.

1.4 Download and Install Hibernate

You can read this tutorial in order to download and install Hibernate in the Eclipse IDE.

1.5 Download and Install MySQL

You can watch this video in order to download and install the MySQL database on your Windows operating system.

Now, open up the Eclipse IDE and let’s see how to implement batch processing in Hibernate!

2. Hibernate Batch Processing Example

2.1 Tools Used

We are using Eclipse Kepler SR2, JDK 8, MySQL Database and Maven. Having said that, we have tested the code against JDK 1.7 and it works well.

2.2 Project Structure

Firstly, let’s review the final project structure, in case you are confused about where you should create the corresponding files or folder later!

Fig. 2: Hibernate Batch Processing Application Project Structure

Fig. 2: Hibernate Batch Processing Application Project Structure

2.3 Project Creation

This section will demonstrate on how to create a Java based Maven project with Eclipse. In Eclipse IDE, go to File -> New -> Maven Project.

Fig. 3: Create Maven Project

Fig. 3: Create Maven Project

In the New Maven Project window, it will ask you to select project location. By default, ‘Use default workspace location‘ will be selected. Select the ‘Create a simple project (skip archetype selection)‘ checkbox and just click on next button to proceed.

Fig. 4: Project Details

Fig. 4: Project Details

It will ask you to ‘Enter the group and the artifact id for the project’. We will input the details as shown in the below image. The version number will be by default 0.0.1-SNAPSHOT.

Fig. 5: Archetype Parameters

Fig. 5: Archetype Parameters

Click on Finish and the creation of a maven project is completed. If you observe, it has downloaded the maven dependencies and a pom.xml file will be created. It will have the following code:

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>HibernateBatch</groupId>
	<artifactId>HibernateBatch</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>
</project>

We can start adding the dependencies that developers want like Hibernate, MySQL etc. Let’s start building the application!

3. Application Building

Below are the steps involved in developing this application.

3.1 Database & Table Creation

The following MySQL script is used to create a database called tutorialDb with table: product. Open MySQL terminal or workbench terminal and execute the script.

CREATE DATABASE IF NOT EXISTS tutorialDb;

USE tutorialDb;

DROP TABLE IF EXISTS product;

CREATE TABLE product (
  product_id int(20) NOT NULL AUTO_INCREMENT,
  product_code varchar(255) DEFAULT NULL,
  PRIMARY KEY (product_id)
);

If everything goes well, the table will be shown in the MySQL workbench.

Fig. 6: Database & Table Creation

Fig. 6: Database & Table Creation

3.2 Maven Dependencies

Here, we specify only two dependencies for Hibernate Core and MySQL Connector. Rest dependencies will be automatically resolved by Maven, such as Hibernate JPA and Hibernate Commons Annotations. The updated file will have the following code:

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>HibernateBatch</groupId>
	<artifactId>HibernateBatch</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>
	<dependencies>
		<!-- Hibernate 4.3.6 Final -->
		<dependency>
			<groupId>org.hibernate</groupId>
			<artifactId>hibernate-core</artifactId>
			<version>4.3.6.Final</version>
		</dependency>
		<!-- Mysql Connector -->
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<version>5.1.21</version>
		</dependency>
	</dependencies>
	<build>
		<finalName>${project.artifactId}</finalName>
	</build>
</project>

3.3 Java Class Creation

Let’s create the required Java files. Right click on src/main/java folder, New -> Package.

Fig. 7: Java Package Creation

Fig. 7: Java Package Creation

A new pop window will open where we will enter the package name as: com.jcg.hibernate.batch.processing.

Fig. 8: Java Package Name (com.jcg.hibernate.batch.processing)

Fig. 8: Java Package Name (com.jcg.hibernate.batch.processing)

Once the package is created in the application, we will need to create the model and implementation classes. Right-click on the newly created package: New -> Class.

Fig. 9: Java Class Creation

Fig. 9: Java Class Creation

A new pop window will open and enter the file name as Product. The model class will be created inside the package: com.jcg.hibernate.batch.processing.

Fig. 10: Java Class (Product.java)

Fig. 10: Java Class (Product.java)

Repeat the step (i.e. Fig. 9) and enter the filename as AppMain. The implementation class will be created inside the package: com.jcg.hibernate.batch.processing.

Fig. 11: Java Class (AppMain.java)

Fig. 11: Java Class (AppMain.java)

3.3.1 Implementation of Model Class

Consider the following POJO class and add the following code to it:

Product.java

package com.jcg.hibernate.batch.processing;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Table(name = "product")
public class Product {

	@Id
	@Column(name = "product_id")
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	private long productId;

	@Column(name = "product_code")
	private String productCode;

	public Product() { }

	public Product(String productCode) {
		this.productCode = productCode;
	}

	public long getProductId() {
		return productId;
	}

	public void setProductId(long productId) {
		this.productId = productId;
	}

	public String getProductCode() {
		return productCode;
	}

	public void setProductCode(String productCode) {
		this.productCode = productCode;
	}
}

3.3.2 Implementation of Utility Class

This class helps in creating the SessionFactory from the Hibernate configuration file and interacts with the database to perform the batch operation. Here we will use flush() and clear() methods available with the Session object so that Hibernate keep writing the records into the database instead of caching them in the JVM memory. Add the following code to it:

AppMain.java

package com.jcg.hibernate.batch.processing;

import java.util.List;

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.cfg.Configuration;
import org.hibernate.service.ServiceRegistry;

@SuppressWarnings("unchecked")
public class AppMain {

	static Session sessionObj;
	static SessionFactory sessionFactoryObj;

	private static SessionFactory buildSessionFactory() {
		// Creating Configuration Instance & Passing Hibernate Configuration File
		Configuration configObj = new Configuration();
		configObj.configure("hibernate.cfg.xml");

		// Since Hibernate Version 4.x, ServiceRegistry Is Being Used
		ServiceRegistry serviceRegistryObj = new StandardServiceRegistryBuilder().applySettings(configObj.getProperties()).build(); 

		// Creating Hibernate SessionFactory Instance
		sessionFactoryObj = configObj.buildSessionFactory(serviceRegistryObj);
		return sessionFactoryObj;
	}

	public static void main(String[] args) {
		System.out.println(".......Hibernate Batch Processing Example.......\n");
		try {
			sessionObj = buildSessionFactory().openSession();
			sessionObj.beginTransaction();

			int batchSize = 30, totalRecords = 100;
			// - - - - - - - - - - - - - - Hibernate/JPA Batch Insert Example - - - - - - - - - - - - //
			for (int i = 0; i < totalRecords; i++) { Product product = new Product("Product " + i); sessionObj.save(product); if (i % batchSize == 0 && i > 0) {
					// Flush A Batch Of Inserts & Release Memory
					sessionObj.flush();
					sessionObj.clear();
				}
			}
			System.out.println("\n.......Records Saved Successfully To The Database.......\n");

			//  - - - - - - - - - - - - - - Hibernate/JPA Batch Update Example - - - - - - - - - - - - //
			String sqlQuery = "FROM Product";
			List productList = sessionObj.createQuery(sqlQuery).list();
			for (int j = 0; j < productList.size(); j++) { Product projectObj = productList.get(j); projectObj.setProductCode("New Product " + j); sessionObj.update(projectObj); if (j % batchSize == 0 && j > 0) {
					// Flush A Batch Of Updates & Release Memory
					sessionObj.flush();
					sessionObj.clear();
				}
			}
			System.out.println("\n.......Records Updated Successfully In The Database.......\n");

			// Committing The Transactions To The Database
			sessionObj.getTransaction().commit();
		} catch(Exception sqlException) {
			if(null != sessionObj.getTransaction()) {
				System.out.println("\n.......Transaction Is Being Rolled Back.......");
				sessionObj.getTransaction().rollback();
			}
			sqlException.printStackTrace();
		} finally {
			if(sessionObj != null) {
				sessionObj.close();
			}
		}
	}
}

3.4 Hibernate Configuration File

To configure the Hibernate framework, we need to implement a configuration file i.e. hiberncate.cfg.xml. Right-click on src/main/resources folder, New -> Other.

Fig. 12: XML File Creation

Fig. 12: XML File Creation

A new pop window will open and select the wizard as an XML file.

Fig. 13: Wizard Selection

Fig. 13: Wizard Selection

Again, a pop-up window will open. Verify the parent folder location as HibernateOneToOneMapping/src/main/resources and enter the file name as hibernate.cfg.xml. Click Finish.

Fig. 14: hibernate.cfg.xml

Fig. 14: hibernate.cfg.xml

Once the file is created, we will include the database configuration and mapping classes details. For batch processing in Hibernate, developers need to set the JDBC batch size in this file using hibernate.jdbc.batch_size property. The ideal value of this property is between 10 to 50. Add the following code to it:

hibernate.cfg.xml

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE hibernate-configuration PUBLIC
        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
        "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
	<session-factory>
		<!-- SQL Dialect -->
		<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>

		<!-- Database Connection Settings -->
		<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
		<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/tutorialDb</property>
		<property name="hibernate.connection.username">root</property>
		<property name="hibernate.connection.password"></property>
		<property name="show_sql">true</property>

		<!-- Specifying Session Context -->
		<property name="hibernate.current_session_context_class">org.hibernate.context.internal.ThreadLocalSessionContext</property>

		<!-- Batching Size Settings -->
		<property name="hibernate.jdbc.batch_size">30 </property>
		<property name="hibernate.order_inserts">true </property>
		<property name="hibernate.order_updates">true </property>
		<property name="hibernate.jdbc.batch_versioned_data">true </property>

		<!-- Mapping With Model Class Containing Annotations -->
		<mapping class="com.jcg.hibernate.batch.processing.Product" />
	</session-factory>
</hibernate-configuration>

Notes:

  • Here, we instructed Hibernate to connect to a MySQL database named tutorialDb and the Mapping classes to be loaded
  • We have also instructed Hibernate framework to use MySQLDialect i.e. Hibernate will optimize the generated SQL statements for MySQL
  • This configuration will be used to create a Hibernate SessionFactory object
  • show_sql: This tag will instruct the Hibernate framework to log all the SQL statements on the console
  • hibernate.jdbc.batch_size: This tag controls the maximum number of statements that Hibernate will batch together before asking the driver to execute the batch. Zero or a negative number disables this feature
  • hibernate.jdbc.batch_versioned_data: Some JDBC drivers return incorrect row counts when a batch is executed. If your JDBC driver falls into this category, this setting should be set to false. Otherwise, it is safe to enable this which will allow Hibernate to still batch the DML for versioned entities and still use the returned row counts for the optimistic lock checks

4. Run the Application

To run the Hibernate application, Right click on the AppMain class -> Run As -> Java Application.

Fig. 15: Run Application

Fig. 15: Run Application

5. Project Demo

On executing the AppMain class, you will see the records in product table. Developers can debug the example and see what happens in the database after every step. Enjoy!

Fig. 16: Application Output

Fig. 16: Application Output

Below is the snapshot of MySQL Database after execution of the above program.

Product Table

Fig. 17: Author Table Records (Insert Operation)

Fig. 17: Author Table Records (Insert Operation)

Fig. 18: Author Table Records (Update Operation)

Fig. 18: Author Table Records (Update Operation)

That’s all for this post. Happy Learning!!

6. Conclusion

Hibernate Batch processing is powerful but it has many pitfalls that developers must be aware of in order to use it properly and efficiently. That’s all for Hibernate batch processing example tutorial and I hope this article served you whatever you were looking for.

7. Download the Eclipse Project

This was an example of Hibernate Batch Processing.

Download
You can download the full source code of this example here: Hibernate Batch

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

 

1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design

 

and many more ....

 

Receive Java & Developer job alerts in your Area

 

1
Leave a Reply

avatar
1 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
1 Comment authors
Harish Sharma Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
Harish Sharma
Guest

The JDBC batching in your example code do not work for the INSERT queries. It only works for UPDATE queries. You can verify the same by setting “hibernate.generate_statistics” property to true. The issue with JDBC batching not running in case of INSERT is because of the “@GeneratedValue(strategy = GenerationType.IDENTITY)” with MySQL database only. Since MySQL database does not have the concept of Sequence and your database table primary key has “AUTO_INCREMENT” enabled, using “IDENTITY” generation type causes it to fetch the value of id in separate statements. And in case of updates, since you already have the “product_id” value with… Read more »