Elasticsearch is a highly scalable open-source which can be used for data store, text search and analytics engine. Every instance of ElasticSearch is called a node and several nodes can be grouped together in a cluster.
In this article, we will see how we can use spring-data-elasticsearch module which integrates spring-data and elasticsearch.
1. Dependencies
Include <spring-core>, <spring-context> and <spring-data-elasticsearch> in your pom.xml.
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.javacodegeeks.data.elasticsearch</groupId> <artifactId>springDataElasticsearchExample</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>SpringElasticsearchExample</name> <description>Example of spring elasticsearch</description> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>4.1.5.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>4.1.5.RELEASE</version> </dependency> <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-elasticsearch</artifactId> <version>1.3.2.RELEASE</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
2. Start Elasticsearch Sever
Download the latest Elasticsearch and unzip the file. Go to elasticsearch working folder/bin, in our case elasticsearch-2.1.1\bin and run elasticsearch command. elasticsearch.yml is the main configuration file for ElasticSearch. We can set here the node name and cluster name.
C:\elasticsearch-2.1.1\bin<elasticsearch
[2016-01-17 20:20:22,821][WARN ][bootstrap ] unable to install sy
scall filter: syscall filtering not supported for OS: 'Windows 7'
[2016-01-17 20:20:23,043][INFO ][node ] [Wilbur Day] version
[2.1.1], pid[9784], build[40e2c53/2015-12-15T13:05:55Z]
[2016-01-17 20:20:23,043][INFO ][node ] [Wilbur Day] initial
izing ...
[2016-01-17 20:20:23,123][INFO ][plugins ] [Wilbur Day] loaded
[], sites []
[2016-01-17 20:20:23,149][INFO ][env ] [Wilbur Day] using [
1] data paths, mounts [[OSDisk (C:)]], net usable_space [24.2gb], net total_spac
e [476gb], spins? [unknown], types [NTFS]
[2016-01-17 20:20:25,551][INFO ][node ] [Wilbur Day] initial
ized
[2016-01-17 20:20:25,552][INFO ][node ] [Wilbur Day] startin
g ...
[2016-01-17 20:20:25,903][INFO ][transport ] [Wilbur Day] publish
_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2016-01-17 20:20:25,912][INFO ][discovery ] [Wilbur Day] elastic
search/d5McLMFpTNGpnYEZDacPvg
[2016-01-17 20:20:29,945][INFO ][cluster.service ] [Wilbur Day] new_mas
ter {Wilbur Day}{d5McLMFpTNGpnYEZDacPvg}{127.0.0.1}{127.0.0.1:9300}, reason: zen
-disco-join(elected_as_master, [0] joins received)
[2016-01-17 20:20:30,002][INFO ][gateway ] [Wilbur Day] recover
ed [0] indices into cluster_state
[2016-01-17 20:20:30,160][INFO ][http ] [Wilbur Day] publish
_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2016-01-17 20:20:30,160][INFO ][node ] [Wilbur Day] started
3. Elasticsearch Storage Structure
Before we start our spring data elasticsearch example, its important to understand the Elasticsearch storage structure.
- Index – This is the main data container it is analogous to database in SQL
- Mappings – Data is organized as data types called mappings. The equivalent structure in SQL is table.
- Field – A mapping contains records which in rurn are composed of fields.
- Object – This is the format of a record which is in form of JSON object
4. Store and retrieve Data
In order to have a working system, all we need to do is define domain entities and a repository class for the support of CRUD machinery. In order to mark a POJO class as domain entity, we just need to add org.springframework.data.elasticsearch.annotations.Document to our index object. indexing your objects to Elasticsearch is to add the @Document annotation to them and create a Repository interface extending ElasticsearchRepository.
Let’s first define index and mapping.
Employee:
package com.javacodegeeks.spring.elasticsearch;
import java.util.List;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
@Document(indexName = "resource", type = "employees")
public class Employee {
@Id
private String id;
private String name;
private Integer age;
@Field( type = FieldType.Nested)
private List<Skill> skills;
public Employee(){}
public Employee(String id, String name, int age) {
this.id = id;
this.name = name;
this.age = age;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
public List<Skill> getSkills() {
return skills;
}
public void setSkills(List<Skill> skills) {
this.skills = skills;
}
public String toString() {
return "Employee [(" + getId() + ", " + getName() + ", " + age + "), skills: " + getSkills() + "]";
}
}
It depends on POJO skill which is our embedded object so its type is defined as FieldType.NESTED.
Skill:
package com.javacodegeeks.spring.elasticsearch;
public class Skill {
private String name;
private int experience;
public Skill() {
}
public Skill(String name, int experience) {
this.name = name;
this.experience = experience;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getExperience() {
return experience;
}
public void setExperience(int experience) {
this.experience = experience;
}
public String toString() {
return "Skill(" + name + ", " + experience + ")";
}
}
5. Employee Repository
EmployeeRepository extends spring data provided ElasticsearchRepository which is the base repository class for elasticsearch based domain classes. Since it extends Spring based repository classes, we get the benefit of avoiding boilerplate code required to implement data access layers for various persistence stores.
Repository is the central markup interface in Spring Data repository. It takes the domain class to manage as well as the id type of the domain class as type arguments. Its main purpose is to make the repository typed. The next main interface is CrudRepository which provides sophisticated CRUD functionality for the entity class that is being managed. On top of the CrudRepository there is a PagingAndSortingRepository abstraction that adds additional methods to ease paginated access to entities.
Declare query methods on the interface. Since we are using Spring JPA repository we don’t need to write implementation for it.
EmployeeRepository:
package com.javacodegeeks.spring.elasticsearch;
import java.util.List;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
public interface EmployeeRepository extends ElasticsearchRepository {
List findEmployeesByAge(int age);
List findEmployeesByName(String name);
List findEmployeesBySkillsIn(List skills);
}
6. Spring Configuration
The Spring Data Elasticsearch module contains a custom namespace allowing definition of repository beans as well as elements for instantiating a ElasticsearchServer.
xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch
Spring is instructed to scan com.javacodegeeks.spring.elasticsearch and all its sub-packages for interfaces extending Repository or one of its sub-interfaces.
Next, we are using a Node Client element to register an instance of Elasticsearch Server in the context.
<elasticsearch:node-client id="client" local="true"/>
If you want to create NodeClient programatically, you can do it using node builder.
private static NodeClient getNodeClient() {
return (NodeClient) nodeBuilder().clusterName(UUID.randomUUID().toString()).local(true).node()
.client();
}
applicationContext.xml:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
xsi:schemaLocation="http://www.springframework.org/schema/data/elasticsearch http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch-1.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:annotation-config/>
<bean name="mainBean" class="com.javacodegeeks.spring.elasticsearch.SpringElasticsearchExample"/>
<elasticsearch:repositories base-package="com.javacodegeeks.spring.elasticsearch"/>
<elasticsearch:node-client id="client" local="true"/>
<bean name="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
<constructor-arg name="client" ref="client"/>
</bean>
</beans>
7. Configuration using Annotation
The Spring Data Elasticsearch repositories scan also be activated using annotation @EnableElasticsearchRepositories.
SpringElasticsearchExampleUsingAnnotation:
@Configuration("mainBean")
@EnableElasticsearchRepositories(basePackages = "com.javacodegeeks.spring.elasticsearch")
public class SpringElasticsearchExampleUsingAnnotation {
@Autowired
private EmployeeRepository repository;
@Autowired
private ElasticsearchTemplate template;
@Bean
public ElasticsearchTemplate elasticsearchTemplate() {
return new ElasticsearchTemplate(getNodeClient());
}
...
}
8. Run the Example
SpringElasticsearchExample loads the spring context. It next gets the SpringElasticsearchExample bean and adds few employees. We then execute several finder methods to list the employees.
The the repository instance EmployeeRepository is injected into it using @Autowired.
We also inject bean ElasticsearchTemplate which is the central class that spring provides using which we save our domain entities.
SpringElasticsearchExample:
package com.javacodegeeks.spring.elasticsearch;
import java.net.URISyntaxException;
import java.util.Arrays;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.query.IndexQuery;
@Configuration
public class SpringElasticsearchExample {
@Autowired
private EmployeeRepository repository;
@Autowired
private ElasticsearchTemplate template;
public static void main(String[] args) throws URISyntaxException, Exception {
ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext(
"applicationContext.xml");
try {
System.out.println("Load context");
SpringElasticsearchExample s = (SpringElasticsearchExample) ctx
.getBean("mainBean");
System.out.println("Add employees");
s.addEmployees();
System.out.println("Find all employees");
s.findAllEmployees();
System.out.println("Find employee by name 'Joe'");
s.findEmployee("Joe");
System.out.println("Find employee by name 'John'");
s.findEmployee("John");
System.out.println("Find employees by age");
s.findEmployeesByAge(32);
} finally {
ctx.close();
}
}
public void addEmployees() {
Employee joe = new Employee("01", "Joe", 32);
Skill javaSkill = new Skill("Java", 10);
Skill db = new Skill("Oracle", 5);
joe.setSkills(Arrays.asList(javaSkill, db));
Employee johnS = new Employee("02", "John S", 32);
Employee johnP = new Employee("03", "John P", 42);
Employee sam = new Employee("04", "Sam", 30);
template.putMapping(Employee.class);
IndexQuery indexQuery = new IndexQuery();
indexQuery.setId(joe.getId());
indexQuery.setObject(joe);
template.index(indexQuery);
template.refresh(Employee.class, true);
repository.save(johnS);
repository.save(johnP);
repository.save(sam);
}
public void findAllEmployees() {
repository.findAll().forEach(System.out::println);
}
public void findEmployee(String name) {
List empList = repository.findEmployeesByName(name);
System.out.println("Employee list: " + empList);
}
public void findEmployeesByAge(int age) {
List empList = repository.findEmployeesByAge(age);
System.out.println("Employee list: " + empList);
}
}
Output:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Load context Add employees Find all employees Employee [(04, Sam, 30), skills: null] Employee [(01, Joe, 32), skills: [Skill(Java, 10), Skill(Oracle, 5)]] Employee [(02, John S, 32), skills: null] Employee [(03, John P, 42), skills: null] Find employee by name 'Joe' Employee list: [Employee [(01, Joe, 32), skills: [Skill(Java, 10), Skill(Oracle, 5)]]] Find employee by name 'John' Employee list: [Employee [(02, John S, 32), skills: null], Employee [(03, John P, 42), skills: null]] Find employees by age Employee list: [Employee [(01, Joe, 32), skills: [Skill(Java, 10), Skill(Oracle, 5)]], Employee [(02, John S, 32), skills: null]]
9. Annotation based main class
We can recreate the above example using just the annotations. If you notice, we have created ElasticsearchTemplate</code. programatically.
@Bean
public ElasticsearchTemplate elasticsearchTemplate() {
return new ElasticsearchTemplate(getNodeClient());
}
annotationApplicationContext.xml:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
xsi:schemaLocation="http://www.springframework.org/schema/data/elasticsearch http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch-1.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:annotation-config/>
<elasticsearch:node-client id="client" local="true"/>
<bean name="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
<constructor-arg name="client" ref="client"/>
</bean>
</beans>
SpringElasticsearchExampleUsingAnnotation:
package com.javacodegeeks.spring.elasticsearch;
import static org.elasticsearch.node.NodeBuilder.nodeBuilder;
import java.net.URISyntaxException;
import java.util.Arrays;
import java.util.List;
import java.util.UUID;
import org.elasticsearch.client.node.NodeClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.query.IndexQuery;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;
@Configuration("mainBean")
@EnableElasticsearchRepositories(basePackages = "com.javacodegeeks.spring.elasticsearch")
public class SpringElasticsearchExampleUsingAnnotation {
@Autowired
private EmployeeRepository repository;
@Autowired
private ElasticsearchTemplate template;
@Bean
public ElasticsearchTemplate elasticsearchTemplate() {
return new ElasticsearchTemplate(getNodeClient());
}
public static void main(String[] args) throws URISyntaxException, Exception {
AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext();
try {
ctx.register(SpringElasticsearchExampleUsingAnnotation.class);
ctx.refresh();
System.out.println("Load context");
SpringElasticsearchExampleUsingAnnotation s = (SpringElasticsearchExampleUsingAnnotation) ctx
.getBean("mainBean");
System.out.println("Add employees");
s.addEmployees();
System.out.println("Find all employees");
s.findAllEmployees();
System.out.println("Find employee by name 'Joe'");
s.findEmployee("Joe");
System.out.println("Find employee by name 'John'");
s.findEmployee("John");
System.out.println("Find employees by age");
s.findEmployeesByAge(32);
} finally {
ctx.close();
}
}
public void addEmployees() {
Employee joe = new Employee("01", "Joe", 32);
Skill javaSkill = new Skill("Java", 10);
Skill db = new Skill("Oracle", 5);
joe.setSkills(Arrays.asList(javaSkill, db));
Employee johnS = new Employee("02", "John S", 32);
Employee johnP = new Employee("03", "John P", 42);
Employee sam = new Employee("04", "Sam", 30);
template.putMapping(Employee.class);
IndexQuery indexQuery = new IndexQuery();
indexQuery.setId(joe.getId());
indexQuery.setObject(joe);
template.index(indexQuery);
template.refresh(Employee.class, true);
repository.save(johnS);
repository.save(johnP);
repository.save(sam);
}
public void findAllEmployees() {
repository.findAll().forEach(System.out::println);
}
public void findEmployee(String name) {
List<Employee> empList = repository.findEmployeesByName(name);
System.out.println("Employee list: " + empList);
}
public void findEmployeesByAge(int age) {
List<Employee> empList = repository.findEmployeesByAge(age);
System.out.println("Employee list: " + empList);
}
private static NodeClient getNodeClient() {
return (NodeClient) nodeBuilder().clusterName(UUID.randomUUID().toString()).local(true).node()
.client();
}
}
10. Elasticsearch as RESTFul Server
ElasticSearch can also be used as a RESTful server, the main protocol is the HTTP, listening on port number 9200 (default).
To view the index type: Enter http://localhost:9200/resource/employees/_search
{"resource":{"aliases":{},"mappings":{"employees":{"properties":{"age":{"type":"long"},"id":{"type":"string"},"name":{"type":"string"},"skills":{"type":"nested","properties":{"experience":{"type":"long"},"name":{"type":"string"}}}}}},"settings":{"index":{"refresh_interval":"1s","number_of_shards":"5","creation_date":"1453094779722","store":{"type":"fs"},"uuid":"7YWl_3VBTq-eluY74GU4sQ","version":{"created":"1050299"},"number_of_replicas":"1"}},"warmers":{}}}
To find all employees, enter http://localhost:9200/resource/employees/_search. Here’s the result JSON object.
{"took":6,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":4,"max_score":1.0,"hits":[{"_index":"resource","_type":"employees","_id":"04","_score":1.0,"_source":{"id":"04","name":"Sam","age":30,"skills":null}},{"_index":"resource","_type":"employees","_id":"01","_score":1.0,"_source":{"id":"01","name":"Joe","age":32,"skills":[{"name":"Java","experience":10},{"name":"Oracle","experience":5}]}},{"_index":"resource","_type":"employees","_id":"02","_score":1.0,"_source":{"id":"02","name":"John S","age":32,"skills":null}},{"_index":"resource","_type":"employees","_id":"03","_score":1.0,"_source":{"id":"03","name":"John P","age":42,"skills":null}}]}}
You can also restrict the employees per page. For example, enter http://localhost:9200/resource/employees/_search?page=1&size=2 to fetch two employees in the first page.
{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":4,"max_score":1.0,"hits":[{"_index":"resource","_type":"employees","_id":"04","_score":1.0,"_source":{"id":"04","name":"Sam","age":30,"skills":null}},{"_index":"resource","_type":"employees","_id":"01","_score":1.0,"_source":{"id":"01","name":"Joe","age":32,"skills":[{"name":"Java","experience":10},{"name":"Oracle","experience":5}]}}]}}
You can also fetch the employee by ID. For example, enter http://localhost:9200/resource/employees/01
{"_index":"resource","_type":"employees","_id":"01","_version":3,"found":true,"_source":{"id":"01","name":"Joe","age":32,"skills":[{"name":"Java","experience":10},{"name":"Oracle","experience":5}]}}
11. Download the Eclipse Project
This was an example about spring data Elasticsearch.
You can download the full source code of this example here: springDataElasticsearch.zip
