DJL Spring Boot Example

In this article, we will discuss what the DJL in Spring Boot is (Deep Java Library) and its uses. We will also see an example of making use of an existing model to detect an object using a spring boot application.

1.What is DJL in Spring Boot

Deep Java Library is one of Java’s libraries that provides a platform for Deep Learning. We can use DJL to train, build, and run deep learning models. It uses the existing deep learning framework to predict and develop models. It provides a simple API to use deep learning by abstracting out all the complexity.
DJL also provides automatic processing choices(CPU/GPU) based on the hardware configuration from a hardware perspective.
It uses MXNet, PyTorch, TensorFlow, and several other popular deep-learning/machine learning frameworks under the hood and provides a very simple java API with a small learning curve.

2. Sample Application

Here we will create a simple application which will use the existing model-zoo models to detect objects in a given image. In this application we will upload any image with different objects like car, traffic, people,etc.

Once we feed the image to the model, the model will run and draw a box surrounding the objects it has detected with the name given.

We will be using springboot to build the application with maven. Lets get started

2.1 The pom file

We will add all the dependency required to run the application

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>ai.djl</groupId>
    <artifactId>image-object-detection</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
 
    </properties>
    <repositories>
        <repository>
            <id>djl.ai</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
        </repository>
    </repositories>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>ai.djl</groupId>
                <artifactId>bom</artifactId>
                <version>0.9.0-SNAPSHOT</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>2.3.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>ai.djl</groupId>
            <artifactId>model-zoo</artifactId>
            <version>0.9.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-model-zoo</artifactId>
            <version>0.9.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-engine</artifactId>
            <version>0.9.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-native-auto</artifactId>
            <scope>runtime</scope>
        </dependency>
    </dependencies>
</project>

In the above file, we have the web springboot starter dependency and others are the dependencies for the Deep Java Library. Which consists of the existing training models and also the engine for processing and analyzing the model into the input passed.

2.2 Code for the object detection

This is the controller which will accept the uploaded image and process it using the DJL engines and provide the output in the browser ImageDetectController.java

package com.jcg.djl;
import ai.djl.Application;
import ai.djl.ModelException;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelZoo;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.training.util.ProgressBar;
import ai.djl.translate.TranslateException;
import org.apache.commons.compress.utils.IOUtils;
import org.springframework.core.io.ClassPathResource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
@RestController
public class ImageDetectController {
    @PostMapping(value = "/upload", produces = MediaType.IMAGE_PNG_VALUE)
    public ResponseEntity diagnose(@RequestParam("file") MultipartFile file) throws ModelException, TranslateException, IOException {
        byte[] bytes = file.getBytes();
        Path imageFile = Paths.get(file.getOriginalFilename());
        Files.write(imageFile, bytes);
        return predict(imageFile);
    }
    public ResponseEntity predict(Path imageFile) throws IOException, ModelException, TranslateException {
        Image img = ImageFactory.getInstance().fromFile(imageFile);
        Criteria criteria =
                Criteria.builder()
                        .optApplication(Application.CV.OBJECT_DETECTION)
                        .setTypes(Image.class, DetectedObjects.class)
                        .optFilter("backbone", "resnet50")
                        .optProgress(new ProgressBar())
                        .build();
        try (ZooModel model = ModelZoo.loadModel(criteria)) {
            try (Predictor predictor = model.newPredictor()) {
                DetectedObjects detection = predictor.predict(img);
                return saveBoundingBoxImage(img, detection);
            }
        }
    }
    private ResponseEntity saveBoundingBoxImage(Image img, DetectedObjects detection)
            throws IOException {
        Path outputDir = Paths.get("src/main/resources");
        Files.createDirectories(outputDir);
        Image newImage = img.duplicate(Image.Type.TYPE_INT_ARGB);
        newImage.drawBoundingBoxes(detection);
        Path imagePath = outputDir.resolve("detected.png");
        newImage.save(Files.newOutputStream(imagePath), "png");
        String fileDownloadUri = ServletUriComponentsBuilder.fromCurrentContextPath()
                .path("get")
                .toUriString();
        return ResponseEntity.ok(fileDownloadUri);
    }
    @GetMapping(
            value = "/get",
            produces = MediaType.IMAGE_PNG_VALUE
    )
    public @ResponseBody
    byte[] getImageWithMediaType() throws IOException {
        InputStream in = new ClassPathResource(
                "detected.png").getInputStream();
        return IOUtils.toByteArray(in);
    }
}

This file contains the actual code which uses the model-zoo model to detect the objects in the image passed

2.3 Application Main File

This is the main class which will run the springboot application ImageObjectDetectionApplication.java

package com.jcg.djl;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class ImageObjectDetectionApplication {
    public static void main(String[] args) {
        SpringApplication.run(ImageObjectDetectionApplication.class, args);
    }
}

3. Running the application

You can run the application using the command below from the root folder

./mvnw spring-boot:run

Once the springboot application has started, you can go to the http://localhost:8080/index.html

Now we will upload the sample file as shown below

Once after uploading the image, you can click the upload button. It processes the image and provides you with the link as shown below

Enter the link into the browser and you will get the processed image with the object detected as a person, car, etc. into the browser as shown below.

4. Summary

In this article, we discussed the DJL Spring Boot and its uses. We also created a spring boot application to use the existing training model and applied that into an image and verified the object detected.

You can read more Spring Boot articles here.

5. Download the source code

That was a DJL spring boot example.