Spring Batch With MongoDB Example
In this example, we will build a simple Batch application using Spring Batch with MongoDB in Java Spring Boot.
Spring Batch is an open source, lightweight framework which is designed for use in developing robust batch applications. Batch applications are usually required by systems that need to process large volume of data on daily basis.
Spring Batch is not designed for use as a scheduling framework. However, it can be used in combination with a scheduling framework such as Quartz, Control-M, etc.
Spring Boot provides spring-boot-starter-batch dependency.
This sample batch application that we will build in this example will do the following:
- Read data from a CSV file.
- Transform the data.
- Write the final results to a MongoDB database.
Complete the following steps to build our sample batch application:
Create a Spring Boot Application
- Go to Spring Initializr at https://start.spring.io and create a Spring Boot application with details as follows:
- Project: Choose Gradle Project or Maven Project.
- Language: Java
- Spring Boot: Latest stable version of Spring Boot is selected by default. So leave it as is.
- Project Metadata: Provide group name in the Group field. The group name is the id of the project. In Artifact field, provide the name of your project. In the package field, provide package name for your project. Next, select your preferred version of Java that is installed on your computer and is available on your local environment.
- Dependencies: Add dependencies for Spring Web, Spring Boot DevTools, Spring Batch, and Spring Data MongoDB.
- Click the GENERATE button and save/download the project zip bundle.
- Extract the project to your preferred working directory.
- Import the project in your preferred Java development IDE such as Eclipse or IntelliJ IDEA.
Refer to the image below for example:

The following listing shows Spring Batch and Spring Data MongoDB dependencies for Gradle and Maven project:
With Gradle
implementation 'org.springframework.boot:spring-boot-starter-batch'
implementation 'org.springframework.boot:spring-boot-starter-data-mongodb'
With Maven
Add Application Configurations
Open the application.properties file and copy the configurations listed below. Do not forget to update the configuration values to make them relevant to your project:
# Server port
server.port = 8080
# Mongo Configuration
spring.data.mongodb.host = localhost
spring.data.mongodb.port = 27017
spring.data.mongodb.database = spring_batch_test
spring.data.mongodb.username = test
spring.data.mongodb.password = 12345678
#disabled job run at startup
spring.batch.job.enabled=false
Create a CSV file
The application will read data from a CSV file, so let's create a user-sample-data.csv file within the src/main/resources folder of the project and add the following values to it:
"[email protected]", "John", "William", "90001234555"
"[email protected]", "Jay", "Watson", "90001234556"
"[email protected]", "Ryan", "Watson", "90001234557"
"[email protected]", "Jake", "Watson", "90001234558"
Create a Model class
Create a UserDetail.java Java class to represent the data of the above CSV file. The following code shows how to do so:
package com.example.model;
public class UserDetail {
private String email;
private String firstName;
private String lastName;
private String mobileNumber;
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getMobileNumber() {
return mobileNumber;
}
public void setMobileNumber(String mobileNumber) {
this.mobileNumber = mobileNumber;
}
}
Create a Domain class
After reading the CSV file, the application will save the file data in a MongoDB database. To perform the insert operation, let's create a User.java domain class and annotate it with @Document(collection = "table_name"). The @Document annotation identifies a class as being a domain object that we want to persist to the database. The attributes of the domain class are mapped to the columns of the database table. The following code shows how to do so:
package com.example.domain;
import java.time.LocalDateTime;
import org.springframework.data.annotation.CreatedDate;
import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.LastModifiedDate;
import org.springframework.data.mongodb.core.mapping.Document;
@Document(collection = "user")
public class User {
@Id
private String id;
private String email;
private String firstName;
private String lastName;
private String mobileNumber;
@CreatedDate
private LocalDateTime createdDate;
@LastModifiedDate
private LocalDateTime lastModifieDate;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getMobileNumber() {
return mobileNumber;
}
public void setMobileNumber(String mobileNumber) {
this.mobileNumber = mobileNumber;
}
public LocalDateTime getCreatedDate() {
return createdDate;
}
public void setCreatedDate(LocalDateTime createdDate) {
this.createdDate = createdDate;
}
public LocalDateTime getLastModifieDate() {
return lastModifieDate;
}
public void setLastModifieDate(LocalDateTime lastModifieDate) {
this.lastModifieDate = lastModifieDate;
}
}
Create a Processor class
Create a UserItemProcessor.java Java class and implement the ItemProcessor<I, O> interface of the Spring Batch framework.
The ItemProcessor<I, O> interface takes two arguments, Input type and Output type. Both doesn't need to be of the same type. You may provide input of one type and return output of some other type after it has been read.
The ItemProcessor<I, O> has a public method that takes an argument of object with data that is read. Inside this method is where the read data must be transformed. The following code shows how to do so:
package com.example.processor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemProcessor;
import com.example.domain.User;
import com.example.model.UserDetail;
public class UserItemProcessor implements ItemProcessor<UserDetail, User> {
private static final Logger log = LoggerFactory.getLogger(UserItemProcessor.class);
@Override
public User process(UserDetail item) throws Exception {
log.info("processing user data.....{}", item);
User transformedUser = new User();
transformedUser.setEmail(item.getEmail());
transformedUser.setFirstName(item.getFirstName());
transformedUser.setLastName(item.getLastName());
transformedUser.setMobileNumber(item.getMobileNumber());
return transformedUser;
}
}
Create a Job Notification Listener class
Create a JobCompletionNotificationListener.java Java class and extend the class with the JobExecutionListenerSupport class from the Spring Batch framework. The JobExecutionListenerSupport class has callback methods which can be called before starting or after completion of a job.
Let's override the afterJob method of JobExecutionListenerSupport class which will help the application to notify when the job completes.
package com.example.listener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.listener.JobExecutionListenerSupport;
import org.springframework.stereotype.Component;
@ Component
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {
private static final Logger log =
LoggerFactory.getLogger(JobCompletionNotificationListener.class);
@Override
public void afterJob(JobExecution jobExecution) {
if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
log.info("!!! JOB FINISHED! Time to verify the results");
}
}
}
Create a BatchConfiguration class
Create a BatchConfiguration.java batch configuration Java class and annotate it with @Configuration and @EnableBatchProcessing annotations. The @EnableBatchProcessing enables Spring Batch features and provide a base configuration for setting up batch jobs in an @Configuration class. The following code shows how to do so:
package com.example.config;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.data.MongoItemWriter;
import org.springframework.batch.item.data.builder.MongoItemWriterBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.mongodb.core.MongoTemplate;
import com.example.domain.User;
import com.example.listener.JobCompletionNotificationListener;
import com.example.model.UserDetail;
import com.example.processor.UserItemProcessor;
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public FlatFileItemReader<UserDetail> reader() {
return new FlatFileItemReaderBuilder<UserDetail>().name("userItemReader")
.resource(new ClassPathResource("user-sample-data.csv")).delimited()
.names(new String[] {"email", "firstName", "lastName", "mobileNumber"})
.fieldSetMapper(new BeanWrapperFieldSetMapper<UserDetail>() {
{
setTargetType(UserDetail.class);
}
}).build();
}
@Bean
public MongoItemWriter<User> writer(MongoTemplate mongoTemplate) {
return new MongoItemWriterBuilder<User>().template(mongoTemplate).collection("user")
.build();
}
@Bean
public UserItemProcessor processor() {
return new UserItemProcessor();
}
@Bean
public Step step1(FlatFileItemReader<UserDetail> itemReader, MongoItemWriter<User> itemWriter)
throws Exception {
return this.stepBuilderFactory.get("step1").<UserDetail, User>chunk(5).reader(itemReader)
.processor(processor()).writer(itemWriter).build();
}
@Bean
public Job updateUserJob(JobCompletionNotificationListener listener, Step step1)
throws Exception {
return this.jobBuilderFactory.get("updateUserJob").incrementer(new RunIdIncrementer())
.listener(listener).start(step1).build();
}
}
Update Main Application class
Annotate the main Application Class with @EnableMongoAuditing and exclude DataSourceAutoConfiguration.class as shown below:
package com.example.batch.mongodb;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.data.mongodb.config.EnableMongoAuditing;
@EnableMongoAuditing
@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
public class BatchMongodbSampleApplication {
public static void main(String[] args) {
SpringApplication.run(BatchMongodbSampleApplication.class, args);
}
}
Run the Application
Run your batch application. The application should read the CSV file and save the data to the MongoDB database.
Summary
Congratulations! you have learned how to build a Spring Batch application with MongoDB and Spring Boot.