Developing first spring batch job

Time needed to complete  ~45 minutes
Prerequisite for this tutorial is Configure Spring Batch Admin to use MySQL

1.Introduction

After configuring Spring Batch Admin project now it is time to write first batch job and run it in eclipse, then run the job from Spring Batch Admin web application. The application is simple, reading from csv file and writing to database table.

2.What is used in this tutorial:

1- Maven 3

2- Jdk 1.7

3- Tomcat 7.0.55

4- Eclipse Luna 4.4

5- Spring Core 3.2.9

6- Spring Batch 2.2.7

7- Spring Batch Admin Manager 1.3.0

8- MySQL 5 Database

3.Project Structure

Picture 1: Project structure

4. Configuration  files

- The database configuration where the data will be wiritten is in file database.xml path springBatchAdminTC2/src/main/resources/META-INF/spring/batch/dbConfig/database.xml, this

is not the same as the database used for spring batch admin to store job repository data. First create the database named company_db and in the database.xml change db_username and db_password to suite company_db username and password.

- The data while be written in tel_order table, the DDL for creating tel_order is in file src/main/resources/tel_order_ddl.sql

Listing 1: DDL for creating tel_order table

CREATE TABLE tel_order (
id INTEGER(11) NOT NULL AUTO_INCREMENT,
code VARCHAR(5) NOT NULL,
service_type varchar(32) NOT NULL,
quantity INTEGER(11) NOT NULL,
offer VARCHAR(32) NOT NULL,
bill_type VARCHAR(32) NOT NULL,
order_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (id)
) ENGINE=InnoDB;

- Each job in spring batch is a collection of step(s) that are executed, each step can do read,process and write, Picture 2,

picture 2: spring batch reference module -link to Image source-

- The job definition that will read,process and write is in src/main/resources/META-INF/spring/batch/jobs/job-order.xml , Listing 2. Order class Listing 3.

Listing 2: Batch job definition in job-order.xml file

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:batch="http://www.springframework.org/schema/batch" xmlns:task="http://www.springframework.org/schema/task"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/batch
  http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
  http://www.springframework.org/schema/beans 
  http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">

 <!-- database configuration -->
 <import resource="../dbConfig/database.xml" />

 <!-- bean which data will be written to order table -->
 <bean id="order" class="com.web.model.Order" scope="prototype" />

 <!-- Batch job -->
 <batch:job id="orderJob">
  <batch:step id="step1">
   <batch:tasklet>
    <batch:chunk reader="cvsFileItemReader" processor="orderProcessor"
     writer="mysqlItemWriter" commit-interval="5">
    </batch:chunk>
   </batch:tasklet>
  </batch:step>
 </batch:job>

 <bean id="orderProcessor" class="com.web.processor.OrderProcessor" />
 <bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
  scope="step">

  <property name="linesToSkip" value="1" />
  <!-- Read order csv file -->
  <property name="resource" value="#{jobParameters['inputDataFile']}" />
  <property name="lineMapper">
   <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">

    <!-- split fileds -->
    <property name="lineTokenizer">
     <bean
      class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
      <property name="names"
       value="code,serviceType,quantity,offer,billType,orderDate" />
     </bean>
    </property>
    <property name="fieldSetMapper">
     <!-- map to an object -->
     <bean
      class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
      <property name="prototypeBeanName" value="order" />
     </bean>
    </property>
   </bean>
  </property>
 </bean>

 <bean id="mysqlItemWriter"
  class="org.springframework.batch.item.database.JdbcBatchItemWriter">
  <property name="dataSource" ref="dataSource2" />
  <property name="sql">
   <value>
            <![CDATA[        
             insert into tel_order(code,service_type,quantity,offer,bill_type,order_date) 
    values (:code, :serviceType, :quantity, :offer, :billType,STR_TO_DATE(:orderDate,'%d-%m-%Y'))
            ]]>
   </value>
  </property>
  <!-- mapping java object property to sql parameter -->
  <property name="itemSqlParameterSourceProvider">
   <bean
    class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
  </property>
 </bean>

</beans>

Listing 3: Order class

package com.web.model;  
   
 import java.io.Serializable;  
   
 public class Order implements Serializable {  
   
      private static final long serialVersionUID = 1853355651832301663L;  
   
      private String code;  
      private String serviceType;  
      private int quantity;  
      private String offer;  
      private String billType;  
      private String orderDate;  
   
 // setter and getter omitted  

- Job id is orderJob with cvsFileItemReader reader that will receive file location as parameter

#{jobParameters['inputDataFile']} skip the first line that is header then read the file data line by line, split each line to fields, map fields to order object in the order that appears in property names. the writer will store data in tel_order table. The processor does not do much only convert offer to uppercase, listing 4.

Listing 4: OrderProcessor class

 package com.web.processor;  
   
 import org.springframework.batch.item.ItemProcessor;  
 import com.web.model.Order;  
   
 public class OrderProcessor implements ItemProcessor<Order, Order> {  
   
      @Override  
      public Order process(Order item) throws Exception {  
   
           item.setOffer(item.getOffer().toUpperCase());  
           return item;  
      }  
 }  

5. CSV file

Listing 5: CSV file of orders

 code,service_type,quantity,offer,bill_type,order_date  
 234ac,mobile,1,small,prepaid,12-08-2014  
 235ab,fix,1,medium,postpaid_mail,14-08-2014  
 237ar,bundle,1,small,postpaid_email,10-08-2014  
 236as,bundle,1,small,postpaid_email,12-08-2014  
 239ac,fix,1,large,postpaid_mail,12-08-2014  
 230mb,mobile,2,small,prepaid,12-08-2014  
 244rx,mobile,1,large,postpaid_email,12-08-2014  
 274cb,mobile,5,medium,prepaid,12-08-2014  
 228wb,mobile,1,large,postpaid_email,12-07-2014  
 284mn,mobile,1,small,prepaid,12-08-2014  
 285rn,fix,1,large,prepaid,12-08-2014  
 292wr,bundle,1,medium,postpaid_email,12-08-2014  
 293un,mobile,1,medium,prepaid,12-08-2014  
 304po,bundle,1,large,postpaid_email,12-08-2014  
 305is,bundle,1,large,postpaid_email,12-08-2014  
 311jr,bundle,1,small,postpaid_mail,12-08-2014  
 312ps,internet,1,fiber32,postpaid_email,12-08-2014  
 317tr,mobile,3,small,prepaid,12-08-2014  
 329qv,mobile,1,medium,prepaid,12-08-2014  

6. Running orderJob In eclipse

- After creating company_db database and tel_order table, configure xml to use company_db, create a job definition in xml, code the Order class and orderProcessor, it is time to run the application using code in File2DBApp Listing 5.

Listing 6: File2DBApp to run orderJob

 package com.web.app;  
   
 import java.util.Date;  
   
 import org.apache.log4j.Logger;  
 import org.springframework.batch.core.Job;  
 import org.springframework.batch.core.JobExecution;  
 import org.springframework.batch.core.JobParameters;  
 import org.springframework.batch.core.JobParametersBuilder;  
 import org.springframework.batch.core.launch.JobLauncher;  
 import org.springframework.context.support.ClassPathXmlApplicationContext;  
   
 public class File2DBApp {  
   
      private static final Logger log = Logger.getLogger(File2DBApp.class);  
      private static ClassPathXmlApplicationContext context;  
   
      public static void main(String[] args) {  
   
           String exitStatus = "Exit Status: ";  
   
           String[] springConfiguration = { "launch-context.xml",  
                     "META-INF/spring/batch/jobs/job-order.xml" };  
   
           context = new ClassPathXmlApplicationContext(springConfiguration);  
   
           JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");  
           Job job = (Job) context.getBean("orderJob");  
   
           try {  
   
                JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();  
                jobParametersBuilder.addString("dateTime", (new Date()).toString());  
                jobParametersBuilder.addString("inputDataFile",  
                          "classpath:cvs/orders_12082014.csv");  
   
                JobParameters param = jobParametersBuilder.toJobParameters();  
   
                JobExecution execution = jobLauncher.run(job, param);  
   
                exitStatus += execution.getStatus().toString();  
   
           } catch (Exception e) {  
                e.printStackTrace();  
                log.info(e.getStackTrace());  
           } finally {  
                if (context != null)  
                     context.close();  
           }  
   
           log.info(exitStatus);  
           log.info("Job Finished");  
   
      }  
 }  
   

Note: If preferring Java 7 try-with-resources the code for running application will be

Listing 7: File2DBApp try with resources

  package com.web.app;  
   
 import java.util.Date;  
   
 import org.apache.log4j.Logger;  
 import org.springframework.batch.core.Job;  
 import org.springframework.batch.core.JobExecution;  
 import org.springframework.batch.core.JobParameters;  
 import org.springframework.batch.core.JobParametersBuilder;  
 import org.springframework.batch.core.launch.JobLauncher;  
 import org.springframework.context.support.ClassPathXmlApplicationContext;  
   
 public class File2DBApp {  
   
      private static final Logger log = Logger.getLogger(File2DB2App.class);  
        
      public static void main(String[] args) {  
   
           String exitStatus = "Exit Status: ";  
             
           String[] springConfig = { "launch-context.xml",  
                     "META-INF/spring/batch/jobs/job-order.xml" };  
   
           try(ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(springConfig)){  
   
           JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");  
           Job job = (Job) context.getBean("orderJob");  
   
        
               JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();  
               jobParametersBuilder.addString("dateTime", (new Date()).toString());  
               jobParametersBuilder.addString("inputDataFile","classpath:cvs/orders_12082014.csv");  
   
               JobParameters param = jobParametersBuilder.toJobParameters();  
   
               JobExecution execution = jobLauncher.run(job, param);  
   
               exitStatus += execution.getStatus().toString();  
   
           } catch (Exception e) {  
                log.info(e.getStackTrace());  
           }  
   
           log.info(exitStatus);  
           log.info("Job Finished");  
   
      }  
 }  
   
   

- The result after job finished and querying table company_db.tel_order is shown in picture 3.

Picture 3: Data in table tel_order after executing orderJob

7. Running orderJob From Spring Batch Admin

- In pom.xml change username_tc and password_tc to suite your local tomcat. 

- Start tomcat. 

- Deploy application from run configuration in goal type clean install tomcat7:redeploy , click apply and 

Run buttons.

Note: If these steps are unfamiliar read steps 4 and 5 deploying spring batch admin Link How To

- open address localhost:8080 /springBatchAdminMysql/jobs  in web browser , picture 4

Picture 4: orderJob is listed in spring batch admin job list

- Select orderJob, click launch button an error will appear, picture 5. In spring batch job can't
run more than once with the same parameters.

Picture 5: Error when trying to launch orderJob

- Change dateTime parameter a couple of second and click launch button again, job will completed.

- Click on the top executions under ID column -> number under ID column -> COMPLETED under Status column to see job summary,  picture 6.

Picture 6: job summary

- Finally query the company_db.tel_order to get the inserted orders, Picture 3.

8. Eclipse Project

      Download project source code