Dealing with empty inputs

Time needed to complete  ~20 minutes
Prerequisite for this tutorial is Configure Spring Batch Admin to use MySQL

1.Introduction

Errors in programming are the norm not the exception. Finding and fixing errors is not an easy task, specially when user reports an error that don't exist, both code and data are correct. A common situation in batch processing is empty file, no input. Clients see no output result, the conclusion is batch job didn't finished, report bug. To prevent this situation spring batch defines a listener after step is finished to check the number of reading. 

2.What is used in this tutorial:

1- Maven 3

2- Jdk 1.7

3- Tomcat 7.0.55

4- Eclipse Luna 4.4

5- Spring Core 3.2.9

6- Spring Batch 2.2.7

7- Spring Batch Admin Manager 1.3.0

8- MySQL 5 Database

3.Project Structure

Picture 1: Project structure

4. Configuration  files

- The database configuration where the data will be written is in file database.xml path src/main/resources/META-INF/spring/batch/dbConfig/database.xml, this is not the same as the database used by spring batch admin to store job repository data. First create the database named company_db and in the database.xml change db_username and db_password to suite company_db username and password.

Listing 1: database.xml

 <beans xmlns="http://www.springframework.org/schema/beans"  
      xmlns:jdbc="http://www.springframework.org/schema/jdbc"   
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://www.springframework.org/schema/beans   
           http://www.springframework.org/schema/beans/spring-beans-3.2.xsd  
           http://www.springframework.org/schema/jdbc   
           http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd">  
   <!-- connect to database -->  
      <bean id="dataSource2"  
           class="org.springframework.jdbc.datasource.DriverManagerDataSource">  
           <property name="driverClassName" value="com.mysql.jdbc.Driver" />  
           <property name="url" value="jdbc:mysql://localhost:3306/company_db" />  
           <property name="username" value="db_username" /> 
           <property name="password" value="db_password" />
      </bean>  
      <bean id="transactionManager"  
           class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />  
 </beans>  

- The job definition that will read,process and write is in src/main/resources/META-INF/spring/batch/jobs/job-emptyFile.xml , Listing 2.

Listing 2: Batch job definition job-emptyFile.xml

 <beans xmlns="http://www.springframework.org/schema/beans"  
      xmlns:batch="http://www.springframework.org/schema/batch" xmlns:task="http://www.springframework.org/schema/task"  
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://www.springframework.org/schema/batch  
           http://www.springframework.org/schema/batch/spring-batch-2.2.xsd  
           http://www.springframework.org/schema/beans   
           http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">  
      <import resource="../dbConfig/database.xml" />  
      <bean id="order" class="com.web.model.Order" scope="prototype" />  
      <bean id="afterStepListener" class="com.web.listener.EmptyFileHandler" />
      <bean id="afterJobListener" class="com.web.listener.JobListener" />   
      <batch:job id="orderEmptyFileJob">  
           <batch:step id="step1">  
                <batch:tasklet>  
                     <batch:chunk reader="cvsFileItemReader" processor="orderProcessor"  
                          writer="mysqlItemWriter" commit-interval="3">  
                     </batch:chunk>  
                      <batch:listeners>    
                          <batch:listener ref="afterStepListener" />    
                      </batch:listeners>    
                </batch:tasklet>  
           </batch:step>
           <batch:listeners>
           <batch:listener ref="afterJobListener" />
           </batch:listeners>  
      </batch:job>  
      <bean id="orderProcessor" class="com.web.processor.OrderProcessor" />  
      <bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"  
           scope="step">  
           <property name="resource" value="#{jobParameters['inputDataFile']}" />  
           <property name="lineMapper">  
                <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">  
                     <property name="lineTokenizer">  
                          <bean  
                               class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">  
                               <property name="names" value="id,name,surname,age,accountNumber" />  
                          </bean>  
                     </property>  
                     <property name="fieldSetMapper">  
                          <bean  
                               class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">  
                               <property name="prototypeBeanName" value="order" />  
                          </bean>  
                     </property>  
                </bean>  
           </property>  
      </bean>  
      <bean id="mysqlItemWriter"  
           class="org.springframework.batch.item.database.JdbcBatchItemWriter">  
           <property name="dataSource" ref="dataSource2" />  
           <property name="sql">  
                <value>  
       <![CDATA[      
            insert into tel_order(code,service_type,quantity,offer,bill_type,order_date)  
            values (:code, :serviceType, :quantity, :offer, :billType,STR_TO_DATE(:orderDate,'%d-%m-%Y'))  
       ]]>  
                </value>  
           </property>  
           <property name="itemSqlParameterSourceProvider">  
                <bean  
                     class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />  
           </property>  
      </bean>  
 </beans>  

- A bean afterStepListener is defined and referenced in the orderEmptyFileJob job definition, afterStepListener will use code in a class com.web.listener.EmptyFileHandler, Listing 3.

Listing 3: EmptyFileHandler class

 package com.web.listener;  
 import org.apache.log4j.Logger;  
 import org.springframework.batch.core.ExitStatus;  
 import org.springframework.batch.core.StepExecution;  
 import org.springframework.batch.core.annotation.AfterStep;  
 public class EmptyFileHandler {  
      private static final Logger log = Logger.getLogger(EmptyFileHandler.class);  
      @AfterStep  
      public ExitStatus afterStep(StepExecution execution) {  
           if (execution.getReadCount() > 0) {  
                return execution.getExitStatus();  
           } else {  
                log.error("-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-");  
                log.error("EMPTY FILE");  
                log.error("-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-");  
                return ExitStatus.FAILED;  
           }  
      }  
 }  

- afterStep method is annotated with @afterStep, and the method will be executed after step, checking the readcount and if it is not greater than zero will log a message that file is empty and return job status failed.

- A bean afterJobListener is defined and referenced in the orderEmptyFileJob job definition, afterJobListener will use code in a class com.web.listener.JobListener, Listing 4.

Listing 4: JobListener class

 package com.web.listener;  
   
 import java.util.Collection;  
 import java.util.Iterator;  
   
 import org.springframework.batch.core.ExitStatus;  
 import org.springframework.batch.core.JobExecution;  
 import org.springframework.batch.core.StepExecution;  
 import org.springframework.batch.core.annotation.AfterJob;  
   
 public class JobListener {  
      private static final String STEP_NAME = "step1";  
      private static final String EXIT_MSG = "EMPTY FILE -";  
   
      @AfterJob 
      public void afterJob(JobExecution jobExecution) {  
   
           Collection<StepExecution> stepExecutionCol = jobExecution.getStepExecutions();  
           Iterator<StepExecution> itr = stepExecutionCol.iterator();  
   
           // Only one step, go for it directly no loop  
           StepExecution stepexe = itr.next();  
   
           if (STEP_NAME.equals(stepexe.getStepName())     && stepexe.getReadCount() < 1) {  
                ExitStatus es = jobExecution.getExitStatus();  
                jobExecution.setExitStatus(new ExitStatus(es.getExitCode(),     EXIT_MSG));  
           }  
      }  
 }  
   

5. CSV file

- The input empty file that will be used src/main/resources/cvs/emptyFile.csv
- For non empty file test use data in listing 5.1.

Listing 5.1: Testing data for non empty file

 234ac,mobile,1,small,prepaid,12-08-2014  
 235ab,fix,1,medium,postpaid_mail,14-08-2014  
 237ar,bundle,1,small,postpaid_email,10-08-2014  

6. Running orderEmptyFileJob in eclipse

- To run the application use code in class src/main/java/com/web/app/EmptyFileHandlingApp.java Listing 5.

Listing 5: EmptyFileHandlingApp to run orderEmptyFileJob

 package com.web.app;  
   
 import java.util.Date;  
   
 import org.apache.log4j.Logger;  
 import org.springframework.batch.core.Job;  
 import org.springframework.batch.core.JobExecution;  
 import org.springframework.batch.core.JobParameters;  
 import org.springframework.batch.core.JobParametersBuilder;  
 import org.springframework.batch.core.launch.JobLauncher;  
 import org.springframework.context.support.ClassPathXmlApplicationContext;  
   
 public class EmptyFileHandlingApp {  
   
      private static final Logger log = Logger  
                .getLogger(EmptyFileHandlingApp.class);  
      private static ClassPathXmlApplicationContext context;  
   
      public static void main(String[] args) {  
   
           String exitStatus = "Exit Status: ";  
   
           String[] springConfig = { "launch-context.xml",  
                     "META-INF/spring/batch/jobs/job-emptyFile.xml" };  
   
           context = new ClassPathXmlApplicationContext(springConfig);  
   
           JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");  
           Job job = (Job) context.getBean("orderEmptyFileJob");  
   
           try {  
                JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();  
   
                jobParametersBuilder.addString("dateTime", (new Date()).toString());  
                jobParametersBuilder.addString("inputDataFile",  
                          "classpath:cvs/emptyFile.csv");  
   
                JobParameters param = jobParametersBuilder.toJobParameters();  
                JobExecution execution = jobLauncher.run(job, param);  
                exitStatus += execution.getStatus().toString();  
   
           } catch (Exception e) {  
                log.info(e.getStackTrace());  
           } finally {  
                if (context != null)  
                     context.close();  
           }  
   
           log.info(exitStatus);  
           log.info("Finished");  
   
      }  
   
 }  
   

- The result of running is Listing 6.

Listing 6: EmptyFileHandlingApp console output in eclipse

 .  
 .  
 .  
  INFO main job.SimpleStepHandler:139 - Executing step: [step1]  
  ERROR main listener.EmptyFileHandler:17 - -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-  
  ERROR main listener.EmptyFileHandler:18 - EMPTY FILE  
  ERROR main listener.EmptyFileHandler:19 - -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-  
 .  
 .  
 .  
  INFO main app.EmptyFileHandlingApp:49 - Exit Status: FAILED  
  INFO main app.EmptyFileHandlingApp:50 - Finished  

7. Running orderJob From Spring Batch Admin

- In pom.xml change username_tc and password_tc to suite your local tomcat. 

- Start tomcat. 

- Deploy application from run configuration in goal type clean install tomcat7:redeploy , click apply and 

Run buttons.

Note: If these steps are unfamiliar read steps 4 and 5 deploying spring batch admin Link How To

- open address localhost:8080 /springBatchAdminMysql/jobs  in web browser , picture 2

Picture 2: orderEmptyfileJob is listed in spring batch admin job list

- Click orderEmptyFileJob, Change dateTime parameter, click launch button, picture 3.  job will completed with FAILED status.

Picture 3: Job Launch screen

- Click on executions then ID number of job execution, to see exit message EMPTY FILE, picture 4.

Picture 4: EMPTY FILE message

8. Eclipse Project

 Download project source code