Input Check

Overview

In this section, the validation check of the input data for the job (hereinafter referred to as input validation) is explained.

Usage of this function is the same for chunk model and tasklet model.

In general, input validation in batch processing is often carried out to confirm that data received from other systems etc. is valid in its own system.
Conversely, it can be said that it is unnecessary to perform input validation on reliable data in its own system (for example, data stored in the database).

Please refer to input Validation in TERASOLUNA Server 5.x Development Guideline because the input validation duplicates the contents of TERASOLUNA Server 5.x. Explain the main comparisons below.

Main comparison list
Comparison target	TERASOLUNA Server 5.x	TERASOLUNA Batch 5.x
Available input validation rules	Same as TERASOLUNA Server 5.x
The target to which the rule is attached	`form class`	`DTO`
Validation execute method	`Give @Validated annotation to the Controller`	`Call the API of Validator class`
Error message settings	Same as Definition of error messages in TERASOLUNA Server 5.x Development Guideline.
Error message output destination	View	Log etc.

The input validation to be explained in this section mainly covers data obtained from ItemReader.
For checking job parameters, refer to Validation check of parameters.

Classification of input validation

The input validation is classified into single item check and correlation item check.

List of setting contents
Type	Description	Example	Implementation method
Single item check	Check to be completed with a single field	Required input check Digit check Type check	Bean Validation (using Hibernate Validator as implementation library)
Correlation item check	Check to compare multiple fields	Comparison of numerical values Comparison of dates	`Validation` class that implements `org.springframework.validation.Validator` interface or Bean Validation

Spring supports Bean Validation which is a Java standard. For this single item check, this Bean Validation is used. For correlation item check, use Bean Validation of the org.springframework.validation.Validator interface provided by Spring.

In this respect, same as Classification of input validation in TERASOLUNA Server 5.x Development Guideline.

Overview of Input Validation

The timing of input validation in the chunk model and tasklet model is as follows.

For chunk model, use ItemProcessor
For tasklet model, use Tasklet#execute() at an arbitrary timing.

In the chunk model and tasklet model, the implementation method of input validation is the same, so here, explain the case where input validation is done in ItemProcessor of the chunk model.

First, explain an overview of input validation. The relationships of classes related to input validation are as follows.

Related class of input validation

Inject org.springframework.batch.item.validator.SpringValidator which is the implementation of org.springframework.batch.item.validator.Validator in ItemProcessor and execute the validate method.
- SpringValidator internally holds org.springframework.validation.Validator and execute the validate method.
  It can be said that it is a wrapper for org.springframework.validation.Validator.
  The implementation of org.springframework.validation.Validator is org.springframework.validation.beanvalidation.LocalValidatorFactoryBean. Use Hibernate Validator through this class.
Implement org.springframework.batch.item.ItemCountAware in the input DTO to determine where the input validation error has occured in any data record.

Setting the number of data

ItemCountAware#setItemCount is set by AbstractItemCountingItemStreamItemReader. Therefore, if you do not use ItemReader in the tasklet model, it will not be updated. In this case, it is necessary for the user to set what error occurred in the data.

Validators such as javax.validation.Validator or org.springframework.validation.Validator should not be used directly.

Validators such as javax.validation.Validator or org.springframework.validation.Validator should not be used directly, use org.springframework.batch.item.validator.SpringValidator.

SpringValidator is wrapper of org.springframework.validation.Validator.
SpringValidator wraps the raised exception in BindException and throws it as ValidationException.
Therefore, BindException can be accessed via ValidationException which makes flexible handling easier.

On the other hand, if validators such as javax.validation.Validator and org.springframework.validation.Validator are used directly, it will be complicated logic to process the information that caused the validation error.

Do not use org.springframework.batch.item.validator.ValidatingItemProcessor

The input validation by org.springframework.validation.Validator can also be realized by using ValidatingItemProcessor provided by Spring Batch.

However, depending on the circumstances, it is necessary to extend it because of the following reasons, so do not use it from the viewpoint of unifying the implementation method.

The input validation error cannot be handled and processing cannot be continued.
It is not possible to flexibly deal with data that has become an input validation error.
- It is assumed that the processing of the data that resulted in input validation error will vary depending on the user (only log output, save error data to another file, etc.).

How to use

As mentioned earlier, the implementation method of input validation is the same as TERASOLUNA Server 5.x as follows.

single item check uses the Bean Validation.
correlation item check uses Bean Validation or the org.springframework.validation.Validator interface provided by Spring.

Explain the method of input validation in the following order.

Various settings
Input validation rule definition
Input validation execution
Input validation error handling

Various settings

Use Hibernate Validator for input validation. Confirm that the definition of Hibernate Validator is in the library dependency and that the required bean definition exists. These have already been set in the blank project provided by TERASOLUNA Batch 5.x.

Setting example of dependent library

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
</dependency>

launch-context.xml

<bean id="validator" class="org.springframework.batch.item.validator.SpringValidator"
      p:validator-ref="beanValidator"/>

<bean id="beanValidator"
      class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />

Error message setting

As mentioned earlier, for setting of error messages, refer to Definition of error messages in TERASOLUNA Server 5.x Development Guideline.

Input validation rule definition

The target of implementing the rule of input validation is the DTO obtained through ItemReader. Implement the DTO obtained through ItemReader as follows.

Implement org.springframework.batch.item.ItemCountAware in the input DTO to determine where the input validation error has occured in any data record.
- In the setItemCount method, hold a numerical value in the class field indicating the number of items read in the currently processed item received as an argument.
Define the input validation rule.
- refer to Input Validation in TERASOLUNA Server 5.x Development Guideline.

Show an example of a DTO defining an input validation rule below.

An example of a DTO defining an input validation rule

public class VerificationSalesPlanDetail implements ItemCountAware {  // (1)

    private int count;

    @NotEmpty
    @Size(min = 1, max = 6)
    private String branchId;

    @NotNull
    @Min(1)
    @Max(9999)
    private int year;

    @NotNull
    @Min(1)
    @Max(12)
    private int month;

    @NotEmpty
    @Size(min = 1, max = 10)
    private String customerId;

    @NotNull
    @DecimalMin("0")
    @DecimalMax("9999999999")
    private BigDecimal amount;

    @Override
    public void setItemCount(int count) {
        this.count = count;  // (2)
    }

    // omitted getter/setter
}

List of setting contents
Sr. No.	Description
(1)	Implement the `ItemCountAware` class and override the `setItemCount` method. `ItemCountAware#setItemCount()` is passed to the argument as to what the data read by `ItemReader` is.
(2)	Holds the `count` received in the argument in the class field. This value is used to determine the number of items of data that caused an input validation error.

Input validation execution

Explain how to implement input validation. Implement input validation execution as follows.

Execute org.springframework.batch.item.validator.Validator#validate() in the implementation of ItemProcessor.
- Use an instance of SpringValidator by injecting it as Validator field.
Handle input validation error. For details, refer to Input validation error handling.

Show an implementation example of input validation below.

An implementation example of input validation

@Component
public class ValidateAndContinueItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
    @Inject  // (1)
    Validator<VerificationSalesPlanDetail> validator;

    @Override
    public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
        try {  // (2)
            validator.validate(item);  // (3)
        } catch (ValidationException e) {
          // omitted exception handling
        }

        SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
        // omitted business logic

        return salesPlanDetail;
    }
}

List of setting contents
Sr. No.	Description
(1)	Inject `SpringValidator` instance. For the type argument of `org.springframework.batch.item.validator.Validator`, set the DTO to be acquired via `ItemReader`.
(2)	Handle input validation error. In the example, exception is handled by catching with try/catch. For details, refer to Input validation error handling.
(3)	Execute `Validator#validate()` with the DTO obtained through `ItemReader` as an argument.

Input validation error handling

There are following 2 ways to handle input validation error.

Processing was aborted at the time when an input validation error occurs and the job is abnormally terminated.
Record the occurrence of an input validation error in the log, etc and continue processing the subsequent data. Thereafter, the job is terminated by specifying a warning at the end of the job.

Abnormal Termination of Processing

In order to abnormally terminate processing when an exception occurs, it throws java.lang.RuntimeException or its subclass.

There are two ways to perform processing such as log output when an exception occurs.

Catch exceptions with try/catch and do it before throwing an exception.
Do not catch exceptions with try/catch, implement ItemProcessListener and do it with the onProcessError method.
- ItemProcessListener#onProcessError() can be implemented using the @OnProcessError annotation. For details, refer to Listener.

Following is an example of logging exception information and abnormally terminating processing when an exception occurs.

An error handling example with try/catch

@Component
public class ValidateAndAbortItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
    /**
     * Logger.
     */
    private static final Logger logger = LoggerFactory.getLogger(ValidateAndAbortItemProcessor.class);

    @Inject
    Validator<VerificationSalesPlanDetail> validator;

    @Override
    public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
        try {  // (1)
            validator.validate(item);  // (2)
        } catch (ValidationException e) {
            // (3)
            logger.error("Exception occurred in input validation at the {} th item. [message:{}]",
                    item.getCount(), e.getMessage());
            throw e;  // (4)
        }

        SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
        // omitted business logic

        return salesPlanDetail;
    }
}

List of setting contents
Sr. No.	Description
(1)	Catch exceptions with try/catch.
(2)	Execute input validation.
(3)	Perform log output processing before throwing an exception.
(4)	Throw exceptions Since `org.springframework.batch.item.validator.ValidationException` is a subclass of `RuntimeException`, it can be thrown as it is.

An error handling example with ItemProcessListener#OnProcessError

@Component
public class ValidateAndAbortItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {

    /**
     * Logger.
     */
    private static final Logger logger = LoggerFactory.getLogger(ValidateAndAbortItemProcessor.class);

    @Inject
    Validator<VerificationSalesPlanDetail> validator;

    @Override
    public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
        validator.validate(item);  // (1)

        SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
        // omitted business logic

        return salesPlanDetail;
    }

    @OnProcessError  // (2)
    void onProcessError(VerificationSalesPlanDetail item, Exception e) {
        // (3)
        logger.error("Exception occurred in input validation at the {} th item. [message:{}]", item.getCount() ,e.getMessage());
    }
}

List of setting contents
Sr. No.	Description
(1)	Execute input validation.
(2)	Implement `ItemProcessListener#onProcessError()` using `@OnProcessError` annotation.
(3)	Perform log output processing before throwing an exception.

Note on using ItemProcessListener#onProcessError()

Using of the onProcessError method is useful for improving the readability of source code, maintainability, etc. since it enables to separate business process and exception handling.
However, when an exception other than ValidationException performing handling processing in the above example occurs, the same method is executed, so it is necessary to be careful.

When outputting log output in ItemProcessor#process() by exception, it is necessary to judge the kind of exception caused by the onProcessError method and handle exception. If this is cumbersome, it is good to share responsibility so that only input validation errors are handled by handling with try / catch and others are handed over to listeners.

Skipping Error Records

After logging the information of the record where input validation error occurred, skip the record where the error occurred and continue the processing of the subsequent data as follows.

Catch exceptions with try/catch.
Perform log output etc. when an exceptions occurs.
Return null as the return value of ItemProcessor#process().
- By returning null, records in which an input validation error occurs are no longer included in subsequent processing targets (output with ItemWriter).

A skipping example with ItemProcessor

@Component
public class ValidateAndContinueItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
    /**
     * Logger.
     */
    private static final Logger logger = LoggerFactory.getLogger(ValidateAndContinueItemProcessor.class);

    @Inject
    Validator<VerificationSalesPlanDetail> validator;

    @Override
    public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
        try {  // (1)
            validator.validate(item);  // (2)
        } catch (ValidationException e) {
            // (3)
            logger.warn("Skipping item because exception occurred in input validation at the {} th item. [message:{}]",
                    item.getCount(), e.getMessage());
            // (4)
            return null;  // skipping item
        }

        SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
        // omitted business logic

        return salesPlanDetail;
    }
}

List of setting contents
Sr. No.	Description
(1)	Catch exceptions with try/catch
(2)	Execute the input validation.
(3)	Perform log output processing before returning `null`.
(4)	Return `null` to skip this data and move on to the next data processing.

Setting the exit code

When an input validation error occurs, in order to distinguish between the case where input validation error did not occur and the state of the job, be sure to set an exit code that is not a normal termination.
If data with input validation error is skipped, setting of exit code is required even when abnormal termination occurs.

For details on how to set the exit code, refer to Job Management.

Output of error messages

Any error message can be output by using MessageSource when input check error occurs. For the settings of error message, refer Definition of error message in TERASOLUNA Server 5.x Development Guideline. Implement as follows when error message is output.

There are 2 methods to output error message in 1 record as shown below.

Collectively output 1 error message.
Output error message for each field of record.

In the example, it is implemented by the method of output of error message for each field.

Catch exception ValidationException that occurs in input check error by try/catch.
Fetch org.springframework.validation.BindException by getCause method of ValidationException.
- FiledError can be fetched since BindException performs implements of BindResult.
Fetch FiledError repetitively one by one for the number of errors by getFieldErrors method.
Output error message one by one by MessageSource by considering the fetched FieldError as an argument.

Error message output example by MessageSource

@Component
public class ValidateAndMessageItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
    /**
     * Logger.
     */
    private static final Logger logger = LoggerFactory.getLogger(ValidateAndMessageItemProcessor.class);

    @Inject
    Validator<VerificationSalesPlanDetail> validator;

    @Inject
    MessageSource messageSource;  // (1)

    @Override
    public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
        try {  // (2)
            validator.validate(item);  // (3)
        } catch (ValidationException e) {
            // (4)
            BindException errors = (BindException) e.getCause();

            // (5)
            for (FieldError fieldError : errors.getFieldErrors()) {
                // (6)
                logger.warn(messageSource.getMessage(fieldError, null) +
                                "Skipping item because exception occurred in input validation at the {} th item. [message:{}]",
                                    item.getCount(), e.getMessage());
            // (7)
            return null;  // skipping item
        }

        SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
        // omitted business logic

        return salesPlanDetail;
    }
}

Item list of setting contents
Sr. No.	Explanation
(1)	Inject the instance of `ResourceBundleMessageSource`. For Bean definition of `MassageSorce`, refer Message management.
(2)	Catch exceptions with try/catch.
(3)	Execute input validation.
(4)	Fetch `org.springframework.validation.BindException` with `getCause()`.
(5)	Fetch `FiledError` for each record with `getFieldErrors()`.
(6)	Perform output process of error message with `messageSource` by considering the fetched `FieldError` as an argument. If there are 3 errors in 1 record, output 3 error messages repeatedly.
(7)	By returning `null`, skip this data and move to the next data process.