Overview
In this section, Explain the validation check of job input data (hereinafter referred to as input validation).
This function is the same usage for chunk model and tasklet model.
In general, input validation in batch processing is often carried out
to confirm that data received from other systems etc. is valid in its own system.
Conversely, it can be said that it is unnecessary to perform input validation
on reliable data in its own system (for example, data stored in the database).
Please refer to input Validation in TERASOLUNA Server 5.x Development Guideline because the input validation duplicates the contents of TERASOLUNA Server 5.x. Explain the main comparisons below.
Comparison target | TERASOLUNA Server 5.x | TERASOLUNA Batch 5.x |
---|---|---|
Available input validation rules |
Same as TERASOLUNA Server 5.x |
|
The target to which the rule is attached |
|
|
Validation execute method |
|
|
Setting error messages |
Same as Definition of error messages in TERASOLUNA Server 5.x Development Guideline. |
|
Error message output destination |
View |
Log etc. |
The input validation to be explained in this section mainly covers data obtained from ItemReader
.
For checking job parameters, refer to Validation check of parameters.
Classification of input validation
The input validation is classified into single item check and correlation item check.
Type | Description | Example | Implementation method |
---|---|---|---|
Single item check |
Check to be completed with a single field |
Required input check |
Bean Validation (using Hibernate Validator as implementation library) |
Correlation item check |
Check to compare multiple fields |
Comparison of numerical values |
|
Spring supports Bean Validation which is a Java standard.
For this single item check, this Bean Validation is used.
For correlation item check, use Bean Validation of the org.springframework.validation.Validator
interface provided by Spring.
In this respect, same as Classification of input validation in TERASOLUNA Server 5.x Development Guideline.
Overview of Input Validation
The timing of input validation in the chunk model and tasklet model is as follows.
-
For chunk model, use
ItemProcessor
-
For tasklet model, use
Tasklet#execute()
at an arbitrary timing.
In the chunk model and tasklet model, the implementation method of input validation is the same,
so here, explain the case where input validation is done in ItemProcessor
of the chunk model.
First, explain an overview of input validation. The relationships of classes related to input validation are as follows.
-
Inject
org.springframework.batch.item.validator.SpringValidator
which is the implementation oforg.springframework.batch.item.validator.Validator
inItemProcessor
and execute the validate method.-
SpringValidator
internally holdsorg.springframework.validation.Validator
and execute the validate method.
It can be said that it is a wrapper fororg.springframework.validation.Validator
.
The implementation oforg.springframework.validation.Validator
isorg.springframework.validation.beanvalidation.LocalValidatorFactoryBean
. Use Hibernate Validator through this class.
-
-
Implement
org.springframework.batch.item.ItemCountAware
in the input DTO to determine where the input validation error occured.
Setting the number of data
|
Validators such as javax.validation.Validator or org.springframework.validation.Validator should not be used directly.
Validators such as
On the other hand, if validators such as |
Do not use org.springframework.batch.item.validator.ValidatingItemProcessor
The input validation by However, depending on the circumstances, it is necessary to extend it because of the following reasons, so do not use it from the viewpoint of unifying the implementation method.
|
How to use
As mentioned earlier, the implementation method of input validation is the same as TERASOLUNA Server 5.x as follows.
-
single item check uses Bean Validation.
-
correlation item check uses Bean Validation or the
org.springframework.validation.Validator
interface provided by Spring.
Explain The method of input validation in the following order.
Various settings
Use Hibernate Validator for input validation. Confirm that the definition of Hibernate Validator is in the library dependency and that the required bean definition exists. These have already been set in the blank project provided by TERASOLUNA Batch 5.x.
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<bean id="validator" class="org.springframework.batch.item.validator.SpringValidator"
p:validator-ref="beanValidator"/>
<bean id="beanValidator"
class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />
As mentioned earlier, for setting of error messages, refer to Definition of error messages in TERASOLUNA Server 5.x Development Guideline.
Input validation rule definition
The target of implementing the rule of input validation is the DTO obtained through ItemReader
.
Implement the DTO obtained through ItemReader
as follows.
-
Implement
org.springframework.batch.item.ItemCountAware
in the input DTO to determine where the input validation error occured.-
In the
setItemCount
method, hold a numerical value in the class field indicating the number of items read in the currently processed item received as an argument.
-
-
Define the input validation rule.
-
refer to Input Validation in TERASOLUNA Server 5.x Development Guideline.
-
Show an example of a DTO defining an input validation rule below.
public class VerificationSalesPlanDetail implements ItemCountAware { // (1)
private int count;
@NotEmpty
@Size(min = 1, max = 6)
private String branchId;
@NotNull
@Min(1)
@Max(9999)
private int year;
@NotNull
@Min(1)
@Max(12)
private int month;
@NotEmpty
@Size(min = 1, max = 10)
private String customerId;
@NotNull
@DecimalMin("0")
@DecimalMax("9999999999")
private BigDecimal amount;
@Override
public void setItemCount(int count) {
this.count = count; // (2)
}
// omitted getter/setter
}
Sr. No. | Description |
---|---|
(1) |
Implement the |
(2) |
Holds the |
Input validation execution
Explain how to implement input validation. Implement input validation execution as follows.
-
Execute
org.springframework.batch.item.validator.Validator#validate()
in the implementation ofItemProcessor
.-
Use an instance of
SpringValidator
by injecting it asValidator
field.
-
-
Handle input validation error. For details, refer to Input validation error handling.
Show an implementation example of input validation below.
@Component
public class ValidateAndContinueItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
@Inject // (1)
Validator<VerificationSalesPlanDetail> validator;
@Override
public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
try { // (2)
validator.validate(item); // (3)
} catch (ValidationException e) {
// omitted exception handling
}
SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
// omitted business logic
return salesPlanDetail;
}
}
Sr. No. | Description |
---|---|
(1) |
Inject |
(2) |
Handle input validation error. |
(3) |
Execute |
Input validation error handling
There are following 2 ways to handle input validation error.
-
Processing is aborted at the time when input validation error occurs, abnormally end the job.
-
Leave the occurrence of input validation error in the log etc. and continue processing the subsequent data. Thereafter, at the end of the job, the job is ended by specifying a warning.
Abnormal Termination of Processing
In order to abnormally terminate processing when an exception occurs, it throws java.lang.RuntimeException
or its subclass.
There are two ways to perform processing such as log output when an exception occurs.
-
Catch exceptions with try/catch and do it before throwing an exception.
-
Do not catch exceptions with try/catch, implement
ItemProcessListener
and do it with the onProcessError method.-
ItemProcessListener#onProcessError()
can be implemented using the@OnProcessError
annotation. For details, refer to Listener.
-
Following is an example of logging exception information and abnormally terminating processing when an exception occurs.
@Component
public class ValidateAndAbortItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
/**
* Logger.
*/
private static final Logger logger = LoggerFactory.getLogger(ValidateAndAbortItemProcessor.class);
@Inject
Validator<VerificationSalesPlanDetail> validator;
@Override
public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
try { // (1)
validator.validate(item); // (2)
} catch (ValidationException e) {
// (3)
logger.error("Exception occurred in input validation at the {} th item. [message:{}]",
item.getCount(), e.getMessage());
throw e; // (4)
}
SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
// omitted business logic
return salesPlanDetail;
}
}
Sr. No. | Description |
---|---|
(1) |
Catch exceptions with try/catch. |
(2) |
Execute input validation. |
(3) |
Perform log output processing before throwing an exception. |
(4) |
Throw exceptions |
@Component
public class ValidateAndAbortItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
/**
* Logger.
*/
private static final Logger logger = LoggerFactory.getLogger(ValidateAndAbortItemProcessor.class);
@Inject
Validator<VerificationSalesPlanDetail> validator;
@Override
public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
validator.validate(item); // (1)
SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
// omitted business logic
return salesPlanDetail;
}
@OnProcessError // (2)
void onProcessError(VerificationSalesPlanDetail item, Exception e) {
// (3)
logger.error("Exception occurred in input validation at the {} th item. [message:{}]", item.getCount() ,e.getMessage());
}
}
Sr. No. | Description |
---|---|
(1) |
Execute input validation. |
(2) |
Implement |
(3) |
Perform log output processing before throwing an exception. |
Note on using ItemProcessListener#onProcessError()
Using of the onProcessError method is useful for improving the readability of source code, maintainability, etc. since business process and exception handling can be separated. When outputting log output in |
Skipping Error Records
After logging the information of the record where input validation error occurred, skip the record where the error occurred and continue the processing of the subsequent data as follows.
-
Catch exceptions with try/catch.
-
Perform log output etc. when an exceptions occurs.
-
Return
null
as the return value ofItemProcessor#process()
.-
By returning
null
, records in which an input validation error occurs are no longer included in subsequent processing targets (output withItemWriter
).
-
@Component
public class ValidateAndContinueItemProcessor implements ItemProcessor<VerificationSalesPlanDetail, SalesPlanDetail> {
/**
* Logger.
*/
private static final Logger logger = LoggerFactory.getLogger(ValidateAndContinueItemProcessor.class);
@Inject
Validator<VerificationSalesPlanDetail> validator;
@Override
public SalesPlanDetail process(VerificationSalesPlanDetail item) throws Exception {
try { // (1)
validator.validate(item); // (2)
} catch (ValidationException e) {
// (3)
logger.warn("Skipping item because exception occurred in input validation at the {} th item. [message:{}]",
item.getCount(), e.getMessage());
// (4)
return null; // skipping item
}
SalesPlanDetail salesPlanDetail = new SalesPlanDetail();
// omitted business logic
return salesPlanDetail;
}
}
Sr. No. | Description |
---|---|
(1) |
Catch exceptions with try/catch |
(2) |
Execute the input validation. |
(3) |
Perform log output processing before returning |
(4) |
Return |
Setting the exit code
When an input validation error occurs, in order to distinguish between the case where input validation error did not occur and the state of the job, be sure to set an exit code that is not a normal termination.
If data with input validation error is skipped, setting of exit code is required even when abnormal termination occurs.
For details on how to set the exit code, refer to Job Management.