Premise
As explained in How to proceed in the tutorial,
it is a format to add implementation of exception handling for jobs that validate input data
Note that, various methods like try-catch or ChunkListener are used as exception handling methods. |
Overview
Create a job which performs exception handling by ChunkListener.
Note that, since this section is explained based on TERASOLUNA Batch 5.x Development guideline, refer to Exception handling by ChunkListener interface for details.
Regarding usage of listener
By using a listener, exception handling is implemented by checking that the exception has occurred after execution of step here. However, since the use of listener is not restricted to exception handling, refer Listener for details. refer Listener for details. |
Background, process overview and business specifications of Explanation of application to be created are listed below.
Background
Some mass retail stores issue point cards to the members.
Membership types include "Gold members", "Normal members" and the services are provided based on membership type.
As a part of the service, 100 points are added for "gold members" and 10 points are added for "normal members" at the end of the month,
for the members who have purchased a product during that month.
Process overview
TERASOLUNA Batch 5.x will be using an application as a monthly batch process which adds
points based on the membership type.
A process to validate verification for checking whether the input data exceeds upper limit value of points is additionally implemented,
a log is output at the time of error and the process is abnormally terminated.
Business specifications
Business specifications are as shown below.
-
Check that the input data points do not exceed 1,000,000 points
-
When an error occurs during a process, an error message is output in a log and process is terminated abnormally
-
Error message is handled through subsequent processing by using a listener
-
Message content is read as "The Point exceeds 1000000."
-
-
When the product purchasing flag is "1"(process target), points are added based on membership type
-
Add 100 points when membership type is "G"(gold member),and add 10 points when membership type is "N" (normal member).
-
-
Product purchasing flag is updated to "0" (initial status) after adding points
-
Upper limit of points is 1,000,000 points
-
If the points exceed 1,000,000 points after adding points, they are adjusted to 1,000,000 points.
Table specifications
Specifications of member information table acting as an input and output resource are shown below.
Since it acts as an explanation for the job which accesses the database as per Premise
refer File specifications for resource specifications of input and output in case of a job accessing the file.
No | Attribute name | Column name | PK | Data type | Number of digits | Explanation |
---|---|---|---|---|---|---|
1 |
Member Id |
id |
CHAR |
8 |
Indicates a fixed 8 digit number which uniquely identifies a member. |
|
2 |
Membership type |
type |
- |
CHAR |
1 |
Membership types are as shown below. |
3 |
Product purchasing flag |
status |
- |
CHAR |
1 |
Indicates whether you have purchased a product during the month. |
4 |
Point |
point |
- |
INT |
7 |
Indicates points retained by the member. |
Job overview
Process flow and process sequence are shown below in order to understand the overview of the job which performs input check created here.
Since it acts as an explanation for the job which accesses the database as per Premise, it must be noted that parts different from that of a process flow and process sequence in case of a job accessing the file exists.
- Process flow overview
-
Process flow overview is shown below.
- Process sequence in case of a chunk model
-
Process sequence is explained in case of a chunk model.
Process sequence is explained in case of a chunk model.
Since this job is explained assuming usage of abnormal data,
the sequence diagram indicates that error (abnormal termination) has occurred during input check.
When input check is successful, process sequence after input check is same as sequence diagram
(Refer Job overview) of database access.
Orange object indicates a class to be implemented at that time.
-
Step is executed from the job.
-
Step opens a resource.
-
MyBatisCursorItemReader
fetches all the member information from member_info table (issue select statement).-
Repeat subsequent processes until input data is exhausted.
-
Start a framework transaction in chunk units.
-
Repeat processes from 4 to 12 until chunk size is reached.
-
-
Step fetches 1 record of input data from
MyBatisCursorItemReader
. -
MyBatisCursorItemReader
fetches 1 record of input data from member_info table. -
member_info table returns input data to
MyBatisCursorItemReader
. -
MyBatisCursorItemReader
returns input data to step. -
Step performs a process for input data by
PointAddItemProcessor
. -
PointAddItemProcessor
requests an input check process toSpringValidator
. -
SpringValidator
performs input check based on input check rules and throws an exception (ValidationException) when an error occurs during checking. -
PointAddItemProcessor
adds the points by reading input data. -
PointAddItemProcessor
returns process results to step. -
Step outputs chunk size data in
MyBatisBatchItemWriter
. -
MyBatisBatchItemWriter
updates member information for member_info table (issue update statement) .
When an exception occurs in the processes from 4 to 14, perform the subsequent process.
-
Step rolls back a framework transaction.
-
Step executes
ChunkErrorLoggingListener
. -
ChunkErrorLoggingListener
outputs ERROR log. -
Step returns exit code (here, abnormal termination:255) to the job.
- Process sequence in case of a tasklet model
-
Process sequence in case of a tasklet model is explained.
Since this job is explained assuming usage of abnormal data, the sequence diagram indicates that error (abnormal termination) has occurred during input check.
When input check is successful, process sequence after input check is same as sequence diagram of (Refer Job overview) of database access.
Orange object indicates a class to be implemented at that time.
-
Step is executed from the job.
-
Step starts a framework transaction.
-
-
Step executes
PointAddTasklet
. -
PointAddTasklet
opens a resource. -
MyBatisCursorItemReader
fetches all the member information from member_info table (issue select statement).-
Repeat processes from 5 to 13 until input data is exhausted.
-
Repeat the processes from 5 to 11 until a certain number of records is reached.
-
-
PointAddTasklet
fetches 1 record of input data fromMyBatisCursorItemReader
. -
MyBatisCursorItemReader
fetches 1 record of input data from member_info table. -
member_info table returns input data to
MyBatisCursorItemReader
. -
MyBatisCursorItemReader
returns input data to tasklet. -
PointAddTasklet
requests input check process toSpringValidator
. -
SpringValidator
performs input check based on input check rules and throws an exception (ValidationException) when an error occurs during the checking. -
PointAddTasklet
adds points by reading input data. -
PointAddTasklet
outputs a certain records of data byMyBatisBatchItemWriter
. -
MyBatisBatchItemWriter
updates member information for member_info table (issue update statement).
When an exception occurs in the processes from 2 to 13, perform the subsequent process.
-
PointAddTasklet
throws an exception (Here, ValidationException) in step. -
Step rolls back the framework transaction.
-
Step executes
ChunkErrorLoggingListener
. -
ChunkErrorLoggingListener
outputs ERROR log. -
Step returns an exit code (here, abnormal termination:255) to the job.
How to implement for chunk model and tasklet model are further explained.
Implementation in chunk model
Processes from creation to execution of job which performs input check in chunk model are implemented with the following processes.
Adding message definition
Log message uses message definition and is used at the time of log output to make it easier to design prevention of variations in the code system and extraction of keywords to be monitored.
Since it is used as common in the chunk model / tasklet model, it can be skipped if created already.
Set application-messages.properties
and launch-context.xml
as shown below.
launch-context.xml
is already configured in TERASOLUNA Batch 5.x.
# (1)
errors.maxInteger=The {0} exceeds {1}.
<!-- omitted -->
<bean id="messageSource" class="org.springframework.context.support.ResourceBundleMessageSource"
p:basenames="i18n/application-messages" /> <!-- (2) -->
<!-- omitted -->
Sr. No. | Explanation |
---|---|
(1) |
Set the message to be output when the upper limit of points is exceeded. |
(2) |
Set |
Implementation of exception handling
Implement exception handling process.
Implement following processes.
Implementation of ChunkErrorLoggingListener class
Perform exception handling by using ChunkListener interface.
Implement the process which output ERROR log when an exception occurs, as an implementation class of ChunkListener interface.
package org.terasoluna.batch.tutorial.common.listener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.ChunkListener;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.item.validator.ValidationException;
import org.springframework.context.MessageSource;
import org.springframework.stereotype.Component;
import javax.inject.Inject;
import java.util.Locale;
@Component
public class ChunkErrorLoggingListener implements ChunkListener {
private static final Logger logger = LoggerFactory.getLogger(ChunkErrorLoggingListener.class);
@Inject
MessageSource messageSource; // (1)
@Override
public void beforeChunk(ChunkContext chunkContext) {
// do nothing.
}
@Override
public void afterChunk(ChunkContext chunkContext) {
// do nothing.
}
@Override
public void afterChunkError(ChunkContext chunkContext) {
Exception e = (Exception) chunkContext.getAttribute(ChunkListener.ROLLBACK_EXCEPTION_KEY); // (2)
if (e instanceof ValidationException) {
logger.error(messageSource
.getMessage("errors.maxInteger", new String[] { "Point", "1000000" }, Locale.getDefault())); // (3)
}
}
}
Sr. No. | Explanation |
---|---|
(1) |
Inject an instance of |
(2) |
Fetch the value of the exception occurred which is set using the key |
(3) |
Fetch a message with message ID |
Configuring Job Bean definition file
Configuration of job Bean definition file to perform exception handling with ChunkListener is shown below.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:batch="http://www.springframework.org/schema/batch"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:mybatis="http://mybatis.org/schema/mybatis-spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd
http://mybatis.org/schema/mybatis-spring http://mybatis.org/schema/mybatis-spring.xsd">
<!-- omitted -->
<context:component-scan base-package="org.terasoluna.batch.tutorial.dbaccess.chunk,
org.terasoluna.batch.tutorial.common.listener"/> <!-- (1) -->
<!-- omitted -->
<batch:job id="jobPointAddChunk" job-repository="jobRepository">
<batch:step id="jobPointAddChunk.step01">
<batch:tasklet transaction-manager="jobTransactionManager">
<batch:chunk reader="reader"
processor="pointAddItemProcessor"
writer="writer" commit-interval="10"/>
<batch:listeners>
<batch:listener ref="chunkErrorLoggingListener"/> <!--(2)-->
</batch:listeners>
</batch:tasklet>
</batch:step>
</batch:job>
</beans>
Sr. No. | Explanation |
---|---|
(1) |
Configure a base package subjected to component scanning. |
(2) |
Set implementation class of |
Job execution and results verification
Execute created job on STS and verify the results.
Execute job from execution configuration
Execute job from already created execution configuration.
Here, execute job by using abnormal system data.
Since how to change input data varies based on the resource (database or file) which handles the job of implementing input check,
execute as below.
- When input check is to be implemented for a job which inputs or outputs data by accessing the database
-
Execute job by using execution configuration created in Execute job from execution configuration of a job which inputs or outputs data by accessing a database.
Comment out script of normal system data and cancel comment out of abnormal system data script by Database Initialize of batch-application.proeprties
in order to use abnormal system data.
# Database Initialize
tutorial.create-table.script=file:sqls/create-member-info-table.sql
#tutorial.insert-data.script=file:sqls/insert-member-info-data.sql
tutorial.insert-data.script=file:sqls/insert-member-info-error-data.sql
- When input check is implemented for the job which inputs or outputs data by accessing a file
-
Execute job by using execution configuration created in Execute job from execution configuration of the job which inputs or outputs data by accessing a file.
Change input file (inputFile) path from normal system data (insert-member-info-data.csv) to abnormal system data (insert-member-info-error-data.csv), from the arguments configured by execution configuration in order to use abnormal system data.
Verifying console log
Open Console View and verify that log of following details is output.
-
Process has terminated abnormally (FAILED)
-
org.springframework.batch.item.validator.ValidationException
has occurred -
org.terasoluna.batch.tutorial.common.listener.ChunkErrorLoggingListener
outputs following message as an ERROR log-
"The Point exceeds 1000000."
-
[2017/09/12 11:13:52] [main] [o.t.b.t.c.l.ChunkErrorLoggingListener] [ERROR] The Point exceeds 1000000.
[2017/09/12 11:13:52] [main] [o.s.b.c.s.AbstractStep] [ERROR] Encountered an error executing step jobPointAddChunk.step01 in job jobPointAddChunk
org.springframework.batch.item.validator.ValidationException: Validation failed for org.terasoluna.batch.tutorial.common.dto.MemberInfoDto@6c8a68c1:
Field error in object 'item' on field 'point': rejected value [1000001]; codes [Max.item.point,Max.point,Max.int,Max]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [item.point,point]; arguments []; default message [point],1000000]; default message [must be less than or equal to 1000000]
at org.springframework.batch.item.validator.SpringValidator.validate(SpringValidator.java:54)
(.. omitted)
[2017/09/12 11:13:52] [main] [o.s.b.c.l.s.SimpleJobLauncher] [INFO ] Job: [FlowJob: [name=jobPointAddChunk]] completed with the following parameters: [{jsr_batch_run_id=480}] and the following status: [FAILED]
[2017/09/12 11:13:52] [main] [o.s.c.s.ClassPathXmlApplicationContext] [INFO ] Closing org.springframework.context.support.ClassPathXmlApplicationContext@735f7ae5: startup date [Tue Sep 12 11:13:50 JST 2017]; root of context hierarchy
Verifying exit code
Verify that the process has terminated abnormally, using exit codes.
For verification, refer Job execution and results verification.
Verify that the exit code (exit value) is 255 (abnormal termination).
Verifying output resource
Verify an output resource (database or file) by a job which implements input check.
In case of a chunk model, verify the update is confirmed upto the chunk just prior the error location since intermediate commit is being adopted.
Verifying member information table
Verify member information table by using Data Source Explorer.
Compare the contents of member information table before and after update, and verify that the contents are in accordance with the verification details.
For verification details, refer Refer database by using Data Source Explorer.
- Verification details
-
-
Regarding records from 1 to 10 (records with member ID from "00000001" to "00000010")
-
status column
-
Records with "0"(initial status) should not exist
-
-
point column
-
Points should be added according to membership type, for points addition
-
100 points when type column is "G" (gold member)
-
10 points when type column is "N"(normal member)
-
-
-
-
Regarding records from 11 to 15 (records with member ID from "00000011" to "00000015")
-
Should not be updated
-
-
Contents of member information table before and after update are shown below.
Verifying member information file
Compare input and output contents of member information file and verify that the contents are in accordance with verification details.
- Verification details
-
-
Member information file should be output in the output directory
-
Output file: files/output/output-member-info-data.csv
-
-
Output records should be only for records 1 to 10 (records with member ID from "00000001" to "00000010")
-
Regarding output records
-
status column
-
Records with "0"(initial status) should not exist
-
-
point column
-
Points should be added based on membership type, for point addition
-
100 points when type column is "G"(gold member)
-
10 points when type column is "N"(normal member)
-
-
-
-
Input and output details of member information file are shown below.
File fields are output in the sequence of id (member id), type (membership type), status(product purchasing flag) and point(points).
Implementation in tasklet model
Implement the processes from creation to execution of a job which performs input check in tasklet model using following procedures.
Adding message definition
Log message uses message definition and is used at the time of log output to make it easier to design prevention of variations in the code system and extraction of keywords to be monitored.
Since it is used as a common in the chunk model / tasklet model, it can be skipped if created already.
Configure application-messages.properties
and launch-context.xml
as shown below.
launch-context.xml
is already configured in TERASOLUNA Batch 5.x.
# (1)
errors.maxInteger=The {0} exceeds {1}.
<!-- omitted -->
<bean id="messageSource" class="org.springframework.context.support.ResourceBundleMessageSource"
p:basenames="i18n/application-messages" /> <!-- (2) -->
<!-- omitted -->
Sr. No. | Explanation |
---|---|
(1) |
Configure a message to be output when the upper limit of points is exceeded. |
(2) |
Set |
Implementation of exception handling
Implement exception handling process.
Implement following processes.
Implementation of ChunkErrorLoggingListener class
Perform exception handling by using ChunkListener interface.
Implement the process which output ERROR log when an exception occurs, as an implementation class of ChunkListener interface.
package org.terasoluna.batch.tutorial.common.listener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.ChunkListener;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.item.validator.ValidationException;
import org.springframework.context.MessageSource;
import org.springframework.stereotype.Component;
import javax.inject.Inject;
import java.util.Locale;
@Component
public class ChunkErrorLoggingListener implements ChunkListener {
private static final Logger logger = LoggerFactory.getLogger(ChunkErrorLoggingListener.class);
@Inject
MessageSource messageSource; // (1)
@Override
public void beforeChunk(ChunkContext chunkContext) {
// do nothing.
}
@Override
public void afterChunk(ChunkContext chunkContext) {
// do nothing.
}
@Override
public void afterChunkError(ChunkContext chunkContext) {
Exception e = (Exception) chunkContext.getAttribute(ChunkListener.ROLLBACK_EXCEPTION_KEY); // (2)
if (e instanceof ValidationException) {
logger.error(messageSource
.getMessage("errors.maxInteger", new String[] { "Point", "1000000" }, Locale.getDefault())); // (3)
}
}
}
Sr. No. | Explanation |
---|---|
(1) |
Inject an instance of |
(2) |
Fetch the value of the exception occurred which is set using the key |
(3) |
Fetch a message with message ID |
Configuring job Bean definition file
Configuration of job Bean definition file which performs exception handling by ChunkListener.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:batch="http://www.springframework.org/schema/batch"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:mybatis="http://mybatis.org/schema/mybatis-spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd
http://mybatis.org/schema/mybatis-spring http://mybatis.org/schema/mybatis-spring.xsd">
<!-- omitted -->
<context:component-scan base-package="org.terasoluna.batch.tutorial.dbaccess.tasklet,
org.terasoluna.batch.tutorial.common.listener"/> <!-- (1) -->
<!-- omitted -->
<batch:job id="jobPointAddTasklet" job-repository="jobRepository">
<batch:step id="jobPointAddTasklet.step01">
<batch:tasklet transaction-manager="jobTransactionManager"
ref="pointAddTasklet">
<batch:listeners>
<batch:listener ref="chunkErrorLoggingListener"/> <!-- (2) -->
</batch:listeners>
</batch:tasklet>
</batch:step>
</batch:job>
</beans>
Sr. No. | Explanation |
---|---|
(1) |
Configure a base package subjected to component scanning. |
(2) |
Configure implementation class of |
Job execution and results verification
Execute created job on STS and verify results.
Execute a job from execution configuration
Execute a job from execution configuration which is created already.
Here, execute a job by using abnormal data.
Since how to change input data vary depending on resource (database or file) which handle job implementing input check,
execute as below.
- When input check is implemented for the job which inputs or outputs data by accessing a database
-
Execute a job by using execution configuration created in Execute job from execution configuration of a job which inputs or outputs data by accessing a database.
Comment out script of normal system data and cancel comment out of abnormal system data script by Database Initialize of batch-application.proeprties
in order to use abnormal system data.
# Database Initialize
tutorial.create-table.script=file:sqls/create-member-info-table.sql
#tutorial.insert-data.script=file:sqls/insert-member-info-data.sql
tutorial.insert-data.script=file:sqls/insert-member-info-error-data.sql
- When input check is implemented for a job which inputs or outputs data by accessing a file
-
Execute a job by using execution configuration created in Execute job from execution configuration of job which inputs or outputs data by accessing a file.
Change input file (inputFile) path from normal system data (insert-member-info-data.csv) to abnormal system data (insert-member-info-error-data.csv), from the arguments configured by execution configuration in order to use abnormal system data.
Verifying console log
Open Console View and verify that following details are output in a log.
-
Process has terminated abnormally (FAILED)
-
org.springframework.batch.item.validator.ValidationException
should occur -
org.terasoluna.batch.tutorial.common.listener.ChunkErrorLoggingListener
should output following message as an ERROR log-
"The Point exceeds 1000000."
-
[2017/09/12 10:22:22] [main] [o.t.b.t.c.l.ChunkErrorLoggingListener] [ERROR] The Point exceeds 1000000.
[2017/09/12 10:22:22] [main] [o.s.b.c.s.AbstractStep] [ERROR] Encountered an error executing step jobPointAddTasklet.step01 in job jobPointAddTasklet
org.springframework.batch.item.validator.ValidationException: Validation failed for org.terasoluna.batch.tutorial.common.dto.MemberInfoDto@6e4ea0bd:
Field error in object 'item' on field 'point': rejected value [1000001]; codes [Max.item.point,Max.point,Max.int,Max]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [item.point,point]; arguments []; default message [point],1000000]; default message [must be less than or equal to 1000000]
at org.springframework.batch.item.validator.SpringValidator.validate(SpringValidator.java:54)
(.. omitted)
[2017/09/12 10:22:22] [main] [o.s.b.c.l.s.SimpleJobLauncher] [INFO ] Job: [FlowJob: [name=jobPointAddTasklet]] completed with the following parameters: [{jsr_batch_run_id=478}] and the following status: [FAILED]
[2017/09/12 10:22:22] [main] [o.s.c.s.ClassPathXmlApplicationContext] [INFO ] Closing org.springframework.context.support.ClassPathXmlApplicationContext@735f7ae5: startup date [Tue Sep 12 10:22:20 JST 2017]; root of context hierarchy
Verifying exit codes
Verify that the process has terminated abnormally, using exit codes.
For verification procedure, refer Job execution and results verification.
Verify that the exit code (exit value) is 255 (abnormal termination).
Verifying output resource
Verify an output resource (database or file) by a job which performs input check.
In case of a tasklet model, confirm that nothing has been updated when an error occurs since collective commit is being adopted.
Verifying member information table
Verify member information table by using Data Source Explorer.
Compare contents of member information table before and after update, and verify that the contents are in accordance with verification details.
For verification procedure, refer Refer database by using Data Source Explorer.
- Verification details
-
-
Data should not be updated for all the records
-
Contents of member information table in initial status are shown below.
Verifying member information file
Compare output contents of member information file and verify that the contents are in accordance with verification details.
- Verification details
-
-
Member information file should be output in the output directory as Blank file
-
Output file: files/output/output-member-info-data.csv
-
-