The development of batch application is explained in the following flow.
What is blank project
Blank project is the template of development project wherein various settings are made in advance such as Spring Batch, MyBatis3
and is the start point of application development.
In this guideline, a blank project with a single project structure is provided.
Refer to Project structure for the explanation of structure.
Difference from TERASOLUNA Server 5.x
Multi-project structure is recommended forTERASOLUNA Server 5.x. The reason is mainly to enjoy the following merits.
However, in this guideline, a single project structure is provided unlike TERASOLUNA Server 5.x. This point should be considered for batch application also, however,
by providing single project structure, accessing the resources related to one job is given priority. |
Creation of project
How to create a project using archetype:generate
of Maven Archetype Plugin
is explained.
Regarding prerequisites of creating environment
Prerequisites are explained below.
|
Execute the following commands in the directory where project is created.
C:\xxx> mvn archetype:generate^
-DarchetypeGroupId=org.terasoluna.batch^
-DarchetypeArtifactId=terasoluna-batch-archetype^
-DarchetypeVersion=5.0.0.RELEASE
$ mvn archetype:generate \
-DarchetypeGroupId=org.terasoluna.batch \
-DarchetypeArtifactId=terasoluna-batch-archetype \
-DarchetypeVersion=5.0.0.RELEASE
Next, set the following to Interactive mode in accordance with the status of the user.
-
groupId
-
artifactId
-
version
-
package
An example of setting and executing the value is shown below.
Item name | Setting example |
---|---|
groupId |
com.example.batch |
artifactId |
batch |
version |
1.0.0-SNAPSHOT |
package |
com.example.batch |
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:2.4:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:2.4:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO]
[INFO] --- maven-archetype-plugin:2.4:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
(.. omitted)
Define value for property 'groupId': : com.example.batch
Define value for property 'artifactId': : batch
Define value for property 'version': 1.0-SNAPSHOT: : 1.0.0-SNAPSHOT
Define value for property 'package': com.example.batch: :
Confirm properties configuration:
groupId: com.example.batch
artifactId: batch
version: 1.0.0-SNAPSHOT
package: com.example.batch
Y: : y
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: terasoluna-batch-archetype:5.0.0-SNAPSHOT
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.example.batch
[INFO] Parameter: artifactId, Value: batch
[INFO] Parameter: version, Value: 1.0.0-SNAPSHOT
[INFO] Parameter: package, Value: com.example.batch
[INFO] Parameter: packageInPathFormat, Value: com/example/batch
[INFO] Parameter: package, Value: com.example.batch
[INFO] Parameter: version, Value: 1.0.0-SNAPSHOT
[INFO] Parameter: groupId, Value: com.example.batch
[INFO] Parameter: artifactId, Value: batch
[INFO] project created from Archetype in dir: C:\workspaces\zzz\batch
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:56 min
[INFO] Finished at: 2017-02-07T17:09:52+09:00
[INFO] Final Memory: 16M/240M
[INFO] ------------------------------------------------------------------------
The creation of project is completed by the above execution.
It can be confirmed whether the project was created properly by the following points.
$ mvn clean dependency:copy-dependencies -DoutputDirectory=lib package
$ java -cp 'lib/*:target/*' org.springframework.batch.core.launch.support.CommandLineJobRunner \
META-INF/jobs/job01/job01.xml job01
It is created properly if the following output is obtained.
$ mvn clean dependency:copy-dependencies -DoutputDirectory=lib package
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building TERASOLUNA Batch Framework for Java (5.x) Blank Project 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
(.. omitted)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.618 s
[INFO] Finished at: 2017-02-07T17:32:27+09:00
[INFO] Final Memory: 26M/250M
[INFO] ------------------------------------------------------------------------
$ java -cp 'lib/*;target/*' org.springframework.batch.core.launch.support.CommandLineJobRunner META-INF/jobs/job01/job01.xml job01
[2017/02/07 17:35:26] [main] [o.s.c.s.ClassPathXmlApplicationContext] [INFO ] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@62043840: startup date [Tue Feb 07 17:35:26 JST 2017]; root of context hierarchy
(.. ommited)
[2017/02/07 17:35:27] [main] [o.s.b.c.l.s.SimpleJobLauncher] [INFO ] Job: [FlowJob: [name=job01]] launched with the following parameters: [{jsr_batch_run_id=1}]
[2017/02/07 17:35:27] [main] [o.s.b.c.j.SimpleStepHandler] [INFO ] Executing step: [job01.step01]
[2017/02/07 17:35:27] [main] [o.s.b.c.l.s.SimpleJobLauncher] [INFO ] Job: [FlowJob: [name=job01]] completed with the following parameters: [{jsr_batch_run_id=1}] and the following status: [COMPLETED]
[2017/02/07 17:35:27] [main] [o.s.c.s.ClassPathXmlApplicationContext] [INFO ] Closing org.springframework.context.support.ClassPathXmlApplicationContext@62043840: startup date [Tue Feb 07 17:35:26 JST 2017]; root of context hierarchy
Project structure
Project structure that was created above, is explained. Project structure should be made by considering the following points.
-
Implement the job independent of startup method
-
Save the efforts of performing various settings such as Spring Batch, MyBatis
-
Make the environment dependent switching easy
The structure is shown and each element is explained below.
(It is explained based on the output at the time of executing the above mvn archetype:generate
to easily understand.)
Sr. No. | Explanation |
---|---|
(1) |
root package that stores various classes of the entire batch application. |
(2) |
Package that stores various classes of 1 job. You can customize it with reference to default state however, consider making it easier to judge the resources specific to job. |
(3) |
Configuration file of the entire batch application. |
(4) |
Configuration file of Logback(log output). |
(5) |
Configuration file that defines messages to be displayed when an error occurs during the input check using BeanValidation. |
(6) |
Mapper XML file that pairs with Mapper interface of MyBatis3. |
(7) |
Property file that defines messages used mainly for log output. |
(8) |
Directory that stores job-specific Bean definition file. |
(9) |
Directory that stores Bean definition file related to the entire batch application. |
(10) |
Bean definition file that describes settings related to asynchronous execution (DB polling) function. |
(11) |
Bean definition file to reduce various settings by importing in a job-specific Bean definition file. |
(12) |
Bean definition file for setting Spring Batch behavior and common jobs. |
Relation figure of each file is shown below.
Flow of development
Series of flow of developing job is explained.
Here, we will focus on understanding general flow and not the detailed explanation.
Import to IDE
Since the generated project is as per the project structure of Maven,
import as Maven project using various IDEs.
Detailed procedures are omitted.
Setting of entire application
Customize as follows depending on user status.
How to customize settings other than these by individual functions is explained.
Project information of pom.xml
As the following information is set with temporary values in the POM of the project, values should be set as per the status.
-
Project name(name element)
-
Project description(description element)
-
Project URL(url element)
-
Project inception year(inceptionYear element)
-
Project license(licenses element)
-
Project organization(organization element)
Database related settings
Database related settings are at many places, so each place should be modified.
<!-- (1) -->
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
# (2)
# Admin DataSource settings.
admin.jdbc.driver=org.h2.Driver
admin.jdbc.url=jdbc:h2:mem:batch-admin;DB_CLOSE_DELAY=-1
admin.jdbc.username=sa
admin.jdbc.password=
# (2)
# Job DataSource settings.
#jdbc.driver=org.postgresql.Driver
#jdbc.url=jdbc:postgresql://localhost:5432/postgres
#jdbc.username=postgres
#jdbc.password=postgres
jdbc.driver=org.h2.Driver
jdbc.url=jdbc:h2:mem:batch;DB_CLOSE_DELAY=-1
jdbc.username=sa
jdbc.password=
# (3)
# Spring Batch schema initialize.
data-source.initialize.enabled=true
spring-batch.schema.script=classpath:org/springframework/batch/core/schema-h2.sql
terasoluna-batch.commit.script=classpath:org/terasoluna/batch/async/db/schema-commit.sql
<!-- (3) -->
<jdbc:initialize-database data-source="adminDataSource"
enabled="${data-source.initialize.enabled:false}"
ignore-failures="ALL">
<jdbc:script location="${spring-batch.schema.script}" />
<jdbc:script location="${terasoluna-batch.commit.script}" />
</jdbc:initialize-database>
<!-- (4) -->
<bean id="adminDataSource" class="org.apache.commons.dbcp2.BasicDataSource"
destroy-method="close"
p:driverClassName="${admin.jdbc.driver}"
p:url="${admin.jdbc.url}"
p:username="${admin.jdbc.username}"
p:password="${admin.jdbc.password}"
p:maxTotal="10"
p:minIdle="1"
p:maxWaitMillis="5000"
p:defaultAutoCommit="false"/>
<!-- (4) -->
<bean id="jobDataSource" class="org.apache.commons.dbcp2.BasicDataSource"
destroy-method="close"
p:driverClassName="${jdbc.driver}"
p:url="${jdbc.url}"
p:username="${jdbc.username}"
p:password="${jdbc.password}"
p:maxTotal="10"
p:minIdle="1"
p:maxWaitMillis="5000"
p:defaultAutoCommit="false" />
<!-- (5) -->
<bean id="jobSqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"
p:dataSource-ref="jobDataSource" >
<property name="configuration">
<bean class="org.apache.ibatis.session.Configuration"
p:localCacheScope="STATEMENT"
p:lazyLoadingEnabled="true"
p:aggressiveLazyLoading="false"
p:defaultFetchSize="1000"
p:defaultExecutorType="REUSE" />
</property>
</bean>
<!-- (5) -->
<bean id="adminSqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"
p:dataSource-ref="adminDataSource" >
<property name="configuration">
<bean class="org.apache.ibatis.session.Configuration"
p:localCacheScope="STATEMENT"
p:lazyLoadingEnabled="true"
p:aggressiveLazyLoading="false"
p:defaultFetchSize="1000"
p:defaultExecutorType="REUSE" />
</property>
</bean>
Sr. No. | Explanation |
---|---|
(1) |
In pom.xml, define dependency relation of JDBC driver for connecting to the database to be used. |
(2) |
Set JDBC driver connection. |
(3) |
Define whether or not to execute the initialization of database used by Spring Batch or TERASOLUNA Batch 5.x, and the script to be used. |
(4) |
Set datasource. |
(5) |
Set MyBatis behavior. |
Build and execution of project
Build and execution of project is explained.
Build of application
Move to the root directory of the project and execute the following command.
$ mvn clean dependency:copy-dependencies -DoutputDirectory=lib package
The following is generated by this.
-
<Root directory>/target/<archetypeId>-<version>.jar
-
Jar of the created batch application is generated
-
-
<Root directory>/lib/(Dependent Jar file)
-
A set of dependent Jar files is copied
-
When deploying to the test environment and the commercial environment, these Jar files can be copied to an arbitrary directory.
Switching of configuration file according to the environment
In the pom.xml of the project, the following Profile is set as the default value.
<profiles>
<!-- Including application properties and log settings into package. (default) -->
<profile>
<id>IncludeSettings</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<exclude-property/>
<exclude-log/>
</properties>
</profile>
<!-- Excluding application properties and log settings into package. -->
<profile>
<id>ExcludeSettings</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<exclude-property>batch-application.properties</exclude-property>
<exclude-log>logback.xml</exclude-log>
</properties>
</profile>
</profiles>
Here, Whether to include environment dependent configuration file
is switched.
By utilizing this setting, it is possible to absorb the environmental difference by separately placing the configurationfile at the time of environment deployment.
Moreover, by applying this, it is possible to change the configuration file to be included in Jar in the test environment and the commercial environment.
An example is shown below.
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
<resource>
<directory>${project.root.basedir}/${project.config.resource.directory.rdbms}</directory>
</resource>
</resources>
</build>
<profiles>
<profile>
<id>postgresql9-local</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
<properties>
<project.config.resource.directory.rdbms>config/rdbms/postgresql9/local</project.config.resource.directory.rdbms>
</properties>
</profile>
<profile>
<id>postgresql9-it</id>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
<properties>
<project.config.resource.directory.rdbms>config/rdbms/postgresql9/it</project.config.resource.directory.rdbms>
</properties>
</profile>
</profiles>
Maven Profile can be activated at the time of executing command as follows.
Multiple Profiles can be activated. Use effectively whenever required.
$ mvn -P profile-1,profile-2
Execution of application
An example of executing the job based on the above-mentioned build result, is shown.
Replace archetypeId
and version
in accordance with the user’s environment.
C:\xxxx> java -cp target\archetypeId-version.jar;lib\*^
org.springframework.batch.core.launch.support.CommandLineJobRunner^
META-INF/jobs/job01.xml job01
$ java -cp 'target/archetypeId-version.jar:lib/*' \
org.springframework.batch.core.launch.support.CommandLineJobRunner \
META-INF/jobs/job01.xml job01
Necessity to handle exit code returned by java command
In the actual system,
rather than issuing a java command directly when issuing a job from the job scheduler,
It is common to start by inserting shell script for starting java. This is for setting the environment variables before starting the java command and for handling the exit code of the java command.
It is recommended that
An example of handling exit code is shown below. Example of handling exit code
|