Overview
Exclusive control is a process performed to maintain consistency of data when update processing is performed simultaneously for the same resource from multiple transactions. In the case where there is a possibility that updating processing is performed simultaneously for the same resource from multiple transactions, it is basically necessary to perform exclusive control.
Multiple transaction means following in this chapter.
-
Transaction at the time of simultaneous execution of multiple jobs
-
Transaction at the time of simultaneous execution with online processing
Exclusive control of multiple jobs
When multiple jobs are executed at the same time, it is fundamental to design jobs so that exclusive control is not required. This means that it is basic to divide the resources to be accessed and the processing target for each job. |
Since the concept of exclusive control is the same as online processing, please refer to Exclusive Control in TERASOLUNA Server 5.x Development Guideline
Here, focus on the part not described in TERASOLUNA Server 5.x.
The usage method of this function is same in the chunk model as well as tasklet model.
Necessity of Exclusive Control
For the necessity of exclusive control, please refer to Necessity of Exclusive Control in TERASOLUNA Server 5.x Development Guideline.
Exclusive Control for File
Exclusive control for file is generally implemented by file locking.
- File Locking
-
File locking is a mechanism for restricting reading and writing from other programs while using files with a certain program. The outline of file lock processing is as follows.
-
The batch process A acquires the lock of the file and starts the file updating process.
-
Batch process B attempts to update the same file and attempts to acquire the file lock fails.
-
The batch process A ends the processing and unlocks the file
-
The Batch Process A tries to acquire the lock of the Target File.
-
The Batch Process A succeeds in acquiring the lock of the Target File.
-
The Batch Process B tries to acquire the lock of the Target File.
-
The Batch Process A writes the Target File.
-
Since the Batch Process A is locked the Target File, the Batch Process B fails to acquire the lock of the Target File.
-
The Batch Process B performs processing of file update failure.
-
The Batch Process A releases the lock of the Target File.
Prevention of Deadlock
Even in a file,in the same as a database, when acquiring a lock on multiple files, deadlock may occur in some cases.
Therefore, it is important to make a rule the update order of files. |
Exclusive Control of Database
For details About Exclusive Control of Database, refer to Exclusive control using database locking in TERASOLUNA Server 5.x Development Guideline.
Choose Exclusive Control Scheme
Explain the locking scheme and suitable situation for TERASOLUNA Batch 5.x.
Lock scheme | Suitable situation |
---|---|
The case where process can be continued with the update result of another transaction being out of processing targets in a transaction at the time of concurrent execution. |
|
Process in which the processing time is long and it is difficult to redo because the situation of the target data has changed during processing |
Relationship between Exclusive Control and Components
The relationship between each component provided by TERASOLUNA Batch 5.x and exclusive control is as follows.
- Optimistic lock
Processing model | Component | File | Database |
---|---|---|---|
Chunk |
ItemReader |
- |
Acquires data including a column that can confirm that the same data is obtained at the time of acquiring and updating such as Version column. |
ItemProcessor |
- |
Exclusive control is unnecessary. |
|
ItemWriter |
- |
Check the difference between acquisition and update, confirm that it is not updated by other processing, then update. |
|
Tasklet |
Tasklet |
- |
When acquiring data, execute the processing described in the ItemReader section, and when updating the data, the processing described in ItemWriter section. |
Optimistic lock on files
Because of the characteristic of the file, do not apply optimistic lock on files. |
- Pessimistic lock
Processing model | Component | File | Database |
---|---|---|---|
Chunk |
ItemReader |
- |
Use FOR UPDATE of SQL statement. |
ItemProcessor |
Since it is fundamental to handle locked data, in principle, exclusive control is not performed here. |
||
ItemWriter |
- |
Update data without conscious of exclusion. |
|
Tasklet |
Tasklet |
Get a file lock right after opening a file with ItemStreamReader. |
When acquiring data, execute the processing described in the ItemReader section, and when updating the data, the processing described in ItemWriter section. |
Pessimistic lock on file
Pessimistic lock on files should be implemented in the tasklet model. In the chunk model, due to its structure, there is a period that can not be excluded in the gap of chunk processing. Also, it is assumed that file access is done by Injecting ItemStreamReader / ItemStreamWriter. |
Waiting time due to Pessimistic lock in database
When pessimistic locking is performed, the wait time for processing due to contention may be prolonged. In that case, it is reasonable to use the pessimistic lock by specifying the NO WAIT option and the timeout time. |
How to use
Explain how to use exclusive control by resource.
Exclusive control of file
Exclusive control of file with TERASOLUNA Batch 5.x is realized by implementing Tasklet.
As a means of achieving exclusion, exclusive control is performed by file lock acquisition using the java.nio.channels.FileChannel
class.
Details of the FileChannel class
For details and how to use |
Show an example of using FileChannel
class to get a file lock.
@Component
@Scope("step")
public class FileExcusiveTasklet implements Tasklet {
private String targetPath = null; // (1)
@Inject
ItemStreamReader<SalesPlanDetail> reader;
@Inject
ItemStreamWriter<SalesPlanDetailWithProcessName> writer;
@Override
public RepeatStatus execute(StepContribution contribution,
ChunkContext chunkContext) throws Exception {
// omitted.
FileChannel fc = null;
FileLock fileLock = null;
try {
reader.open(executionConetxt);
writer.open(executionConetxt); // (2)
try {
File file = new File(targetPath);
fc = FileChannel.open(file.toPath(), StandardOpenOption.WRITE,
StandardOpenOption.CREATE,
StandardOpenOption.APPEND); // (3)
fileLock = fc.tryLock(); // (4)
} catch (IOException e) {
logger.error("Failure other than lock acquisition", e);
throw new FailedOtherAcquireLockException(
"Failure other than lock acquisition", e);
}
if (fileLock == null) {
logger.error("Failed to acquire lock. [processName={}]", processName);
throw new FailedAcquireLockException("Failed to acquire lock");
}
// (5)
SalesPlanDetail item;
List<SalesPlanDetailWithProcessName> items = new ArrayList<>();
while ((item = reader.read()) != null) {
// omitted.
items.add(item);
if (items.size() >= 10) {
writer.write(items);
items.clear();
}
}
if (items.size() > 0) {
writer.write(items);
}
} finally {
if (fileLock != null) {
try {
fileLock.release(); // (6)
} catch (IOException e) {
logger.warn("Lock release failed.", e);
}
}
if (fc != null) {
try {
fc.close();
} catch (IOException e) {
// ignore
}
}
writer.close(); // (7)
reader.close();
}
return RepeatStatus.FINISHED;
}
// (8)
@Value("#{jobParameters[outputFile]}")
public void setTargetPath(String targetPath) {
this.targetPath = targetPath;
}
}
Sr. No. | Description |
---|---|
(1) |
File path to be exclusively controlled. |
(2) |
Open file to be exclusively controlled. |
(3) |
Get file channel. |
(4) |
Get file lock. |
(5) |
Execute business logic with file output. |
(6) |
Release file lock. |
(7) |
Close file to be exclusively controlled. |
(8) |
Set file path. |
About the method of FileChannel used for lock acquisition
It is recommended to use the |
Exclusive control between threads in the same VM
Attention must be paid to exclusive control between threads in the same VM.
When processing files between threads in the same VM, the lock function using the |
About appendAllowed property of FlatFileItemWriter
When creating (overwriting) a file, exclusive control can be realized by setting the |
Exclusive Control of Database
Explain exclusive control of database in TERASOLUNA Batch 5.x.
The exclusive control implementation of the database is basically How to implement while using MyBatis3 in TERASOLUNA Server 5.x Development Guideline. In this guideline, Explain it on the premise that How to implement while using MyBatis3 is done.
As shown in Relationship between Exclusive Control and Components, there are variations due to combination of processing model and component.
Exclusive control scheme |
Processing model |
Component |
Optimistic lock |
Chunk model |
ItemReader/ItemWriter |
Tasklet model |
ItemReader/ItemWriter |
|
Mapper interface |
||
Pessimistic lock |
Chunk model |
ItemReader/ItemWriter |
Tasklet model |
ItemReader/ItemWriter |
|
Mapper interface |
When using the Mapper interface in tasklet model, the explanation is omitted. Refer to How to implement while using MyBatis3.
When using ItemReader/ItemWriter in tasklet model, the calling part in the Mapper interface is replaced by ItemReader/ItemWriter, so the explanation is also omitted.
Therefore, exclusive control of chunk model will be explained here.
Optimistic Lock
Explain Optimistic lock in chunk model.
Since the behavior of the job changes according to the setting of the assertUpdates
property of MyBatisBatchItemWriter, it is necessary to set it appropriately according to the business requirements.
Show the job definition for optimistic lock.
<!-- (1) -->
<bean id="reader"
class="org.mybatis.spring.batch.MyBatisCursorItemReader" scope="step"
p:queryId="org.terasoluna.batch.functionaltest.ch05.exclusivecontrol.repository.ExclusiveControlRepository.branchFindOne"
p:sqlSessionFactory-ref="jobSqlSessionFactory"/>
<property name="parameterValues">
<map>
<entry key="branchId" value="#{jobParameters[branchId]}"/>
</map>
</property>
</bean>
<!-- (2) --->
<bean id="writer"
class="org.mybatis.spring.batch.MyBatisBatchItemWriter" scope="step"
p:statementId="org.terasoluna.batch.functionaltest.ch05.exclusivecontrol.repository.ExclusiveControlRepository.branchExclusiveUpdate"
p:sqlSessionTemplate-ref="batchModeSqlSessionTemplate"
p:assertUpdates="true" /> <!-- (3) -->
<batch:job id="chunkOptimisticLockCheckJob" job-repository="jobRepository">
<batch:step id="chunkOptimisticLockCheckJob.step01">
<batch:tasklet transaction-manager="jobTransactionManager">
<batch:chunk reader="reader" processor="branchEditItemProcessor"
writer="writer" commit-interval="10" />
</batch:tasklet>
</batch:step>
</batch:job>
Sr. No. | Description |
---|---|
(1) |
Set SQLID of data acquisition by optimistic lock. |
(2) |
Set SQLID of data update by optimistic lock. |
(3) |
Set whether to check the number of batch updates. |
Pessimistic Lock
Explain pessimistic lock in chunk model.
Show the job definition for pessimistic lock.
<!-- (1) -->
<bean id="reader"
class="org.mybatis.spring.batch.MyBatisCursorItemReader" scope="step"
p:queryId="org.terasoluna.batch.functionaltest.ch05.exclusivecontrol.repository.ExclusiveControlRepository.branchFindOneWithNowWaitLock"
p:sqlSessionFactory-ref="jobSqlSessionFactory">
<property name="parameterValues">
<map>
<entry key="branchId" value="#{jobParameters[branchId]}"/>
</map>
</property>
</bean>
<!-- (2) -->
<bean id="writer"
class="org.mybatis.spring.batch.MyBatisBatchItemWriter" scope="step"
p:statementId="org.terasoluna.batch.functionaltest.ch05.exclusivecontrol.repository.ExclusiveControlRepository.branchUpdate"
p:sqlSessionTemplate-ref="batchModeSqlSessionTemplate"
p:assertUpdates="#{new Boolean(jobParameters[assertUpdates])}" />
<batch:job id="chunkPessimisticLockCheckJob" job-repository="jobRepository">
<batch:step id="chunkPessimisticLockCheckJob.step01">
<batch:tasklet transaction-manager="jobTransactionManager">
<batch:chunk reader="reader" processor="branchEditItemProcessor"
writer="writer" commit-interval="10" />
</batch:tasklet>
</batch:step>
<batch:listeners>
<batch:listener ref="jobExecutionLoggingListener"/>
</batch:listeners>
</batch:job>
Sr. No. | Description |
---|---|
(1) |
Set SQLID of data acquisition by pessimistic lock. |
(2) |
Set the same SQLID as SQL of data update without exclusive control. |
Behavior during exclusive control
If performing pessimistic lock by setting NO WAIT or timeout, when excluded by another transaction, an exception is thrown in the |