CN114296886B - A parallel execution method and device based on range row lock - Google Patents

Abstract

The invention relates to a parallel execution method and device based on range line lock. The method mainly comprises the following steps: merging the single transactions into multiple transactions, setting commit LSNs and parallel LSNs of the multiple transactions, and registering the multiple transactions to an execution thread linked list; single transaction processing step: the execution thread extracts the next single transaction in the multi-transaction to start execution and takes the commit LSN of the single transaction as a separation line LSN; entering a corresponding step by judging whether the operation type, the parallel LSN size, the separation line LSN size and the ROWID range lock conflict or not; thread submitting step: judging whether the current single transaction is the last single transaction, if so, executing thread submitting operation, and if not, re-entering the single transaction processing step. The method adopted by the invention can optimize the parallelism of the conflict-free data synchronization operation, thereby improving the data synchronization performance and reducing the occupation of ROWID to the memory.

Description

A parallel execution method and device based on range row lock

Technical Field

The present invention relates to the technical field of database data processing, and in particular to a parallel execution method and device based on range row locks.

Background Art

Currently, heterogeneous database replication technology based on log parsing is widely used. This technology captures database logs at the source end, parses the INSERT, UPDATE, and DELETE operations in the logs, and then sends them to the target end. The target end reversely generates the log information and restores it to SQL statements. Then, it uses the database general interface and applies it to the target database to achieve data replication. Therefore, in the process of real-time database replication, the execution efficiency of the target end is one of the important factors affecting data synchronization performance.

When executing synchronization transactions on the target database, the transaction operations need to be executed based on the transaction submission order. Otherwise, errors will occur when executing related operations, thus affecting the correctness of synchronization. In order to pursue data synchronization performance, the transaction merging strategy is often adopted when executing synchronization transactions. Small transactions are merged into large transactions. This can not only reduce the number of executions by merging the same operations of different transactions, but also reduce the number of transaction submissions, thereby achieving the purpose of improving synchronization performance. In addition, the synchronization performance can be further improved by increasing the number of threads executing transactions and adopting a multi-threaded parallel strategy to simultaneously store multiple merged transactions. However, after adopting the transaction merging and parallel execution strategies at the same time, it is necessary to prevent and resolve data conflicts caused by parallel execution. In the existing technical solution, a ROWID conflict detection method is used for single transaction parallelism to prevent data conflicts. When executing a later submitted transaction, a conflict detection is performed on the ROWID involved in the earlier submitted transaction. This solution is to extract all ROWIDs of the operations involved in the transaction when the transaction is executed. This is effective for scenarios with a small transaction scale. However, when the transaction scale is large, obtaining all ROWIDs of the transaction at one time will not only affect the performance of data synchronization, but also consume a large amount of memory to store ROWID information, posing the risk of memory exhaustion.

In view of this, how to overcome the defects of the prior art and solve the above-mentioned technical problems is a difficult problem to be solved in this technical field.

Summary of the invention

In view of the above defects or improvement needs of the prior art, the present invention provides a parallel execution method and device based on range row locks, and formulates a feasible solution for the efficient parallel execution of multiple transactions formed after the single transaction is merged. The parallel execution interval is mainly divided based on the commit LSN of each single transaction in the multiple transactions (that is, the dividing line LSN is set), and the operations in the first parallel interval (that is, the interval before the first dividing line LSN, the first dividing line LSN is called the parallel LSN) are executed in parallel, and the ROWID of the operation is extracted during execution, and then the ROWID merging method is adopted to merge multiple ROWIDs into one range to reduce the memory usage of ROWID information. When executing other parallel intervals except the first parallel interval, it is necessary to ensure that the operations in the previous interval have been executed, and then use the ROWID of the current operation and the ROWID range of the previous execution thread to perform conflict detection, and execute after confirming that there is no conflict; if there is a conflict, it is necessary to wait for the previous multiple transactions to be committed before executing.

The embodiment of the present invention adopts the following technical solution:

In a first aspect, the present invention provides a parallel execution method based on range row locks, comprising:

Merge a single transaction into multiple transactions and set the commit LSN and parallel LSN of the multiple transactions, and register the multiple transactions to the execution thread list;

Single transaction processing step: The execution thread extracts the next single transaction from the multiple transactions and starts executing it, and uses the commit LSN of the single transaction as the separator LSN;

Enter the corresponding step by judging the operation type, parallel LSN size, separator LSN size, and whether the ROWID range lock conflicts;

Thread commit step: Determine whether the current single transaction is the last single transaction. If so, execute the thread commit operation. If not, re-enter the single transaction processing step.

Further, the step of merging a single transaction into multiple transactions, setting commit LSNs and parallel LSNs of the multiple transactions, and registering the multiple transactions into an execution thread linked list specifically includes:

After the target-side data synchronization service receives the source-side operation, the transactions to be executed are stored in the pending execution linked list in the order of the LSN size of the submitted operation;

The execution thread extracts N single transactions from the pending execution linked list in the order of transaction commit LSNs and merges them into a multi-transaction, sets the commit LSN of the last single transaction as the commit LSN of the multi-transaction, and then registers it in the execution thread linked list according to the commit LSN of the multi-transaction;

The execution thread sets the commit LSN of the first single transaction in the multi-transaction as the parallel LSN. If the LSN of the operations to be executed by other execution threads after the execution thread is smaller than the parallel LSN, they are executed in parallel.

Furthermore, the single transaction processing step also includes: determining the wake-up linked list of the current execution thread, and waking up the execution thread whose wake-up type is execution waiting and whose execution LSN is less than the dividing line LSN.

Furthermore, the step of entering corresponding steps by judging whether the operation type, the parallel LSN size, the separator LSN size, and the ROWID range lock conflict specifically includes:

Operation type determination step: Select to enter the parallel LSN determination step or thread submission step according to the operation type of the next operation of the current single transaction;

Parallel LSN determination step: selecting to enter the operation type determination step or the separator line LSN determination step according to the LSN of the current operation and the size of the parallel LSN of the execution thread before the current execution thread;

Separation line LSN judgment step: select to enter the conflict detection step or wait for wake-up according to the LSN of the current operation and the size of the separation line LSN of the execution thread before the current execution thread;

Conflict detection step: determine whether there is a conflict based on the ROWID of the current operation and the ROWID range lock of the execution thread before the current execution thread to choose to enter the operation type judgment step or wait for wake-up.

Furthermore, the operation type determination step specifically includes: determining the operation type of the next operation of the current single transaction, and if the operation type is a commit operation, entering a thread commit step; otherwise, entering a parallel LSN determination step.

Furthermore, the parallel LSN determination step specifically includes: determining whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads that are arranged before the current execution thread; if so, executing the current operation and extracting the ROWID of the current operation and merging it into the ROWID range lock of the current execution thread, and then entering the operation type determination step; if not, entering the separator line LSN determination step.

Furthermore, the dividing line LSN judgment step specifically includes: judging whether the LSN of the current operation is smaller than the dividing line LSNs of all execution threads arranged before the current execution thread, and if so, entering the conflict detection step; if not, searching from back to front for the first execution thread whose dividing line LSN is smaller than the LSN of the current operation, and adding the current execution thread to the wake-up linked list of the found execution thread, and marking the wake-up type as execution wait.

Furthermore, the conflict detection step specifically includes: extracting the ROWID of the current operation, and determining whether the ROWID falls within the ROWID range lock of the execution thread before the current execution thread. If so, it indicates that there is a conflict, then the current execution thread is added to the wake-up list of the execution thread with the conflict, and the wake-up type is marked as commit wait; if not, it indicates that there is no conflict, then the current operation is executed, and the ROWID of the current operation is extracted and incorporated into the ROWID range lock of the current execution thread, and then the operation type determination step is entered.

Furthermore, the thread submission operation in the thread submission step specifically includes: submitting the current execution thread and waking up all other execution threads waiting in the wake-up list, and then removing the current execution thread from the execution thread list and releasing the ROWID range lock.

On the other hand, the present invention provides a parallel execution device based on range row locks, specifically: including at least one processor and a memory, at least one processor and the memory are connected via a data bus, the memory stores instructions that can be executed by at least one processor, and after the instructions are executed by the processor, they are used to complete the parallel execution method based on range row locks in the first aspect.

Compared with the prior art, the beneficial effects of the present invention are: first, when multiple execution threads execute multiple transactions in parallel, the commit LSN of a single transaction is used to divide different parallel intervals (that is, set the separator LSN), and in the first divided parallel interval (that is, the interval before the first separator LSN, the first separator LSN is called the parallel LSN), the operations in multiple execution threads whose operation LSNs fall in this interval can be unconditionally parallelized because the resources accessed by these operations do not conflict.

Secondly, starting from the second parallel interval (that is, the interval after the first separator LSN), the execution thread with the later submission LSN may access the data modified by the execution thread with the earlier submission LSN in the first parallel interval in this interval. However, the modification of these data has not been submitted at this time, so the later execution thread may not be able to access the data it wants. In this case, the later execution thread must first ensure that the execution thread in the previous parallel interval has constructed the ROWID range lock of the interval when executing in parallel, and then use the ROWID conflict detection method. If the ROWID of the current operation to be executed falls within the ROWID range lock of the previous execution thread, it means that there is a data conflict. Otherwise, it can be executed in parallel with the previous transaction.

The above solution is not only applicable to the scenario where transaction merging is used to optimize synchronization performance, but also adopts a construction-while-execution approach when constructing the ROWID range lock, without the need to construct it before execution. Compared with the existing solution, this solution reduces the traversal process of transaction operations, can optimize the parallelism of those conflict-free data synchronization operations, thereby improving the performance of data synchronization, and also reduces the memory usage of ROWID.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a parallel execution method based on range row locks provided in Embodiment 1 of the present invention;

FIG2 is a specific flow chart of step 100 provided in Example 1 of the present invention;

FIG3 is a specific flow chart of step 300 provided in Embodiment 1 of the present invention;

FIG4 is a specific flow chart of step 400 provided in Embodiment 1 of the present invention;

FIG5 is a specific flow chart of step 500 provided in Embodiment 1 of the present invention;

FIG6 is a specific flow chart of step 600 provided in Embodiment 1 of the present invention;

FIG7 is a specific transaction execution flow chart provided by Embodiment 1 of the present invention;

FIG8 is a schematic diagram of the structure of a parallel execution device based on range row locks provided in Example 3 of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

For the sake of convenience, the embodiments of the present invention refer to transactions in the source log as "single transactions", and transactions formed by merging multiple single transactions as "multiple transactions". In addition, in the database log, each operation has an LSN. These LSNs are located in different positions, so they will naturally have different names, otherwise they cannot be distinguished. The following is a unified explanation of the LSN names that may appear in this article.

Current operation LSN: The LSN of the operation in the log.

Execution LSN: The LSN of the current operation.

Commit LSN: The LSN of each transaction commit operation.

Parallel LSN: The commit LSN of the first single transaction in multiple transactions.

Separator LSN: The commit LSN of the single transaction to which the currently executed operation belongs.

As shown in FIG. 1 , an embodiment of the present invention provides a parallel execution method based on range row locks, and the specific steps are as follows.

Step 100: Merge a single transaction into multiple transactions, set commit LSNs and parallel LSNs for the multiple transactions, and register the multiple transactions into an execution thread linked list.

Step 200 (single transaction processing step): the execution thread extracts the next single transaction in the multiple transactions and starts to execute it and uses the commit LSN of the single transaction as the separator LSN. In this step, since the execution thread executes multiple transactions formed by merging multiple single transactions, it must execute the single transactions in the order of merging in the multiple transactions when executing. The next single transaction in this embodiment refers to the next single transaction to be executed. For example, when the execution just starts, the first single transaction is the "next single transaction". When a single transaction is executed later and the step is re-entered, the original second single transaction is the current "next single transaction", and so on.

Step 300-Step 600: Enter the corresponding step by judging whether the operation type, parallel LSN size, separator LSN size and ROWID range lock conflict.

Step 700 (thread commit step): determine whether the current single transaction is the last single transaction, if so, execute the thread commit operation, if not, re-enter the single transaction processing step.

The above steps 300 to 600 specifically include step 300, step 400, step 500 and step 600. Each step is as follows:

Step 300 (operation type determination step): selecting to enter the parallel LSN determination step or the thread submission step according to the operation type of the next operation of the current single transaction;

Step 400 (parallel LSN determination step): select to enter the operation type determination step or the separator line LSN determination step according to the LSN of the current operation and the size of the parallel LSN of the execution thread before the current execution thread;

Step 500 (separation line LSN determination step): select to enter the conflict detection step or wait for wake-up according to the LSN of the current operation and the size of the separation line LSN of the execution thread before the current execution thread;

Step 600 (conflict detection step): determine whether there is a conflict based on the ROWID of the current operation and the ROWID range lock of the execution thread before the current execution thread to choose to enter the operation type determination step or wait for wake-up.

Through the above steps 100-700, when multiple execution threads execute multiple transactions in parallel, the embodiment of the present invention uses the commit LSN of a single transaction to divide different parallel intervals (i.e., sets the separator LSN), and then determines whether to execute in parallel through operation type judgment, parallel LSN judgment, separator LSN judgment, and conflict detection. Since there are multiple single transactions in the multiple transactions executed by the execution threads, the above scheme uses the commit LSN of each single transaction to divide different parallel intervals. When multiple execution threads execute, the operations falling in the same parallel interval can be detected through ROWID conflicts, and the conflict-free operations can be executed in parallel to improve the efficiency of data synchronization. Compared with the existing scheme, this embodiment reduces the traversal process of transaction operations once, and can optimize the parallelism of those conflict-free data synchronization operations, thereby improving the performance of data synchronization.

It should be noted that before starting to execute transactions, this embodiment also needs to perform the following preparations: deploy a log parsing module of the data synchronization service in the source database, which is responsible for parsing log operations; deploy a data synchronization receiving service in the target database, which is responsible for receiving operations sent by the source data synchronization service and warehousing them.

After the target end synchronous receiving service is started, it is necessary to initialize the pending execution linked list, execution thread and execution thread linked list. Among them, the pending execution linked list is used to store the pending single transactions in the linked list in the order of the LSN size of the transaction submission operation, waiting for execution and storage. Each execution thread is used to extract multiple single transactions from the pending execution linked list and merge them into multiple transactions, set the submission LSN of the last single transaction as the submission LSN of the multiple transactions, and then execute and store them. The execution thread linked list is used to register the execution threads in the linked list in the order of the size of the submission LSN of the current multiple transactions to be executed. The operation of executing multiple transactions can only be started after the registration is completed.

It should also be noted that each execution thread in this embodiment needs to initialize a ROWID range lock and set a maximum interval N for ROWID merging. When the difference between the ROWID of the current operation and the existing ROWID range is less than N, the ROWID is merged into the existing ROWID range, thereby reducing the memory usage of ROWID. It should be noted that the difference between the ROWID of the current operation and the existing ROWID range refers to the difference between the maximum value or minimum value of the ROWID of the current operation and the existing ROWID range. For example, if the range is [2,3] and the ROWID of the current operation is 1, then the difference is 2-1=1; if the range is [2,3] and the ROWID of the current operation is 5, then the difference is 5-3=2.

Each execution thread also needs to initialize a wake-up list to store the execution threads waiting for their own execution or submission. The waiting type (wake-up type) includes two types: submission waiting and execution waiting. Among them, submission waiting means that when the operating execution thread submits, other execution threads marked with submission waiting type will be awakened; execution waiting means that when the operating execution thread completes a single transaction, the submission LSN of the next single transaction is extracted as the separator LSN. If the separator LSN is greater than the LSN of the operation to be executed on the wake-up list, the execution thread to be executed will be awakened.

As shown in FIG. 2 , in this preferred embodiment, step 100 (merging a single transaction into multiple transactions and setting the commit LSN and parallel LSN of the multiple transactions, and registering the multiple transactions into the execution thread linked list) specifically includes the following steps:

Step 101: After receiving the source operation, the target data synchronization service stores the transactions to be executed in the pending execution linked list in the order of the LSN size of the submitted operation. In this step, after receiving the source operation, the target data synchronization service also classifies and manages the transaction ID of the operation. After a transaction is submitted, the transaction is stored in the pending execution linked list in the order of the LSN size of the submitted operation.

Step 102: The execution thread extracts N single transactions from the to-be-executed linked list in the order of transaction commit LSNs and merges them into a multi-transaction, sets the commit LSN of the last single transaction as the commit LSN of the multi-transaction, and then registers the multi-transaction in the execution thread linked list according to the commit LSN of the multi-transaction.

Step 103: The execution thread sets the commit LSN of the first single transaction in the multi-transaction as the parallel LSN (refer to step 200, the commit LSN of the single transaction is used as the separator LSN, so the parallel LSN here is also the first separator LSN). If the LSN of the operations to be executed by other execution threads after the execution thread is smaller than the parallel LSN, they are executed in parallel. The reason why these operations smaller than the parallel LSN can be executed in parallel is that these operations are also executed in parallel at the source end.

In this preferred embodiment, step 200 (single transaction processing step) not only starts executing the next single transaction and determines the current separation line LSN, but also includes: judging the wake-up linked list of the current execution thread, and waking up the execution thread whose wake-up type is execution waiting and whose execution LSN is less than the separation line LSN. In this way, the execution threads whose execution LSN is less than the separation line LSN can be processed in parallel.

As shown in FIG. 3 , in this preferred embodiment, step 300 (operation type determination step) specifically includes the following steps:

Step 301: Determine the operation type of the next operation of the current single transaction. The operation type refers to the DML operation of the database: including INSERT, UPDATE, DELETE, COMMIT and ROLLBACK operations.

Step 302: If the operation type is a commit operation, then enter the thread commit step (step 700); otherwise enter the parallel LSN determination step (step 400).

As shown in FIG. 4 , in this preferred embodiment, step 400 (parallel LSN determination step) specifically includes the following steps:

Step 401: Determine whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads that are arranged before the current execution thread. This step can also be understood as: obtaining the LSN of the current operation, and determining whether the parallel LSNs of the execution threads that are arranged before the current execution thread in the execution thread linked list are all larger than the LSN of the execution operation.

Step 402: If yes, execute the current operation and extract the ROWID of the current operation and merge it into the ROWID range lock of the current execution thread, and then enter the operation type judgment step (step 300); if no, enter the separation line LSN judgment step (step 500). In this step, when merging the ROWID of the current operation into the ROWID range lock of the current execution thread, this ROWID is merged into the existing ROWID range lock only when the difference between the ROWID and the existing ROWID range is less than N.

As shown in FIG. 5 , in this preferred embodiment, step 500 (separation line LSN determination step) specifically includes the following steps:

Step 501: Determine whether the LSN of the current operation is less than the LSN of the dividing lines of all execution threads that are arranged before the current execution thread. This step can also be understood as: obtaining the LSN of the current operation, and determining whether the LSN of the dividing lines of the execution threads that are arranged before the current execution thread in the execution thread linked list are all greater than the LSN of the execution operation.

Step 502: If yes, then enter the conflict detection step (step 600); if not, find the first execution thread whose dividing line LSN is smaller than the LSN of the current operation from back to front, and add the current execution thread to the wake-up list of the found execution thread, and mark the wake-up type as execution waiting, and then suspend waiting to be awakened. In this step, since the execution thread in the previous parallel interval has not yet completed the operation in the interval, the ROWID information of its operation in the parallel interval has not yet been extracted. At this time, when the parallel interval behind executes the operation, it cannot judge the operation conflict through the ROWID conflict detection mechanism. It needs to wait for the execution thread in the previous parallel interval to complete the execution and construct the ROWID range lock of the interval before conflict detection can be performed. That is to say, only when the dividing line LSN of the execution thread before the current execution thread is greater than the LSN of the execution operation, can the conflict detection step be entered.

As shown in FIG. 6 , in this preferred embodiment, step 600 (conflict detection step) specifically includes the following steps:

Step 601: extract the ROWID of the current operation, and determine whether the ROWID falls within the ROWID range lock of the execution thread before the current execution thread. This step can be understood as: extract the ROWID of the current operation, use the ROWID to locate in the ROWID range lock of the execution thread in front of the execution thread linked list, to determine whether the ROWID falls within the above ROWID range lock.

Step 602: If yes, that means there is a conflict, then add the current execution thread to the wakeup list of the conflicting execution thread, and mark the wakeup type as submission waiting, and then suspend waiting to be awakened; if no, that means there is no conflict, then execute the current operation, extract the ROWID of the current operation and merge it into the ROWID range lock of the current execution thread, and then enter the operation type judgment step (step 300). In this step, when merging the ROWID of the current operation into the ROWID range lock of the current execution thread, when the difference between the ROWID and the existing ROWID range is less than N, this ROWID is merged into the existing ROWID range lock.

In this preferred embodiment, the thread submission operation in step 700 (thread submission step) specifically includes: submitting the current execution thread and waking up all other execution threads waiting in the wake-up list, and then removing the current execution thread from the execution thread list and releasing the ROWID range lock.

The above is a detailed description of each step of the method provided in this embodiment, and the specific transaction execution process after these steps are combined into one is shown in Figure 7. The specific transaction execution process is as follows: the target end synchronization service starts working; the execution thread merges the single transaction into multiple transactions and registers them to the execution thread linked list; then the commit LSN of the first single transaction is set to the parallel LSN; then the next single transaction is extracted, its LSN is used as the separator LSN, and the execution threads that meet the conditions in the wake-up linked list are awakened at the same time; then the next operation of the current single transaction is extracted to determine whether it is a commit operation. If so, then determine whether it is the last single transaction. If so, commit and wake up all the execution threads in the wake-up linked list, and the process of the execution thread is completed. If it is not the last single transaction, re-enter the step of "extracting the next single transaction and using its LSN as the separator LSN"; when determining whether it is a commit operation, if not, then determine whether the execution LSN are all smaller than the previous parallel LSN. If so, the operation is executed and stored in the warehouse and the operation ROWID is extracted, and it is merged into the ROWID range lock, and then the step of "extracting the next operation of the current single transaction" is re-entered; if it is judged that the execution LSN is smaller than the previous parallel LSN, the result is no, then the execution LSN is judged that the execution LSN is smaller than the previous dividing line LSN, if not, the execution thread is added to the wake-up list of the previous execution thread, and the wake-up type is execution wait; if it is judged that the execution LSN is smaller than the previous dividing line LSN, the result is yes, then a conflict detection is performed in the ROWID range lock of the execution thread in front, if there is a conflict, the execution thread is added to the wake-up list of the conflicting execution thread, and the wake-up type is set to commit wait, if there is no conflict, the operation is executed and stored in the warehouse, and the operation ROWID is extracted, and it is merged into the ROWID range lock, and then the step of "extracting the next operation of the current single transaction" is re-entered.

In summary, when multiple execution threads execute multiple transactions in parallel, the present embodiment uses the submission LSN of a single transaction to divide different parallel intervals (i.e., set the separation line LSN). In the divided first parallel interval (i.e., the interval before the first separation line LSN), the operations of multiple execution threads whose operation LSN falls in this interval can be unconditionally parallel, because the resources accessed by these operations do not conflict. Secondly, starting from the divided second parallel interval (i.e., the interval after the first separation line LSN), the execution thread whose submission LSN is ranked behind may access the data modified by the execution thread whose submission LSN is ranked ahead in the first parallel interval in this interval, but the modification of these data has not been submitted at this time, so the subsequent execution thread may not be able to access the data it wants. In view of this situation, the subsequent execution thread must first ensure that the execution thread in the previous parallel interval has constructed the ROWID range lock of the interval when executing in parallel, and then use the ROWID conflict detection method. If the ROWID of the current operation to be executed falls into the ROWID range lock of the previous execution thread, it means that there is a data conflict. Otherwise, it can be executed in parallel with the previous transaction.

The above solution is not only applicable to the scenario where transaction merging is used to optimize synchronization performance, but also adopts a construction-while-execution approach when constructing the ROWID range lock, without the need to construct it before execution. Compared with the existing solution, the solution of this embodiment reduces the traversal process of transaction operations once, can optimize the parallelism of those conflict-free data synchronization operations, thereby improving the performance of data synchronization, and also reducing the memory usage of ROWID.

Embodiment 2:

Based on the parallel execution method based on range row locks provided in Example 1, this Example 2 describes the present invention in more detail through a specific application scenario.

In this embodiment, the maximum interval N of ROWID range lock merging is set to 10.

In this embodiment, table T1 (DATA INT) of the source database contains 3 rows of data, and its data is distributed in the database as follows:

ROWID DATA column 1 1 2 2 3 3

The source side has the following log operations:

Specifically, based on the above application scenario, the steps for establishing the execution thread embodied in Embodiment 1 in Embodiment 2 are as follows:

1. Create two execution threads A and B.

2. Execution thread A extracts transactions with single transaction IDs 1 and 2 and merges them into a multi-transaction. It uses the commit LSN 7 of single transaction 2 as the commit LSN of the multi-transaction and registers it in the execution thread linked list. (The commit LSN of each single transaction refers to the LSN corresponding to the COMMIT operation)

3. Execution thread B extracts transactions with single transaction IDs 3 and 4 and merges them into a multi-transaction. It uses the commit LSN 9 of single transaction 4 as the commit LSN of the multi-transaction and registers it in the execution thread linked list. At this time, execution thread A already exists in the execution thread linked list. Since B's commit LSN is larger than A's, A is ranked ahead of B in the execution thread linked list.

4. Execution thread A divides the parallel intervals into [1-3] and [3-7] according to the commit LSN of the single transaction, and sets the commit LSN 3 of the first single transaction (single transaction 1) as the parallel LSN.

5. Execution thread B divides the parallel intervals into [7-8] and [8-9] according to the commit LSN of the single transaction, and sets the commit LSN 8 of the first single transaction (single transaction 3) as the parallel LSN.

6. Execution threads A and B start executing in parallel.

After the above execution threads A and B are established and the preparation work is completed, they officially enter the parallel execution stage. The process of this stage is referred to the following table:

Through this embodiment, it can be seen from the above parallel execution process that when execution thread B executes an operation with LSN of 2, it can be executed in parallel unconditionally because it is smaller than the parallel LSN of execution thread A. This is a point to improve synchronization performance. When executing an operation with LSN of 5, it needs to wait for execution thread A to complete all operations of single transaction 1. After constructing the ROWID range lock, the ROWID of the current operation is detected for conflict in execution thread A. After detecting that there is no conflict, parallel execution is performed. This is another point to improve synchronization performance. When executing an operation with LSN of 6, the ROWID conflict detects that there is a conflict in accessing data with execution thread A, and adds itself to the wake-up linked list of execution thread A. After execution thread A submits, it is executed again, thereby ensuring the consistency of data synchronization during parallel execution. (It should be noted that the blank spaces in the above table can be operated. This table is paused to highlight the characteristics of this solution, so that the interaction between the two threads can be clearly displayed)

By comparing this embodiment with the existing solutions, it can be seen that this embodiment uses the commit LSN of a single transaction in multiple transactions to divide the parallel intervals, and then combines the ROWID conflict detection method to execute non-conflicting operations in parallel to improve the synchronization performance. This solution not only uses the optimization method of transaction merging, but also uses the method of extracting ROWID while executing and combining ROWID range locks to reduce the performance impact of ROWID extraction on data synchronization and reduce the memory usage of ROWID.

Embodiment 3:

On the basis of the parallel execution method based on range row locks provided in the above-mentioned embodiments 1 and 2, the present invention further provides a parallel execution device based on range row locks that can be used to implement the above-mentioned method, as shown in FIG8 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The parallel execution device based on range row locks of this embodiment includes one or more processors 21 and a memory 22. Among them, FIG8 takes one processor 21 as an example.

The processor 21 and the memory 22 may be connected via a bus or other means, and FIG8 takes the connection via a bus as an example.

The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the parallel execution method and system based on range row locks in Embodiments 1 and 2. The processor 21 executes various functional applications and data processing of the parallel execution device based on range row locks by running the non-volatile software programs, instructions and modules stored in the memory 22, that is, implements the parallel execution method based on range row locks in Embodiments 1 and 2.

The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The program instructions/modules are stored in the memory 22, and when executed by one or more processors 21, the parallel execution method based on range row locks in the above-mentioned embodiments 1 to 2 is executed, for example, the various steps shown in Figures 1 to 7 described above are executed.

A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (8)

1. A range line lock based parallel execution method, comprising:

Merging the single transactions into multiple transactions, setting commit LSNs and parallel LSNs of the multiple transactions, and registering the multiple transactions to an execution thread linked list; the method specifically comprises the following steps: after receiving the source operation, the target data synchronization service stores the transaction to be executed into a linked list to be executed according to the LSN size sequence of the submitting operation; the execution thread extracts N single transactions from the linked list to be executed according to the sequence of the transaction commit LSNs, merges the N single transactions into a multi-transaction, sets the commit LSN of the last single transaction as the commit LSN of the multi-transaction, and registers the multi-transaction commit LSN in the execution thread linked list; the execution thread sets the commit LSN of the first single transaction in the multi-transaction as a parallel LSN, and if the LSN of the operation which is ready to be executed by the execution thread after the execution thread is smaller than the parallel LSN, the execution thread performs parallel execution;

single transaction processing step: the execution thread extracts the next single transaction in the multi-transaction to start execution and takes the commit LSN of the single transaction as a separation line LSN;

Entering a corresponding step by judging whether the operation type, the parallel LSN size, the separation line LSN size and the ROWID range lock conflict or not; the method specifically comprises the following steps: an operation type judging step: selecting to enter a parallel LSN judging step or a thread submitting step according to the operation type of the next operation of the current single transaction; parallel LSN judging step: selecting an entering operation type judging step or a separation line LSN judging step according to the LSN of the current operation and the sizes of the parallel LSNs of all execution threads arranged before the current execution thread; and a separation line LSN judgment step: selecting to enter a conflict detection step or wait for awakening according to the LSN of the current operation and the size of a separation line LSN of an execution thread arranged before the current execution thread; and a conflict detection step: judging whether conflict exists according to ROWID of the current operation and ROWID range locks of the execution threads before the current execution thread so as to select to enter an operation type judging step or wait for awakening;

thread submitting step: judging whether the current single transaction is the last single transaction, if so, executing thread submitting operation, and if not, re-entering the single transaction processing step.

2. The range-line lock based parallel execution method of claim 1, wherein the single transaction step further comprises: judging a wake-up linked list of the current execution thread, and waking up the execution thread with the wake-up type of execution waiting and execution LSN smaller than that of the separation line LSN.

3. The parallel execution method based on range line lock according to claim 1, wherein the operation type determining step specifically includes: judging the operation type of the next operation of the current single transaction, and entering a thread submitting step if the operation type is a submitting operation; otherwise, enter the parallel LSN judgement step.

4. The parallel execution method based on range line lock according to claim 3, wherein the parallel LSN determining step specifically includes: judging whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads arranged in front of the current execution thread, if so, executing the current operation, extracting ROWID of the current operation, merging the ROWID range lock of the current execution thread, and then entering an operation type judging step; if not, entering a separation line LSN judgment step.

5. The parallel execution method based on range line lock according to claim 4, wherein the separation line LSN determining step specifically includes: judging whether the LSN of the current operation is smaller than the LSN of all the partition lines of the execution threads arranged before the current execution thread, if so, entering a conflict detection step; if not, searching the execution thread with the first separation line LSN smaller than the LSN of the current operation from the back to the front, adding the current execution thread into a wake-up linked list of the searched execution thread, and marking the wake-up type as execution waiting.

6. The parallel execution method based on range line lock according to claim 5, wherein the collision detection step specifically includes: extracting ROWID of the current operation, judging whether the ROWID falls in a ROWID range lock of an execution thread before the current execution thread, if yes, adding the current execution thread into a wakeup linked list of the execution thread with conflict, and marking the wakeup type as commit waiting; if not, the current operation is executed, and ROWID of the current operation is extracted and incorporated into the ROWID range lock of the current execution thread, and then an operation type judging step is carried out.

7. The parallel execution method based on range line lock according to any one of claims 1 to 6, wherein the thread commit operation in the thread commit step specifically includes: and submitting the execution thread, waking up all other execution threads waiting in the wake-up linked list, and then removing the execution thread from the execution thread linked list and releasing ROWID range locks.

8. A range line lock based parallel execution device, characterized by:

Comprising at least one processor and a memory connected by a data bus, the memory storing instructions for execution by the at least one processor, the instructions, when executed by the processor, for performing the range-line lock based parallel execution method of any of claims 1-7.