CN109189608B - A method for ensuring the consistency of replicated transactions and a corresponding replication device - Google Patents

Abstract

The invention provides a method for ensuring consistency of replication affairs and a corresponding replication device, comprising the following steps: acquiring an initial log serial number of an active transaction which occurs earliest in a source database, and marking the initial log serial number of the active transaction which occurs earliest as a recovery initial point; acquiring initial log serial numbers of two adjacent active transactions of a source database, and marking the initial log serial numbers of the two adjacent active transactions as range check points when the difference value of the initial log serial numbers of the two adjacent active transactions is greater than a preset threshold value; acquiring a source database transaction log, and deleting the transaction log with the sequence number of the commit log not larger than the recovery starting point and the transaction log with the sequence number of the commit log falling into a range check point; data recovery is performed according to a recovery start point, a range checkpoint, and committed transaction records. The invention adopts a range check point mechanism to compress the submitted transaction table, and can effectively reduce the scale of the submitted transaction table.

Description

A method for ensuring the consistency of replicated transactions and a corresponding replication device

【Technical field】

The invention relates to the technical field of database synchronization, in particular to a method for ensuring the consistency of replicated transactions and a corresponding replication device.

【Background technique】

Real-time synchronization of database data is a technical solution to improve the availability of information systems and ensure business continuity. Through real-time data synchronization, the business data of the target database and the source database are kept consistent in real time. When the source database fails and interrupts the service, the application system can quickly switch to the target database to ensure business continuity.

The real-time database data replication technology based on log analysis has the characteristics of small impact on the performance and data mode of the source database, support for heterogeneous operating systems and database platforms, and high data replication performance. , data migration and other fields are widely used. This technology captures the online log or archived log of the source database through the log capture process of the source end, and then analyzes the INSERT (insert), UPDATE (update) and DELETE (deletion) operations of the database into message packets in an internal specific format, and then The message packet is sent to the target end of the replication system through the TCP/IP (Transmission Control Protocol/Internet Protocol, abbreviated TCP/IP) network. After the target end receives the message packet, it unpacks and restores the transaction information of the source end to the corresponding SQL ( Structured QueryLanguage, abbreviated SQL) statement, performs real-time replication in the target database through the local database interface to achieve database data synchronization.

In order to ensure the data consistency between the source database and the target database, the database data replication technology based on log analysis usually takes the transaction of the source database as the smallest replication unit, and performs real-time data replication in strict accordance with the transaction sequence of the source database to ensure the transaction between the target database and the source database. The integrity and consistency of the target database ensure that the target database conforms to the transaction logic of the source database. Therefore, in the database data replication technology based on log analysis, the technical key lies in how to ensure the transaction consistency of the source database and the target database, especially after the replication system fails, the target database can comply with the requirements of transaction integrity and consistency Do a proper recovery.

At present, in order to ensure the integrity and consistency of the replication transaction between the target database and the source database, the general method is as follows: create a commit transaction table in the target database to record the completed commit transaction information, and use the checkpoint mechanism to check the commit transaction. The table is maintained to record the recovery starting point information. The main technical features of the checkpoint mechanism are: first, take out the minimum log sequence number of active transactions in the source database; then delete all committed transaction information less than or equal to the minimum log sequence number in the commit transaction table of the target database; The log sequence number is inserted into the commit transaction table as a checkpoint record. When the replication system fails to recover, it starts from the minimum log sequence number recorded in the checkpoint to ensure transaction consistency after failure recovery.

However, the above method based on the checkpoint mechanism to ensure transaction consistency after failure recovery of the replication system has certain shortcomings: when there is a long transaction that has not been committed for a long time in the source database, the starting log sequence number of the long transaction is written into the source database. After the transaction table is committed, the replication system may have performed a large number of transaction operations before the failure occurred. Because the long transaction has not been submitted, the log sequence number corresponding to the checkpoint cannot be advanced, and the expired transaction information in the submitted transaction table cannot be deleted in time, resulting in an excessively large scale of the submitted transaction table. During failure recovery, the target-side replication process needs to load and submit transaction table information to determine whether the transaction after the recovery starting point needs to be redone. Excessive submission of the transaction table will result in a long time to load the submitted transaction information, increase the recovery time of the replication system, and also increase the memory resource consumption of the replication system.

In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the technical field.

[Content of the invention]

The technical problem to be solved by the present invention is that due to the existence of long transactions, a large amount of expired transaction information is stored in the submission transaction table, which causes the memory of the submission transaction table to be too large and occupies resources. At the same time, when the replication system fails to recover, the submission of the transaction table will take too long to load the submitted transaction information, which increases the recovery time of the replication system and also increases the memory resource consumption of the replication system.

The embodiment of the present invention adopts the following technical solutions:

In a first aspect, the present invention provides a method for ensuring the consistency of replicated transactions, and the method for ensuring the consistency of replicated transactions includes:

Obtain the initial log sequence number of the earliest active transaction in the source database, and mark the initial log sequence number of the earliest active transaction as the recovery start point;

Obtain the starting log sequence numbers of two adjacent active transactions in the source database, and when the difference between the starting log sequence numbers of the two adjacent active transactions is greater than a preset threshold, assign the two adjacent active transactions The starting log sequence number of marked as a range checkpoint;

Obtain the source database transaction log, delete the transaction log whose commit log sequence number is not greater than the recovery starting point and the transaction log whose commit log sequence number falls within the range checkpoint to obtain the committed transaction record;

During failure recovery, data recovery is performed according to the recovery starting point, the range checkpoint and the committed transaction record, so as to ensure transaction consistency between the source database and the target database.

Preferably, the obtaining the initial log sequence number of the earliest active transaction in the source database, and marking the initial log sequence number of the earliest active transaction as the recovery starting point includes:

Use a timer to regularly obtain the starting log sequence number of the earliest active transaction in the source database;

Determine whether the start log sequence number of the earliest active transaction in the source database at the current moment has changed relative to the start log sequence number of the earliest active transaction in the source database at the previous moment;

If a change occurs, the recovery starting point is reset according to the starting log sequence number of the earliest active transaction in the source database at the current moment.

Preferably, the acquisition of the starting log sequence numbers of two adjacent active transactions in the source database, and when the difference between the starting log sequence numbers of the two adjacent active transactions is greater than a preset threshold, The starting log sequence numbers of adjacent two active transactions marked as range checkpoints include:

Obtain the starting log sequence numbers of two adjacent active transactions in the source database in turn;

Determine whether the difference between the initial log sequence numbers of the two adjacent active transactions is greater than a preset threshold;

If the difference between the initial log sequence numbers of the two adjacent active transactions is greater than a preset threshold, marking the initial log sequence numbers of the two adjacent active transactions as a range checkpoint;

Store range checkpoints as a two-dimensional array, where one row of the two-dimensional array corresponds to one of the range checkpoints.

Preferably, during failure recovery, performing data recovery according to the recovery starting point, the range checkpoint and the committed transaction record includes:

sending the recovery starting point to the source database and receiving the operation log sent by the source database, wherein the log sequence number of the operation log is greater than the log sequence number corresponding to the recovery starting point;

Analyze the operation log to obtain the commit log sequence number of the transaction corresponding to the operation log, and determine whether the commit log sequence number falls within the range checkpoint;

If the serial number of the commit log falls within the range checkpoint, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

Preferably, the method for ensuring the consistency of replicated transactions further includes:

If the serial number of the commit log does not fall within the range checkpoint, the transaction corresponding to the operation log is not loaded into the target database before the failure occurs, and the operation log needs to be synchronized in the target database.

Preferably, if the serial number of the commit log does not fall within the range checkpoint, the transaction corresponding to the operation log is not loaded into the target database before the failure occurs, and the operation log needs to be synchronized in the target database to include :

When receiving the commit log of the transaction corresponding to the operation log, adding the transaction log corresponding to the operation log to the committed transaction record;

Data synchronization is performed on the target database according to the transaction log corresponding to the operation log.

Preferably, the method for ensuring the consistency of replicated transactions further includes:

Parsing the operation log to obtain a transaction identification code corresponding to the operation log;

Judging to determine whether the transaction identification code exists in the submitted transaction record;

If it exists, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored;

If it does not exist, the operation log is parsed to obtain the commit log sequence number of the transaction corresponding to the operation log, and it is determined whether the commit log sequence number falls within the range checkpoint.

Preferably, the analyzing the operation log to obtain the commit log sequence number of the transaction corresponding to the operation log, and judging and determining whether the commit log sequence number falls within the range checkpoint includes:

Analyzing the operation log to obtain the commit log sequence number of the transaction corresponding to the operation log;

A binary search algorithm is used to locate in a two-dimensional array including multiple range checkpoints, and it is determined whether the serial number of the commit log falls within the range checkpoint.

Preferably, before the acquiring the initial log sequence number of the earliest active transaction in the source database and marking the initial log sequence number of the earliest active transaction as the recovery start point, the method further includes:

A committed transaction table and an uncommitted transaction table are created, where the committed transaction table is used to store recovery starting points, range checkpoints, and committed transaction records, and the uncommitted transaction table is used to store transaction logs of uncommitted transactions.

In a second aspect, the present invention provides a copying apparatus, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor , the instructions are set by the program to execute the method for ensuring the consistency of replicated transactions described in the first aspect.

In a third aspect, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions, the computer-executable instructions being executed by one or more processors for completing the first The method described in the aspect guarantees the consistency of replicated transactions.

Compared with the prior art, the beneficial effects of the embodiments of the present invention are: the present invention sets a range checkpoint according to a preset threshold, deletes the transaction log whose commit log sequence number falls within the range checkpoint, and uses the range checkpoint mechanism to mark Submitted transactions, that is, using the log sequence number range to represent batches of submitted transactions, even if there are long uncommitted transactions in the database, the expired submitted transaction information can be deleted effectively and timely, and the submitted transaction table can be processed. For compression, the size of the committed transaction table can be effectively reduced. At the same time, when the replication system fails to restore, it can be determined whether the corresponding transaction has been synchronized based on the range checkpoint, which can effectively shorten the recovery time of the replication system and the consumption of memory resources.

【Description of drawings】

In order to describe the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic structural diagram of a replication system provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for ensuring the consistency of replicated transactions provided by an embodiment of the present invention;

3a is a schematic diagram of a data structure of a submission transaction table provided by an embodiment of the present invention;

3b is a schematic diagram of a two-dimensional array data structure including multiple range checkpoints according to an embodiment of the present invention;

3c is a schematic diagram of a data structure of an uncommitted transaction table provided by an embodiment of the present invention;

Fig. 4 is the specific flow chart of step 204 in Fig. 2;

FIG. 5 is a schematic structural diagram of a copying apparatus according to an embodiment of the present invention.

【Detailed ways】

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Embodiment 1 :

Referring to FIG. 1, in this embodiment, the replication system includes a source database 1, a replication device and a target database 2, and the replication device is respectively connected with the source database 1 and the target database 2, so as to ensure the source database 1 through the data replication technology based on log analysis It is consistent with the transaction of the target database 2, thereby realizing data synchronization.

Specifically, the replication device includes a source-side replication tool 31 and a target-side replication tool 32 . The source-side replication tool 31 is connected to the source database 1, and the source-side replication tool 31 accesses the source database 1 through the source replication process; the target-side replication tool 32 is connected to the target database 2, and the target-side replication tool 32 accesses the target database through the target replication process 2. The source replication process and the target replication process interact to replicate the data of the source database 1 to the target database 2, thereby realizing the function of data synchronization between the source database 1 and the target database 2.

The source database 1 and the target database 2 may be homogeneous databases or heterogeneous databases, which may be designed according to specific conditions, which are not specifically limited here. Wherein, the source database 1 and the target database 2 may respectively correspond to any one of the SQLSERVER database, the MYSQL database, the DB2 database, the DM6 database, the DM7 database, the POSTGRESQL database or the ORACLE database.

In an actual application scenario, when the target-side replication tool 32 is started, it first searches the target database 2 for whether there is a commit transaction table through the target replication process, and if not, triggers the target database 2 to create a commit transaction table. The target database 2 performs corresponding operations on the object corresponding to the transaction according to the transaction log in the submitted transaction table, so as to realize data synchronization. Among them, the committed transaction table is used to store committed transaction records, recovery starting points, and checkpoint ranges. The commit transaction table can be understood as a data registration table, which is used to record the transaction operations that have been synchronized.

The data memory structure of the commit transaction table is shown in Figure 3a. The data memory structure of the commit transaction table includes three field names TID, SEQID and END_SCN. Among them, TID is the identification code of the transaction, which is the unique identifier of the transaction, SEQID is the transaction start operation, and END_SCN is the transaction commit operation. There are three kinds of different record information in the commit transaction table of the present embodiment, and the data memory structure is different based on the different record information, and the details are as follows:

The first type of record information: The first type of record information is used to record the log sequence number corresponding to the recovery start point, the content of the TID field is 0, the SEQID field is the log sequence number corresponding to the recovery start point, and the END_SCN value is 0. In other embodiments, the TID may also be other special characters, and the record information can be distinguished as the recovery starting point according to the special characters. Similarly, END_SCN can also be other values.

The second type of record information: the second type of record information is used to record the submitted transaction, and the content of the TID field is the identification code of the submitted transaction. The identification code is the unique identification of the transaction, which is extracted from the log records of the source database 1. is a positive integer; SEQID is the starting log sequence number of the committed transaction, extracted from the log record of source database 1, and the log sequence number is continuously increasing with the database operation; END_SCN is the commit log sequence number of the committed transaction, Extracted from the log records of source database 1, the log sequence number is continuously incremented with database operations.

The third type of record information: The third type of record information is used to record the log sequence number corresponding to the range checkpoint. The content of the TID field is -1, SEQID is the starting sequence number LSN1 of the active transaction that occurs earlier in the two adjacent active transactions, and END_SCN is the starting sequence number of the active transaction that occurs later in the two adjacent active transactions. LSN2, where LSN1 and LSN2 satisfy the following condition: LSN2-LSN1>RANGE, where RANGE is a preset threshold. The preset threshold is determined according to the actual situation, and may be 400, 500, 1000 or other values. In other embodiments, the TID may also be other special characters, and the record information can be distinguished as a range checkpoint according to the special characters.

The target replication process needs to load the above three kinds of record information in the commit transaction table into the memory data structure of the target database 2 when starting. The target-side replication tool 32 uses the following query statement to first query the commit transaction table information from the target database 2: SELECT SEQID, TID, END_SCN FROM "commit transaction table" ORDER BYSEQID. Then, the first type of record and the second type of record are loaded into the memory hash structure with the TID field as the search key, and the third type of record is stored in the form of a two-dimensional array. The data memory structure of the two-dimensional array is specifically shown in Figure 3b. The two-dimensional array includes two columns, the number of rows depends on the number of range checkpoints, and each row of the two-dimensional array corresponds to a range checkpoint. Among them, LSN _A and LSN _B form a range checkpoint, LSN _C and LSN _D form a range checkpoint, LSN _E and LSN _F form a range checkpoint, LSN _X and LSN _Y form a range checkpoint, and so on. The value of the log sequence number in the left column corresponds to the value of the element whose index is [0] in the array, and the value of the log sequence number in the right column corresponds to the value of the element whose index is [1] in the array. In this way, the two log sequence numbers that satisfy the conditions are correspondingly set as the maximum value and the minimum value of the range checkpoint, so as to form the range checkpoint interval.

At the same time, the target-side replication tool 32 creates an uncommitted transaction table in its memory. Among them, the uncommitted transaction table is used to store the transaction log of active transactions (uncommitted transactions). The transaction log includes an initial operation log (the first operation log) of the transaction, at least one intermediate operation log and a commit operation log (the last operation log). When the commit operation log is received, the transaction log corresponding to the transaction is stored in the commit transaction table. The data memory structure of the uncommitted transaction table is shown in Figure 3c. The data memory structure of the uncommitted transaction table includes two field names, TID and SEQID. Wherein, TID is the identification code of the transaction, and SEQID is the initial log sequence number of the uncommitted transaction, which is extracted from the log record of the source database 1, and the log sequence number is continuously increased with the operation of the database.

After the committed transaction table and the uncommitted transaction table are created, the replication device obtains the recovery starting point and range checkpoint in real time according to the actual situation to update the data stored in the committed transaction table and the uncommitted transaction table, so that in the event of a failure of the replication system, the Commit the recovery start point, range checkpoint, and committed transaction records within the transaction table for data recovery.

In practical application scenarios, the replication system may fail. In order to ensure the transaction consistency between the source database 1 and the target database 2 when the failure is restored, and to ensure that the replication system can recover accurately and efficiently, this embodiment provides a A method to ensure the consistency of replicated transactions, see Figure 2 for details.

Step 201: Obtain the initial log sequence number of the earliest active transaction in the source database, and mark the initial log sequence number of the earliest active transaction as a recovery start point.

Among them, the log sequence number is used to represent a physical transaction generated inside the database system. The log sequence number is globally unique in the log file of the database system, and the log sequence number gradually increases with the operation of the database to distinguish different physical transactions. . At present, most database management systems use LSN (Log sequence number, LSN for short) to represent a physical transaction generated inside the database system, for example, database management systems such as SQLSERVER, MYSQL, DB2, DM6, DM7, etc.

Among them, the active transaction refers to the uncommitted transaction. When an active transaction is committed, it is changed to a committed transaction, and the operations corresponding to the transaction are updated to the database.

In this embodiment, in the process of data synchronization between the source database and the target database, the source database may always have transaction operations being performed, for example, a new transaction is generated or a certain active transaction is committed and so on. That is to say, as time goes by, the object referred to by the earliest active transaction will change, and accordingly, the starting log sequence number corresponding to the earliest active transaction will also change.

Therefore, in order to ensure that the most accurate recovery starting point is obtained (the recovery starting point corresponding to the time before the fault and the closest to the fault occurrence time point is the most accurate recovery starting point), in a preferred embodiment, a timer is used to obtain the source The starting log sequence number of the earliest active transaction in the database, and it is determined whether the starting log sequence number of the earliest active transaction in the source database at the current moment is relative to the starting log sequence number of the earliest active transaction in the source database at the previous moment. If a change occurs, the recovery starting point is reset according to the starting log sequence number of the earliest active transaction in the source database at the current moment. And update the reset recovery starting point to the commit transaction table.

Specifically, it is determined whether the initial log sequence number of the earliest active transaction in the source database at the current moment is greater than the initial log sequence number of the earliest active transaction in the source database at the previous moment; The starting log sequence number of the active transaction that occurred resets the recovery starting point, and updates the reset recovery starting point to the commit transaction table.

In the actual application scenario, it is assumed that the starting log sequence number of the earliest active transaction in the source database at the current moment is LSN0, and the starting log sequence number of the earliest active transaction in the source database at the previous moment is LSN. When LSN0 is greater than LSN, Delete all records whose END_SCN is less than or equal to the LSN0 and records whose TID is equal to 0 in the commit transaction table. Specifically, the following statement can be executed:

BEGIN

DELET FROM "commit transaction table" WHERE END_SCN <= LSN0OR TID=0;

INSERT INTO "commit transaction table" VALUES(0, LSN0-1, 0);

END

It can be understood that, after the initial recovery point is reset, the transaction logs whose commit log sequence number is not greater than the reset initial recovery point will be deleted from the commit transaction table.

Step 202: Obtain the starting log sequence numbers of two adjacent active transactions in the source database, and when the difference between the starting log sequence numbers of the two adjacent active transactions is greater than a preset threshold The starting log sequence number of an active transaction is marked as a range checkpoint.

In this embodiment, the starting log sequence numbers of two adjacent active transactions in the source database are sequentially obtained, and it is determined whether the difference between the starting log sequence numbers of the two adjacent active transactions is greater than a preset threshold. If the difference between the start log sequence numbers of the two adjacent active transactions is greater than the preset threshold, the start log sequence numbers of the two adjacent active transactions are marked as a range checkpoint, and multiple range checkpoints are marked. Stored as a 2D array, where one row of the 2D array corresponds to one of the range checkpoints.

For example, there are n active transactions in the uncommitted transaction table, the starting log sequence number of the first active transaction is LSN1, the starting log sequence number of the second active transaction is LSN2, and the starting log sequence number of the third active transaction is LSN2. The sequence number is LSN3...the starting log sequence number of the nth active transaction is LSNn. First, it is determined whether the difference between LSN2 and LSN1 is greater than the preset threshold. If it is greater than the preset threshold, the starting log sequence number of the first active transaction is LSN1 and the starting log sequence of the second active transaction. The number LSN2 is marked as the first range checkpoint, and the following SQL statement can be executed:

BEGIN

DELETE FROM "Commit Transaction Table" WHERE SEQID>LSN1AND SEQID<LSN2AND END_SCN<LSN2AND TID>0;

INSERT INTO "commit transaction table" VALUES(-1, LSN1, LSN2);

END

Then, it is determined whether the difference between LSN3-LSN2 is greater than the preset threshold, and if it is greater than the preset threshold, set the starting log sequence number of the second active transaction as the starting log sequence of LSN2 and the third active transaction No. LSN3 marked as the second range checkpoint, and so on, to get all range checkpoints.

In this embodiment, all the aforementioned range checkpoints are stored in the form of a two-dimensional array, wherein a row of the two-dimensional array corresponds to one of the range checkpoints. The element with the index [0] of the array records the smaller starting log sequence number, and the element with the index [1] of the array records the larger starting log sequence number. By setting the range checkpoint in the foregoing manner, and then marking whether the transaction has been committed according to the range checkpoint, the memory occupied by the commit transaction table can be effectively reduced.

Step 203: Obtain the source database transaction log, delete the transaction log whose commit log sequence number is smaller than the recovery starting point and the transaction log whose commit log sequence number falls within the range checkpoint to obtain the committed transaction record.

In this embodiment, the source-side replication tool implements access to the source database to obtain the source database transaction log, and sends the obtained transaction log to the target-side replication tool, and the target-side replication tool executes the execution according to the size of the log sequence number of the transaction commit log corresponding action.

Specifically, the target-side replication tool first deletes the transaction log whose commit log sequence number is smaller than the recovery starting point, and then deletes the transaction log whose commit log sequence number falls within the range checkpoint, and simultaneously deletes the two starting points corresponding to the corresponding range point. The log sequence number is inserted into the commit transaction table. In this way, a row of range checkpoint record information is used to represent batch submitted transaction information, and the submitted transaction table is compressed, so as to avoid the occurrence of an excessively large scale of the submitted transaction table caused by long transactions.

It should be noted here that when the log sequence number of at least one transaction log falls within a range checkpoint, the range checkpoint is inserted into the commit transaction table, indicating batch commit transaction information. When the log sequence numbers of no transaction logs fall within a range checkpoint, the range checkpoint need not be recorded in the commit transaction table.

Step 204: During fault recovery, perform data recovery according to the recovery starting point, the range checkpoint, and the committed transaction record, so as to ensure transaction consistency between the source database and the target database.

In this embodiment, when the replication system fails to recover, the target-side replication tool first loads the submitted transaction table information into the memory structure, and stores the range checkpoint in the form of a two-dimensional array. Restore the starting point to restore.

Please refer to FIG. 3 for details, step 204: during fault recovery, perform data recovery according to the recovery starting point, the range checkpoint and the committed transaction record, so as to ensure the transaction consistency of the source database and the target database, including the following: step:

Step 2041: Send the recovery starting point to the source database and receive the operation log sent by the source database, wherein the log sequence number of the operation log is greater than the log sequence number corresponding to the recovery starting point.

In this embodiment, when the replication system performs fault recovery, the source database must obtain the recovery starting point, and send the transaction log to the target database according to the recovery starting point. The recovery starting point is the starting log sequence number of the earliest active transaction in the source database when the failure occurs.

Specifically, the source-side replication tool starts the source-side replication process, and sends a command message requesting to obtain the recovery starting point to the target-side replication tool, so as to obtain the starting log sequence number for log analysis. After receiving the request command from the source replication tool, the target replication tool executes the following query: SELECT SEQID FROM "commit transaction table" WHERE TID=0, and then obtains the log sequence number corresponding to the recovery starting point from the submission transaction table. The target-side replication tool sends the obtained log sequence number corresponding to the recovery starting point to the source-side replication tool. After receiving the log sequence number corresponding to the recovery starting point, the source-side replication tool locates the log in the source database according to the recovery starting point. Analysis point. That is, the source-side replication tool obtains from the source database an operation log whose log sequence number is greater than the log sequence number corresponding to the recovery starting point, and sends the operation log to the target-side replication tool.

Step 2042: Parse the operation log to obtain a transaction identification code corresponding to the operation log.

In this embodiment, the target-side replication tool parses the operation log to obtain the transaction identification code corresponding to the operation log.

Step 2043: Determine whether the transaction identification code exists in the submitted transaction record, if so, execute step 2044; if not, execute step 2045.

In this embodiment, after receiving the commit log of the transaction, firstly use the transaction identification code as the search key to search the content of the submitted transaction table to determine whether the transaction identification code exists in the submitted transaction record. If it exists, go to step 2044; if not, go to step 2045.

Step 2044: The transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

If the transaction identification code exists in the committed transaction record, it means that the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

Step 2045: Analyze the operation log to obtain the commit log sequence number of the transaction corresponding to the operation log, and determine whether the commit log sequence number falls within the range checkpoint.

If the transaction identifier does not exist in the committed transaction record, the operation log is parsed to obtain the commit log sequence number of the transaction corresponding to the operation log, and it is determined whether the commit log sequence number falls within the range checkpoint. In order to improve the accuracy of the judgment, in a preferred embodiment, if the transaction identification code does not exist in the committed transaction record, the operation log is parsed to obtain the commit log sequence number and the start log sequence of the transaction corresponding to the operation log. number, and determine whether the commit log sequence number and the start log sequence number fall within the range checkpoint. If both the commit log sequence number and the start log sequence number fall within the range checkpoint, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

In a preferred embodiment, since there may be multiple range checkpoints in the commit transaction table, in order to improve the search speed, a binary search algorithm can be used to locate in a two-dimensional array containing multiple range checkpoints, and it is determined that the Whether the commit log sequence number falls within the range checkpoint.

Step 2046: If the serial number of the commit log falls within the range checkpoint, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

In this implementation, if the serial number of the commit log falls within the range checkpoint, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and the operation log is ignored.

For example, suppose the commit log sequence number for this committing transaction is LSN_COMMIT. The smaller log sequence number in the range checkpoint is LSN_C, and the larger log sequence number is LSN_D.

If LSN_C<LSN_COMMIT<LSN_D, the commit log sequence number LSN_COMMIT falls within the range checkpoint, indicating that the commit transaction has been loaded into the target database before the failure occurs, and the operation log is ignored.

In order to improve the accuracy of the judgment, in another embodiment, if both the serial number of the commit log and the serial number of the starting log fall within the range checkpoint, the transaction corresponding to the operation log has been loaded into the target database before the failure occurs, and it is ignored. Operation log.

For example, suppose the start log sequence number of the commit transaction is LSN_START, and the commit log sequence number of the commit transaction is LSN_COMMIT. The smaller log sequence number in the range checkpoint is LSN_C, and the larger log sequence number is LSN_D.

If LSN_C<LSN_START<LSN_D, LSN_C<LSN_COMMIT<LSN_D, then the start log sequence number is LSN_START and the commit log sequence number LSN_COMMIT both fall within the range checkpoint, indicating that the commit transaction has been loaded into the target database before the failure occurs, ignoring The operation log.

Step 2047: If the serial number of the commit log does not fall within the range checkpoint, the transaction corresponding to the operation log is not loaded into the target database before the failure occurs, and the operation log needs to be synchronized in the target database.

If the serial number of the commit log does not fall within the range checkpoint, the transaction corresponding to the operation log is not loaded into the target database before the failure occurs, and the operation log is synchronized in the target database. Specifically, when receiving the commit log of the transaction corresponding to the operation log, first determine whether the serial number of the commit log falls within the range checkpoint, and if it does not fall within the range checkpoint, add the transaction log corresponding to the operation log to the The transaction record has been submitted, and then data synchronization is performed on the target database according to the transaction log corresponding to the operation log.

Different from the prior art, the present invention sets a range checkpoint according to a preset threshold, deletes the transaction log whose commit log sequence number falls within the range checkpoint, and adopts the range checkpoint mechanism to mark the committed transactions, that is, uses the log sequence The number range is used to represent batches of committed transactions. Even if there are long uncommitted long-term transactions in the database, the expired committed transaction information can be deleted in a timely manner, and the committed transaction table is compressed, which can effectively reduce the number of committed transaction tables. size. At the same time, when the replication system fails to restore, it can be determined whether the corresponding transaction has been synchronized based on the range checkpoint, which can effectively shorten the recovery time of the replication system and the consumption of memory resources.

Example 2 :

Please refer to FIG. 5. FIG. 5 is a schematic structural diagram of a copying apparatus according to an embodiment of the present invention. The replication apparatus of this embodiment includes one or more processors 51 and a memory 52 . Among them, one processor 51 is taken as an example in FIG. 5 .

The processor 51 and the memory 52 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 5 .

The memory 52, as a non-volatile computer-readable storage medium based on data synchronization, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, such as ensuring the consistency of replication transactions in Embodiment 1. method and corresponding program instructions. The processor 51 executes various functional applications and data processing of the method for ensuring the consistency of replicated transactions by running the non-volatile software programs, instructions and modules stored in the memory 52, thereby realizing the guaranteed replication transaction consistency of Embodiment 1 function of the method.

The memory 52 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 52 may optionally include memory located remotely from processor 51 , which may be connected to processor 51 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

For the method of ensuring the consistency of the replicated transaction, please refer to FIG. 1 to FIG. 4 and the related text descriptions, which will not be repeated here.

It is worth noting that the information exchange, execution process and other contents between the modules and units in the above-mentioned device and the system are based on the same concept as the processing method embodiments of the present invention. For details, please refer to the descriptions in the method embodiments of the present invention. , and will not be repeated here.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: Read Only Memory (abbreviated as ROM), random access memory (Random Access Memory, abbreviated as RAM), magnetic disk or optical disk, etc.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (9)

1. a kind of method for guaranteeing duplication transaction consistency, which is characterized in that the method packet for guaranteeing duplication transaction consistency It includes:

The origin log sequence number for obtaining the active transaction that source database occurs earliest, by the active transaction occurred earliest Origin log sequence number is labeled as recovery starting point；

Obtain the origin log sequence number of the two neighboring active transaction of source database, and rising in the two neighboring active transaction When the difference of beginning log serial number is greater than preset threshold, the origin log sequence number of the two neighboring active transaction is labeled as Range check point；

Obtain source database transaction journal, delete submit log serial number no more than it is described restore starting point transaction journal and Log serial number is submitted to fall into the transaction journal of range check point to obtain having been filed on transaction journal；

In fault recovery, according to the recovery starting point, range check point and it is described have been filed on transaction journal carry out Data are restored, to guarantee the transaction consistency of source database and target database；

Wherein, in fault recovery, remember according to the recovery starting point, range check point and the affairs that have been filed on Record carries out data recovery

The recovery starting point is sent to the source database and receives operation log transmitted by the source database, In, the log serial number of the operation log is greater than the corresponding log serial number of the recovery starting point；

The submission log serial number that parsing obtains the corresponding affairs of the operation log is carried out to the operation log, and is judged true Whether the fixed submission log serial number falls into the range check point；

If the submission log serial number is fallen into the range check point, the corresponding affairs of the operation log are sent out in failure It has been loaded into target database before death, has ignored the operation log.

2. the method according to claim 1 for guaranteeing duplication transaction consistency, which is characterized in that the acquisition source database The origin log sequence number of the active transaction occurred earliest, by the origin log sequence number mark of the active transaction occurred earliest Being denoted as recovery starting point includes:

Using the origin log sequence number for the active transaction that timer timing acquisition source database occurs earliest；

Judge the origin log sequence number for the active transaction for determining that current time source database occurs earliest relative to last moment Whether the origin log sequence number for the active transaction that source database occurs earliest changes；

If changing, the origin log sequence number for the active transaction that foundation current time source database occurs earliest is set again Set recovery starting point.

3. the method according to claim 1 for guaranteeing duplication transaction consistency, which is characterized in that the acquisition source database The origin log sequence number of two neighboring active transaction, and the difference of the origin log sequence number in the two neighboring active transaction When value is greater than preset threshold, include: labeled as range check point by the origin log sequence number of the two neighboring active transaction

Successively obtain the origin log sequence number of the two neighboring active transaction of source database；

Judge to determine whether the difference of the origin log sequence number of the two neighboring active transaction is greater than preset threshold；

If the difference of the origin log sequence number of the two neighboring active transaction is greater than preset threshold, will be described two neighboring The origin log sequence number of active transaction is labeled as a range check point；

Range check point is stored in the form of two-dimensional array, wherein a line of two-dimensional array corresponds to one of range inspection It makes an inventory of.

4. the method according to claim 1 for guaranteeing duplication transaction consistency, which is characterized in that the guarantee replicates affairs The method of consistency further include:

If the submission log serial number is not fallen in the range check point, the corresponding affairs of the operation log are in failure It is not loaded onto target database before occurring, the operation log is added to and does not submit transaction table.

5. the method according to claim 4 for guaranteeing duplication transaction consistency, which is characterized in that the guarantee replicates affairs The method of consistency further include:

When receiving the submission log of the corresponding affairs of the operation log, the corresponding transaction journal of the operation log is added Transaction journal is had been filed on described in being added to；

It is synchronous that data are carried out to target database according to the operation log corresponding transaction journal.

6. the according to claim 1, method of guarantee duplication transaction consistency described in 4 or 5, which is characterized in that described to guarantee again The method of transaction consistency processed further include:

Parsing is carried out to the operation log and obtains the corresponding Transaction Identifier code of the operation log；

Judgement determines that the Transaction Identifier code whether there is in having been filed in transaction journal；

If it exists, then the corresponding affairs of the operation log have been loaded into target database before failure occurs, and ignore described Operation log；

If it does not exist, then it executes and the submission log that parsing obtains the corresponding affairs of the operation log is carried out to the operation log Sequence number, and judge to determine whether the submission log serial number falls into the range check point.

7. the according to claim 1, method of guarantee duplication transaction consistency described in 4 or 5, which is characterized in that described to described Operation log carries out parsing and obtains the submission log serial number of the corresponding affairs of the operation log, and judges to determine the submission Whether log serial number falls into the range check point

The submission log serial number that parsing obtains the corresponding affairs of the operation log is carried out to the operation log；

It is positioned in the two-dimensional array comprising multiple range check points using binary chop algorithm, judges to determine the submission Whether log serial number falls into range check point.

8. the method according to claim 1 for guaranteeing duplication transaction consistency, which is characterized in that the acquisition source database The origin log sequence number of the active transaction occurred earliest, by the origin log sequence number mark of the active transaction occurred earliest It is denoted as before restoring starting point further include:

Creation submits transaction table and does not submit transaction table, and the submission transaction table is used for recovery of stomge starting point, range check Transaction journal is put and has been filed on, it is described not submit transaction table for storing the transaction journal for not submitting affairs.

9. a kind of reproducing unit, which is characterized in that including at least one processor；And it is logical at least one described processor Believe the memory of connection；Wherein, the memory is stored with the instruction that can be executed by least one described processor, described instruction It is arranged to carry out by program and guarantees the method for replicating transaction consistency as claim 1 ~ 8 is described in any item.