CN115509694B

CN115509694B - Transaction processing method, device, electronic equipment and storage medium

Info

Publication number: CN115509694B
Application number: CN202211237963.4A
Authority: CN
Inventors: 邓云
Original assignee: Beijing Volcano Engine Technology Co Ltd
Current assignee: Beijing Volcano Engine Technology Co Ltd
Priority date: 2022-10-08
Filing date: 2022-10-08
Publication date: 2024-04-30
Anticipated expiration: 2042-10-08
Also published as: CN115509694A

Abstract

The present disclosure provides a transaction processing method, an apparatus, an electronic device, and a storage medium, where a first write set of a currently executed transaction is determined according to transaction record log information of a main database, where the first write set includes key values of multiple types of keys, and is used to uniquely characterize the currently executed transaction and each statement in the currently executed transaction; determining a transaction to be executed in an executable state, and determining a second write set of the transaction to be executed according to transaction record log information of the main database, wherein the second write set comprises key values of a plurality of types of keys and is used for uniquely characterizing the transaction to be executed and each statement in the transaction to be executed; judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and executing any statement in advance when determining that any statement does not conflict with the current execution transaction.

Description

Transaction processing method, device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of data processing, and in particular relates to a transaction processing method, a transaction processing device, electronic equipment and a storage medium.

Background

The core problem of data replication among instances such as a database MySQL, a data transmission service (Data Transmission Service, DTS) and the like, which affects the synchronization performance of the instances, is concurrency analysis of the transactions, so that the conflicts among the transactions need to be coordinated, and the concurrency of the playback transactions can be improved.

In the related technology, the method mainly comprises the steps that based on a write set (WriteSet) and a logic clock on a master node, transaction update data and transaction concurrency related information are recorded in a binary log (Binlog), and then a slave node plays back the transaction in concurrency according to the Binlog of the master node, so that the aim of the data consistency with the master node is fulfilled, but the slave node cannot perceive WriteSet existence, and plays back each transaction in concurrency by the slave node only depending on the transaction concurrency related information of the Binlog.

Disclosure of Invention

The embodiment of the disclosure at least provides a transaction processing method, a transaction processing device, electronic equipment and a storage medium.

In a first aspect, an embodiment of the present disclosure provides a transaction processing method, including:

Determining a first write set of a currently executed transaction according to transaction record log information of a main database, wherein the first write set comprises key values of a plurality of types of keys and is used for uniquely representing the currently executed transaction and each statement in the currently executed transaction;

Determining a transaction to be executed in an executable state, and determining a second write set of the transaction to be executed according to transaction record log information of the main database, wherein the second write set comprises key values of various types of keys and is used for uniquely characterizing the transaction to be executed and each statement in the transaction to be executed;

Judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction.

In an alternative embodiment, taking the first write set or the second write set as a target write set and the currently executed transaction or the to-be-executed transaction as a target transaction, determining the target write set of the target transaction according to the transaction record log information of the main database, including:

determining global keys, table keys and main keys and/or unique keys of each statement, which are included in a target write set of the target transaction, according to transaction record log information of the main database;

The global key represents a global lock of the target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and lock priority is that the global key is larger than the table key, which is larger than the primary key and the unique key.

In an alternative embodiment, determining, according to the transaction log information of the master database, a global key, a table key, and a master key and/or a unique key of each statement included in a target write set of the target transaction, where the method includes:

if a preset target condition exists, determining a global key or a table key included in a target write set of the target transaction; if the target condition does not exist, determining the main key and/or the unique key of each statement included in the target write set of the target transaction;

Wherein the target conditions include: if the target transaction comprises a database definition language DDL statement or is the first transaction in a transaction log file included in the transaction log information, determining that a target write set of the target transaction comprises the global key; if the target transaction does not have a main key table or comprises at least one of the following target type tables, determining that the target write set of the target transaction at least comprises the table key, wherein the target type table is: a table with unique index statements, a parent table of foreign key constraints.

In an alternative embodiment, determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction according to the key value of each type key in the first write set and the key value of each type key in the second write set includes:

respectively comparing a second write set corresponding to each statement included in the transaction to be executed with the first write set of the current transaction to be executed;

and sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed are intersected with the first write set according to the lock priority, determining that corresponding sentences collide with the current transaction under the condition that intersection is determined, and determining that corresponding sentences do not collide with the current transaction under the condition that intersection is not determined.

In an optional implementation manner, the determining, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed have intersections with the first write set in sequence, if yes, determining that the corresponding statement has a conflict with the currently executed transaction, and if no intersection is determined, determining that the corresponding statement does not conflict with the currently executed transaction includes:

According to the lock priority, if the global key of the transaction to be executed has an intersection with the first write set, determining that all sentences included in the transaction to be executed are in conflict with the current transaction to be executed;

If the global key of the transaction to be executed does not have an intersection with the first write set, and the table key of the transaction to be executed has an intersection with the first write set, all sentences under the table key with the intersection of the transaction to be executed collide with the current transaction to be executed;

if the global key, the table key and the first write set of the transaction to be executed do not have intersection, and the unique key or the main key of the transaction to be executed and the first write set have intersection, determining that the sentence corresponding to the unique key or the main key with intersection conflicts with the currently executed transaction.

In an alternative embodiment, before determining the first write set of the currently executed transaction according to the transaction log information of the master database, the method further includes:

and after the execution of any one transaction is completed, determining the current execution transaction according to the logic clock of the transaction record log information of the main database, wherein the current execution transaction at least comprises the transaction of the next execution sequence corresponding to the completed execution of the any one transaction.

In an alternative embodiment, determining a transaction to be executed in an executable state includes:

And determining the transaction in the next execution sequence corresponding to the current execution transaction as the transaction to be executed in an executable state according to the logic clock of the transaction record log information of the main database.

In a second aspect, embodiments of the present disclosure further provide a transaction processing apparatus, including:

The first determining module is used for determining a first write set of a current execution transaction according to transaction record log information of a main database, wherein the first write set comprises key values of a plurality of types of keys and is used for uniquely representing the current execution transaction and each statement in the current execution transaction;

The second determining module is used for determining a transaction to be executed in an executable state, and determining a second write set of the transaction to be executed according to transaction record log information of the main database, wherein the second write set comprises key values of various types of keys and is used for uniquely representing the transaction to be executed and each statement in the transaction to be executed;

And the execution module is used for judging whether each statement included in the transaction to be executed conflicts with the current execution transaction according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction.

In a third aspect, an optional implementation manner of the disclosure further provides an electronic device, including a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, where the machine-readable instructions, when executed by the processor, perform the steps in the first aspect, or any possible implementation manner of the first aspect.

In a fourth aspect, an alternative implementation of the present disclosure further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the first aspect, or any of the possible implementation manners of the first aspect.

In the embodiment of the disclosure, a first write set of a currently executed transaction is determined according to transaction record log information of a main database, a transaction to be executed in an executable state is determined, and a second write set of the transaction to be executed is determined according to the transaction record log information of the main database, and further, according to key values of all types of keys in the first write set and key values of all types of keys in the second write set, whether all sentences included in the transaction to be executed collide with the currently executed transaction or not is judged, and when any sentence is determined to not collide with the currently executed transaction, the any sentence is pre-executed, so that the write set can be constructed according to the transaction record log information of the main database, each sentence in the transaction to be executed can be compared with the currently executed transaction in a collision mode, a transaction concurrency analysis method of the transaction with granularity is realized, the sentence to be executed in front of the transaction to be executed without waiting for the sentence to have no conflict, the transaction to be executed is not required to be executed again, the conflict between the sentence and the transaction to be executed is reduced, the concurrency time is not guaranteed, and the transaction is not executed is not in conflict, and the playback condition is not is improved to the furthest.

The effect descriptions of the above transaction processing device, the electronic apparatus, and the computer readable storage medium are referred to the description of the above transaction processing method, and are not repeated here.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the aspects of the disclosure.

The foregoing objects, features and advantages of the disclosure will be more readily apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for the embodiments are briefly described below, which are incorporated in and constitute a part of the specification, these drawings showing embodiments consistent with the present disclosure and together with the description serve to illustrate the technical solutions of the present disclosure. It is to be understood that the following drawings illustrate only certain embodiments of the present disclosure and are therefore not to be considered limiting of its scope, for the person of ordinary skill in the art may admit to other equally relevant drawings without inventive effort.

FIG. 1 is a schematic diagram of a MySQL master-slave replication principle in the related art;

FIG. 2 illustrates a flow chart of a transaction processing method provided by an embodiment of the present disclosure;

FIG. 3 illustrates a lock priority schematic provided by an embodiment of the present disclosure;

FIG. 4 illustrates an application scenario diagram of a transaction provided by an embodiment of the present disclosure;

FIG. 5 illustrates a schematic diagram of a transaction device provided by an embodiment of the present disclosure;

Fig. 6 shows a schematic diagram of an electronic device provided by an embodiment of the disclosure.

Detailed Description

It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, but not all embodiments. The components of the disclosed embodiments generally described and illustrated herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be made by those skilled in the art based on the embodiments of this disclosure without making any inventive effort, are intended to be within the scope of this disclosure.

For the convenience of understanding the technical solutions of the present disclosure, technical terms in the embodiments of the present disclosure will be described first:

Data transfer service (Data Transmission Service, DTS): the real-time data stream service provided by a manufacturer supports data interaction among data sources such as a relational database, a non-relational database, data multidimensional analysis and the like, and comprises services such as data synchronization, migration, subscription, integration, processing and the like.

Transaction: the system is a logic unit of the database management system in the process of executing operation, is formed by a limited database operation sequence, is the minimum execution unit of the database management system operation, can be understood as a minimum non-subdividable working unit, generally corresponds to a complete service, generally is jointly completed by a plurality of data manipulation language (Data Manipulation Language, DML) sentences, and generally comprises a plurality of tables in practice, and a plurality of sentences can be included in one table.

Binary log (Binlog): binlog is a very important log of MySQL, which records all database definition language (Data Definition Language, DDL) and DML statements (except for the data query statement select), and also contains the time consumed by the statement execution, and generally, binlog mainly includes two types of files: the binary log index file is used for recording all binary files and binary log files, and is used for recording all DDL and DML (except data query statement select) statement events of the database, for example, the transaction log information of the main database in the embodiment of the present disclosure may be Binlog, and the transaction log information includes a plurality of transaction log files.

DDL statement: the main commands include create (create), modify (alter), drop (drop), etc., and DDL is mainly used for defining or changing the initial work of the structure of the table, data type, connection and constraint between tables, etc., and is usually used when building the table.

DML statement: the primary commands include select (select), update (update), insert (insert), delete (delete), etc. for operating on the data in the database.

Log sequence number (Log sequence number, LSN): the logical sequence number representing the transaction is mainly used for identifying the position in the Binlog and maintaining the consistency and the integrity of the database, and plays a very important role in implementing and rollback operations, crashing and rollback recovery and database operation synchronization in the partitioned database environment, the value of LSN gradually increases along with the writing of each transaction, and the value of LSN in each new Binlog starts counting again.

Log sequence number of last commit complete (Last Committed LSN): the LSN that indicates the largest committed before the current transaction is ready to complete (Commit), i.e., the LSN of the last transaction that the current transaction depends on, can be simply understood as meaning that if a transaction equal to or less than current transaction Last Committed LSN has completed execution, then the current transaction can begin execution, last Committed LSN identical transactions can be grouped together, indicating that no conflicts with each other can be executed or replayed in parallel.

Write set (WriteSet): writeSet is a transaction conflict detection technology with finer granularity, which processes Last Committed LSN of a transaction on the basis of a logic clock, and the basic principle is as follows: a) Transactions without data collision can be written concurrently; b) The existence of an intersection of transactions WriteSet indicates that there is a data conflict; c) WriteSet denotes a set of row key Hash values (RowKey Hash) of transaction changes or dependencies, including in particular the following expressions: if a transaction is T, which includes n rows (n statements), then t= { r ₁,…,r_n }, Where r _i is the ith row of transaction T, priKey represents a primary key, uniqKey represents a unique key, and ForeignKey represents an external key.

D) For ease of calculation, the Hash value may be calculated by exporting each RowKey into a set format, for example, the following set format may be used:

Table 1.

Main key (PRIMARY KEY): if a column uniquely identifies each row in a table, that column may be referred to as a primary key of the table, only one primary key is allowed to be used in a table, i.e., only one column can be designated as a primary key, and the primary key does not accept duplicate values and NULL values, i.e., the primary keys of each row are not identical in key value and are not NULL values.

Unique Key (Unique Key): the unique key may also uniquely identify each row in the table, unlike the primary key, which allows for NULL values, and there may be multiple unique keys in a table, i.e., multiple columns may be used as unique keys.

Foreign Key (Foreign Key): foreign keys are used to associate with other tables.

The research shows that the core problem affecting the synchronous performance of the data replication among instances such as a database MySQL, a data transmission service (Data Transmission Service, DTS) and the like is concurrency analysis of the transactions, so that the conflict among the transactions needs to be coordinated, and the concurrency of the playback transactions needs to be improved.

In the related art, a logic clock and WriteSet can be introduced on the basis of Binlog to process multi-transaction concurrency, for example, referring to fig. 1, which is a schematic diagram of MySQL master-slave replication principle in the related art, a master node (master) records transaction update and transaction concurrency related information in Binlog, and there may be multiple binlogs, when a slave node (slave) is normally connected with the master node, the master node may synchronize the updated Binlog to the slave node, an Input/Output (I/O) thread of the slave node may read the Binlog of the master node and copy the same into a local relay log (RelayLog), an SQL thread of the slave node reads RelayLog locally, and a transaction submitted on the master node may be played back locally, so as to implement the transaction through multiple working (Worker) threads, thereby achieving the purpose of keeping consistent with data of a master database of the master node.

To improve transaction concurrency performance, mysql5.7 introduced a Logic Clock (LC) mechanism, the principle of which was to add two hint (hit) fields, LSN and Last CommittedLSN, in Binlog for providing data concurrency information to slave nodes. After MySQL5.7.21, a WriteSet mechanism is newly added on the basis of LC, and WriteSet mechanism is a switchable function, if it is turned on, the master node can perform WriteSet analysis on the transaction to promote the concurrency effect of the logic clock, the basic principle is that by calculating PRIMARY KEY, UNIQUEKEY and ForeignKey of the transaction, writeSet of the transaction is obtained, then by WriteSet comparison to judge the conflict dependency between the transactions, if WriteSet of the two transactions have intersection, the two transactions have conflict, but because some legacy characteristics of MySQL are only based on WriteSet analysis, such as operations of Cascade, UPDATE (UPDATE), DELETE (DELETE) and the like of the external key, so MYSQL WRITESET in the related art is only an optimization method used on the master node, mySQL optimizes Last Committed LSN information in Binlog through WriteSet, slave nodes cannot perceive the existence of WriteSet, and the effect is actually small Last Committed LSN, so that the non-conflicting transactions can have the same promotion Last Committed LSN and can be executed.

However, in the related art method, the slave node only depends on the Binlog logic clock to determine the execution sequence of each transaction, for the transaction with conflict, the transaction needs to be executed after the execution of the transaction with conflict is completed, but one transaction may involve operations such as reading data, checking and updating data, etc., and the whole execution process may take a long time, so that the service processing throughput is limited, the concurrency of the transaction playback is limited, and the processing efficiency is reduced.

Based on the above study, the disclosure provides a transaction processing method, according to the transaction record log information of the main database, a first write set of a currently executed transaction is determined, and a transaction to be executed in an executable state is determined, and according to the transaction record log information of the main database, a second write set of the transaction to be executed is determined, further according to the key value of each type key in the first write set and the key value of each type key in the second write set, whether each statement included in the transaction to be executed conflicts with the currently executed transaction or not is judged, and when any statement is determined to not conflict with the currently executed transaction, the statement is executed in advance, so that conflict analysis of the statement of the transaction to be executed and the write set of the transaction to be executed in the executable state can be realized, the statement can be executed concurrently without waiting for all the conflicting transactions to be executed completely, the conflict in the transaction to be executed can be executed in advance, the conflict playback time is greatly reduced, and the concurrency of the transaction is promoted.

The present invention is directed to a method for manufacturing a semiconductor device, and a semiconductor device manufactured by the method.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.

For the sake of understanding the present embodiment, first, a detailed description will be given of a transaction processing method disclosed in an embodiment of the present disclosure, where an execution body of the transaction processing method provided in the embodiment of the present disclosure is generally an electronic device with a certain computing capability, and the electronic device includes, for example: the terminal device or server or other processing device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc., where the Personal digital assistant is a handheld electronic device, has some functions of an electronic computer, may be used to manage Personal information, may browse internet, send and receive e-mails, etc., and may be called a palm computer. In some possible implementations, the transaction method may be implemented by way of a processor invoking computer readable instructions stored in a memory.

The transaction processing method provided by the embodiment of the present disclosure is described below by taking an execution body as a server as an example.

Referring to fig. 2, a flowchart of a transaction processing method according to an embodiment of the disclosure is shown, where the method includes:

S201: and determining a first write set of the currently executed transaction according to the transaction record log information of the main database, wherein the first write set comprises key values of various types of keys and is used for uniquely representing the currently executed transaction and each statement in the currently executed transaction.

In the related art, only the conflict analysis at the transaction level can be realized in the process of playing back the transaction, but in the embodiment of the disclosure, in order to improve the concurrency to the maximum extent, writeSet of the transaction is directly reconstructed to realize the conflict judgment at the statement level.

In particular, when this step S201 is performed, one possible way is provided in the embodiments of the present disclosure: and acquiring transaction record log information of the main database, and determining a first write set of the currently executed transaction according to the transaction record log information of the main database.

The transaction log information is, for example, binlog, in which all data changes in the master database are recorded, and therefore WriteSet may be calculated based on Binlog in the embodiments of the present disclosure.

Of course, in the embodiment of the present disclosure, if there are multiple current execution transactions, the first write set of each current transaction needs to be calculated separately, and then when conflict analysis is performed, the to-be-executed transaction and the multiple current execution transactions may be determined to be in conflict, for example, it is determined that multiple transactions may be executed concurrently, there will be multiple current execution transactions, and generally, the number of transactions in the transaction queue is limited, and at most, there are multiple transactions satisfying concurrency, so the calculation amount of WriteSet is also controllable.

S202: and determining a to-be-executed transaction in an executable state, and determining a second write set of the to-be-executed transaction according to transaction record log information of a main database, wherein the second write set comprises key values of various types of keys and is used for uniquely representing the to-be-executed transaction and each statement in the to-be-executed transaction.

When this step S202 is executed, it includes:

s1, determining a transaction to be executed in an executable state.

In particular the present disclosure provides one possible implementation: and determining the transaction in the next execution sequence corresponding to the currently executed transaction as the transaction to be executed in the executable state according to the logic clock of the transaction record log information of the main database.

For example, the log information of the transaction record is Binlog, the LSNs and Last Committed LSN of each transaction are recorded in the Binlog, the order of each transaction can be determined according to the value of the LSNs, the transaction with Last Committed LSN identical indicates that no conflict can be executed concurrently, that is, the transaction with Last Committed LSN identical is satisfied, so that the transaction to be executed in the executable state, that is, the transaction to be executed in the executable state can be determined according to the LSNs of the currently executed transaction and the LSNs and Last Committed LSN of all the non-executed transactions, which can be understood as the transaction with the currently executed transaction not satisfying the logical clock concurrency condition, that is, the transaction with Last Committed LSN identical to the currently executed transaction and the determined next execution order corresponding to the maximum LSN of the currently executed transaction, that is, the transaction marked as the executable state.

In addition, the transaction log information may include a plurality of transaction log files, each transaction log file has a file identifier, and when the LSN of the transaction in each transaction log file can restart counting, the execution sequence of each transaction is determined, not only based on the values of LSN and Last Committed LSN, but also according to the transaction log file in which the transaction is located.

S2, determining a second writing set of the transaction to be executed.

Specifically, one possible implementation manner is included: and determining a second write set of the transaction to be executed according to the transaction record log information of the main database.

S203: judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and when any statement is determined to not conflict with the current execution transaction, executing any statement in advance.

In the embodiment of the disclosure, the first write set of the transaction to be executed and the second write set of the transaction to be executed can uniquely characterize the transaction and the sentences included in the transaction, if the intersections are the same key values, the conflicts are indicated, if the intersections are not the same, the conflicts are indicated, if the conflicts are not the intersections are not the same, the conflict judgment can be carried out on each sentence in the transaction to be executed and the transaction to be executed instead of the conflict judgment of the transaction to be executed and the transaction to be executed, so that the transaction can be started first, and the sentence to be executed can be executed in advance as long as the sentence to be executed does not conflict with the transaction to be executed.

That is, in the embodiment of the present disclosure, any statement in a transaction to be executed can be executed requiring the following condition to be satisfied: 1) The transaction to which any statement belongs is in an executable state, for example, whether the transaction is in the executable state can be determined based on a logic clock; 2) The either statement has no conflict with all transactions that are currently executing and have LSNs less than Last Committed LSN of the belonging transactions.

Further, in an embodiment of the present disclosure, a trigger condition for executing the method is further provided, that is, before executing the step S201 to determine, according to the transaction log information of the master database, the first write set of the currently executed transaction, the method further includes: and after the execution of any one of the transactions is completed, determining the currently executed transaction according to the logic clock of the transaction record log information of the main database, wherein the currently executed transaction at least comprises the next execution sequence of the transaction corresponding to the completed transaction.

That is, in the embodiment of the present disclosure, each time there is a transaction execution completion, the scheduler is triggered to recalculate the first write set of the currently executed transaction and notify the transaction to be executed in the executable state, which is also because there is a new transaction or new statement to be executed when there is a transaction execution completion, and the currently executed transaction and the transaction to be executed in the executable state are changed, so that the calculation needs to be recalculated to perform conflict analysis to determine the statement in the concurrently executable transaction to be executed, so that the non-conflicting statement can be pre-executed.

In the embodiment of the disclosure, a first write set of a currently executed transaction is determined according to transaction record log information of a main database, a to-be-executed transaction in an executable state is determined, and a second write set of the to-be-executed transaction is determined according to the transaction record log information of the main database, so that each transaction is not required to be restarted according to key values of each type of key in the first write set and key values of each type of key in the second write set, whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction or not is judged, and when any statement is determined to not conflict with the currently executed transaction, the any statement is executed in advance, so that by constructing the write set, a transaction concurrency analysis method of statement granularity is realized, each statement in the to-be-executed transaction can be compared with the currently executed transaction in an executable state, each statement does not conflict with the currently executed transaction until the associated transaction is not started, the transaction can be started, the statement in the to be executed with the currently executed transaction is not in advance, the statement in the conflict with the currently executed transaction is avoided, the waiting time is greatly reduced, and under the condition that no conflict exists between the statement and the concurrency is improved to the maximum degree is ensured.

Based on the foregoing embodiments, in the embodiment of the present disclosure, in a data replication scenario, when a replication node or a slave node plays back a transaction, a first write set of a currently executed transaction and a second write set of a transaction to be executed in an executable state may be determined, so that a statement granularity conflict comparison may be performed, so as to maximize concurrency, and for how to determine the write set and how to perform conflict judgment based on the write set, the present disclosure further provides a possible embodiment, which is specifically described below respectively.

To facilitate understanding, the basic principle of how to determine the write set in the embodiments of the present disclosure is described first, and the basic principle of the method of MYSQL WRITESET in the related art is that the master node may optimize LSN and Last Committed LSN information about transaction concurrency in Binlog based on WriteSet, and then the copy node or slave node may play back the transaction concurrently through these information in Binlog, which in the embodiments of the present disclosure consider that, with reference to the idea of MYSQL WRITESET in the related art, writeSet of the transaction may be reconstructed by Binlog on the copy node or slave node for transaction concurrency analysis, but it is found that WriteSet cannot be directly constructed based on Binlog, because some implicit semantics in MySQL are not included in Binlog, and when calculating WriteSet, there may be cases of information insufficiency, for example, several possible cases of information insufficiency in Binlog: 1) Unique function index statements, such as a statement: CREATE TABLE t1 (c 1 int, c2 int, UNIQUE INDEX ((c1+c2))) records only the values of c1 and c2 in Binlog, UNIQUE INDEX (c1+c2) is stored as a hidden generation column in MySQL, but not Binlog, so when the function INDEX is the only INDEX, UNIQUE INDEX (c1+c2) is hard to restore, especially the expression can support embedded MySQL functions. 2) There is a foreign key constraint statement, such as a cascades command, whose delete operation automatically deletes the secondary table when the primary table is deleted, but in Binlog, only the delete record of the primary table is deleted, and there is no delete record of the sub table. 3) With UNIQUE multi-value INDEX statements, such as UNIQUE INDEX idx1 ((CAST 1- > $. Arr' AS UNSIGNED ARRAY)), it is only the data type attribute value that is indexed, and repeated insertion errors occur when values are inserted multiple times.

Therefore, in the embodiment of the disclosure, aiming at the problem occurring when WriteSet is constructed based on Binlog, writeSet in the related technology is improved, the semantics of WriteSet are expanded, and the concepts of global keys and table keys are provided, so that aiming at the scene of incomplete information in Binlog, the semantics of sentences can be properly amplified to ensure that the concurrently executed transactions have no conflict.

Specifically, in the embodiments of the present disclosure, for determining a first write set of a currently executed transaction and determining a second write set of a to-be-executed transaction, a possible specific implementation manner is provided, where, for convenience of unified description, a first write set or a second write set is taken as a target write set, a currently executed transaction or a to-be-executed transaction is taken as a target transaction, and the target write set and the target transaction are described as examples, where the first write set corresponds to the currently executed transaction, and the second write set corresponds to the to-be-executed transaction, and then, specifically, determining, according to transaction record log information of a main database, the target write set of the target transaction may include the following implementation manners:

and determining global keys, table keys and main keys and/or unique keys of each statement, which are included in a target write set of the target transaction, according to the transaction record log information of the main database.

The global key characterizes a global lock of a target transaction, and the table key characterizes a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize the statement lock; and the lock priority is that the global key is larger than the table key, and the table key is larger than the main key and the unique key.

In the embodiment of the disclosure, the target write set includes key values of multiple types of keys, for example, the key values can be global keys, table keys, primary keys and unique keys, and different types of keys can represent locks with different semantic levels.

According to the transaction log information of the main database, determining a global key, a table key and a main key and/or a unique key of each statement included in a target write set of a target transaction, and providing a possible implementation manner in the embodiment of the present disclosure, which can be specifically divided into the following scenarios:

first scenario: if the preset target condition exists, determining a global key or a table key included in the target write set of the target transaction.

The target conditions include: 1) If the target transaction includes a DDL statement or is the first transaction in a transaction log file included in the transaction log information, then it is determined that the target write set of the target transaction includes a global key.

For example, since DDL statements are typically initial jobs for defining or updating structures of tables, data types, connections and constraints between tables, etc., DDL statements typically involve multiple transactions or tables, to avoid concurrent execution of conflicting transactions, global keys may be set for DDL statements that represent conflicts and cannot be executed concurrently, e.g., a global key may be a randomly generated string.

2) If the target transaction does not have the main key table or comprises at least one of the following target type tables, determining that the target write set of the target transaction at least comprises a table key, wherein the target type table is: a table with unique index statements, a parent table of foreign key constraints.

The unique index statement is, for example, a unique function index statement or a unique multi-value index statement.

That is, in the embodiment of the present disclosure, for a table, when information insufficiency may occur in the table, a table lock of the table may be constructed, semantics of a table level lock may be increased, and when the table lock is calculated, a possible implementation manner is provided, 1) a primary key and/or a unique key of a statement that information insufficiency may occur may be used as a table key, for example, if a transaction includes Unique Functional Index rows, the primary key and the unique key of the row may be directly updated to the table key, so in the embodiment of the present disclosure, when a transaction conflict is analyzed, it may be considered that all transactions that update the table may have a conflict. 2) If the table without the main key is a table without the main key, a hidden main key is added by default, and in Binlog, a table key can be generated to identify the whole table so as to avoid data operation conflict in the table.

In the embodiment of the disclosure, for the implicit semantics which are not recorded in Binlog, the semantics of the current record can be properly amplified, and global keys and table keys are added on the basis of the prior WriteSet, so that when sentences cannot be uniquely determined or the situation of incomplete records caused by external keys, the semantics of global locks or table locks of transactions can be expanded, thereby avoiding the concurrent conflicts of the transactions and ensuring the consistency and the integrity of data.

The second scenario: if the target condition does not exist, determining a main key and/or a unique key of each statement included in the target write set of the target transaction.

That is, for a statement with certain semantics, the primary key and/or the unique key of the statement can be directly constructed to implement the lock on the statement.

Based on the above embodiments, a lock priority definition is further provided in the embodiments of the present disclosure, specifically referring to fig. 3, which is a schematic lock priority diagram in the embodiments of the present disclosure, as can be seen from fig. 3, mutual exclusion relationships between different types of keys may be defined by a tree-shaped lock structure, where a lock priority is that a global key is larger than a table key, a table key is larger than a primary key and a unique key, there is a conflict between transactions of the same global key, there is a conflict between tables of the same table key, there is a conflict between sentences of the same unique key or primary key, and when concurrent conflict analysis is performed, the lock priority may be sequentially compared according to the lock priority, for example, for a lock priority including only a global key (RootKey, RK) may be defined as a write lock: WLock (RK), the lock priority for a key comprising a Table Key (TK) may be defined as: read lock RLock (RK) - > write lock WLock (TK), lock priorities for a Key comprising a primary Key (PRIMARY KEY, PK) and/or a Unique Key (UK) may be defined as: read Lock RLock (RK) - > read Lock RLock (TK) - > Lock (PK/UK).

In this way, in the embodiment of the disclosure, the reconstruction of the write set during the transaction playback is realized, the write set in the related technology is improved, and for the case of incomplete information in Binlog, the lock semantics can be enlarged, so that the concurrent playback transaction is ensured not to conflict, the accuracy of the transaction conflict analysis is improved, and the consistency and the integrity of the data in the main database are ensured.

Further, in the embodiment of the present disclosure, conflict analysis may be performed based on the determined write set and each type key in the write set, specifically, in step S203, according to the key value of each type key in the first write set and the key value of each type key in the second write set, whether each statement included in the transaction to be executed conflicts with the currently executed transaction is determined, and the present disclosure provides a possible implementation manner:

s1, respectively comparing a second write set corresponding to each statement included in the transaction to be executed with a first write set of the transaction currently executed.

In the embodiment of the disclosure, when the transaction conflict analysis is performed, the statement of the transaction to be executed and the current execution transaction can be compared and analyzed, so that whether the statement of the transaction to be executed and the current execution transaction can be executed concurrently or not is judged.

S2, judging whether the global key, the table key, the main key or the unique key of the transaction to be executed and the first write set have intersections or not in sequence according to the lock priority, determining that corresponding sentences collide with the current transaction under the condition that the intersections are determined, and determining that the corresponding sentences do not collide with the current transaction under the condition that the intersections are not determined.

The method specifically comprises the following cases:

1) And according to the lock priority, if the global key of the transaction to be executed has an intersection with the first write set, determining that all sentences included in the transaction to be executed conflict with the current transaction to be executed.

That is, in the embodiment of the present disclosure, when performing transaction conflict analysis, according to the lock priority, it is first determined whether the global key of the transaction to be executed conflicts with the first write set, if the global key conflicts, it is no longer necessary to determine the table key, the primary key and the unique key, and it may be considered that all statements under the global key transaction conflict with the currently executed transaction.

2) If the global key of the transaction to be executed does not have an intersection with the first write set, and the table key of the transaction to be executed has an intersection with the first write set, all sentences under the table key with the intersection of the transaction to be executed are in conflict with the currently executed transaction.

After the global keys are different, judging whether the table keys of the transaction to be executed have conflicts, if the table keys are the same, considering that the transaction which only updates the table has conflicts, and considering that the statement related to the table has conflicts and cannot be executed concurrently with the currently executed transaction.

3) If the global key, the table key and the first write set of the transaction to be executed do not have intersections, and the unique key or the main key of the transaction to be executed and the first write set have intersections, determining that the sentence corresponding to the unique key or the main key with the intersections conflicts with the currently executed transaction.

In this way, in the embodiment of the disclosure, according to the transaction record log information of the main database, a write set of the transaction can be constructed, and when the write set is constructed, the problem of incomplete information in the transaction record log information is solved by expanding semantics of the write set, and further, transaction concurrency analysis between sentences of the transaction to be executed and the transaction currently executed can be performed based on the constructed write set, so that the non-conflicting sentences can be executed in advance, the transaction can be started first without waiting until the transaction currently executed does not conflict, the transaction can be executed first as long as the sentences in the transaction to be executed do not conflict with the transaction currently executed, the conflict waiting time is greatly reduced, and compared with the transaction concurrency analysis which is realized by only depending on a logic clock in the related technology, the performance is improved.

The following describes the above embodiments with reference to a specific application scenario, and fig. 4 is a schematic diagram of an application scenario of a transaction processing in an embodiment of the disclosure, as shown in fig. 4, for example, there are nine transactions, which are Trx1, trx2, …, trx9, respectively, and the following describes a transaction processing method in an embodiment of the disclosure with reference to the scenario.

1) At time t1, trx2, trx3 are concurrently executed, and the currently executed transaction includes Trx1, trx2, trx3.

2) If Trx1 and Trx3 are executed at time t2, trx2 is still executing, it is determined that Trx4 can continue to execute according to the logic clock of the transaction log information of the master database, trx5 does not satisfy the logic clock concurrency condition because of the logic clock dependency, and Trx5 is marked as executable but cannot execute concurrently, at this time, the currently executed transaction is updated to Trx2 and Trx4, and the transaction to be executed in the executable state is updated to Trx5.

For example, as in fig. 4, trx5 contains two statements: insert r3 operation I (r 3) and update r1 operation U (r 1), trx2 includes a statement: insert r1 operation I (r 1), trx4 includes a statement: inserting r2 operation I (r 2), based on the transaction processing method in the embodiment of the present disclosure, by comparing the statement included in Trx5 with the current execution transactions Trx2 and Trx4, it is determined that the I (r 3) statement in Trx5 has no conflict with both Trx2 and Trx4, and it is determined that the U (r 1) statement in Trx5 has a conflict with Trx2, then the I (r 3) statement in Trx5 may be executed concurrently, and the U (r 1) statement in Trx5 is not executed.

In this way, in the embodiment of the disclosure, based on the transaction concurrency information in Binlog, for the transactions meeting the logic clock concurrency condition, each transaction meeting the logic clock concurrency condition is directly and concurrently executed, for the transactions not meeting the logic clock concurrency condition and marked as an executable state, statement conflict analysis is performed, and the statement without conflict can be executed first, that is, the first statement of Trx5 in the executable state, without waiting for the execution of the currently executed transaction to be completed, and then executed concurrently with the currently executed transaction, and the conflict analysis is performed with the statement granularity, thereby improving the concurrency degree and also improving the processing efficiency and performance.

It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.

Based on the same inventive concept, the embodiments of the present disclosure further provide a transaction processing device corresponding to the transaction processing method, and since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to that of the transaction processing method in the embodiments of the present disclosure, the implementation of the device may refer to the implementation of the method, and the repetition is omitted.

Referring to fig. 5, a schematic diagram of a transaction provided in an embodiment of the disclosure, the apparatus includes:

a first determining module 51, configured to determine a first write set of a currently executed transaction according to transaction record log information of a main database, where the first write set includes key values of multiple types of keys, and is configured to uniquely characterize the currently executed transaction and each statement in the currently executed transaction;

A second determining module 52, configured to determine a transaction to be executed in an executable state, and determine a second write set of the transaction to be executed according to transaction record log information of the master database, where the second write set includes key values of multiple types of keys, and is configured to uniquely characterize the transaction to be executed and each statement in the transaction to be executed;

And an execution module 53, configured to determine whether each statement included in the transaction to be executed conflicts with the current execution transaction according to the key value of each type key in the first write set and the key value of each type key in the second write set, and when it is determined that any statement does not conflict with the current execution transaction, pre-execute the any statement.

In an alternative embodiment, when the first write set or the second write set is taken as a target write set and the currently executed transaction or the to-be-executed transaction is taken as a target transaction, and the target write set of the target transaction is determined according to the transaction record log information of the main database, the first determining module 51 or the second determining module 52 is configured to:

In an alternative embodiment, when determining, according to the transaction log information of the master database, a global key, a table key, and a master key and/or a unique key of each statement included in the target write set of the target transaction, the first determining module 51 or the second determining module 52 are configured to:

In an alternative embodiment, according to the key value of each type of key in the first write set and the key value of each type of key in the second write set, the executing module 53 is configured to, when determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction, determine that:

In an optional implementation manner, according to the lock priority, the global key, the table key, and whether the primary key or the unique key of the transaction to be executed and the first write set have intersections are sequentially determined, where in a case where an intersection is determined, a conflict exists between a corresponding statement and the currently executed transaction, and in a case where no intersection is determined, where in a case where it is determined that the corresponding statement does not conflict with the currently executed transaction, the execution module 53 is configured to:

In an alternative embodiment, the first determining module 51 is further configured to: before determining a first write set of a currently executing transaction from transaction log information of a master database, performing the following operations:

In an alternative embodiment, when determining a transaction to be executed in an executable state, the second determining module 52 is configured to: and determining the transaction in the next execution sequence corresponding to the current execution transaction as the transaction to be executed in an executable state according to the logic clock of the transaction record log information of the main database.

The process flow of each module in the apparatus and the interaction flow between the modules may be described with reference to the related descriptions in the above method embodiments, which are not described in detail herein.

The embodiment of the disclosure further provides an electronic device, as shown in fig. 6, which is a schematic structural diagram of the electronic device provided by the embodiment of the disclosure, including:

A processor 61 and a memory 62; the memory 62 stores machine readable instructions executable by the processor 61, the processor 61 being configured to execute the machine readable instructions stored in the memory 62, the machine readable instructions when executed by the processor 61, the processor 61 performing the steps of:

In an alternative embodiment, when the first write set or the second write set is taken as a target write set and the currently executed transaction or the to-be-executed transaction is taken as a target transaction, and the target write set of the target transaction is determined according to the transaction record log information of the main database, the processor 61 is configured to:

In an alternative embodiment, when determining, according to the transaction log information of the master database, a global key, a table key, and a master key and/or a unique key of each statement included in the target write set of the target transaction, the processor 61 is configured to:

In an alternative embodiment, according to the key value of each type of key in the first write set and the key value of each type of key in the second write set, when determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction, the processor 61 is configured to:

In an optional implementation manner, according to the lock priority, the global key, the table key, and whether the primary key or the unique key of the transaction to be executed and the first write set have intersections are sequentially determined, where in the case of determining that there is an intersection, a conflict exists between a corresponding statement and the currently executed transaction, and in the case where in any case it is determined that there is no intersection, where in the case where in any case it is determined that there is no conflict between a corresponding statement and the currently executed transaction, the processor 61 is configured to:

In an alternative embodiment, processor 61 is further configured to: before determining a first write set of a currently executing transaction from transaction log information of a master database, performing the following operations:

In an alternative embodiment, when determining a transaction to be executed in an executable state, the processor 61 is configured to: and determining the transaction in the next execution sequence corresponding to the current execution transaction as the transaction to be executed in an executable state according to the logic clock of the transaction record log information of the main database.

The memory 62 includes a memory 621 and an external memory 622; the memory 621 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 61 and data exchanged with the external memory 622 such as a hard disk, and the processor 61 exchanges data with the external memory 622 via the memory 621.

The specific execution process of the above instruction may refer to the steps of the transaction processing method described in the embodiments of the present disclosure, which is not described herein.

The disclosed embodiments also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the transaction method described in the method embodiments above. Wherein the storage medium may be a volatile or nonvolatile computer readable storage medium.

Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program code, where instructions included in the program code may be used to perform steps of a transaction processing method described in the foregoing method embodiments, and specific reference may be made to the foregoing method embodiments, which are not described herein.

Wherein the above-mentioned computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present disclosure. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that: the foregoing examples are merely specific embodiments of the present disclosure, and are not intended to limit the scope of the disclosure, but the present disclosure is not limited thereto, and those skilled in the art will appreciate that while the foregoing examples are described in detail, it is not limited to the disclosure: any person skilled in the art, within the technical scope of the disclosure of the present disclosure, may modify or easily conceive changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features thereof; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the disclosure, and are intended to be included within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method of transaction processing, comprising:

Judging whether each statement included in the transaction to be executed conflicts with the current execution transaction according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction;

The determining, by taking the first write set or the second write set as a target write set and the current executing transaction or the transaction to be executed as a target transaction, the target write set of the target transaction according to transaction record log information of a main database includes:

determining global keys, table keys and main keys and/or unique keys of all sentences included in a target write set of the target transaction according to transaction record log information of the main database;

2. The method according to claim 1, wherein determining global keys, table keys, and main and/or unique keys of each statement included in a target write set of the target transaction according to transaction record log information of the main database comprises:

3. The method of claim 1, wherein determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction based on the key value of each type key in the first write set and the key value of each type key in the second write set comprises:

4. A method according to claim 3, wherein said sequentially determining, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed have intersections with the first write set, determining that the corresponding sentence has a conflict with the currently executed transaction if it is determined that there is an intersection, and determining that the corresponding sentence does not have a conflict with the currently executed transaction if it is determined that there is no intersection, comprises:

5. The method of claim 1, wherein prior to determining the first write set for the currently executing transaction based on the transaction log information of the master database, further comprising:

6. The method of any of claims 1-5, wherein determining a transaction to be performed in an executable state comprises:

7. A transaction processing device, comprising:

the execution module is used for judging whether each statement included in the transaction to be executed conflicts with the current execution transaction according to the key value of each type key in the first writing set and the key value of each type key in the second writing set, and executing any statement in advance when determining that any statement does not conflict with the current execution transaction;

8. An electronic device, comprising: a processor, a memory storing machine readable instructions executable by the processor for executing machine readable instructions stored in the memory, which when executed by the processor, perform the steps of the transaction method of any of claims 1 to 6.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the transaction method according to any of claims 1 to 6.