CN117076211A - Data guarantee method, device, equipment and storage medium - Google Patents
Data guarantee method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117076211A CN117076211A CN202311100861.2A CN202311100861A CN117076211A CN 117076211 A CN117076211 A CN 117076211A CN 202311100861 A CN202311100861 A CN 202311100861A CN 117076211 A CN117076211 A CN 117076211A
- Authority
- CN
- China
- Prior art keywords
- library
- slave
- data
- master
- slave library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2056—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
- G06F11/2082—Data synchronisation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2315—Optimistic concurrency control
- G06F16/2329—Optimistic concurrency control using versioning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data guaranteeing method, a data guaranteeing device, data guaranteeing equipment and a storage medium. The method comprises the following steps: acquiring a main library version number of business data corresponding to a transaction request in a main library; the method comprises the steps that a master library version number is sent to a slave library, when the slave library detects that the master library version number is smaller than a slave library version number stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that data loss occurs when the master library is switched with a standby library; and when version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from the service data which is pre-backed up in the slave library. And when the transaction request is acquired, the version number of the master library corresponding to the transaction request is sent to the slave library. And comparing the version numbers of the master library and the slave library, and judging whether the data loss occurs in the master library. And carrying out data restoration on the master library by utilizing the data of the slave library. The method and the device solve the technical problem that lost data can be retrieved through reliable backup after the database primary and backup are switched.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a data security method, apparatus, device, and storage medium.
Background
The database master-slave technology is a common high-guarantee technology means, and when the master database fails or needs maintenance, the backup database can take over the work of the master database, so that the continuity of the service is ensured. In order to copy data from a primary database to a backup database, the current mainstream data replication technology mainly has two modes of strong synchronization and asynchronization.
The realization of the technologies mainly depends on the functions and characteristics of a database management system, and the prior art uses an asynchronous mode to maintain data and cannot effectively avoid data loss; the performance loss of the strong synchronous mode is larger, and the optimal balance between the performance and the reliability cannot be achieved; the data maintenance is only processed in the database, and the business layer cannot effectively identify the problem data and intercept the transaction. After the database is switched between the primary and the backup, the lost data can be retrieved through reliable backup.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a data guaranteeing method, a device, equipment and a storage medium, which aim to solve the technical problem of how to retrieve lost data through reliable backup after the primary and backup of a database are switched.
In order to achieve the above object, the present invention provides a data security method, which includes the steps of:
acquiring a main library version number of business data corresponding to a transaction request in the main library;
the version number of the master library is sent to the slave library, when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is subjected to data loss when the master library and the slave library are subjected to master-slave switching;
and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library.
Optionally, before the step of obtaining the master library version number of the master library of the service data corresponding to the transaction request, the method further includes:
when a transaction request is received, acquiring a synchronous switch value;
when the synchronous switch value is a first slave library state value or a second slave library state value, inserting synchronous control records with a first notification mark in states of the master library and the slave library;
when the synchronous switch value is a third slave library state value, a synchronous control record with a second notification mark is inserted into the master library;
The synchronization control record is used for recording the synchronization state of the transaction request, and the synchronization state is used for indicating the master library and the slave library to perform data synchronization;
the first slave library state value represents that the slave library is normally synchronizable, the second slave library state value represents that the slave library is abnormally unsynchronizable in the data recovery process, the first notification mark represents that the slave library is notified to be unsynchronized, and the second notification mark represents that the slave library is not notified to be unsynchronized.
Optionally, after the step of inserting the synchronization control record with the state of the master library being the second notification identifier when the synchronization switch value is the third slave library state value, the method further includes:
obtaining a transaction result corresponding to the transaction request;
when the transaction result is that the transaction is successful, the synchronous control records of the master library and the slave library are not processed;
and when the transaction result is that the transaction fails, clearing the transaction data of the master library and the synchronous control record, and not processing the data of the slave library.
Optionally, the step of acquiring the synchronous switch value when the transaction request is received includes:
detecting availability of the slave library upon receipt of a transaction request;
Setting the synchronization switch value to the third slave library state value upon detecting that the slave library is not available;
setting the synchronization switch value to the second slave library state value upon detecting that the slave library is restored to available from unavailable;
and setting the synchronous switch value of the slave library to the first slave library state value when the synchronous switch value of the slave library is the second slave library state value and the synchronous switch value of the slave library is synchronous with the master library.
Optionally, the sending the master library version number to the slave library, where the slave library, when detecting that the master library version number is smaller than a slave library version number stored in the slave library, feeds back version exception information, where the version exception information is used to characterize that data loss occurs in the master library when the master library and the slave library are switched, and the method further includes:
when the slave library is not available, marking the business data corresponding to the transaction request as a non-notification asynchronous state;
and when the slave library is in the data recovery process, sending the service data marked as the un-notified asynchronous state to the slave library.
Optionally, the step of sending the service data marked as the undelayed state to the slave library when the slave library is in the data recovery process includes:
When the slave library is in the data recovery process, comparing the current master library version number of the service data in the master library with the current slave library version number of the service data in the slave library;
and sending the business data in the master library with the version number of the master library larger than that of the slave library to the slave library.
Optionally, after the step of rejecting the response to the transaction request by the master library and repairing the data of the master library from the service data backed up in advance in the slave library when the version exception information is received, the method further includes:
receiving the query request sent from the library, and searching service data corresponding to the query request;
and sending business data corresponding to the query request to the slave library so that the slave library is updated based on the business data.
In addition, to achieve the above object, the present invention also proposes a data security device including:
the data acquisition module is used for acquiring a main library version number of business data corresponding to a transaction request in the main library when the transaction request is received;
the data sending module is used for sending the version number of the master library to the slave library, and when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is lost when the master library and the slave library are switched;
And the exception repairing module is used for refusing to respond to the transaction request through the master library when receiving the version exception information, and repairing the data of the master library from the service data which is backed up in advance in the slave library.
In addition, in order to achieve the above object, the present invention also proposes a data security device, the device comprising: a memory, a processor and a data assurance program stored on the memory and executable on the processor, the data assurance program configured to implement the steps of the data assurance method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a data assurance program that, when executed by a processor, implements the steps of the data assurance method as described above.
The method comprises the steps of obtaining a main library version number of business data corresponding to a transaction request in the main library; the version number of the master library is sent to the slave library, when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is subjected to data loss when the master library and the slave library are subjected to master-slave switching; and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library. And adding a version number in service data corresponding to the transaction request, and sending the version number of the master library corresponding to the transaction request to the slave library when the transaction request is acquired. And judging whether the data loss occurs in the master library or not by comparing the master library version number with the slave library version number. And further, data restoration is carried out on the master library by utilizing the data of the slave library. The method and the device solve the technical problem that lost data can be retrieved through reliable backup after the database primary and backup are switched.
Drawings
FIG. 1 is a schematic diagram of a data security device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of the data security method of the present invention;
FIG. 3 is a flow chart of a second embodiment of the data security method of the present invention;
FIG. 4 is a schematic diagram of the overall architecture of a system to which the data assurance method of the present invention is applied;
FIG. 5 is a timing diagram of a transaction flow for a data assurance method of the present invention;
FIG. 6 is a timing diagram of a forward transaction flow for the data assurance method of the present invention;
FIG. 7 is a timing diagram of a process flow of notifying an unsynchronized record in the data security method of the present invention;
FIG. 8 is a timing diagram of an online synchronous exception handling process for the data security method of the present invention;
fig. 9 is a block diagram of the data securing apparatus of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic diagram of a data protection device structure of a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the data securing apparatus may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a display (Di splay), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) Memory or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 does not constitute a limitation on the data security device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a data security program may be included in the memory 1005 as one type of storage medium.
In the data security device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the data security device of the present invention may be disposed in the data security device, and the data security device calls the data security program stored in the memory 1005 through the processor 1001 and executes the data security method provided by the embodiment of the present invention.
An embodiment of the present invention provides a data security method, referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data security method of the present invention.
In this embodiment, the data security method is applied to a data security architecture including a master library, a standby library and a slave library, and the data security method includes the following steps:
Step S10: and acquiring a master library version number of the business data corresponding to the transaction request in the master library.
It should be noted that, the execution body of the method of the embodiment may be a computing service device with database query and query functions, such as a server, a tablet computer, a personal computer, etc.; the above-described data securing apparatus having the same or similar functions may also be used. The present embodiment and the following embodiments will be described taking a data security device (hereinafter referred to as a "security device") as an example.
It will be appreciated that the transaction request may be a transaction operation request transmitted over a computer system or network. Such as online shopping, requests for online banking transfers, etc.
The service data may be information and data related to a transaction request. These data are used to record, process and track various aspects of the transaction request. For example, it may include transaction principals, transaction details, transaction status, transaction logs, and the like.
It can be understood that the main database can be a main database for transaction request processing, has higher read-write performance and uses a high-performance and high-availability architecture.
It should be noted that the master library version number may be a number or a character string indicating a version of the service data in the master library. It is typically composed of numbers, letters or symbols for distinguishing between different versions of the business data.
In a specific implementation, it will be appreciated that since a switch between the primary and secondary libraries may occur, it is first necessary to determine the primary database that stores the transaction request corresponding to the business data. The master database is typically the core system responsible for storing and managing the transaction data. And obtaining a master library version number corresponding to the business data corresponding to the transaction request. The version number may be stored in the data as a field or attribute, and version number information may be obtained by reading the value of the field or attribute.
Step S20: and sending the version number of the master library to the slave library, wherein when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is subjected to data loss when the master library and the slave library are subjected to master-slave switching.
It will be appreciated that the secondary repository may be the primary database for transaction request backups, with higher read data performance, and cost effective devices may be used.
It should be noted that the slave library version number may be a number or a character string indicating a version of the service data in the slave library. It is typically composed of numbers, letters or symbols for distinguishing between different versions of the business data.
It is understood that the version exception information may be exception or error information related to the version of the data. It generally indicates that the traffic data in the master library is inconsistent with the version of the traffic data in the slave library.
In a specific implementation, in the master library, the master library version number is sent to the slave library periodically. The version number may be communicated to the slave library via network communication or otherwise. After the slave library receives the version number sent by the master library, the version number is compared with the slave library version number stored by the slave library. If the slave library detects that the master library version number is less than the slave library version number stored by the slave library, the master library is likely to lose data. At this point, a version of the anomaly information may be generated from the library for feedback to the associated monitoring system or operator.
Step S30: and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library.
It can be understood that when the version exception information is received, it is indicated that the data stored in the main library is wrong, so that the transaction request should not be responded, the data in the main library is repaired first, and the consistency of the data is ensured and then the data is processed.
In a specific implementation, a monitor program of the slave library may be configured to periodically receive the version number sent by the master library and compare the version number with the version number stored by the slave library. If the slave library monitoring program detects that the version numbers are inconsistent, version exception information is generated, and then the response of the master library to the transaction request is suspended. And simultaneously, acquiring the pre-backed-up service data from the backup database. The pre-backed up business data is applied to the main library to carry out data restoration. The specific data repair process may depend on the support of the database system and the backup restoration tool. Backup data may be applied to the primary repository using a backup restoration tool, database replication, or other corresponding means.
The embodiment obtains the version number of the main library in the main library of the business data corresponding to the transaction request; the version number of the master library is sent to the slave library, when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is subjected to data loss when the master library and the slave library are subjected to master-slave switching; and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library. And adding a version number in service data corresponding to the transaction request, and sending the version number of the master library corresponding to the transaction request to the slave library when the transaction request is acquired. And judging whether the data loss occurs in the master library or not by comparing the master library version number with the slave library version number. And further, data restoration is carried out on the master library by utilizing the data of the slave library. The method and the device solve the technical problem that lost data can be retrieved through reliable backup after the database primary and backup are switched.
Further, in order to more precisely control the master library to perform data synchronization with the slave library, before step S10, the method further includes:
step S01: upon receiving a transaction request, a synchronous switch value is obtained.
The synchronous switch value may be an identifier indicating an operation state of the slave library.
In a specific implementation, a synchronous switching variable is defined in the system to indicate whether synchronous operation is currently required. When a transaction request is received, the value of the synchronous switch is acquired before the request is processed.
Step S02: and inserting a synchronous control record identified for a first notification in the master library and the slave library when the synchronous switch value is a first slave library state value or a second slave library state value.
It should be noted that the synchronization control record may be a record for recording a synchronization status of the transaction request, where the synchronization status is used to indicate that the master library performs data synchronization with the slave library. The synchronization status may be a data identification representing a synchronization of data in the database.
In a specific implementation, a common synchronous switch identifier is arranged in a master library and a slave library database. When the value of the synchronization switch is either the first slave bank state value or the second slave bank state value, it is indicated that the slave banks can be synchronized. An insertion status identification field is introduced into the synchronization control record and set to a first notification identification value.
Step S03: and when the synchronous switch value is a third slave library state value, inserting a synchronous control record with a second notification identifier in the master library.
In a specific implementation, when the value of the synchronous switch is the third slave state value, two insertion state identification fields are introduced into the synchronous control record and set as the first notification identification value.
The first slave library state value represents that the slave library is normally synchronizable, the second slave library state value represents that the slave library is abnormally unsynchronizable in the data recovery process, the first notification mark represents that the slave library is notified to be unsynchronized, and the second notification mark represents that the slave library is not notified to be unsynchronized.
The above manner of the present embodiment obtains the synchronous switching value when a transaction request is received; when the synchronous switch value is a first slave library state value or a second slave library state value, inserting synchronous control records with a first notification mark in states of the master library and the slave library; when the synchronous switch value is a third slave library state value, a synchronous control record with a second notification mark is inserted into the master library; the synchronization control record is used for recording the synchronization state of the transaction request, and the synchronization state is used for indicating the master library and the slave library to perform data synchronization; the first slave library state value represents that the slave library is normally synchronizable, the second slave library state value represents that the slave library is abnormally unsynchronizable in the data recovery process, the first notification mark represents that the slave library is notified to be unsynchronized, and the second notification mark represents that the slave library is not notified to be unsynchronized. Thereby enabling more accurate control of the master library for data synchronization with the slave library.
Further, in order to save the data according to the result of the transaction, after the step S03, the method further includes:
step S04: and obtaining a transaction result corresponding to the transaction request.
It will be appreciated that the transaction result may be data of whether the transaction was successful after the transaction request was processed.
Step S05: and when the transaction result is that the transaction is successful, the synchronous control records of the master library and the slave library are not processed.
In a specific implementation, after the transaction result is obtained, whether the result is a successful transaction state is determined. And checking the synchronous control records of the master library and the slave library when the transaction result is judged to be successful. If the transaction result is successful and the synchronization control record has no mark to be processed, the step of processing the synchronization control records of the master library and the slave library is skipped.
Step S06: and when the transaction result is that the transaction fails, clearing the transaction data of the master library and the synchronous control record, and not processing the data of the slave library.
In a specific implementation, after the transaction result is obtained, it is determined whether the result is a transaction failure state. And if the transaction result is that the transaction fails, executing the operation of clearing the transaction data in the main library. The specific operation may be deleting the corresponding database record or performing data resetting, so as to ensure that the transaction data in the main library is cleared. At the same time as the master database data is purged, the synchronization control records associated with the transaction need to be purged to ensure that the synchronization control records are purged as well. In the event of a transaction failure, the data of the slave library is not processed, i.e., the data in the slave library is kept unchanged.
The above manner of this embodiment obtains the transaction result corresponding to the transaction request; when the transaction result is that the transaction is successful, the synchronous control records of the master library and the slave library are not processed; and when the transaction result is that the transaction fails, clearing the transaction data of the master library and the synchronous control record, and not processing the data of the slave library. The data can be stored according to the result of the transaction, and the invalid data can be prevented from being stored.
Further, in order to obtain an accurate synchronous switching value, the step S01 includes:
step S011: upon receipt of a transaction request, availability of the slave library is detected.
In particular implementations, a monitoring mechanism may be provided to periodically or on demand detect availability from the library. The monitoring can be performed by selecting the modes of heartbeat detection, network connection detection and the like. When the system receives a transaction request, an operation of probing from library availability is triggered. The system sends a probe signal to the slave library, which may be in a "ping" like manner or other custom availability probe manner. The availability of the slave library is judged by detecting whether the slave library can respond normally. And judging the availability of the slave library according to the response condition of the slave library to the detection signal. If the slave library is able to respond normally, it is determined to be available; if the slave library fails to respond normally, it is determined as unavailable.
Step S012: and setting the synchronous switch value to the third slave library state value when the slave library is detected to be unavailable.
Step S013: and setting the synchronous switch value to the second slave library state value when the slave library is detected to be restored to be available from unavailable.
Step S014: and setting the synchronous switch value of the slave library to the first slave library state value when the synchronous switch value of the slave library is the second slave library state value and the synchronous switch value of the slave library is synchronous with the master library.
The above-described manner of this embodiment is by detecting the availability of the slave library upon receipt of a transaction request; setting the synchronization switch value to the third slave library state value upon detecting that the slave library is not available; setting the synchronization switch value to the second slave library state value upon detecting that the slave library is restored to available from unavailable; and setting the synchronous switch value of the slave library to the first slave library state value when the synchronous switch value of the slave library is the second slave library state value and the synchronous switch value of the slave library is synchronous with the master library. So that an accurate synchronous switching value can be obtained according to the state of the slave library.
Further, based on the above embodiment, a second embodiment of the present application is proposed. Referring to fig. 3, fig. 3 is a flow chart of a second embodiment of the data security method of the present application.
In this embodiment, in order to ensure the integrity of the slave database data, after the step S20, the method further includes:
step S21: and when the slave library is not available, marking the business data corresponding to the transaction request as a non-notification asynchronous state.
It should be noted that, the un-notified unsynchronized state may be a state representing the business data in the master library and the save state in the slave library.
In a specific implementation, when the slave library is not available, marking the service data, and marking the service data corresponding to the transaction request as an undeployed unsynchronized state in the master library.
Step S22: and when the slave library is in the data recovery process, sending the service data marked as the un-notified asynchronous state to the slave library.
In particular implementations, the data recovery process may be entered when recovering from the library from an unavailable state. And sending all the service data marked as the un-notified asynchronous state to the slave library.
In this embodiment, when the slave library is not available, marking the service data corresponding to the transaction request as a non-notification unsynchronized state; and when the slave library is in the data recovery process, sending the service data marked as the un-notified asynchronous state to the slave library. After the slave library is recovered from the unavailable state, the data to be saved can be quickly saved, and the integrity of the data of the slave library is ensured.
Further, in order to accurately transmit the data required by the slave library, the step S22 includes:
step S221: and when the slave library is in the data recovery process, comparing the current master library version number of the service data in the master library with the current slave library version number of the service data in the slave library.
In a specific implementation, the current master and slave version numbers of the master and slave may be determined first, e.g., using corresponding database commands, to obtain the current version numbers of the master and slave. And then checking whether the business data of the master library and the slave library are consistent.
Step S222: and sending the business data in the master library with the version number of the master library larger than that of the slave library to the slave library.
In a specific implementation, the version numbers of the master library and the slave library are compared to determine whether the version number of the master library is greater than the version number of the slave library. If the master library version number is greater than the slave library version number, it is indicated that there is traffic data that needs to be sent to the slave library. And further determining the service data range to be transmitted through the master library version number and the slave library version number. The database replication tool is then used for data synchronization, and replication processes can also be monitored to ensure that data can be successfully sent from the master library to the slave library. Copy status, error log, or other monitoring tools may be reviewed to verify the status of the data sync.
In the embodiment, when the slave library is in the data recovery process, comparing the current master library version number of the service data in the master library with the current slave library version number of the service data in the slave library; and sending the business data in the master library with the version number of the master library larger than that of the slave library to the slave library. The data required by the slave library can be accurately transmitted.
Further, in order to better ensure data consistency, after the step S30, the method further includes:
step S31: and receiving the query request sent from the library, and searching the service data corresponding to the query request.
It should be noted that, the query request may be the request that is actively sent from the library to update data.
In particular implementations, an interface or endpoint may be provided on the master repository that may receive query requests, such that the master repository may be able to receive query requests from other systems. When the main library receives the inquiry request, the request data is obtained from the network interface, the content of the inquiry request is analyzed, the parameters required by the inquiry are extracted, the database inquiry language (such as SQL) of the main library is used, the corresponding inquiry operation is executed according to the inquiry parameters, and the corresponding service data is searched.
Step S32: and sending business data corresponding to the query request to the slave library so that the slave library is updated based on the business data.
In a specific implementation, determining a service data range needing to be updated, and updating service data obtained by inquiring from a master library into a slave library.
In this embodiment, by receiving the query request sent from the library, service data corresponding to the query request is searched; and sending business data corresponding to the query request to the slave library so that the slave library is updated based on the business data. The data consistency can be better ensured.
Based on the above embodiments, a third embodiment of the present application is presented.
In this embodiment, referring to fig. 4 and 5, fig. 4 is a schematic diagram of the overall architecture of a system to which the data security method of the present application is applied, and fig. 5 is a timing chart of a transaction flow of the data security method of the present application.
In this embodiment, an application a is installed in the main library, and is used to provide a main library transaction data maintenance service and a query service. The slave library is provided with an application M and an application T, wherein the application M is responsible for receiving transaction data from the application A, and the application T is responsible for providing transaction data query service.
After receiving the transaction request, the master library uses the heterogeneous read library expansion method to store a service backup data in the slave library. And manages the state of the service data using the self-growing version number. The version number of the slave library service data is additionally checked before each transaction. If the slave library version number is less than the received version, allowing the master library transaction; if the slave library version number is greater than the received version, the master library transaction is rejected and the technician is notified to manually compare and update the data. Alternatively, the data versions are compared by the program and the data is automatically restored.
And adding the step of detecting the abnormality of the main library in the main library, recording the last time available for the main library before the main library and the standby library are switched, and recording the earliest time available for the main library after the main library and the standby library are switched.
And adding an exception handling program in the slave library, and triggering after the master library and the slave library are switched. The exception handler scans the slave library for transaction data submitted during this period of time and compares it to the master library data. If the data version of the slave library is greater than the master library, the technician is notified to manually compare and update the data. Alternatively, the data versions are compared by the program and the data is automatically restored.
Further, referring to fig. 6, fig. 6 is a timing chart of a forward transaction flow of the data security method according to the present invention.
In this embodiment, a shared state value is used as a synchronization switch, Y represents that the slave library is normally synchronizable, N represents that the slave library is abnormally unsynchronizable, and W represents that the slave library is abnormally unsynchronizable during data recovery. Before the main library receives an external transaction request to prepare updating the transaction data of the main library, a synchronous control record is inserted into the database according to the value of the synchronous switch, and each control record corresponds to one transaction data.
When the switch value is Y or W, respectively inserting synchronous control records with the state P into the master library and the slave library; and checks the version number of the slave library service data. If the slave library version number is less than the received version, allowing the master library transaction; if the slave library version number is greater than the received version, the master library transaction is rejected and the technician is notified to manually compare and update the data. Alternatively, the data versions are compared by the program and the data is automatically restored.
When the switch value is N, only the synchronous control record with the state of U is inserted in the main library (the state value and meaning of the synchronous control record are that P is notified of the non-synchronization, U is not notified of the non-synchronization, C is notified of the synchronization, and R is notified of the rollback). The synchronization control record inserted in the master library is placed in the same database transaction as the transaction data, and submitted or rolled back as the transaction succeeds or fails.
At this time, in the process of inserting the control record into the slave library, the slave library is called by the master library, and if the call fails, the whole transaction is considered to fail.
And after the transaction of the main library is completed, updating the synchronous control record of the database according to the final result of the transaction.
If the final result is that the transaction is successful, and the current value of the synchronous switch is Y or W, respectively updating transaction data of the master library and the slave library, and updating the state of the synchronous control record to be C;
if the final result is that the transaction is successful and the current value of the synchronous switch is N, the control records of the master library and the slave library are not processed; (in this case, the master library must have a synchronization control record of P or U, and the slave library may have a synchronization control record of state P);
if the final result is that the transaction fails, and the current value of the synchronous switch is Y or W, the transaction data and the synchronous control record of the main library are all rollback cleared along with the failure of the transaction, and the state of the synchronous control record of the slave library is changed into R;
If the final result is that the transaction fails, and the current value of the synchronous switch is N, the transaction data and the synchronous control record of the master library are all rollback and cleared along with the failure of the transaction, but any data of the slave library is not processed.
The updating of the transaction data and the synchronous control records of the slave library is also completed by the master library, but the updating is different from the process of notifying the slave library to insert the synchronous control records, and the master library is only called once after the slave library is called successfully or failed, and the abnormal transaction data and the abnormal synchronous control records in the slave library are submitted to the online synchronous abnormal processing process for repairing, so that the normal transaction process is not blocked, the high-concurrency redundant updating operation is avoided, and the data storm is avoided.
For a transaction data query request received from the outside, receiving the transaction data query request from a slave library, judging the value of a synchronous switch and the synchronous control record state corresponding to the transaction from the slave library, and if the value of the synchronous switch is Y and the corresponding transaction does not have the synchronous control record with the state of P, querying the transaction data from the slave library; if the value of the synchronous switch is N or W, or the corresponding transaction exists in the synchronous control record with the state of P, transaction data is inquired from a main library.
To ensure that the slave library does not affect the normal execution of the service, the slave library needs to actively probe the availability of the slave library. And determining a shared synchronous switch state value according to the detection result. If the value of the synchronous switch is Y, any application detects that the system is not available, the value of the synchronous switch is changed to N, and other applications are actively notified.
The state value of the synchronous switch may not change from N directly to Y. If the system is restored to the available state after the fault is cleared, the system can automatically detect the availability and set the value of the synchronous switch to W, or the value of the synchronous switch can be manually modified after the fault is cleared.
The value W of the synchronous switch indicates the data recovery process. The recovery means that the master library synchronizes all the records with the synchronization control record as U to the slave library and stores the transaction data corresponding to the synchronization control record to the slave library. If the U state record is not existed for a certain time after all the data are synchronized, the master library sets the value of the shared synchronous switch to Y. Here, before the value of the synchronous switch is modified to Y, it must be checked that the original value is W, and if the original value is N, it cannot be modified to Y.
Further, referring to fig. 7, fig. 7 is a timing chart of a processing flow of the data guaranteeing method according to the present invention, in which the un-notified un-synchronized recording is not performed.
When the slave library is not available, the synchronous switch on the master library will be closed. The master library will not synchronize the transaction data with the slave library at this time, leaving only a synchronization control record in the U (un-notified un-synchronized) state in the master library. After the slave library is restored to be available, the master library should actively send these non-notified unsynchronized records to the slave library, updating the slave library's transaction data.
When the value of the synchronous switch is W (in the process of data recovery), the timing scanning function on the main library inquires all synchronous control records with the state U on the main library, and groups according to specific transactions. The transaction data has a self-increasing version number, the master library compares the version number on the synchronous control record with the version number on the master library transaction data, and the synchronous control record state corresponding to the transaction data and having a version number smaller than the version number of the transaction data is directly set as C. If there is any synchronization control record (hereinafter referred to as "current synchronization control record") having the same version number as the transaction data and a state of U, the timing scanning function of the master library synchronizes the corresponding transaction data to the slave library, updates the slave-library transaction data by the slave library, and sets the synchronization control record of the slave library to C (if not present, a synchronization control record of C state is newly added). At this time, if the master library calls the slave library successfully, the master library sets the state of the current synchronous control record to C; if the master library calls the slave library unsuccessfully, the current synchronous control record is not processed, the timing scanning function processes the next transaction data, and the next processing of the timing scanning function on the master library is waited.
Further, referring to fig. 8, fig. 8 is a timing chart of an online synchronization exception handling procedure of the data guaranteeing method according to the present invention.
The master library, after transaction processing, either successfully or failed, only performs the call to the slave library once. When execution fails, a non-final state synchronization control record with state P (notified of the lack of synchronization) will be left from the library database. In this case, the transaction data change cannot be updated to the slave library. The transaction data and synchronization control records of the master library may have failed to roll back and no corresponding synchronization control records will be left in the master library, so exception handling needs to be initiated by the slave library's synchronization control record timing scanning function at the slave library.
The slave library is scanned for more than 5 minutes (which can be controlled by parameters) of the synchronization control record which is still P-state. And for each P-state synchronous control record, inquiring the synchronous control record state of the main library according to the transaction information. There are two cases at this time:
one case is that there is no synchronization control record in the master, which means that the transaction data on the master has been rolled back or has not yet been committed, and a corresponding R (notified of the rolled back) state synchronization control record is inserted to identify. If the record is successfully inserted, returning to the R state; if the insertion fails, the transaction on the main library is not submitted yet, and the error is reported back to wait for the next processing.
Another case is that there is a control record in the master library, and then the control state of the synchronous control record is directly returned.
And for the call returned by the error report, the slave library does not process and waits for the next initiation. For the call of the normal return control state, the slave library performs specific processing according to the state, specifically as follows:
p, notified of the unsynchronized state: updating the transaction data of the slave library, updating the synchronous control record state to be C, and updating the synchronous control record state of the master library to be C;
r, the rollback state has been notified: updating the slave library synchronous control state to be R;
c, notified of synchronized state: updating transaction data of the slave library, and updating the synchronous control record state of the slave library to be C;
u, the unsynchronized state is not notified (probability is very small): and updating the transaction data of the slave library, updating the synchronous control record state of the slave library to be C, and updating the control state of the master library to be C.
After the slave library processing is completed, the master library is called to update the control record state. If the master library update fails, the slave library transaction rolls back to wait for the next processing.
In addition, in order to achieve the above object, the present invention further provides a data security device, wherein referring to fig. 9, fig. 9 is a block diagram of a data security device according to the present invention, the data security device includes:
The data acquisition module is used for acquiring a main library version number of business data corresponding to a transaction request in the main library when the transaction request is received;
the data sending module is used for sending the version number of the master library to the slave library, and when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is lost when the master library and the slave library are switched;
and the exception repairing module is used for refusing to respond to the transaction request through the master library when receiving the version exception information, and repairing the data of the master library from the service data which is backed up in advance in the slave library.
The embodiment obtains the version number of the main library in the main library of the business data corresponding to the transaction request; the version number of the master library is sent to the slave library, when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is subjected to data loss when the master library and the slave library are subjected to master-slave switching; and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library. And adding a version number in service data corresponding to the transaction request, and sending the version number of the master library corresponding to the transaction request to the slave library when the transaction request is acquired. And judging whether the data loss occurs in the master library or not by comparing the master library version number with the slave library version number. And further, data restoration is carried out on the master library by utilizing the data of the slave library. The method and the device solve the technical problem that lost data can be retrieved through reliable backup after the database primary and backup are switched.
Further, the data acquisition module is further configured to acquire a synchronous switch value when receiving a transaction request; when the synchronous switch value is a first slave library state value or a second slave library state value, inserting synchronous control records with a first notification mark in states of the master library and the slave library; when the synchronous switch value is a third slave library state value, a synchronous control record with a second notification mark is inserted into the master library; the synchronization control record is used for recording the synchronization state of the transaction request, and the synchronization state is used for indicating the master library and the slave library to perform data synchronization; the first slave library state value represents that the slave library is normally synchronizable, the second slave library state value represents that the slave library is abnormally unsynchronizable in the data recovery process, the first notification mark represents that the slave library is notified to be unsynchronized, and the second notification mark represents that the slave library is not notified to be unsynchronized.
Further, the data acquisition module is further configured to acquire a transaction result corresponding to the transaction request; when the transaction result is that the transaction is successful, the synchronous control records of the master library and the slave library are not processed; and when the transaction result is that the transaction fails, clearing the transaction data of the master library and the synchronous control record, and not processing the data of the slave library.
Further, the data acquisition module is further configured to detect availability of the slave library when a transaction request is received; setting the synchronization switch value to the third slave library state value upon detecting that the slave library is not available; setting the synchronization switch value to the second slave library state value upon detecting that the slave library is restored to available from unavailable; and setting the synchronous switch value of the slave library to the first slave library state value when the synchronous switch value of the slave library is the second slave library state value and the synchronous switch value of the slave library is synchronous with the master library.
The data sending module is further used for marking the business data corresponding to the transaction request as a non-notification and non-synchronization state when the slave library is not available; and when the slave library is in the data recovery process, sending the service data marked as the un-notified asynchronous state to the slave library.
The data sending module is further configured to compare a current master library version number of the service data in the master library with a current slave library version number of the service data in the slave library when the slave library is in a data recovery process; and sending the business data in the master library with the version number of the master library larger than that of the slave library to the slave library.
The abnormality repair module is further used for receiving the query request sent from the library and searching service data corresponding to the query request; and sending business data corresponding to the query request to the slave library so that the slave library is updated based on the business data.
In addition, in order to achieve the above object, the present invention also proposes a storage medium, wherein a data security program is stored on the storage medium, the data security program implementing the steps of the data security method as described above when executed by a processor.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. The data guaranteeing method is applied to a data guaranteeing architecture comprising a master library, a standby library and a slave library, and comprises the following steps:
acquiring a main library version number of business data corresponding to a transaction request in the main library;
the version number of the master library is sent to the slave library, when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, the version abnormality information is used for representing that the master library is lost when the master library and the slave library are subjected to master-slave switching, and the version number of the slave library is increased with time;
and when the version abnormality information is received, refusing to respond to the transaction request through the master library, and repairing the data of the master library from providing the service data which is backed up in advance in the slave library.
2. The data security method as claimed in claim 1, wherein the step of obtaining the master library version number of the service data corresponding to the transaction request in the master library further comprises:
when a transaction request is received, acquiring a synchronous switch value;
when the synchronous switch value is a first slave library state value or a second slave library state value, inserting synchronous control records with a first notification mark in states of the master library and the slave library;
When the synchronous switch value is a third slave library state value, a synchronous control record with a second notification mark is inserted into the master library;
the synchronization control record is used for recording the synchronization state of the transaction request, and the synchronization state is used for indicating the master library and the slave library to perform data synchronization;
the first slave library state value represents that the slave library is normally synchronizable, the second slave library state value represents that the slave library is abnormally unsynchronizable in the data recovery process, the first notification mark represents that the slave library is notified to be unsynchronized, and the second notification mark represents that the slave library is not notified to be unsynchronized.
3. The data securing method as claimed in claim 2, wherein, when the synchronization switch value is a third slave library state value, after the step of inserting the synchronization control record whose state is the second notification flag, further comprising:
obtaining a transaction result corresponding to the transaction request;
when the transaction result is that the transaction is successful, the synchronous control records of the master library and the slave library are not processed;
and when the transaction result is that the transaction fails, clearing the transaction data of the master library and the synchronous control record, and not processing the data of the slave library.
4. The data securing method as claimed in claim 2, wherein the step of acquiring the synchronous switching value upon receiving the transaction request includes:
detecting availability of the slave library upon receipt of a transaction request;
setting the synchronization switch value to the third slave library state value upon detecting that the slave library is not available;
setting the synchronization switch value to the second slave library state value upon detecting that the slave library is restored to available from unavailable;
and setting the synchronous switch value of the slave library to the first slave library state value when the synchronous switch value of the slave library is the second slave library state value and the synchronous switch value of the slave library is synchronous with the master library.
5. The data protection method as claimed in claim 1, wherein the sending the master library version number to the slave library, the slave library feeding back version exception information when detecting that the master library version number is smaller than a slave library version number stored in the slave library, the version exception information being used for characterizing that data loss occurs when the master library is switched with the slave library, the step of increasing the slave library version number with time further comprising:
when the slave library is not available, marking the business data corresponding to the transaction request as a non-notification asynchronous state;
And when the slave library is in the data recovery process, sending the service data marked as the un-notified asynchronous state to the slave library.
6. The data securing method as claimed in claim 5, wherein the step of transmitting the service data marked as the un-notified unsynchronized state to the slave library when the slave library is in a data recovery process, comprises:
when the slave library is in the data recovery process, comparing the current master library version number of the service data in the master library with the current slave library version number of the service data in the slave library;
and sending the business data in the master library with the version number of the master library larger than that of the slave library to the slave library.
7. The data securing method according to any one of claims 1 to 6, characterized in that, upon receiving the version abnormality information, the step of refusing to respond to the transaction request by the master library and repairing the master library from providing the service data backed up in advance in the slave library further comprises:
receiving the query request sent from the library, and searching service data corresponding to the query request;
And sending business data corresponding to the query request to the slave library so that the slave library is updated based on the business data.
8. A data assurance device, the data assurance device comprising:
the data acquisition module is used for acquiring a main library version number of business data corresponding to a transaction request in the main library when the transaction request is received;
the data sending module is used for sending the version number of the master library to the slave library, and when the slave library detects that the version number of the master library is smaller than the version number of the slave library stored by the slave library, version abnormality information is fed back, and the version abnormality information is used for representing that the master library is lost when the master library and the slave library are switched;
and the exception repairing module is used for refusing to respond to the transaction request through the master library when receiving the version exception information, and repairing the data of the master library from the service data which is backed up in advance in the slave library.
9. A data assurance device, the device comprising: a memory, a processor and a data assurance program stored on the memory and executable on the processor, the data assurance program being configured to implement the steps of the data assurance method of any one of claims 1 to 7.
10. A storage medium having stored thereon a data security program which, when executed by a processor, implements the steps of the data security method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311100861.2A CN117076211A (en) | 2023-08-29 | 2023-08-29 | Data guarantee method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311100861.2A CN117076211A (en) | 2023-08-29 | 2023-08-29 | Data guarantee method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117076211A true CN117076211A (en) | 2023-11-17 |
Family
ID=88702093
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311100861.2A Pending CN117076211A (en) | 2023-08-29 | 2023-08-29 | Data guarantee method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076211A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187073A (en) * | 2021-12-13 | 2022-03-15 | 平安证券股份有限公司 | Order information processing method and device, computer equipment and storage medium |
-
2023
- 2023-08-29 CN CN202311100861.2A patent/CN117076211A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187073A (en) * | 2021-12-13 | 2022-03-15 | 平安证券股份有限公司 | Order information processing method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11921748B1 (en) | Method and apparatus for using representations of blocks of data when continuously comparing two databases which are actively being kept synchronized | |
| US9189348B2 (en) | High availability database management system and database management method using same | |
| US7003694B1 (en) | Reliable standby database failover | |
| EP3598317B1 (en) | System and method for massively parallel processing database | |
| US7925633B2 (en) | Disaster recovery system suitable for database system | |
| US7730029B2 (en) | System and method of fault tolerant reconciliation for control card redundancy | |
| US9582382B1 (en) | Snapshot hardening | |
| EP1959347A1 (en) | Non-stop transaction processing system | |
| US20060218204A1 (en) | Log stream validation in log shipping data replication systems | |
| JP3094888B2 (en) | Numbering mechanism, data consistency confirmation mechanism, transaction re-execution mechanism, and distributed transaction processing system | |
| CN109710596B (en) | Data cleaning method, device, equipment and computer readable storage medium | |
| US20230004465A1 (en) | Distributed database system and data disaster backup drilling method | |
| US9330153B2 (en) | System, method, and computer readable medium that coordinates between devices using exchange of log files | |
| CN112711382B (en) | Data storage method and device based on distributed system and storage node | |
| CN117076211A (en) | Data guarantee method, device, equipment and storage medium | |
| CN112765126B (en) | Database transaction management method, device, computer equipment and storage medium | |
| CN111404737B (en) | Disaster recovery processing method and related device | |
| CN114265900A (en) | A data processing method, device, electronic device and storage medium | |
| CN110928945B (en) | Data processing method and device for database and data processing system | |
| JP2003345628A (en) | Failure investigation data collection method, its implementation system, and its processing program | |
| CN117609294A (en) | Data query method, device and equipment | |
| US20220229740A1 (en) | Protecting databases in a distributed availability group | |
| CN113868340B (en) | Distributed transaction processing method, device and electronic equipment | |
| JP2005250819A (en) | Replication DB consistency check method | |
| CN112596867A (en) | Suspension transaction processing method and distributed database system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |