KR20240010939A - Method and system for backup and recovery of distributed databases - Google Patents

Abstract

A method performed by a backup manager server according to an embodiment may comprise the steps of: based on receiving backup input for one or more sharded clusters, obtaining information on a target sharded cluster from a backup waiting queue sorted according to backup priority; determining whether the target sharded cluster can be backed up based on whether one or more shards belonging to the target sharded cluster can be backed up; and based on the target sharded cluster being determined to be able to be backed up, giving a backup command to the one or more shards belonging to the target sharded cluster to perform an operation log backup prior to a snapshot backup.

Description

{METHOD AND SYSTEM FOR BACKUP AND RECOVERY OF DISTRIBUTED DATABASES}

Hereinafter, technologies related to backup and recovery of distributed databases are disclosed.

An application that uses a database as its main program may not allow data processing to stop when a system failure occurs. However, it may be impossible to avoid system failures while the database is operating, so it may be important for the database to continue operating even if a failure occurs. Among the methods for normal operation of the database, regular database backups can prevent data loss in the event of an unintended data loss event. It may be used to back up and/or replicate current data and provide at least some of the services while the primary system is being restored. In other words, remote backup is a method that supports a different form of fault tolerance.

Remote backup can be said to be important for operating a database system because it can protect data and ensure operational continuity in the event of data loss or accident. Remote backup can have the advantage of helping even when computation is not possible on the primary system. As an example, there may be a local backup method to reduce the load. Remote backup can be performed locally to a mirror disk. Local replication may not be appropriate because system errors or failures are likely to propagate from the same physical nodes (e.g., data centers in the same region). To overcome these shortcomings, remote backup can be performed and the backup data can be maintained regardless of failure by physically separating the backup data from the system. Remote backup can have the advantage of being able to isolate not only physical failures but also software failures (e.g., calculation errors, bugs, etc.).

A method performed by a backup manager server according to an embodiment is based on receiving backup input for one or more sharded clusters, selecting a target from a backup waiting queue sorted according to backup priority. Obtaining information on a target sharded cluster, determining whether the target shard cluster can be backed up based on whether one or more shards belonging to the target shard cluster can be backed up. And based on the target shard cluster being determined to be capable of backup, a backup command is given to one or more shards belonging to the target shard cluster to perform an operation log backup prior to a snapshot backup. It may include steps.

The step of acquiring information on the target shard cluster includes waiting for the backup sorted according to the backup priority determined based on one or a combination of two or more of the average backup time length of the shards belonging to the shard cluster or the number of shards belonging to the shard cluster. It may include the step of creating a queue.

The step of determining whether backup of the target shard cluster is possible may include selecting a secondary to be used for backup based on the physical location of the secondary from among one or more secondary of the shard.

The step of determining whether backup of the target shard cluster is possible includes determining whether backup of the corresponding shard is possible based on the number of available backup servers in the data center mapped to the secondary of the shard belonging to the target shard cluster. May include steps.

The step of determining whether the target shard cluster is capable of being backed up includes determining that the target shard cluster is capable of being backed up based on determining that all of one or more shards belonging to the target shard cluster are capable of being backed up, and determining that the target shard cluster is capable of being backed up. It may include determining that the target shard cluster is not capable of being backed up, based on determining that at least one shard belonging to is not capable of being backed up.

The step of ordering a backup may include ordering a backup so that the backup start times of the one or more shards are within a predetermined time range.

The step of ordering a backup may include ordering a backup of the operation log for data replication between the primary and secondary of the shard.

The step of ordering a backup of the computation log may include commanding a backup of the computation log whose computation time follows the logical clock of the target shard cluster.

The step of commanding the backup includes commanding a snapshot backup using a snapshot for the secondary of each shard after a predetermined length of time has elapsed from the start time of the operation log backup. can do.

A method performed by a backup manager server according to an embodiment changes the backup priority of the target shard cluster based on determining that the target shard cluster is not available for backup, and performs the backup according to the changed backup priority. The step of updating the waiting queue may be further included.

The method performed by the backup manager server according to one embodiment may further include determining whether a new target shard cluster can be backed up based on the end of the backup of at least one shard belonging to the target shard cluster. there is.

The method performed by the backup manager server according to one embodiment is based on determining that the target shard cluster is capable of backup, and another shard cluster indicated by information obtained from the backup standby queue is parallel to the target shard cluster. It may further include a step of determining whether backup is possible.

The step of ordering the backup includes commanding the target shard cluster and the other shard cluster to be backed up in parallel, based on determining that the other shard cluster is capable of being backed up in parallel with the target shard cluster, , the method further includes the step of deleting the information of the target shard cluster and the information of the other shard cluster from the backup standby queue, based on determining that another shard cluster is capable of being backed up in parallel with the target shard cluster. It can be included.

The method performed by the backup manager server according to one embodiment is based on determining that the other shard cluster is not capable of being backed up in parallel with the target shard cluster, and extracts information of the target shard cluster from the backup standby queue. The step of deleting and maintaining information about the other shard cluster in the backup waiting queue may be further included.

A method performed by a backup manager server according to an embodiment includes commanding to set a snapshot of the target shard cluster to the initial recovery state of the target shard cluster based on receiving a recovery input for the target shard cluster. And it may further include commanding to adjust the initial recovery state of the set target shard cluster based on the operation log.

The step of commanding to adjust the initial recovery state of the target shard cluster may include commanding to restore the data of the shard by adjusting the initial recovery state of the shard set based on the snapshot using the operation log. there is.

In a method performed by a system for backup and recovery of a distributed database according to an embodiment, the backup priority is determined based on receiving backup input for one or more sharded clusters by a backup manager server. Obtaining information on a target sharded cluster from a backup waiting queue sorted according to the backup waiting queue, based on whether one or more shards belonging to the target sharded cluster can be backed up by the backup manager server. Thus, determining whether the target shard cluster can be backed up, based on the backup manager server determining that the target shard cluster is capable of being backed up, performing a snapshot backup on one or more shards belonging to the target shard cluster. Commanding a backup to perform an operation log backup prior to a snapshot backup, based on receiving a recovery input for the target shard cluster by a recovery manager server, It may include commanding to set a snapshot to the initial recovery state of the target shard cluster, and commanding the recovery manager server to adjust the initial recovery state of the set target shard cluster based on the operation log. there is.

The backup manager server according to one embodiment includes a memory storing computer-executable instructions and a processor that accesses the memory and executes the instructions, and the instructions are stored in one or more shards. Based on receiving the backup input for the sharded cluster, obtain information of the target sharded cluster from the backup waiting queue sorted according to backup priority, and Based on whether or not one or more shards belonging to the target shard cluster can be backed up, it is determined whether the target shard cluster can be backed up, and based on the target shard cluster being determined to be capable of being backed up, to one or more shards belonging to the target shard cluster. , It may be configured to command a backup to perform an operation log backup prior to a snapshot backup.

In a system for backing up and restoring a distributed database according to an embodiment, the memory of the backup manager server where computer-executable instructions are stored, and the memory of the backup manager server are accessed. A backup manager server including a processor of the backup manager server that executes the instructions, a memory of the recovery manager server storing computer-executable instructions, and a processor of the recovery manager server that accesses the memory of the recovery manager server and executes the instructions. a recovery manager server including a backup manager server, wherein the commands stored in a memory of the backup manager server include: a backup standby queue sorted according to backup priority, based on receiving backup input for one or more sharded clusters; Obtain information about the target sharded cluster from (backup waiting queue), and determine whether the target shard cluster can be backed up based on whether one or more shards belonging to the target shard cluster can be backed up, and Based on the target shard cluster being determined to be capable of backup, to command one or more shards belonging to the target shard cluster to perform an operation log backup prior to a snapshot backup. Commands configured and stored in the memory of the recovery manager server command, based on receiving a recovery input for the target shard cluster, to set a snapshot of the target shard cluster to the recovery initial state of the target shard cluster, Based on the operation log, it may be configured to command to adjust the initial recovery state of the set target shard cluster.

1 is a diagram for explaining the operation of a system for backup and recovery according to an embodiment.
Figure 2 is a diagram for explaining a distributed database according to an embodiment.
Figure 3 is a diagram for explaining a backup command operation for a shard cluster according to an embodiment.
FIG. 4 is a diagram illustrating an operation of determining whether a target shard cluster can be backed up according to an embodiment.
Figure 5 is a diagram for explaining a backup command operation of a backup manager server according to an embodiment.
FIG. 6 is a diagram illustrating an operation of backing up a plurality of shard clusters in parallel according to an embodiment.
Figure 7 is a diagram for explaining a recovery operation of a backup and recovery system according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific disclosed embodiments, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of the described features, numbers, steps, operations, components, parts, or combinations thereof, and are intended to indicate the presence of one or more other features or numbers, It should be understood that this does not exclude in advance the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

1 is a diagram for explaining the operation of a system for backup and recovery according to an embodiment.

The input module 110 may transmit input to the system 120 for backup and recovery. The input is an input that instructs the system 120 for backup and recovery to perform a specific operation (e.g., backup command operation, recovery command operation, etc.), and may include, for example, backup input, recovery input, etc. You can. The input module 110 according to one embodiment may be implemented using Jenkins.

The backup and recovery system 120 is a system for performing backup and recovery of the distributed database 130, and includes a backup manager server 121 and a recovery manager server 122. , and may include a tracker server (tracker server) 123.

The backup manager server 121 may command the distributed database 130 to perform a backup based on receiving a backup input from the input module 110. The backup manager server 121 according to one embodiment may include a memory in which computer-executable instructions are stored, and a processor that accesses the memory and executes the instructions. As will be described later, the distributed database 130, which has received a backup command, can perform the backup by transmitting backup data to the storage 140. The backup command operation by the backup manager server 121 will be described later with reference to FIGS. 3 to 6.

The recovery manager server 122 may command the distributed database 130 to perform recovery based on receiving a recovery input from the input module 110. The recovery manager server 122 according to one embodiment may include a memory in which computer-executable instructions are stored, and a processor that accesses the memory and executes the instructions. As will be described later, the distributed database 130 that has received a recovery command may receive backup data from the storage 140 and restore the distributed database 130 based on the received backup data. The recovery command operation by the recovery manager server 122 will be described later with reference to FIG. 7.

The tracker server 123 checks the status during the backup operation and/or recovery operation of the distributed database 130, and when an error is detected, reports the detected error to the backup manager server 121 (or recovery manager server 122). Information can be conveyed. The tracker server 123 may monitor the occurrence of errors by checking the status of the backup operation and/or recovery operation of the distributed database 130 according to a predetermined sampling period. For example, when the tracker server 123 detects that at least one shard cluster of the distributed database 130 has failed to backup and/or restore, the backup manager server 121 (or recovery manager server 122) Information about errors can be conveyed to

The distributed database 130 may have a database structure in which one logical database is physically distributed across multiple computers on a network but is logically integrated and shared so that the client can recognize it as one database. As will be described later in FIG. 2, the distributed database 130 includes one or more sharded clusters, and each shard cluster may include one or more shards.

The storage 140 may store backup data received from the distributed database 130 by the backup server. Storage 140 according to one embodiment may be implemented through a storage medium included in a data center.

Figure 2 is a diagram for explaining a distributed database according to an embodiment.

The distributed database 200 (e.g., the distributed database 130 of FIG. 1) according to an embodiment may include one or a combination of two or more of a shard cluster 210 and a single replica set 220.

Each shard cluster can contain one or more shards. Exemplarily, as shown in FIG. 2, the first shard cluster 210 may include three shards 211, 212, and 213.

Although not explicitly shown in FIG. 2, each shard cluster may further include a configuration server (config server) and a router server along with one or more shards. The Config server can store metadata about the distributed state of data stored in a shard cluster. The router server can perform the operation of transmitting a command from the user to each shard and transmitting the result of the command to the user. By way of example, a shard cluster may be implemented through MongoDB.

Each shard can contain one primary and one or more secondary. Each of the primary and secondary shards can be implemented as physically separate servers to provide common data. Exemplarily, as shown in FIG. 2, the shard 211 may include a primary 211a, a first secondary 211b, and a second secondary 211c.

Among the primary and secondary of a shard, the primary can change data by applying change commands to the data (e.g., create, update, delete). The secondary may not synchronize data with the primary synchronously when the data in the primary changes. Instead, the secondary's data can be changed asynchronously based on an operation log for recording operations on the primary. A checkpoint is an operation to write data in memory to a disk, and may refer to the point in time when an operation to synchronize data in memory and data on a disk is performed at one node. Illustratively, the checkpoint may be set to repeat at a period of 60 seconds. In other words, the operation log is a log for data consistency between the primary and secondary of one shard and can be used at each checkpoint. Data consistency may include the property that, in one shard, data obtainable from the primary can be obtained based on secondary data and operation logs.

The operation log may have an operation time that follows a logical clock. The logical clock may represent a clock for synchronization between shards when a plurality of shards belong to a shard cluster. A logical clock can express the precedence relationship between operations that occur in a shard cluster. For example, the first operation log may include the first operation time expressed by the logical clock of the first operation, and the second operation log may include the second operation time expressed by the logical clock of the second operation. there is. The magnitude relationship between the first operation time and the second operation time indicates the precedence relationship between the first operation and the second operation, but the difference between the first operation time and the second operation time is the occurrence time of the first operation and the second operation time. It may be independent of the difference between occurrence times. In other words, if the first operation time is smaller than the second operation time, it can only be determined that the first operation is applied before the second operation, and the time difference between the first operation and the second operation is different from the first operation time. The calculation time and the second calculation time may be unrelated to the time difference between the first calculation time and the second calculation time.

A shard can adjust the shard's clock by comparing the computation time included in the computation log with the shard's time (e.g., the time expressed by the shard's physical clock). For example, a shard can receive operation instructions. Operation commands can be delivered to the corresponding shard by the config server. The operation instruction may include an operation time expressed as a logical clock. For the shard that received the operation command, the physical time of the shard may be smaller than the operation time of the operation command. If the shard's physical time is smaller than the operation time of the operation command, after processing the operation command in the primary, the shard's physical time is set to a time greater than the operation time (for example, in advance of the operation time of the operation command). can be changed to the value added to the determined value). In other words, the shard can adjust the shard's time according to the sequential relationship between the shard's time and the operation time of the operation command. By adjusting the shard's clock, the shard can be synchronized via the logical clock of the shard cluster. After applying an operation instruction in the primary, a shard can record an operation log that includes the operation time following the logical clock. Therefore, whenever a shard in a shard cluster receives an operation command, the shard's time can be synchronized with the logical clock of the shard cluster. As a result, if a shard in a shard cluster has received at least one operation command, it can be in a synchronized state with the logical clock of the shard cluster.

Hereinafter, a method of operating a system for backing up and restoring a distributed database including one or more shard clusters will be described in FIGS. 3 to 7.

Figure 3 is a diagram for explaining a backup command operation for a shard cluster according to an embodiment.

At step 310, the backup manager server may create a backup waiting queue ordered by backup priority. The backup standby queue may have information on at least part of the distributed database sorted according to backup priority. For example, a backup standby queue may have information from one or more shard clusters. Information on the shard cluster may include one or a combination of two or more of the identification of each shard cluster, the ID of the shard belonging to each shard cluster, or the geographical location of the secondary of each shard. The backup manager server can determine the backup priority of the shard cluster. The backup manager server may create a backup standby queue based on the determined backup priority.

According to one embodiment, the backup manager server may determine backup priority based on the average backup time length between one or more shards belonging to a shard cluster. The backup manager server may calculate the average backup time length of the shards belonging to each shard cluster based on the stored past backup time length (e.g., the backup time length of each shard in the immediately preceding backup operation). The backup manager server may create a backup standby queue by sorting one or more shard clusters (e.g., in ascending or descending order) based on the average backup time length of the shards.

According to one embodiment, the backup manager server may determine the backup priority based further on the number of shards belonging to the shard cluster (also expressed as the size of the shard cluster) along with the average backup time length. The backup manager server can create a backup queue by sorting the initial backup queue based on the average backup time length and adjusting the initial backup queue based on the number of shards. For example, the backup manager server can create an initial backup queue by sorting shard clusters in descending order based on average backup time length. In other words, the backup manager server can determine the initial backup priority of the shard cluster to be higher as the average backup time length of the shard cluster is longer. The backup manager server sorts the two or more adjacently aligned shard clusters in descending order of shard cluster size, based on the difference between the average backup time lengths of the two or more adjacently aligned shard clusters in the initial backup standby queue being less than or equal to the threshold time length. You can create a backup queue by adjusting the sorted backup queue. For example, the backup manager server may adjust an initial backup standby queue that includes a first shard cluster and a second shard cluster where the difference in average backup time length is less than or equal to a threshold time length. The backup manager server can adjust the backup standby queue by sorting the size of the first shard cluster and the size of the second shard cluster in descending order. In other words, the backup manager server can adjust the backup priority of the shard cluster to a higher priority as the size of the shard cluster becomes larger.

According to one embodiment, the backup manager server may further create and/or manage one or a combination of two or more of a backup execution queue, a backup failure queue, or a backup completion queue along with the backup standby queue. As described above, the backup standby queue includes shard clusters (or shard cluster information) before the backup starts, and can be managed in a sorted state according to backup priority.

The backup execution queue may include a shard cluster (or information on a shard cluster) that is performing a backup, that is, a shard cluster (or information on a shard cluster) where a backup has started but has not yet ended. As an example, the backup execution queue can be managed in a sorted state according to the backup start time of the shard cluster.

The backup completion queue may include a shard cluster (or information on a shard cluster) for which backup has been completed according to the backup input, that is, a shard cluster (or information on the shard cluster) for which the backup of all shards belonging to the shard cluster has been completed according to the backup input. You can. As an example, the backup completion queue may be managed in a sorted state according to the backup completion time of the shard cluster.

The backup failure queue may include shard clusters (or shard cluster information) for which backup was attempted by the backup manager server but the backup failed. In other words, even though the backup manager server determines that the target shard cluster is available for backup, if the backup fails due to errors other than the backup time zone conditions and the number of available backup servers, the shard cluster can be inserted into the backup failure queue. there is. For reference, if the backup manager determines that a shard cluster is not available for backup based on the number of available backup servers or backup time zone conditions, it changes the backup priority of the shard cluster and inserts it back into the backup waiting queue for later use. You can determine whether backup is possible again and not insert the corresponding shard cluster into the backup failure queue.

Management operations of the backup standby queue, backup execution queue, backup completion queue, and backup failure queue of the backup manager server according to one embodiment will be described later in related steps.

In step 320, the backup manager server may obtain information about the target sharded cluster from the backup standby queue based on receiving the backup input. The target shard cluster may refer to a shard cluster for which the backup manager server will command backup, among one or more shard clusters indicated by the backup input.

The backup input may include an input instructing to perform a backup operation on at least a portion of the distributed database (eg, one or more shard clusters). The backup input may be transmitted from an input module (e.g., input module 110 in FIG. 1) to a system for backup and recovery (e.g., system for backup and recovery 120 in FIG. 1). The system for backup and recovery may transmit the backup input to the backup manager server (eg, the backup manager server 121 in FIG. 1) based on receiving the backup input from the input module.

According to one embodiment, the backup input indicates at least a portion of the distributed database as the target of backup, and the system for backup and recovery performs backup to at least a portion of the distributed database indicated by the backup input based on receiving the backup input. can be commanded. According to one embodiment, the backup input may not specify the target of the backup, and the system for backup and recovery orders a backup of the entire distributed database based on receiving the backup input that does not specify the target of the backup. can do.

In step 330, the backup manager server may determine whether the target shard cluster can be backed up based on whether one or more shards belonging to the target shard cluster can be backed up. The backup manager server may determine that the target shard cluster is capable of being backed up based on all of one or more shards belonging to the target shard cluster being capable of being backed up. In other words, the target shard cluster may be determined to be capable of being backed up if all shards belonging to the target shard cluster are capable of being backed up.

According to one embodiment, the backup manager server may determine whether a shard can be backed up based on the shard's secondary and the available backup servers mapped to the secondary. The operation of determining whether a target shard cluster can be backed up is described later in FIG. 4.

Mainly, it has been explained that the backup manager server determines whether a target shard cluster can be backed up based on whether the shard can be backed up, but it is not limited to this. For example, the backup manager server may determine whether the target shard cluster can be backed up based on whether the shard can be backed up and/or the backup time zone.

According to one embodiment, the backup manager server may determine whether backup of the target shard cluster is possible based on the backup available time zone corresponding to the target shard cluster. For example, the backup manager server can obtain the backup available time zone corresponding to each shard cluster. The backup time zone is, by way of example, a predetermined time zone for each shard cluster and may mean a time zone during which backup of the corresponding shard cluster can be performed. The backup manager server may determine that the target shard cluster is available for backup if the backup time zone of the target shard cluster includes the time when determining whether or not the target shard cluster can be backed up. The backup manager server may determine that the target shard cluster is not available for backup if the time for determining whether or not the target shard cluster can be backed up is not included in the backup time zone of the target shard cluster.

According to one embodiment, the backup manager server may determine whether the target shard cluster can be backed up based on whether the shard can be backed up and the backup time zone. For example, the backup manager server configures the target shard cluster based on the fact that all shards belonging to the target shard cluster are available for backup and that the time for determining whether or not the target shard cluster can be backed up is included in the backup available time zone corresponding to the target shard cluster. It can be determined that the cluster is capable of being backed up. The backup manager server may determine that the target shard cluster is not capable of being backed up based on at least one of the shards belonging to the target shard cluster being not capable of being backed up. The backup manager server may determine that the target shard cluster is not available for backup if the time for determining whether or not the target shard cluster can be backed up is not included in the backup time zone of the target shard cluster.

In step 340, the backup manager server may order a backup of one or more shards belonging to the target shard cluster based on the target shard cluster being determined to be capable of backup. The backup manager server may command to perform a backup by performing an operation log backup prior to a snapshot backup.

Operation log backup may refer to a backup for storing an operation log in which operations on the primary are recorded. As described above in FIG. 2, the operation log may include a log for data replication between the primary and secondary of the shard, and the primary data can be obtained by applying the operation log to the secondary data. there is. Additionally, shards in a shard cluster can be synchronized to the logical clock of the shard cluster through the operation time of the computation log.

Snapshot backup may refer to a backup for storing the entire image of at least part of a distributed database (e.g., one or more shard clusters) as a snapshot. A snapshot is a concept designed to preserve the state of a database (e.g., at least part of a shard cluster) at a specific point in time and later restore the database based on the snapshot, and can be used in environments where stability is important. As an example, snapshot backup can be implemented through snapshots provided by a volume manager such as a logical volume manager (LVM).

Snapshot backup stores data at a specific point in time in the shard, especially the secondary in the shard, at the time the snapshot backup was performed, and performs calculations for synchronization with logical clocks between shard clusters and/or data recovery in the primary. Additional log backups may be required. Backup command operations including operation log backup and snapshot backup will be described later with reference to FIG. 5 .

As will be described later in FIG. 5, the shard can perform backup by transmitting backup data to the backup server in the data center based on receiving a backup command from the backup manager server. Backup data may include one or a combination of two or more of operation logs or snapshots.

As described above in FIG. 2, shards can be synchronized to a logical clock while receiving operation instructions from the router. By performing operation log backup first and then snapshot backup, the backup manager server can command shards in a shard cluster to perform snapshot backup with their logical clocks synchronized as they receive operation commands.

The backup manager server according to one embodiment may update one or a combination of two or more of the backup standby queue or backup execution queue based on a backup command to the target shard cluster. For example, when commanding a backup to a target shard cluster, the backup manager server may delete the target shard cluster (or information on the target shard cluster) from the backup waiting queue. Additionally, when commanding a backup to a target shard cluster, the backup manager server manages the queue for the shard cluster performing backup through the backup execution queue by inserting the target shard cluster (or information on the target shard cluster) into the backup execution queue. can do.

At step 350, the backup manager server may update the backup standby queue based on the target shard cluster being determined to be unbackupable. The backup manager server can change the backup priority of the target shard cluster that is not available for backup and sort the backup waiting queue based on the changed backup priority. For example, based on determining that the target shard cluster is not available for backup, the backup manager server changes the backup priority of the target shard cluster to the lowest priority among the shard clusters in the backup standby queue, making the target shard cluster standby for backup. It can be inserted at the last position of the queue.

In step 360, the backup manager server may determine whether a new target shard cluster can be backed up based on the completion of the backup of at least one shard belonging to the target shard cluster. As will be described later in FIG. 4, each backup server performs backup of a shard, so when the backup of one shard is terminated, the backup server that performed the backup of the terminated shard can be treated as an available backup server. In other words, even before the backup of all shards belonging to a shard cluster is terminated, if the backup of one shard is terminated, the number of available backup servers may increase. Therefore, the backup manager server can obtain information about the new target shard cluster from the backup standby queue based on the end of the backup of one shard, even if the backup of the shard cluster is not terminated. . The backup manager server can determine whether a new target shard cluster can be backed up.

According to one embodiment, although not explicitly shown in FIG. 3, the backup manager server creates one or a combination of two or more of the backup execution queue or the backup completion queue based on the completion of the backup of all shards belonging to the target shard cluster. It can be updated. For example, the backup manager server can delete information about the target shard cluster from the backup execution queue. The backup manager server can insert the target shard cluster (or information on the target shard cluster) into the backup completion queue.

FIG. 4 is a diagram illustrating an operation of determining whether a target shard cluster can be backed up according to an embodiment.

At step 410, the backup manager server may select a secondary to be used for backup, from among one or more secondary of the shard. For example, the backup manager server can select the secondary to be used for backup by considering the impact of the backup operation on the distributed database. The backup manager server may select the secondary to be used for backup based on the physical location of the secondary. The shorter the distance between secondary and storage, the less traffic can occur between data centers. Illustratively, the data center's backup servers and/or storage (e.g., storage 140 in FIG. 1) exist in region A, the first secondary resides in region A, and the second secondary resides in region B, which is different from region A. If present, the backup manager server may select the first secondary that exists in the same area as the backup server and/or storage (e.g., area A) as the secondary to be used for backup.

For example, a target shard cluster may have three shards (eg, a first shard, a second shard, and a third shard). Each shard can include one primary and two secondary. If at least one secondary is located in area A, the backup manager server may select the secondary in area A, and if no secondary is located in area A, the backup manager server may select the secondary closest to area A. Illustratively, the backup service manager may select the secondary in region A for the first shard, the secondary in region B for the second shard, and the secondary in region A for the third shard as the secondary to be used for backup.

In step 420, the backup manager server may determine whether a shard can be backed up based on the number of available backup servers. The backup manager server can determine whether backup of the corresponding shard is possible based on the number of available backup servers in the data center mapped to the secondary. According to one embodiment, each secondary may be mapped to a data center based on geographic location. Illustratively, the secondary may be mapped to the data center that exists in the closest location.

For example, the backup service manager may select the secondary in region A for the first shard, the secondary in region B for the second shard, and the secondary in region A for the third shard as the secondary to be used for backup. The data center in region A may have one available backup server, and the data center in region B may have two available backup servers. The backup service manager determines that the first and third shards are not available for backup, based on the fact that the number of available backup servers in the data center in region A is 1 and the number of secondary servers to be used for backup in region A is 2. You can. The backup service manager may determine that the second shard is available for backup, based on the fact that the number of available backup servers in the data center in region B is 2 and the number of secondary to be used for backup in region B is 1.

In step 430, the backup manager server may determine that the shard cluster is capable of being backed up, based on determining that one or more shards belonging to the target shard cluster are all capable of being backed up.

In step 440, the backup manager server may determine that the target shard cluster is not capable of backup based on at least one shard belonging to the target shard cluster being determined to be not capable of backup.

As described above, if all shards belonging to a shard cluster are capable of being backed up, the corresponding shard cluster may be capable of being backed up. In other words, if at least one of the shards belonging to a shard cluster is not capable of being backed up, the corresponding shard cluster may not be capable of being backed up. The backup manager server can reinsert a shard cluster that has been determined as not available for backup into the backup waiting queue and re-determine whether or not it is possible to back up later.

Figure 5 is a diagram for explaining a backup command operation of a backup manager server according to an embodiment.

In step 510, the backup manager server may command the shards belonging to the target shard cluster to start backup at a similar time. The backup manager server may command a backup so that the backup start times of one or more shards are within a predetermined time range.

If the backup start times of one or more shards fall within a predetermined time range, a smaller amount of operation logs may be required for recovery of the shard cluster than would otherwise be the case. In other words, if the backup start times of the shards are similar, the maximum difference between the backup start time and recovery time between the shards may be relatively smaller, and the data at the recovery time from the snapshot can be obtained even through the operation log stored for a short period of time. It may be possible to recover. The recovery time may be indicated by a recovery input, or may be the latest time at which recovery is possible based on backup data. On the other hand, if the backup start times of the shards do not fall within the predetermined time range (or if the backup start times of the shards have a relatively large difference), a larger amount is needed to restore data from the snapshot to the recovery time. Computational logs may be required. The backup manager server according to one embodiment orders the backup so that the backup start times of the shards are within a predetermined time range, so that the backup start time of the shards is not within the predetermined time range, so that the backup manager server orders the backup so that the backup start times of the shards are not within the predetermined time range. For shard cluster recovery, a smaller amount of operation logs can be backed up.

In step 520, the backup manager server may command a backup of the operation log to one or more shards belonging to the target shard cluster. As described above in FIG. 3, the backup manager server may command to perform operation log backup prior to snapshot backup. As described above in FIG. 2, the operation log is a log for data replication between the primary and secondary of the shard, and may have an operation time that follows the logical clock.

According to one embodiment, the backup manager server may command a backup of the operation log to one or more shards belonging to the target shard cluster. The shard stores the operation logs by sending the operation logs to the backup server in the data center (e.g., the backup server in the data center mapped to the secondary selected to be used for backup) based on the command to back up the operation logs from the backup manager server. You can back up. The backup server can store the operation log received from the shard in storage. For reference, the shard directly transmits the computation log to the data center's backup server, and the computation log backup operation may not go through an agent between the shard and the data center's backup server.

In step 530, the backup manager server may command a snapshot backup for one or more shards belonging to the target shard cluster after a predetermined length of time has elapsed from the start time of the operation log backup. The backup manager server can command snapshot backup using snapshots for the secondary of each shard. A shard can perform a snapshot backup by sending a snapshot for the secondary selected to be used for backup to the data center's backup server (e.g., a backup server in the data center mapped to the secondary selected to be used for backup). The backup server can store snapshots received from shards in storage. For reference, like the operation log backup operation, the shard directly transmits the snapshot for the secondary to the data center's backup server, and the snapshot backup operation may not go through an agent between the shard and the data center's backup server.

According to one embodiment, the length of time from the operation log backup start time to the snapshot backup start time may be determined to be equal to or longer than the checkpoint period. From the primary data at the checkpoint, that is, from the snapshot of the secondary acquired after the checkpoint, the primary data at the recovery time can be recovered up to the same time of the shard cluster using the operation log. .

The backup manager server according to one embodiment performs backup by transmitting backup data from the shard to the backup server in the data center without going through an agent between the shard and the backup server in the data center, so it can use a large number of backup servers at the same time. Even with backup, bottlenecks due to insufficient agent throughput can be prevented.

FIG. 6 is a diagram illustrating an operation of backing up a plurality of shard clusters in parallel according to an embodiment.

In step 610, the backup manager server may determine whether another shard cluster is capable of being backed up in parallel with the target shard cluster, based on the target shard cluster being determined to be capable of being backed up. If the target shard cluster and other shard clusters can be backed up in parallel, backing up the target shard cluster and other shard clusters together can maximize the utilization of available backup servers and shorten the length of backup time.

According to one embodiment, the backup manager server may change the number of available backup servers based on the backup server to be used for backup of the target shard cluster, based on the target shard cluster being determined to be capable of backup. For example, if two backup servers from the data center in region A are used to back up the target shard cluster, the backup manager server can reduce the number of available backup servers in the data center in region A by 2. After that, the backup manager server can obtain information about the target shard cluster and other shard clusters from the backup standby queue. In other words, the backup manager server can obtain information indicating another shard cluster from the backup standby queue. The backup manager server can determine whether backup of another shard cluster is possible based on the changed number of available backup servers.

The backup manager server can determine whether another shard cluster can be backed up, similar to the operation of determining whether the target shard cluster can be backed up, as described above in FIG. 4.

In step 620, the backup manager server may command to back up the target shard cluster and the other shard cluster in parallel, based on determining that the other shard cluster is capable of being backed up in parallel with the target shard cluster. The target shard cluster and other shard clusters perform backups by transmitting backup data to the backup servers assigned to each shard belonging to the target shard cluster and the shards belonging to other shard clusters, based on commands to back up in parallel from the backup manager server. It can be done.

In step 630, the backup manager server may delete the information of the target shard cluster and the information of other shard clusters from the backup standby queue, based on determining that the other shard cluster is capable of being backed up in parallel with the target shard cluster.

According to one embodiment, when the backup manager server manages the backup execution queue, information on the target shard cluster and information on other shard clusters can be inserted into the backup execution queue.

In step 640, the backup manager server may delete information of the target shard cluster from the backup standby queue based on determining that another shard cluster is not capable of being backed up in parallel with the target shard cluster. The backup manager server can maintain information of other shard clusters in the backup queue. The backup manager server orders backup for the target shard cluster, and later, when the number of available backup servers increases and other shard clusters become available for backup, it can order backup for other shard clusters.

Figure 7 is a diagram for explaining a recovery operation of a backup and recovery system according to an embodiment.

Hereinafter, a recovery command operation for at least a portion of a distributed database (e.g., distributed database 130 of FIG. 1) performed by a recovery manager server (e.g., recovery manager server 122 of FIG. 1) will be described. However, it has been mainly explained that the recovery command operation is performed by a recovery manager server that is independent of the backup manager server that performs the backup command operation, but it is not limited to this, and the backup command operation (e.g., the backup command operation described in FIGS. 3 to 6 It can be performed by a backup manager server that performs backup command operations). In other words, the recovery manager server can be the same server as the backup manager server. Alternatively, the backup manager server and the recovery manager server may be implemented as one server and perform both backup command operations and recovery command operations.

In step 710, the recovery manager server configures the target shard cluster to recover the target shard cluster based on receiving a recovery input for the target shard cluster from an input module (e.g., input module 110 in FIG. 1). You can command a snapshot to be set to the initial recovery state of the target shard cluster.

The recovery input may include a command instructing to perform a recovery command operation on at least a portion of the distributed database (e.g., one or more shard clusters). The backup input may be input from an input module (e.g., input module 110 of FIG. 1) to a system for backup and recovery (e.g., system for backup and recovery 120 of FIG. 1). The system for backup and recovery may forward the recovery input to a recovery manager server (eg, recovery manager server 122 in FIG. 1) based on receiving a recovery command from the input module.

According to one embodiment, the recovery input indicates at least a portion of the distributed database as a target for recovery, and the system for backup and recovery is configured to restore the at least a portion of the distributed database indicated by the recovery input based on receiving the recovery input. Recovery can be ordered. According to one embodiment, the recovery input may not specify the target of the backup, and the system for backup and recovery orders recovery of the entire distributed database based on receiving the recovery input that does not specify the target of the recovery. can do.

Based on receiving a recovery input from the input module, the recovery manager server may command recovery to one or more shard clusters indicated by the recovery input. The shard cluster can receive backup data stored in storage from the backup server based on a recovery command received from the recovery manager server. The recovery manager server may command one or more shards belonging to the shard cluster to set the initial recovery state of the shard based on a snapshot of the backup data. The recovery manager server can restore the entire image of the shard through a snapshot of the shard's secondary. However, the initial recovery state of the target shard cluster may have secondary data rather than primary data before being adjusted based on the operation log. As will be described later, the initial recovery state of the target shard cluster is adjusted based on the operation log, so that the target shard cluster can be restored with the same data as the primary data.

In step 720, the recovery manager server may command to adjust the initial recovery state of the set target shard cluster based on the operation log. The recovery manager can command the shard to recover the data of the shard by adjusting the initial recovery state of the shard based on the operation log. The recovery manager server can restore the data of the target shard cluster from the secondary data of the snapshot to the primary data by adjusting the initial recovery state of the shard set based on the snapshot using the operation log. For example, the recovery manager server can select the operation log for the operation to apply to the snapshot for the secondary. The recovery manager server can restore the shard cluster to the same point by applying the operation corresponding to the selected operation log to the snapshot. The recovery manager server can select the operation log for the operation to be applied to the snapshot by comparing the operation time of the operation log and the reference time of the snapshot. The recovery manager server can recover shards containing primary data by applying the selected operation log to the shard's snapshot.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using a general-purpose computer or a special-purpose computer, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. A computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination, and the program instructions recorded on the medium may be specially designed and constructed for the embodiment or may be known and available to those skilled in the art of computer software. It may be possible. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or multiple software modules to perform the operations of the embodiments, and vice versa.

As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (20)

In the method performed by the backup manager server,
Based on receiving a backup input for one or more sharded clusters, obtaining information of a target sharded cluster from a backup waiting queue sorted according to backup priority;
Determining whether the target shard cluster can be backed up based on whether one or more shards belonging to the target shard cluster can be backed up; and
Based on the target shard cluster being determined to be capable of backup, commanding a backup to one or more shards belonging to the target shard cluster to perform an operation log backup prior to a snapshot backup. step
How to include .
According to paragraph 1,
The step of obtaining information on the target shard cluster is,
Comprising the step of creating the backup standby queue sorted according to a backup priority determined based on one or a combination of two or more of the average backup time length of the shards belonging to the shard cluster or the number of shards belonging to the shard cluster,
method.
According to paragraph 1,
The step of determining whether the target shard cluster can be backed up is:
Selecting, from among the one or more secondary in the shard, a secondary to be used for backup based on the physical location of the secondary,
method.
According to paragraph 1,
The step of determining whether the target shard cluster can be backed up is:
Comprising the step of determining whether backup of the corresponding shard is possible based on the number of available backup servers in the data center mapped to the secondary of the shard belonging to the target shard cluster,
method.
According to paragraph 1,
The step of determining whether the target shard cluster can be backed up is:
determining that the target shard cluster is capable of being backed up, based on determining that one or more shards belonging to the target shard cluster are all capable of being backed up; and
Based on determining that at least one shard belonging to the target shard cluster is not capable of backup, comprising at least one of the steps of determining that the target shard cluster is not capable of backup,
method.
According to paragraph 1,
The step of commanding the backup is,
Including commanding a backup so that the backup start times of the one or more shards are within a predetermined time range,
method.
According to paragraph 1,
The step of commanding the backup is,
Including the step of ordering a backup of the operation log for data replication between the primary and secondary of the shard,
method.
In clause 7,
The step of commanding a backup of the operation log is,
Including commanding a backup of an operation log having an operation time that follows the logical clock of the target shard cluster,
method.
According to paragraph 1,
The step of commanding the backup is,
After a predetermined length of time has elapsed from the start time of the operation log backup, commanding a snapshot backup using a snapshot for the secondary of each shard,
method.
According to paragraph 1,
Based on the target shard cluster being determined to be unavailable for backup, changing the backup priority of the target shard cluster and updating the backup standby queue according to the changed backup priority.
How to further include .
According to paragraph 1,
Determining whether a new target shard cluster can be backed up based on the end of the backup of at least one shard belonging to the target shard cluster.
How to further include .
According to paragraph 1,
Based on the target shard cluster being determined to be capable of backup, determining whether another shard cluster indicated by information obtained from the backup standby queue is capable of being backed up in parallel with the target shard cluster.
How to further include .
According to paragraph 1,
The step of commanding the backup is,
Based on determining that another shard cluster is capable of being backed up in parallel with the target shard cluster, commanding to back up the target shard cluster and the other shard cluster in parallel,
The method is:
Based on determining that another shard cluster is capable of being backed up in parallel with the target shard cluster, deleting information of the target shard cluster and information of the other shard cluster from the backup standby queue.
How to include more.
According to paragraph 1,
Based on it being determined that the other shard cluster is not capable of being backed up in parallel with the target shard cluster, the information of the target shard cluster is deleted from the backup waiting queue and the information of the other shard cluster is placed in the backup waiting queue. steps to maintain
How to further include .
According to paragraph 1,
Based on receiving a recovery input for the target shard cluster, commanding to set a snapshot of the target shard cluster to the recovery initial state of the target shard cluster; and
Commanding to adjust the initial recovery state of the set target shard cluster based on the operation log
How to further include .
According to clause 15,
The step of commanding to adjust the initial recovery state of the target shard cluster is:
Including commanding to restore the data of the shard by adjusting the initial recovery state of the shard set based on the snapshot using the operation log,
method.
In a method performed by a system for backup and recovery of a distributed database,
By the backup manager server, based on receiving backup input for one or more sharded clusters, the target sharded cluster is retrieved from the backup waiting queue, sorted by backup priority. Obtaining information;
determining, by the backup manager server, whether the target shard cluster can be backed up based on whether one or more shards belonging to the target shard cluster can be backed up;
Based on the backup manager server determining that the target shard cluster is capable of backup, operation log backup is performed on one or more shards belonging to the target shard cluster prior to snapshot backup. Commanding the backup to perform;
Commanding, by a recovery manager server, to set a snapshot of the target shard cluster to the recovery initial state of the target shard cluster, based on receiving a recovery input for the target shard cluster; and
Commanding the recovery manager server to adjust the initial recovery state of the set target shard cluster based on the operation log.
How to include .
A computer program combined with hardware and stored in a computer-readable recording medium to execute the method of any one of claims 1 to 17.
On the backup manager server,
Memory storing computer-executable instructions; and
a processor that accesses the memory and executes the instructions;
Including,
The above commands are:
Based on receiving backup input for one or more sharded clusters, obtain information of the target sharded cluster from a backup waiting queue sorted according to backup priority,
Based on whether one or more shards belonging to the target shard cluster can be backed up, determine whether the target shard cluster can be backed up,
Based on the target shard cluster being determined to be capable of backup, a backup command is given to one or more shards belonging to the target shard cluster to perform an operation log backup prior to a snapshot backup.
configured to,
Backup manager server.
In a system for backup and recovery of a distributed database,
A backup manager server including a memory of the backup manager server storing computer-executable instructions, and a processor of the backup manager server that accesses the memory of the backup manager server and executes the instructions; and
A recovery manager server including a memory of the recovery manager server storing computer-executable instructions, and a processor of the recovery manager server that accesses the memory of the recovery manager server and executes the instructions,
The commands stored in the memory of the backup manager server are:
Based on receiving backup input for one or more sharded clusters, obtain information of the target sharded cluster from a backup waiting queue sorted according to backup priority,
Based on whether one or more shards belonging to the target shard cluster can be backed up, determine whether the target shard cluster can be backed up,
Based on the target shard cluster being determined to be capable of backup, a backup command is given to one or more shards belonging to the target shard cluster to perform an operation log backup prior to a snapshot backup.
It is configured to
The commands stored in the memory of the recovery manager server are:
Based on receiving a recovery input for the target shard cluster, command to set a snapshot of the target shard cluster to the recovery initial state of the target shard cluster,
Based on the operation log, command to adjust the recovery initial state of the target shard cluster set above
configured to,
A system for backup and recovery of distributed databases.

Images