JP2016115036A - Data storage control device, data storage control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To safely store meta information to be used for the restoration of data in a data storage control device for storing data in a plurality of data storage servers in a distributed manner.SOLUTION: A data storage control device includes: data distribution and storage processing means for dividing data to be stored to generate a plurality of divided data, and for storing the plurality of divided data in a plurality of data storage servers in a distributed manner, and for storing meta information to be used for restoring original data from the plurality of divided data in a meta information storage part; meta information distribution and storage processing means for dividing the meta information to generate a plurality of divided meta information, and for storing the plurality of divided meta information in a plurality of data storage servers in a distributed manner; and meta information restoration processing means for acquiring the plurality of divided meta information from the plurality of data storage servers, and for restoring the original meta information from the plurality of divided meta information, and for storing the meta information in the meta information storage part.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for distributing data and storing it in a plurality of storages.

  There is a secret sharing technique as a technique for securely storing important data. For example, in the threshold secret sharing method (see Non-Patent Document 1), the original data is divided into n pieces of divided data (which may be referred to as fragment data), and k (k <n) pieces of divided data can be collected. For example, the original data can be restored, but the original data cannot be restored with k-1 pieces of divided data. Therefore, even if up to k−1 pieces of divided data are stolen, the original data does not leak, and even if up to n−k pieces of divided data are lost or destroyed, the original data can be restored.

  There is provided a service for storing data by distributing to a plurality of storages (data storage servers) using the secret sharing technique as described above. The threshold secret sharing method is an example of a secret sharing technique, and other secret sharing techniques may be used.

  When data is divided as described above and the divided data is stored in a plurality of data storage servers, in order to restore the original data from the divided data, each divided data storage location (stored server address, server Meta information such as secret sharing algorithm used at the time of data division is required.

  The meta information is generated when data is distributedly stored, and is stored in a database in a system that performs data distributed storage, for example. Since the meta information is important information necessary for data restoration, backup data of the meta information is stored in case the meta information is lost due to a failure in the database.

Hiroshi Yamamoto: "(k, L, n) threshold secret sharing system" IEICE Transactions, vol. 168-A, no. 9, pp. 945-952 (Sep. 1985)

  The meta information described above is important information necessary for data restoration, and if the meta information can be known, the original data can be restored. Therefore, the meta information is highly confidential information. is there. However, a storage method that stores meta information in one place and has a backup as in the above prior art cannot be said to be sufficiently safe as a meta information storage method.

  The present invention has been made in view of the above points, and in a data storage system in which data is distributed and stored in a plurality of data storage servers, meta information used for data restoration is safely stored. The purpose is to provide the technology that enables it.

According to an embodiment of the present invention, a data storage control device for storing data distributed to a plurality of data storage servers,
Divide the data to be distributed and generate multiple divided data, distribute the multiple divided data to multiple data storage servers, and use it to restore the original data from multiple divided data Data storage processing means for storing the meta information to be stored in the meta information storage unit;
Meta information distributed storage processing means for dividing the meta information to generate a plurality of divided meta information, and distributing and storing the plurality of divided meta information in a plurality of data storage servers;
Meta information restoration processing means for obtaining a plurality of divided meta information from the plurality of data storage servers, restoring the original meta information from the plurality of divided meta information, and storing the meta information in the meta information storage unit; A data storage control device is provided.

  According to an embodiment of the present invention, there is a technique that enables safe storage of meta information used for data restoration in a data storage system that stores data in a plurality of data storage servers in a distributed manner. Provided.

1 is a configuration diagram of a system according to an embodiment of the present invention. It is a block diagram of the data storage control apparatus 100 which concerns on embodiment of this invention. It is a flowchart for demonstrating a data dispersion | distribution storage process. It is a flow chart for explaining meta information restoration processing. It is a flowchart for demonstrating the other example of a meta-information decompression | restoration process.

  Embodiments of the present invention will be described below with reference to the drawings. The embodiment described below is only an example, and the embodiment to which the present invention is applied is not limited to the following embodiment. For example, in the embodiment described below, a system that stores data by distributing it to a plurality of data storage servers using the secret sharing method is described as an example. However, the present invention is a method other than the secret sharing method. Thus, the present invention can also be applied to a technique in which data is divided and distributed to a plurality of data storage servers.

(Whole system configuration)
FIG. 1 shows the overall configuration of a system according to an embodiment of the present invention. As shown in FIG. 1, the present system has a configuration in which a data storage control device 100 and a plurality of data storage servers 20 are connected via a communication network 30 (eg, the Internet). A user terminal 10 is connected to the data storage control device 100.

  The data storage control device 100 has a function of distributing and storing data to be distributed and received from the user terminal 10 in a plurality of data storage servers 20 using a secret sharing method, and a plurality of divided data stored in a distributed manner In addition to the function of restoring the original data from and providing the restored data to the user terminal 10, the function of distributing and storing the meta information to the plurality of data storage servers 20 using the secret sharing method, and A function of restoring original meta information from a plurality of stored divided meta information is included. Details of the data storage control device 100 will be described later.

  For example, the data storage control device 100 and the user terminal 10 may be connected via a LAN or may be connected via the Internet. Each data storage server 20 may be a physical individual server or a virtual server on the cloud.

(Configuration of data storage control device 100)
FIG. 2 shows a functional configuration of the data storage control device 100 according to the embodiment of the present invention. As shown in FIG. 2, the data storage control device 100 includes a data distribution storage processing unit 101, a data restoration processing unit 102, a meta information distributed storage processing unit 103, a meta information restoration processing unit 104, and a meta information storage unit 105. .

  The data distributed storage processing unit 101 divides the data to be distributed stored received from the user terminal 10 to generate a plurality of divided data, and distributes the plurality of divided data to the plurality of data storage servers 20 and stores them. In the present embodiment, one piece of divided data is stored in one data storage server 20, and each piece of divided data is stored in a different data storage server 20. However, such a storage form is an example.

  Where in which data storage server 20 each divided data is stored may be determined in advance or may be determined by a predetermined rule (eg, randomly). The “data storage server 20 and the storage location in the server” that stores the divided data will be referred to as a divided data storage location.

  The distributed data storage processing unit 101 also has a function of generating meta information including a divided data storage location for each divided data and storing the meta information in the meta information storage unit 105 when performing distributed data storage processing. .

  The data restoration processing unit 102 uses the meta information stored in the meta information storage unit 105 to acquire a plurality of pieces of divided data from the plurality of data storage servers 20 and restores the original data from the plurality of pieces of divided data. To do.

  The meta information distributed storage processing unit 103 reads out the meta information generated by the distributed storage processing and stored in the meta information storage unit 105 immediately after the data distributed storage processing is performed by the data distributed storage processing unit 101, for example. The meta information is divided by the secret sharing method to generate a plurality of pieces of divided meta information, and the plurality of pieces of divided meta information are distributed and stored in the plurality of data storage servers 20. In the present embodiment, one piece of divided meta information is stored in one data storage server 20, and each piece of divided meta information is stored in different data storage servers 20. However, such a storage form is an example.

  It is determined in advance in which data storage server 20 each divided meta information is stored. The “data storage server 20 and the storage location in the server” that stores the divided meta information will be referred to as a divided meta information storage location. In the present embodiment, the storage location of the divided meta information is determined in advance, and therefore it is not necessary to store the meta information for restoring the original meta information from the plurality of divided meta information in a database or the like.

  It should be noted that meta information other than the storage location such as a secret sharing algorithm and parameters used for secret sharing processing is added to the divided meta information as a header, and the divided meta information with the header added is stored in each data storage server 20. Also good. In this method, meta information such as a secret sharing algorithm and parameters can be acquired by referring to the header. However, it is not essential to add such a header. In other words, the header can be made unnecessary by using only predetermined meta information such as secret sharing algorithm and parameters.

  The meta information restoration processing unit 104 acquires a plurality of pieces of divided meta information from the plurality of data storage servers 20 based on a predetermined storage location of the divided meta information, and obtains the original meta information from the plurality of pieces of divided meta information. The meta information is restored and stored in the meta information storage unit 105.

  The data storage control device 100 according to the present embodiment can be realized, for example, by causing one or a plurality of computers (servers) to execute a program describing the processing content described in the present embodiment. That is, the functions of the data storage control device 100 are realized by executing a program corresponding to the processing executed by the data storage control device 100 using hardware resources such as a CPU and a memory built in the computer. Is possible. Further, the program can be recorded on a computer-readable recording medium (portable memory or the like), stored, or distributed. It is also possible to provide the program through a network such as the Internet or electronic mail.

  As a more specific example, the data distributed storage processing unit 101, the data restoration processing unit 102, the meta information distributed storage processing unit 103, and the meta information restoration processing unit 104 are configured by one or a plurality of computers, and the program The meta information storage unit 105 may be configured by another computer (such as a database server).

(Operation example of the data storage control device 100)
Next, an operation example of the data storage control apparatus 100 configured as shown in FIG. 2 will be described with reference to the flowcharts of FIGS.

<Distributed data storage processing>
First, an operation example of the data storage control device 100 in the data distribution storage process will be described with reference to FIG.

  When the data distributed storage processing unit 101 receives the data to be distributed stored from the user terminal 10 (step 101), the data distributed storage processing unit 101 divides the data by the secret sharing method to generate a plurality of divided data (step 102). Are divided and stored in a plurality of data storage servers 20 (step 103). Further, the data distribution storage processing unit 101 stores meta information such as the divided data storage location in the meta information storage unit 105 (step 104).

  The data distributed storage processing unit 101 notifies the meta information distributed storage processing unit 103 that the data distributed storage processing has been completed. Alternatively, when the meta information distributed storage processing unit 103 is configured by a program module or the like, the data distributed storage processing unit 101 calls the meta information distributed storage processing unit 103 when the data distributed storage processing ends. Also good.

  The meta information distributed storage processing unit 103 notified of the completion of the data distributed storage process acquires the meta information from the meta information storage unit 105, and divides the meta information by the secret sharing method to obtain a plurality of pieces of divided meta information. Is generated (step 105), and the plurality of pieces of divided meta information are stored in a predetermined divided meta information storage location (step 106).

  The meta information to be distributed and stored in the above steps 105 and 106 may be all meta information (the entire meta information database), or only meta information related to the current data, or this time. It may be the entire meta information of the user of the data or other units. In addition, when meta information regarding individual data / user is targeted, which data / user's meta information (division meta information) can be identified by, for example, a file name.

  In addition, in the example of FIG. 3, the distributed storage process of meta information is performed by using the distributed storage of data as a trigger. However, the distributed storage process of meta information may be performed according to an explicit instruction, or periodically. Alternatively, meta information distributed storage processing may be performed.

  As described above, the meta information is distributed and stored in the plurality of data storage servers 20 using the secret sharing method, so that the meta information in the meta information storage unit 105 disappears due to a failure or the like, and the plurality of data Even if some of the storage servers have a failure, if the plurality of data storage servers 20 having the same settings regarding the storage of the divided meta information are prepared, the original meta information is obtained from the divided meta information. Can be obtained.

<Meta information restoration process>
Next, an operation example of the data storage control device 100 in the meta information restoration process will be described with reference to FIG.

  For example, when the meta information restoration processing unit 104 receives a meta information restoration instruction from the user terminal 10 (step 201), the meta information restoration processing unit 104 acquires a plurality of pieces of meta information from a predetermined divided meta information storage location (step 202). Then, the meta information restoration processing unit 104 restores the original meta information from the plurality of divided meta information (step 203), and stores the meta information in the meta information storage unit 105 (step 204).

  In addition, as described above, meta information may be restored by an explicit restoration instruction, and it is detected that meta information cannot be used due to loss of meta information or the like when restoring data as described below. The meta information may be automatically restored by using as a trigger. An example of the operation in this case will be described with reference to FIG.

  When the data restoration processing unit 102 receives a data restoration instruction from, for example, the user terminal 10 (step 301), the meta information storage unit 105 is referred to, thereby searching for meta information corresponding to the data (step 302). . If there is meta information (Yes in step 302), the process proceeds to step 306. If there is no meta information (No in step 302) due to loss of the meta information database, the process proceeds to step 303.

  In step 303, a meta information restoration instruction is issued from the data restoration processing unit 102 to the meta information restoration processing unit 104, and the meta information restoration processing unit 104 acquires a plurality of pieces of meta information from a predetermined divided meta information storage location. To do. Then, the meta information restoration processing unit 104 restores the original meta information from the plurality of divided meta information (step 304), stores the meta information in the meta information storage unit 105, and restores the data that the meta information is stored. The processing unit 102 is notified (step 305).

  The data restoration processing unit 102 acquires a plurality of divided data using the meta information and restores the original data (step 306).

  In the case of No in step 302, the data restoration processing unit 102 displays an error on a terminal such as an administrator terminal, transmits an instruction by a command or the like from the terminal to the data restoration processing unit 102, and the instruction The meta information may be restored according to the above.

(Summary of embodiment)
As described above, in the present embodiment, a data storage control device that stores data distributed to a plurality of data storage servers, and divides the data to be distributed and generates a plurality of divided data. In addition, the plurality of divided data is distributed and stored in a plurality of data storage servers, and the meta information used for restoring the original data from the plurality of divided data is stored in the meta information storage unit. A processing unit; a meta information distributed storage processing unit that divides the meta information to generate a plurality of divided meta information; and stores the plurality of divided meta information in a plurality of data storage servers; A meta information that acquires a plurality of pieces of divided meta information from the storage server, restores the original meta information from the plurality of pieces of divided meta information, and stores the meta information in the meta information storage unit. Data storage control device and a retaliation original processing means.

  Each divided meta information may be stored in a predetermined storage location in the data storage server, and the meta information restoration processing means may acquire the divided meta information from a predetermined storage location in each data storage server. .

  Further, according to the present embodiment, there is provided a data storage control method executed by a data storage control device that stores data distributed to a plurality of data storage servers, and divides the data to be distributed and stores a plurality of divided data. Data distribution for generating and storing the plurality of divided data in a plurality of data storage servers and storing meta information used for restoring the original data from the plurality of divided data in the meta information storage unit A storage processing step, a meta information distributed storage processing step of dividing the meta information to generate a plurality of divided meta information, and distributing and storing the plurality of divided meta information in a plurality of data storage servers; Acquire a plurality of divided meta information from the data storage server, restore the original meta information from the plurality of divided meta information, and store the meta information in the meta information Data storage control method and a meta-information restoration processing step of storing in is provided.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

DESCRIPTION OF SYMBOLS 10 User terminal 20 Data storage server 30 Communication network 100 Data storage control apparatus 101 Data dispersion | distribution storage process part 102 Data restoration process part 103 Meta information dispersion | distribution storage process part 104 Meta information restoration process part 105 Meta information storage part

Claims (4)

A data storage control device for storing data distributed to a plurality of data storage servers,
Divide the data to be distributed and generate multiple divided data, distribute the multiple divided data to multiple data storage servers, and use it to restore the original data from multiple divided data Data storage processing means for storing the meta information to be stored in the meta information storage unit;
Meta information distributed storage processing means for dividing the meta information to generate a plurality of divided meta information, and distributing and storing the plurality of divided meta information in a plurality of data storage servers;
Meta information restoration processing means for acquiring a plurality of divided meta information from the plurality of data storage servers, restoring the original meta information from the plurality of divided meta information, and storing the meta information in the meta information storage unit; A data storage control device comprising: Each divided meta information is stored in a predetermined storage position in the data storage server, and the meta information restoration processing means acquires the divided meta information from a predetermined storage position in each data storage server. The data storage control device according to claim 1. A data storage control method executed by a data storage control device for storing data distributed to a plurality of data storage servers,
Divide the data to be distributed and generate multiple divided data, distribute the multiple divided data to multiple data storage servers, and use it to restore the original data from multiple divided data Distributed data storage processing step for storing the meta information to be stored in the meta information storage unit;
A meta information distributed storage processing step of dividing the meta information to generate a plurality of divided meta information, and distributing the plurality of divided meta information to a plurality of data storage servers;
A meta information restoration processing step of acquiring a plurality of divided meta information from the plurality of data storage servers, restoring the original meta information from the plurality of divided meta information, and storing the meta information in the meta information storage unit; A data storage control method comprising:   The program for functioning a computer as each means in the data storage control apparatus of Claim 1 or 2.

Images