CN105094696B - Based on RAID1 and RAID4 mixed structures transfer process data reliability ensuring method and device - Google Patents

Abstract

The present invention relates to the field of computer storage. The present invention proposes a data reliability guarantee method and device based on RAID1 and RAID4 hybrid structure conversion process. When the threshold is reached, the area where the BRD is located is added to the suspend_area list, the area is converted into a sub-area corresponding to multiple data nodes, and a suspend operation request is sent to the sub-area, and the data node receives the After suspending the operation request, suspend the write request of the sub-area; perform RAID4 verification calculation on the data of the sub-area, write the generated verification data into the logical volume device, and send a remapping request to the data node , the data node remaps the BRD of the sub-area to ARP to complete data conversion. The invention reduces the probability that the conversion process from RAID1 to RAID4 and the I/O operation interact with each other in a short time, and ensures the asynchronous degree of conversion process and writing operation.

Description

Data reliability guarantee method and method for conversion process based on RAID1 and RAID4 hybrid structure device

technical field

The invention relates to the field of computer storage, in particular to a data reliability guarantee device for conversion process based on RAID1 and RAID4 hybrid structures.

Background technique

RAID (Redundant Array of Independent Disks) technology is a storage method that provides enhanced redundancy, capacity and storage performance, and has strong manageability, reliability and availability. Through redundant computing, RAID technology can reduce system capacity overhead on the premise of meeting reliability requirements. The research on applying RAID technology to network storage systems has experienced from centralized control to distributed control, from single RAID mode to hybrid RAID During the development of the model, RAID1 technology has high reliability. RAID1 mirroring enables two copies of data in write operations. Even if one of the nodes fails, the copy can also guarantee a correct data. RAID4 redundancy technology occupies less redundant space Store verification information, can tolerate any single node failure, and can restore lost data through data and verification information of other nodes, and the system reliability is relatively good.

In a large-scale network storage system, it is necessary to consider the impact of a complex network environment on performance, so some storage systems use a hybrid RAID mode. AutoRAID puts recently used data on high-performance hard disks stored in RAID1, and less frequently used data on cost-effective hard disks stored in RAID5. DPGADR uses a structure similar to AutoRAID, but is more suitable for distributed network storage environments, and uses replication and delayed check block generation to improve system performance. The Chinese patent No. 201410033455.3 - "RAID1 and RAID4 Mixed Structure Network Storage System and Method" provides a method for centralized redundant management of data nodes at the back-end redundant management node, and the redundant management node is separated from the data channel The front end is moved to the back end of the storage device, so that read and write requests can directly access the storage nodes, reducing the impact of redundant computing on system write performance.

In AutoRAID, RAID1 and RAID5 data are stored separately. Migrating a large amount of data from RAID1 to RAID5 in a network environment will bring huge consumption of network bandwidth and disk bandwidth, which is not suitable for use in a distributed environment. However, the RAID5 part of DPGADR is stored in a degraded mode, and the read and write performance of cold data is poor. The Chinese patent No. 201410033455.3 - "RAID1 and RAID4 mixed structure network storage system and method" proposes a back-end redundant centralized management method to solve the performance bottleneck of the centralized redundant management method, but the data synchronization process and write I The /O operations will affect each other, and the data consistency during the process of converting RAID1 data to RAID4 data cannot be guaranteed.

Based on the above analysis, the hybrid RAID structure network storage technology still has the following limitations: the data conversion process affects the write I/O operation, and the consistency of data conversion cannot be guaranteed.

In view of the above considerations, on the basis of the Chinese patent "RAID1 and RAID4 Mixed Structure Network Storage System and Method" (system structure as shown in Figure 1 and Figure 2), the present invention proposes a conversion process data consistency based on RAID1 and RAID4 mixed structure security method.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention proposes a device for ensuring data reliability during the conversion process based on the mixed structure of RAID1 and RAID4.

The present invention proposes a data reliability guarantee method based on RAID1 and RAID4 hybrid structure conversion process, including:

The suspending step is to obtain the occupied space value of the BRD in the check node, and when the occupied space value reaches the set threshold, add the area where the BRD is located to the suspend_area list, and convert the area into corresponding multiple data nodes and send a suspend operation request to the subarea, and after receiving the suspend operation request, the data node suspends the write request of the subarea;

The conversion step is to perform RAID4 verification calculation on the data of the sub-area, write the generated verification data into a logical volume device, and send a remapping request to the data node, and the data node will transfer the data of the sub-area BRD is remapped to ARP to complete the data conversion.

In the method for ensuring data reliability during the conversion process based on the mixed structure of RAID1 and RAID4, the conversion step includes: performing RAID4 verification calculation on the sub-area that successfully executes the pending operation, and sending the data to the sub-area that successfully executes the pending operation The data node to which it belongs sends a remapping request.

In the method for ensuring data reliability during the conversion process based on the mixed structure of RAID1 and RAID4, the suspending step further includes: sending the suspending operation of the sub-area to the corresponding data node concurrently through multiple tasks.

In the method for ensuring data reliability in the conversion process based on the mixed structure of RAID1 and RAID4, the suspending step further includes: if there is a write request being executed in the sub-area at this time, wait for all write requests to end, and then enter the conversion process. step.

The described data reliability guarantee method based on the RAID1 and RAID4 hybrid structure conversion process also includes a reset step: after the data node completes the remapping operation, the area where the BRD is located is removed from the suspend_area linked list; Send the write request of the suspended sub-area in the data node.

The present invention also proposes a device for ensuring data reliability during conversion based on the mixed structure of RAID1 and RAID4, including:

The suspend module is used to obtain the occupied space value of the BRD in the check node. When the occupied space value reaches the set threshold, the area where the BRD is located is added to the suspend_area list, and the area is converted into a corresponding plurality of a sub-area of the data node, and send a suspend operation request to the sub-area, and after the data node receives the suspend operation request, suspend the write request of the sub-area;

A conversion module, configured to perform RAID4 verification calculation on the data in the sub-area, write the generated verification data into a logical volume device, and send a remapping request to the data node, and the data node will convert the data in the sub-area The BRD is remapped to ARP to complete data conversion.

In the device for ensuring data reliability during conversion based on the hybrid structure of RAID1 and RAID4, the suspend module further includes: a suspend operation of sending sub-areas to corresponding data nodes concurrently through multiple tasks.

In the data reliability guarantee device for conversion process based on RAID1 and RAID4 hybrid structure, the conversion module includes: performing RAID4 verification calculation on the sub-area that successfully executes the pending operation, and sends the data to the sub-area that successfully executes the pending operation The data node to which it belongs sends a remapping request.

In the device for ensuring data reliability during conversion based on the mixed structure of RAID1 and RAID4, the suspending module further includes: if there is a write request being executed in the sub-area at this time, wait for all write requests to be completed before performing the conversion module.

The device for ensuring data reliability during the conversion process based on the mixed structure of RAID1 and RAID4 further includes a reset module, which is used to remove the area where the BRD is located from the suspend_area linked list after the data node completes the remapping operation; Sending the write request of the suspended sub-area in the data node.

As can be seen from the above invention, the present invention has the advantages of:

The present invention reduces the probability that the conversion process from RAID1 to RAID4 and the I/O operation interact with each other in a short time, and ensures the asynchronous degree of the conversion process and the write I/O operation;

On the premise of not affecting the write I/O performance, the data consistency of the conversion from R1 to R4 is guaranteed.

Description of drawings

Fig. 1 is a schematic diagram of a redundancy group of a mixed structure of RAID1 and RAID4;

Fig. 2 is a system structural diagram of hybrid RAID in a redundancy group;

Figure 3 is a schematic diagram of the data node suspend operation process;

Figure 4 is a schematic diagram of the data node resume operation process;

Figure 5 is a schematic diagram of the check node sending suspend and resume operations;

FIG. 6 is a schematic diagram of a data node write request sending query process;

Fig. 7 is a flow diagram of the suspend stage of the conversion process;

Fig. 8 is a schematic diagram of the process flow of the conversion stage of the conversion process;

FIG. 9 is a schematic flow diagram of the resume stage of the conversion process.

Detailed ways

The following are the concrete steps of the inventive method, as follows:

The suspending step is to obtain the occupied space value of the BRD in the check node, and when the occupied space value reaches the set threshold, add the area where the BRD is located to the suspend_area list, and convert the area into corresponding multiple data nodes and send a suspend operation request to the subarea, and after receiving the suspend operation request, the data node suspends the write request of the subarea;

The conversion step is to perform RAID4 verification calculation on the data of the sub-area, write the generated verification data into a logical volume device, and send a remapping request to the data node, and the data node will transfer the data of the sub-area BRD is remapped to ARP to complete the data conversion.

The conversion step includes: performing RAID4 check calculation on the sub-area that successfully executes the pending operation, and sending a remapping request to the data node to which the sub-area that successfully executes the pending operation belongs.

The suspending step further includes: concurrently sending the suspending operation of the sub-area to the corresponding data node through multitasking.

The suspending step further includes: if there are write requests being executed in the sub-area at this time, waiting for all write requests to be completed before entering the conversion step.

It also includes a reset step: after the data node completes the remapping operation, the area where the BRD is located is removed from the suspend_area linked list; the write request of the suspended sub-area in the data node is downloaded send.

The device of the present invention comprises the following modules, as follows:

The suspend module is used to obtain the occupied space value of the BRD in the check node. When the occupied space value reaches the set threshold, the area where the BRD is located is added to the suspend_area list, and the area is converted into a corresponding plurality of a sub-area of the data node, and send a suspend operation request to the sub-area, and after the data node receives the suspend operation request, suspend the write request of the sub-area;

A conversion module, configured to perform RAID4 verification calculation on the data in the sub-area, write the generated verification data into a logical volume device, and send a remapping request to the data node, and the data node will convert the data in the sub-area The BRD is remapped to ARP to complete data conversion.

The suspend module further includes: a suspend operation of concurrently sending sub-areas to corresponding data nodes through multi-task.

The conversion module includes: performing RAID4 verification calculation on the sub-area that successfully executes the pending operation, and sending a remapping request to the data node to which the sub-area that successfully executes the pending operation belongs.

The suspending module further includes: if there is a write request being executed in the sub-area at this time, wait until all write requests are completed, and then enter the conversion step.

The reset module is used for removing the area where the BRD is located from the suspend_area linked list after the data node completes the remapping operation; and downloading the write request of the suspended sub-area in the data node send.

Specific embodiments of the present invention are given below, and the present invention is described in detail in conjunction with the accompanying drawings. The present invention includes two parts: system conversion related structure and conversion process.

System Conversion Related Structure

First, the composition of the RAID1 and RAID4 hybrid structure is described. As shown in the dotted line box of the middle layer in Figure 1, the unit of the RAID1 and RAID4 hybrid structure of the present invention is a redundancy group composed of a group of data structure nodes and a check node. Figure 2 shows the system structure in a redundancy group. The data nodes in the redundancy group are composed of data sending and receiving programs, cache units, mapping units, and logical volumes that actually store data; the check nodes in the redundancy group are composed of data sending and receiving programs. It is composed of program, cache unit, mapping unit, Cache module (high-speed cache module), Suspend module, redundant calculation unit responsible for RAID calculation in the group, and logical volume for storing verification information.

Now explain and explain the main components of the system: Suspend module (suspend module): This module is in the check node, and records a suspend_area list (suspend area list); if the module adds an area to the suspend_area list, Then the Suspend module will notify the data node where the corresponding area is located through MP (Multi Path, multi-path) to suspend the write request for this area; until the Suspend module removes the area from the list, the write request will be sent normally (data node write request The query process is shown in Figure 6).

Mapping unit: The mapping unit includes BRD and ARD devices, and the mapped data are Mirror (mirror) and R4 data respectively.

BRD: Before Raid Device, mirroring device. The device that carries the current data and is also the Prev-RAID consistent data that is about to undergo RAID conversion; that is, the logical volume of the mapping relationship maintained by BRD corresponds to the R1 data that has not been converted into R4.

ARD: After Raid Device, verify the device. The device that carries the Post-RAID consistent data of the data that has completed RAID conversion; that is, the logical volume of the mapping relationship maintained by ARD corresponds to the data that has been converted to R4.

The conversion process of the present invention is as follows:

The conversion process can be divided into three phases: Suspend phase, conversion phase, and Resume phase.

Suspend phase (suspend phase) (as shown in Figure 7):

When the Cache module in the check node finds that the space occupied by the BRD of the check node exceeds a set threshold (the threshold is determined by comprehensively considering the size of the system logical volume, cache size, etc.), the Cache module notifies the Suspend module to add the area to the suspend_area List, perform suspend operation;

After the Suspend module receives the suspend operation, it converts the area into a sub-area of multiple corresponding data nodes, and sends data to the corresponding data nodes concurrently through MP (Multi Path, multi-path, which refers to multi-path sending from the suspend module to each node) Send the suspend operation of the sub-area;

After the data node receives the suspend operation, it suspends the write request of the subsequent corresponding area (the sub-area). If there is a write request in the area at this time, it will wait for all the write requests to end and enter the conversion stage;

Conversion stage (as shown in Figure 8):

The Cache module performs RAID4 verification calculation on the data in the area where the suspend is successfully executed;

After the verification calculation is completed, the generated verification data is written to the logical volume device, and each data node is notified to remap the data area participating in the calculation through WR (Write Remap, which remaps the data in the BRD to the ARD);

After the data node receives the remapping operation sent by WR, it remaps the data in the corresponding area from BRP to ARP. After all data nodes complete the remapping operation, the conversion phase ends;

Resume phase (reset phase) (as shown in Figure 9):

After all data nodes complete the remapping operation, the Cache module notifies the Suspend module to remove the area from the suspend_area linked list;

The Suspend module notifies each data node to resume the original corresponding suspend_area through MP;

After the resume is completed, the write request for the corresponding area suspended in the data node is issued, and the conversion process ends.

Claims (10)

1. one kind is based on RAID1 and RAID4 mixed structure transfer process data reliability ensuring methods, it is characterised in that including：

Step is hung up, the space-consuming value of mirroring device BRD in check-node is obtained, when the space-consuming value reaches setting threshold During value, the region where the mirroring device BRD is added and hangs up zone list suspend_area, the region is converted to The subregion of corresponding multiple back end, and pending operation request is sent to the subregion, the back end receives institute After stating pending operation request, the write request of the subregion is hung up；

Switch process, RAID4 verification calculating is carried out to the data of the subregion, the verification data write-in logical volume of generation is set It is standby, and sent to the back end and remap request, the back end is heavy by the mirroring device BRD of the subregion Calibration equipment ARP is mapped to, to complete data conversion.

2. RAID1 and RAID4 mixed structure transfer process data reliability ensuring methods are based on as claimed in claim 1, its It is characterised by, the switch process includes：RAID4 verifications are carried out to the subregion of successful execution pending operation to calculate, and to institute State the transmission of the back end belonging to the subregion of successful execution pending operation and remap request.

3. RAID1 and RAID4 mixed structure transfer process data reliability ensuring methods are based on as claimed in claim 1, its It is characterised by, the hang-up step also includes：The hang-up that subregion is sent to corresponding back end by Multi-task Concurrency is grasped Make.

4. RAID1 and RAID4 mixed structure transfer process data reliability ensuring methods are based on as claimed in claim 1, its It is characterised by, the hang-up step also includes：If now the subregion has write request and is carrying out, wait all write please After asking end, into the switch process.

5. RAID1 and RAID4 mixed structure transfer process data reliability ensuring methods are based on as claimed in claim 1, its It is characterised by, in addition to reset process：After the back end completes remapping operation, by where the mirroring device BRD Region removes from the suspend_area lists；The write request for the subregion being suspended in the back end is entered Row issues.

6. one kind ensures device based on RAID1 and RAID4 mixed structure transfer processes data reliability, it is characterised in that including：

Suspension module, for obtaining the space-consuming value of mirroring device BRD in check-node, set when the space-consuming value reaches When determining threshold value, the region where the mirroring device BRD is added and hangs up zone list suspend_area, the region is turned The subregion of multiple back end corresponding to being changed to, and pending operation request is sent to the subregion, the data section is checked and accepted After being asked to the pending operation, the write request of the subregion is hung up；

Modular converter, calculated for carrying out RAID4 verifications to the data of the subregion, the verification data write-in logical volume of generation Equipment, and sent to the back end and remap request, the back end is by the mirroring device BRD of the subregion Calibration equipment ARP is remapped to, to complete data conversion.

7. device is ensured based on RAID1 and RAID4 mixed structure transfer processes data reliability as claimed in claim 6, its It is characterised by, the suspension module also includes：The hang-up that subregion is sent to corresponding back end by Multi-task Concurrency is grasped Make.

8. device is ensured based on RAID1 and RAID4 mixed structure transfer processes data reliability as claimed in claim 6, its It is characterised by, the modular converter includes：RAID4 verifications are carried out to the subregion of successful execution pending operation to calculate, and to institute State the transmission of the back end belonging to the subregion of successful execution pending operation and remap request.

9. device is ensured based on RAID1 and RAID4 mixed structure transfer processes data reliability as claimed in claim 6, its It is characterised by, the suspension module also includes：If now the subregion has write request and is carrying out, wait all write please After asking end, the modular converter is performed.

10. device is ensured based on RAID1 and RAID4 mixed structure transfer processes data reliability as claimed in claim 6, its It is characterised by, in addition to reseting module, after completing remapping operation for the back end, by the mirroring device BRD institutes Region removed from the suspend_area lists；Please by writing for the subregion being suspended in the back end Ask and issued.