CN101751415B - Metadata service system, metadata synchronization method and write server update method - Google Patents

Abstract

The invention discloses a metadata service system, a metadata synchronization method and a write server update method. The metadata service system provided by the present invention includes: a write server and a read server; the write server is used to store metadata in the parallel file system and accept read access; and accept write access to modify the metadata in the parallel file system, And synchronously update the modified metadata to the reading server; the reading server is used to store the metadata and accept read access; and accept the synchronous update of the metadata from the writing server; it is also used to monitor the failure of the writing server, Convert to write service. The invention solves the problem of a single point of failure by using the read server and the write server as backups for each other, and can satisfy a large number of high-efficiency concurrent access requirements.

Description

Metadata service system, metadata synchronization method and write server update method

technical field

The invention relates to distributed storage in cluster computing, in particular to a metadata service system for storing, modifying and reading metadata in a parallel file system, a method for synchronizing metadata and a method for updating a write server.

Background technique

A file system is a subsystem used for data storage and reading in a computer system, and is generally built on a disk. In network application scenarios, computing and storage are often separated, that is, distributed file systems are accessed through the network. For high-performance computing and high-load network service applications that use a large number of computers to work in parallel, it is often necessary to use multiple servers dedicated to storage to jointly provide file system services to meet access bandwidth requirements.

A commonly used high-performance parallel file system architecture is to store the "metadata" such as the index information of the file system in the metadata server, and store the data in other data servers. The client program only needs to retrieve a small amount of data from the metadata server. Index information, most of the time, directly interacts with the data server. This architecture is simple in structure, can provide high access bandwidth, and is suitable for most applications.

However, the metadata server of this architecture is a single point, and its failure will cause the entire file system to be unavailable, so there is an availability problem and it cannot provide carrier-class services. At the same time, as a single point, the performance of the metadata server may become a performance bottleneck in the face of high-density peak access.

In the current parallel file system architecture of multi-data servers, most of them are active and standby servers composed of shared storage devices. The synchronization between the active and standby servers is completed by means of shared storage devices. , some state information will be lost.

In addition, some parallel file systems in the prior art access index information located on multiple servers through relay multi-hop queries, such as PVFS2, but this will increase the query delay and make it difficult to ensure that the information on multiple servers in the file system consistency.

To sum up, in the prior art, the metadata management in the parallel file system has the following disadvantages:

1. The existing parallel file system that provides metadata services by a single server does not have high availability, and the failure of the metadata server will cause the entire file system to be unavailable.

2. The metadata server that uses shared storage devices to form a primary and secondary structure will lose state data when a switchover occurs. Moreover, only one server provides services at the same time, and it is difficult to provide sufficient load capacity when facing a large number of concurrent accesses.

3. Multi-node distributed storage of metadata, that is, the way of providing multiple metadata server services by querying each other for information will affect the efficiency of access and increase the delay of access. At the same time, it does not solve the synchronization problem of state data.

Contents of the invention

The invention provides a metadata service system, which solves the problem of single point of failure and can meet the requirements of high-efficiency massive concurrent access.

According to the metadata service system provided by the present invention, the present invention also provides a metadata synchronization method, which realizes the synchronous update of metadata stored between servers and ensures the consistency of data stored by each server.

According to the metadata service system provided by the present invention, the present invention also provides a method for updating a write server. When the write server fails, the read server is converted to the write server to solve the single point failure problem.

The metadata service system provided by the present invention includes: a write server, at least two read servers and an arbitration service device;

The write server is used to store metadata in the parallel file system, accept read access; and accept write access, modify the metadata, and synchronously update the modified metadata to the read server;

The read server is used to store the metadata and accept read access; and accept the synchronous update of the metadata by the write server; it is also used to monitor whether the write server is invalid, and when it is detected that the write server is invalid Each of the read servers is further configured to initiate an arbitration request to the arbitration service device, and according to the succession order determined by the arbitration service device, the read server with the largest update sequence number is converted to the write server;

The arbitration service device is configured to, according to the arbitration request initiated by the read server, arrange for each read server to inherit the succession order of the write server;

Wherein, the update sequence number is the corresponding sequence number assigned by the write server to the metadata updated each time, and the update sequence is carried when the modified metadata is synchronously updated to the read server Number.

The corresponding read server converted to the write server is also used to synchronize the last modified metadata to the rest of the read servers.

The arbitration service device is an entity device independent of the write server and the read server; or

The arbitration service device is integrated in the read server.

When the arbitration service device is an entity device independent of the write server and the read server, it is also used to store the information of the write server and the read server; receive the server information query request initiated by the client, and return to the write server and read server information.

The metadata synchronization method provided by the present invention is applied to the metadata service system provided by the present invention, including:

The write server accepts the write request, writes the modified metadata into the temporary area, and sends a write request notification to the read server;

After the read server receives the write request notification, it prohibits operations on the corresponding metadata before modification, or prohibits operations after waiting for the corresponding metadata that is being read to be read; The write server returns a notification receipt response;

After the write server receives the notification reception response returned by the read server, send the metadata stored in the temporary area to the read server; and use the metadata stored in the temporary area, Update the corresponding metadata records stored locally;

The read server updates the corresponding metadata record stored locally, and releases the prohibited operation after the update is successful.

Also includes:

The write server monitors whether the read server fails;

The write server only sends the write request notification to the current non-failure read server; and after determining that the current non-failure read server returns a notification reception response, sends the metadata stored in the temporary area to the current A non-failed read server.

If the read server detects that the write server is invalid after the prohibition operation is completed, the prohibition operation is lifted.

The write server update method provided by the present invention is applied to the metadata service system provided by the present invention, including:

the read server monitors the current state of the write server;

When the read server detects that the write server fails, each of the read servers initiates an arbitration request to the arbitration service device, and the arbitration service device arranges an inheritance sequence for each read server to inherit the write server, according to the In the order of inheritance, the read server with the largest update sequence number is converted to the write server, wherein the update sequence number is the corresponding sequence number assigned by the write server to the metadata of each update in sequence, and carrying the update sequence number when synchronously updating the modified metadata to the read server

The above write server update method specifically includes:

When the read server detects that the write server is invalid, it initiates an arbitration request for locking to the arbitration service device, carrying the latest update sequence number stored locally;

The arbitration service device stores the latest update serial number of each read server, arranges the inheritance order of each read server to inherit the write server, and assigns a lock to the read server corresponding to the first inheritance order;

The read server that obtains the lock judges whether the latest update sequence number stored locally is the largest update sequence number among them according to the latest update sequence numbers of each read server stored in the arbitration service device; write server; otherwise, yield the lock to the next reader server in the succession order.

When the arbitration service device is integrated in the read server, the arbitration service device integrated in any one of the read servers is preset as an authorized arbitration service device, and the authorized arbitration service device receives the arbitration ask;

When the read server to which the authorized arbitration service device belongs is converted into a write server, the read server randomly selects one of the arbitration service devices integrated in the other read servers as the authorized arbitration service device, and notifies each reader server.

The invention consists of a write server and a read server to form a metadata service system. The write server can not only accept read access, but also accept write access, modify the metadata in the parallel file system, and update the modified metadata to the read server synchronously; the read server stores metadata and accepts read access; and accepts the write server Synchronous update of metadata; when the read server detects that the write server fails, it will switch to the write server. Since the metadata service system provided by the present invention is provided with a read server and a write server for mutual backup, the problem of single point failure is effectively solved. Through data synchronization, each read server stores consistent metadata, which can back up each other and meet a large number of concurrent access requirements; and because each server is set in parallel, there is no need for multi-hop queries, and each read server can directly receive read access from clients , high read efficiency.

Description of drawings

Fig. 1 is one of the structural diagrams of the metadata service system provided by the embodiment of the present invention;

Fig. 2 is the second schematic diagram of the structure of the metadata service system provided by the embodiment of the present invention;

Fig. 3 is the third structural diagram of the metadata service system provided by the embodiment of the present invention;

FIG. 4 is a flowchart of a method for synchronously updating metadata provided by an embodiment of the present invention;

FIG. 5 is one of the flow charts of the write server update method provided by the embodiment of the present invention;

FIG. 6 is the second flowchart of the write server update method provided by the embodiment of the present invention;

Fig. 7 is a signaling flow chart of a specific example of the write server update method.

Detailed ways

Embodiments of the present invention provide a metadata service system, a method for synchronously updating metadata, and a method for updating server writes. Through multiple servers serving as backups for each other, the synchronization update of metadata stored between servers and the update of write servers solve the problem of single point failure problem, and can meet the high efficiency of a large number of concurrent access requirements.

The system and method provided by the present invention will be described in detail below with reference to the accompanying drawings.

Referring to FIG. 1 , it is one of the schematic structural diagrams of the metadata service system provided by the embodiment of the present invention, including: a write server 11 and a read server 12; wherein:

The write server 11 is used to store metadata in the parallel file system and accept read access; and accept write access, modify the metadata in the parallel file system, and synchronously update the modified metadata to the read server 12;

The read server 12 is used to store the metadata in the parallel file system, and accepts read access; and accepts the synchronous update of the metadata by the write server 11; it is also used to monitor whether the write server 11 fails, and when the write server 11 is detected to fail, The read server 12 is converted into a write server.

In the system shown in Figure 1, a write server and a read server are configured. Both the write server and the read server can accept read access; the write server can implement metadata modification and data synchronization; when the write server fails, the read server is converted into a write server to accept read access and write access, solving the single point of failure problem.

In one embodiment, the metadata service system further includes an arbitration service device 13, and at least two read servers 12 are configured, as shown in FIG. The physical device of the server 12. The functions of each part are as follows:

The write server 11 is used to store metadata in the parallel file system and accept read access; and accept write access, modify the metadata in the parallel file system, and synchronously update the modified metadata to each read server 12;

The read server 12 is used to store metadata and accept read access; and accepts the synchronous update of the metadata by the write server 11; it is also used to monitor whether the write server 11 is invalid, and when the write server 11 is detected to be invalid, start the arbitration service device 13 ;

The arbitration service device 13 is configured to determine that one of the read servers 12 is converted into a write server.

In an embodiment, the arbitration service device 13 can also store the relevant information of the write server 11 and the read server 12; receive the server information inquiry request initiated by the client, and return the information of the write server and the read server. The client can initiate a read request to any read server and a write request to the write server according to the returned server information.

In practical applications, the write server and the read server may be servers with the same function. When a server is used as a write server, it needs to perform the related functions of the write server, that is: in addition to storing metadata in the parallel file system and accepting read access, it also accepts write access, modifies the metadata in the parallel file system, and Synchronously update the modified metadata to each read server. When a server is used as a read server, metadata is stored and read access is accepted. Each server used as a read server has a write server-related function, but the related function is not enabled or is temporarily idle when used as a read server. When the write server in the system fails, any read server can be converted into a write server.

Referring to FIG. 3 , it is another structural diagram of a metadata service system provided by an embodiment of the present invention, including: a write server 11 , at least two read servers 12 and an arbitration service device 13 . Wherein, the difference from the structure shown in FIG. 2 is that the arbitration service device 13 is not an independently configured physical device, but is integrated in each read server. The arbitration service device 13 integrated in each read server can be a pure software module, or a corresponding functional module combining software and hardware.

In the following, the metadata synchronous update process and the write server update process when the write server fails will be specifically described in conjunction with the metadata service system provided by the above-mentioned embodiments of the present invention.

Referring to Figure 4, it is a flow chart of the metadata synchronization update method, which specifically includes:

Step S401, the write server accepts the write request, and writes the modified metadata into the temporary area;

Step S402, the write server sends a write request notification to the read server;

Step S403. After receiving the write request notification, the reading server prohibits operations on the corresponding metadata before modification, or prohibits operations after waiting for the corresponding metadata being read to be read;

Step S404, returning a notification reception response to the write server after the prohibition operation is completed;

Step S405, after the write server receives the notification reception response returned by the read server, it packages and sends the metadata stored in the temporary area to the read server; and uses the metadata stored in the temporary area to update the corresponding metadata records stored locally;

Step S406, the reading server updates the corresponding metadata record stored locally, and releases the prohibited operation after the update is successful.

The above step 402 can be executed after the write server accepts the write request, that is, in the process of writing the modified metadata into the temporary area, it can send a write request notification to the read server without waiting for all the modified metadata After being written, step S402 is executed.

In step S403, the corresponding metadata is prohibited from being read; in order to prevent the read operation from being performed simultaneously during the subsequent data update process, causing the read data to be wrong.

In step S406, after the read server successfully updates the corresponding metadata record stored locally, it releases the prohibition operation in time, and accepts the metadata read access again. When more than one read server is configured in the metadata service system, even if the update of other read servers has not been completed, the corresponding metadata of these read servers in an unstable state is not allowed to be accessed, so there will be no client The process accesses inconsistent metadata, thus ensuring the consistency of the file system metadata after modification.

In an embodiment, the write server and each read server monitor each other's status, and can report the monitored status information to each other. In a specific application, a status monitoring program may be run in the write server and each read server (the specific monitoring method is in the prior art, and will not be described in detail here). Before the write server sends a write request notification to the read server, the write server determines whether any read server fails according to the current monitoring result; only sends the write request notification to the currently unfailed read server. When the write server receives the notification reception response returned by the reading server, according to the current monitoring results, after determining that all the reading servers that are not invalid have returned the notification reception response, it will package the metadata stored in the temporary area and send it to the currently unavailable read server. Failed read servers.

By monitoring each other's status between the write server and each read server, it is possible to ensure the smooth progress of metadata synchronization updates, including:

1. If the read server fails during the prohibition operation, after the write server is informed, it does not need to wait for the failed read server to return a notification receipt response, and can continue to synchronize metadata with the rest of the non-failed read servers.

2. If the read server detects that the write server is invalid during the prohibition operation, the read servers will notify each other and release the prohibition operation. This write operation failed without leaving the file system in an unstable state.

3. If the write server fails during the process of data packaging and sending, then the reading server that has received the updated metadata that is packaged and sent can complete the update; if there are still reading servers that have not received the updated metadata, then you can The subsequent write server re-synchronizes to it (detailed in the write server synchronization method described later), to ensure the synchronization of metadata stored in each read server.

4. After the metadata is packaged and sent to the reading server, if a reading server fails, it will not affect the data update of other reading servers, and the writing server can know the failed reading server in time, and can judge without waiting for the reading server to return information The entire data synchronization process is over.

In summary, under various possible circumstances, the metadata synchronization method provided by the present invention can maintain the consistency of metadata in the file system. At the same time, all read servers can provide read access services at the same time, which is highly applicable to a large number of concurrent, especially burst access.

According to the metadata service system provided by the above-mentioned embodiments of the present invention, the present invention also provides a write server update method, the specific implementation process of which is shown in Figure 5, including:

Step S501, the read server monitors the current state of the write server;

Step S502, judging whether the write server is invalid, if not, go to step S501; if the write server is invalid, execute step S503;

Step S503, the read server is converted into a write server.

When the metadata system also includes an arbitration service device, and at least two read servers are configured, and the read server detects that the current write server is invalid, the specific process for updating the write server is shown in Figure 6, including:

Step S601 , the read server initiates an arbitration request to the arbitration service device, carrying the latest updated serial number stored locally.

Among them, the update sequence number is the corresponding sequence number assigned by the write server to the metadata updated each time, and the update sequence number is carried when the modified metadata is synchronously updated to the read server, and each read server locally Save the latest update serial number.

In this embodiment, the read server initiates an arbitration request to the arbitration service device, which is an arbitration request requesting locking.

Step S602, the arbitration service device stores the latest update sequence number of each read server, and arranges for each read server the succession sequence of the write server.

The method for sorting the succession order may be arranged by the arbitration service device according to the order of the received arbitration requests, or may be randomly arranged by the arbitration service device.

Step S603, the arbitration service device allocates locks to the read servers corresponding to the first inheritance sequence.

Step S604 , the read server currently obtaining the lock compares the latest update sequence number stored locally with the latest update sequence number of each read server stored in the arbitration server device.

Step S605, judging whether the latest update serial number stored locally is the largest update serial number; if yes, execute step S606; otherwise, execute step S607;

Step S606, the currently locked read server transforms itself into a write server, and the process ends.

In step S607, the read server that currently obtains the lock yields the lock to the next read server in the succession sequence, and proceeds to step S604.

Referring to FIG. 7 , it is a signaling flowchart of a specific example of a write server update method. Described as follows:

Assuming that the metadata service system is equipped with Read Server 1, Read Server 2, Read Server 3, and Read Server 4, the latest update sequence number stored locally on Read Server 1 is 1009, and the latest update sequence number stored locally on Read Server 2 is The number is 1008, the latest update sequence number stored locally by the reading server 3 is 1008, and the latest update sequence number stored locally by the reading server 4 is 1009. According to the aforementioned metadata synchronization method, the latest sequence number stored in each reading server The numbers differ by at most 1.

If it is detected that the current write server is invalid, the write server needs to be updated, and one server among the read server 1, read server 2, read server 3 and read server 4 is determined as the write server. The specific signaling process is:

Read server 1, read server 2, read server 3, and read server 4 respectively initiate an arbitration request for locking to the arbitration service device, and carry the latest update sequence number stored locally;

The quorum server stores the latest update sequence numbers of Read Server 1, Read Server 2, Read Server 3, and Read Server 4, and schedules succession of Write Servers for Read Server 1, Read Server 2, Read Server 3, and Read Server 4. The sequence is: Read Server 2 is the first inheritance sequence, Read Server 4 is the second inheritance sequence, Read Server 3 is the third inheritance sequence, and Read Server 1 is the fourth inheritance sequence; and assign locks to the first inheritance sequence read server 2;

The arbitration service device can notify each read server of the scheduled order, or store the scheduled order locally, and each read server can actively learn about it;

Read server 2 knows that it is the first succession order and has obtained the lock, and judges whether the latest update sequence number stored locally is the largest update sequence number among them according to the latest update sequence numbers of other read servers stored in the arbitration service device; Since the latest update sequence number stored locally by the reading server 2 is 1008, and the maximum update sequence number stored in the arbitration service device is 1009, it is judged that the latest update sequence number stored locally is not the maximum update sequence number (that is, not the latest update sequence number). update sequence number, because the update sequence number is incremented according to the number of updates of the metadata); the read server 2 abandons the lock, and gives the lock to the next (second inheritance order) read server 4 in the inheritance order;

The read server 4 repeats the comparison and judgment process of the read server 2. Since the latest update sequence number stored locally by the read server 4 is 1009, and the maximum update sequence number stored in the arbitration service device is also 1009, the read server 4 judges that the locally stored The latest update sequence number is the maximum update sequence number stored in the arbitration server device, which converts itself into a write server (as a new write server), and writes in the arbitration server device;

Other read servers find that the existing server (ie, the read server 4) in the arbitration service device is set as the write server, and then set themselves as the synchronized server and keep their own read server identity;

The newly arbitrated write server has the largest update sequence number, and its metadata update is the latest. In order to make the metadata stored by each read server consistent, the newly arbitrated write server (read server 4) will resend Its last updated data is sent to other read servers for synchronous data update; that is: read server 4 resends its last updated data to read server 1, read server 2 and read server 3; among them, read server 2 and read server 3 Accept the update and store the latest update serial number 1009; the latest update serial number saved by the reading server 1 is already 1009, which indicates that the corresponding update has been performed before, and the existing update can be ignored and this update is accepted.

In the example shown in FIG. 7, the arbitration service device is an entity device independent of the write server and the read server. When the arbitration service device is integrated in each read server, the arbitration service device integrated in any read server can be pre-set as the authorized arbitration service device. When the write server fails, the signaling process is basically similar. The authorized arbitration service device initiates an arbitration request.

In order to avoid that after the read server to which the authorized arbitration service device belongs is converted into a write server, if a failure occurs, when a new write server is arbitrated next time, each read server does not know which arbitration service device integrated in the read server should initiate an arbitration request. Therefore, when the read server to which the authorized arbitration service device belongs is converted into a write server, the subordinate read server randomly selects an arbitration service device integrated in the remaining read servers as the next authorized arbitration service device, and notifies each read server.

Those of ordinary skill in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as: ROM/RAM, Diskettes, CDs, etc.

In summary, the present invention consists of a write server, at least two read servers and an arbitration service device to form a metadata service system. Since the metadata service system provided by the present invention is provided with at least two read servers, and when the write server fails, one of the read servers can be converted into a write server, thus effectively solving the single point failure problem. Through data synchronization, multiple read servers store consistent metadata, which can back up each other and meet a large number of concurrent access requirements; and because multiple read servers are set in parallel, multi-hop queries are not required, and each read server can directly receive customers End read access, high read efficiency.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (10)

1. A metadata service system, comprising: a write server, at least two read servers and an arbitration service device; The write server is used to store metadata in the parallel file system, accept read access; and accept write access, modify the metadata, and synchronously update the modified metadata to the read server; The read server is used to store the metadata and accept read access; and accept the synchronous update of the metadata by the write server; it is also used to monitor whether the write server is invalid, and when it is detected that the write server is invalid Each of the read servers is further configured to initiate an arbitration request to the arbitration service device, and according to the succession order determined by the arbitration service device, the read server with the largest update sequence number is converted to the write server; The arbitration service device is configured to, according to the arbitration request initiated by the read server, arrange for each read server to inherit the succession order of the write server; Wherein, the update sequence number is the corresponding sequence number assigned by the write server to the metadata updated each time, and the update sequence is carried when the modified metadata is synchronously updated to the read server Number. 2. The system according to claim 1, wherein the corresponding read server transformed into the write server is also used to synchronize the last modified metadata to other read servers. 3. The system according to claim 2, wherein the arbitration service device is an entity device independent of the write server and the read server; or The arbitration service device is integrated in the read server. 4. The system according to claim 3, wherein when the arbitration service device is an entity device independent of the write server and the read server, it is also used to store the information of the write server and the read server; Receive the server information query request initiated by the client, and return the write server and read server information. 5. A method for synchronizing metadata, applied to the metadata service system according to claim 1, characterized in that, comprising: The write server accepts the write request, writes the modified metadata into the temporary area, and sends a write request notification to the read server; After the read server receives the write request notification, it prohibits operations on the corresponding metadata before modification, or prohibits operations after waiting for the corresponding metadata that is being read to be read; The write server returns a notification receipt response; After the write server receives the notification reception response returned by the read server, send the metadata stored in the temporary area to the read server; and use the metadata stored in the temporary area, Update the corresponding metadata records stored locally; The read server updates the corresponding metadata record stored locally, and releases the prohibited operation after the update is successful. 6. The method of claim 5, further comprising: The write server monitors whether the read server fails; The write server only sends the write request notification to the current non-failure read server; and after determining that the current non-failure read server returns a notification reception response, sends the metadata stored in the temporary area to the current A read server that has not failed. 7. The method according to claim 6, wherein if the read server detects that the write server is invalid after the prohibition operation is completed, the prohibition operation is lifted. 8. A write server update method, applied to the metadata service system according to claim 1, characterized in that it comprises: the read server monitors the current state of the write server; When the read server detects that the write server fails, each of the read servers initiates an arbitration request to the arbitration service device, and the arbitration service device arranges an inheritance sequence for each read server to inherit the write server, according to the In the order of inheritance, the read server with the largest update sequence number is converted to the write server, wherein the update sequence number is the corresponding sequence number assigned by the write server to the metadata of each update in sequence, And carry the update sequence number when synchronously updating the modified metadata to the read server. 9. The method according to claim 8, characterized in that, specifically comprising: When the read server detects that the write server is invalid, it initiates an arbitration request for locking to the arbitration service device, carrying the latest update sequence number stored locally; The arbitration service device stores the latest update serial number of each read server, arranges the inheritance order of each read server to inherit the write server, and assigns a lock to the read server corresponding to the first inheritance order; The read server that obtains the lock judges whether the latest update sequence number stored locally is the largest update sequence number among them according to the latest update sequence numbers of each read server stored in the arbitration service device; write server; otherwise, yield the lock to the next reader server in the succession order. 10. The method according to claim 8 or 9, wherein when the arbitration service device is integrated in the read server, the arbitration service device integrated in any one of the read servers is preset as active an authorized arbitration service device, and the authorized arbitration service device receives the arbitration request; When the read server to which the authorized arbitration service device belongs is converted into a write server, the read server randomly selects one of the arbitration service devices integrated in the other read servers as the authorized arbitration service device, and notifies each reader server.

Images