CN111104283B - A fault detection method, device, equipment and medium for a distributed storage system - Google Patents

Abstract

The present application discloses a fault detection method, device, device and computer-readable storage medium for a distributed storage system. The method includes: determining a fault threshold value by using a corresponding calculation rule according to a storage pool type of the distributed storage system; Acquire the number of reports that each node in the distributed storage system is reported as an abnormal state, and determine the fault condition of the distributed storage system according to the number of reports and the fault threshold. It can be seen that the fault threshold value in this method is determined according to the storage pool type of the storage system and using the corresponding calculation rule. Therefore, the fault condition of the distributed storage system is determined according to the number of reports and the fault threshold value. It can avoid the faulty misjudgment caused by the faulty node of the back-end network misreporting the abnormality of other nodes, improve the accuracy of fault detection of the distributed storage system, and relatively ensure the normal use of the entire distributed storage system.

Description

A fault detection method, device, equipment and medium for a distributed storage system

technical field

The present invention relates to the field of distributed storage systems, and in particular, to a fault detection method, apparatus, device and computer-readable storage medium for a distributed storage system.

Background technique

In a distributed storage system, a daemon process (or service) is set on each node to provide access and monitoring of the hard disks in the storage pool; and through the daemon process (or service) between different nodes Heartbeat message to detect whether the peer daemon (or service) is normal.

For each node, including the front-end network and the back-end network, the front-end network is used for customer business, and the back-end network is used for message communication and data interaction within the cluster; in order to detect the connectivity of the network, the daemon process between nodes will be in the front-end at the same time. The network and the back-end network perform heartbeat detection; each node communicates with the cluster management process through the front-end network. In this case, if the back-end network of an individual node fails (the actual faulty node), causing its back-end network to be unable to communicate with other nodes, other nodes will report these actual faulty nodes to the cluster management process as abnormal; these The actual faulty node will also report the abnormal state of other nodes through its own front-end network because it cannot communicate with other nodes.

In the prior art, a fixed fault threshold value is preset, and then when it is determined that the number of times that a node is reported as an abnormal state exceeds the fault threshold value, the node is determined to be a faulty node. However, there will be a problem with this method: for example, if there are two actual faulty nodes in the current distributed storage system, other nodes will report that these two actual faulty nodes are abnormal, and these two actual faulty nodes will also report all other faulty nodes. The node is abnormal; in this way, each other node is reported abnormally by at least the two actual faulty nodes, and because the number of times that each other node is reported abnormally exceeds the preset threshold, the cluster management process will A node becomes a failed node, rendering the entire cluster unavailable. In reality, with only two actual failed nodes, the cluster may still be available. It can be seen that with the fault detection method for a distributed storage system in the prior art, when the back-end network of a node fails, there will be a false positive abnormal situation, thereby affecting the normal use of the entire distributed storage system.

Therefore, how to improve the accuracy of fault detection of the distributed storage system and relatively ensure the normal use of the distributed storage system is a technical problem to be solved by those skilled in the art at present.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a fault detection method for a distributed storage system, which can improve the accuracy of fault detection of the distributed storage system and relatively ensure the normal use of the distributed storage system; another aspect of the present invention is The purpose is to provide a fault detection device, device and computer-readable storage medium of a distributed storage system, all of which have the above beneficial effects.

In order to solve the above technical problems, the present invention provides a fault detection method for a distributed storage system, including:

According to the storage pool type of the distributed storage system, use the corresponding calculation rule to determine the failure threshold value;

Obtain the number of reports that each node in the distributed storage system is reported as an abnormal state;

The fault condition of the distributed storage system is determined according to each of the reported times and the fault threshold.

Preferably, according to the storage pool type of the distributed storage system, the process of determining the fault threshold value by using the corresponding calculation rule specifically includes:

If the storage pool of the distributed storage system is of the copy type, obtaining the first number of hard disks in the storage pool that belong to the same placement group;

A value greater than half of the first number is set as the fault threshold value.

Preferably, according to the storage pool type of the distributed storage system, the process of determining the fault threshold value by using the corresponding calculation rule specifically includes:

If the storage pool of the distributed storage system is of the erasure type, acquiring the second quantity of redundant data in the storage pool calculated according to the data blocks;

A value greater than the second number is set as the fault threshold value.

Preferably, when there are multiple storage pools on the first node, the method further includes:

respectively acquiring fault thresholds corresponding to a plurality of the storage pools on the first node;

A maximum value among a plurality of the fault threshold values is set as the fault threshold value of the first node.

Preferably, the process of acquiring the number of times that each node in the distributed storage system is reported as an abnormal state is specifically:

The number of times of reporting that each of the nodes in the distributed storage system is reported as an abnormal state is acquired according to a preset time period.

Preferably, after determining the fault condition of the distributed storage system according to each of the reporting times and the fault threshold, the method further includes:

The number of nodes currently judged to be faulty nodes is displayed using a display device.

Preferably, after determining the fault condition of the distributed storage system according to each of the reporting times and the fault threshold, the method further includes:

Send the corresponding prompt message.

In order to solve the above technical problems, the present invention also provides a fault detection device for a distributed storage system, including:

The threshold value determination module is used to determine the fault threshold value by using the corresponding calculation rule according to the storage pool type of the distributed storage system;

an obtaining module, configured to obtain the number of reports that each node in the distributed storage system is reported as an abnormal state;

A fault determination module, configured to determine the fault condition of the distributed storage system according to the number of times of reporting and the fault threshold value.

Preferably, when there are multiple storage pools on the first node, the method further includes:

a first obtaining module, configured to obtain respective fault thresholds corresponding to the plurality of storage pools on the first node;

A setting module, configured to set the maximum value of the plurality of fault threshold values as the fault threshold value of the first node.

Preferably, it further includes:

The display module is used for displaying the number of nodes currently judged as faulty nodes by using a display device.

Preferably, it further includes:

The prompt module is used to send out the corresponding prompt information.

In order to solve the above technical problems, the present invention also provides a fault detection device for a distributed storage system, including:

memory for storing computer programs;

The processor is configured to implement the steps of any of the above-mentioned fault detection methods of the distributed storage system when executing the computer program.

In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any of the above distributed storage systems can be implemented. The steps of the fault detection method.

The invention provides a fault detection method for a distributed storage system. Compared with the prior art, a fixed fault threshold value is preset, and the fault condition of the distributed storage system is determined according to the fixed fault threshold value. The fault threshold value in this method is determined according to the storage pool type of the storage system and using the corresponding calculation rule. Therefore, the fault condition of the distributed storage system is determined according to the number of reports and the fault threshold value. , which can prevent the faulty node of the back-end network from falsely reporting the fault of other nodes, improve the accuracy of fault detection of the distributed storage system, and relatively ensure the normal use of the entire distributed storage system.

In order to solve the above technical problems, the present invention also provides a fault detection device, equipment and computer-readable storage medium of a distributed storage system, all of which have the above beneficial effects.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions of the prior art more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only For some embodiments of the present invention, for those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative efforts.

1 is a flowchart of a method for detecting faults in a distributed storage system according to an embodiment of the present invention;

FIG. 2 is a structural diagram of a fault detection device of a distributed storage system according to an embodiment of the present invention;

FIG. 3 is a structural diagram of a fault detection device of a distributed storage system according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The core of the embodiments of the present invention is to provide a fault detection method for a distributed storage system, which can improve the accuracy of fault detection of the distributed storage system and relatively ensure the normal use of the distributed storage system; another core of the present invention is A fault detection device, device and computer-readable storage medium of a distributed storage system are provided, all of which have the above beneficial effects.

In order to make those skilled in the art better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a flowchart of a fault detection method for a distributed storage system according to an embodiment of the present invention. As shown in Figure 1, a fault detection method for a distributed storage system includes:

S10: According to the storage pool type of the distributed storage system, use a corresponding calculation rule to determine a fault threshold value.

Specifically, in this embodiment, the storage pool type of the distributed storage system needs to be determined first, and the storage pool type includes the erasure type copy type according to its storage fault tolerance mechanism. Then, according to the storage pool type, the corresponding fault threshold value is calculated by using the calculation rule corresponding to the storage pool type.

S20: Acquire the number of times that each node in the distributed storage system is reported as an abnormal state.

Specifically, when there is an actual faulty node in the distributed storage system, if the actual faulty node is a front-end network fault, other normal nodes will report these actual faulty nodes to the cluster management process as abnormal status, and the actual faulty node is due to the front-end network. In this case, the cluster management server can accurately determine the fault of the distributed storage system according to the number of reports.

The main consideration in this embodiment is that the actual faulty node is a back-end network fault. At this time, if the actual faulty node cannot communicate with other nodes, other nodes will report to the cluster management process that the actual faulty node is in an abnormal state; at the same time, the actual faulty node It will also report other nodes as abnormal states to the cluster management process. At this time, the number of reports that each node in the distributed storage system is reported as an abnormal state is acquired.

S30: Determine the fault condition of the distributed storage system according to the number of times of reporting and the fault threshold value.

After determining the fault threshold value and obtaining the number of reports that each node in the distributed storage system is reported as abnormal state, determine the fault threshold value as a reference value according to the storage pool type corresponding to the node, and then use the The number of reports is compared with the fault threshold; if the number of reports of a target node is greater than the fault threshold, the target node is determined as a fault; otherwise, the normal state of the target node is maintained.

In actual operation, after the node whose number of reports is greater than the corresponding fault threshold value is determined as a fault node, the number of nodes of the fault node is determined; then, a preset corresponding to the storage pool type is obtained to determine whether the occurrence The node threshold for cluster failure, to determine whether the number of nodes is greater than the node threshold; if so, it is determined that a cluster failure has occurred in the distributed storage system; if not, it means that although there are faulty nodes in the current storage pool, the storage pool can still be used normally, that is , the cluster is normal.

It should be noted that for different types of storage pools, the tolerance to faulty nodes is different. Therefore, the node thresholds of different types of storage pools are also different. The node thresholds are set as follows:

Specifically, in the erasure type storage pool, according to the specified erasure algorithm, the original data is equally divided into K parts, and then the K parts of the data are used to calculate M parts of redundant data, and finally K +M pieces of data; then write the K+M pieces of data to K+M hard disks in copies, each hard disk saves a different piece of data, and read any K from the K+M pieces of data at any time. The original data can be restored. In other words, for erasure-type storage pools, at least K copies of normal data are required to restore the original data, otherwise data will be lost. Therefore, the node threshold of the erasure type storage pool is (K+M-K), that is, the number M of redundant data.

That is to say, as long as the number of nodes of the faulty node does not exceed the node threshold M, the cluster can be used normally; if the number of nodes of the faulty node exceeds M, for example, the number of nodes is (M+1), then due to the number of normal nodes If it is less than K, the original data cannot be restored, so it is determined that the cluster is faulty.

Specifically, in a copy-type storage pool, the copy is the data that is exactly the same as the original data, and the N copy is to write N copies of the original data to N hard disks, and each hard disk corresponds to a copy of the data that is exactly the same as other hard disks. . Therefore, for a copy-type storage pool, as long as there is a normal hard disk and a copy of normal data, the original data can be obtained. Therefore, the node threshold for replica-type storage pools is (N-1).

In other words, as long as the number of nodes of the faulty node does not exceed the node threshold (N-1), the cluster can be used normally. If the number of nodes of the faulty node exceeds (N-1), that is, the number of nodes is N, then at this time Since the number of normal nodes is less than 1, the original data cannot be obtained, so the cluster fails.

Compared with the method for detecting a fault of a distributed storage system provided by an embodiment of the present invention, a fixed fault threshold value is preset in the prior art, and the fault threshold value of the distributed storage system is determined according to the fixed fault threshold value. The fault condition method, the fault threshold value in this method is determined according to the storage pool type of the storage system and using the corresponding calculation rule. Therefore, according to the number of reports and the fault threshold value, the distributed storage system The fault condition can prevent the faulty node of the back-end network from falsely reporting the fault of other nodes, improve the accuracy of fault detection of the distributed storage system, and relatively ensure the normal use of the entire distributed storage system.

On the basis of the above embodiment, this embodiment further describes and optimizes the technical solution. Specifically, in this embodiment, according to the storage pool type of the distributed storage system, the corresponding calculation rule is used to determine the failure threshold value process, including:

If the storage pool of the distributed storage system is a copy type, obtain the first number of hard disks in the storage pool that belong to the same placement group;

A value greater than half of the first number is set as the fault threshold value.

Specifically, first obtain the first number of hard disks belonging to the same placement group in the storage pool, and the hard disks belonging to the same placement group refer to the hard disks with exactly the same data written, that is, the number N of copies; The value of one half of the first number is set as the fault threshold value, that is, the fault threshold value is at least greater than N/2. As a preferred implementation manner, the failure threshold value corresponding to the storage pool of the copy type may be (N/2+1).

It is understandable that when the back-end network of a node in the distributed storage system is abnormal (the actual faulty node), each node in the cluster will report the abnormal state of other nodes to each other. According to the principle of minority obeying the majority, When (N/2+1) nodes report that a target node is in an abnormal state, that is, when the number of times the target node is reported to be in an abnormal state is (N/2+1), the target node is determined to be faulty node; otherwise, keep the target node in normal state.

If the storage pool of the distributed storage system is of the erasure type, obtain the second quantity of redundant data in the storage pool calculated according to the data blocks;

A value greater than the second number is set as the fault threshold value.

Specifically, according to a piece of original data, it is equally divided into K pieces of data blocks, and the K pieces of data are divided into M pieces of redundant data according to the erasure algorithm, and the quantity M of redundant data is set as the second quantity; then , a value greater than the second number is set as the fault threshold value. As a preferred implementation manner, in this embodiment, the fault threshold value of the erasure type storage pool is set to (M+1).

Specifically, for a cluster whose storage pool is an erasure pool of K+M, if the number of reports of a target node as an abnormal state is greater than (M+1), the target node is determined to be a faulty node. , because (M+1) nodes are required to report that the target node is in an abnormal state, so when there are M or less than M nodes as actual faulty nodes, other nodes report these actual faulty nodes as abnormal states, and these actual faulty nodes are in an abnormal state. Other nodes will also be reported as abnormal, but since the number of actual faulty nodes is less than or equal to M, the cluster management process only determines these actual faulty nodes to be abnormal, and will not consider other normal nodes to be abnormal. The state of the cluster remains normal, and the actual number of faulty nodes is within M, and the number of normal nodes is greater than or equal to K, and the cluster can still work normally.

It can be seen that, according to the method for setting a fault threshold value provided in this embodiment, it is possible to avoid a false alarm of a cluster fault when the distributed storage system can work normally.

On the basis of the above embodiment, this embodiment further describes and optimizes the technical solution. Specifically, in this embodiment, when there are multiple storage pools on the first node, the method further includes:

respectively acquiring fault thresholds corresponding to the multiple storage pools on the first node;

The fault threshold value of the first node is set as the maximum value among the multiple fault threshold values.

It can be understood that, in actual operation, there may be multiple storage pools on a certain node (referred to as the first node) in the distributed storage system, and the maximum value of the multiple fault thresholds is set to the first node. The fault threshold value of the node, the first node is determined to be a faulty node only when the number of times the first node is reported as an abnormal state exceeds the fault threshold value corresponding to the maximum value.

Taking the erasure type storage pool as an example, if there are two erasure pools at the same time, the first one is 2+1 (K=2, M=1) redundant, and the second one is 3+2 (K=3 , M=2) redundancy, the redundancy of the second storage pool shall prevail. Therefore, M+1=2+1=3 nodes must report that the first node is in an abnormal state at the same time. It is determined that the first node is a faulty node.

It can be seen that this embodiment further considers that when there are multiple storage pools on a certain node, the corresponding method for setting the fault threshold value can further improve the accuracy of detecting the fault condition of the distributed storage system.

On the basis of the above embodiment, this embodiment further describes and optimizes the technical solution. Specifically, in this embodiment, the process of obtaining the number of reports that each node in the distributed storage system is reported as an abnormal state, Specifically:

Acquires the number of times that each node in the distributed storage system is reported as an abnormal state according to a preset time period.

Specifically, in this embodiment, the number of reports that each node in the distributed storage system is reported as an abnormal state is obtained according to a preset time period. It can be understood that when acquiring the number of reports of each node, it is generally to acquire the number of reports corresponding to each node in a certain period of time. Set the number of reports within the time period. This embodiment does not limit the length of the preset time period, which is specifically set according to the actual situation.

It can be seen that by obtaining the number of reports that each node in the distributed storage system is reported as an abnormal state according to a preset time period, the obtained situation of each node in the distributed storage system can be updated in time, and the distribution can be determined in a more timely manner. failure conditions in the storage system.

On the basis of the above-mentioned embodiment, this embodiment further describes and optimizes the technical solution. Further includes:

The number of nodes currently judged to be faulty nodes is displayed using a display device.

Specifically, in this embodiment, since the number of times that each node in the distributed storage system is reported as abnormal state is obtained according to a preset time period, the number of nodes determined to be faulty nodes may change. Therefore, the present embodiment further utilizes a display device to display the number of nodes.

Specifically, the display device may be a TFT (Thin Film Transistor, thin film field effect transistor) liquid crystal display screen or a UFB (Ultra Fine&Bright) liquid crystal display screen or an OLED (Organic Light-Emitting Diode, organic light emitting diode) display screen, etc. This embodiment This is not limited.

In addition, this embodiment does not limit the manner of displaying the number of nodes, for example, it may be displayed in the manner of text, image or animation, which is specifically set according to actual requirements.

It can be seen that in this embodiment, the number of nodes currently determined as faulty nodes is further displayed by the display device, so that the user can directly know the number of nodes determined to be faulty in the current distributed storage system, and the user experience is further improved.

On the basis of the above-mentioned embodiment, this embodiment further describes and optimizes the technical solution. Further includes:

Send the corresponding prompt message.

Specifically, in this embodiment, after the fault condition of the distributed storage system is determined according to the number of reports and the fault threshold value, if a cluster fault occurs in the distributed storage system, the prompt device is further used to send out corresponding prompt information, In order to prompt the operator that the current distributed storage system cluster is unavailable, will not be able to read and write or may cause data loss.

It should be noted that the specific type of the prompting device used to send out the corresponding prompting information is not limited in this embodiment. As a preferred embodiment, the prompting device may be a buzzer and/or an indicator light, etc. Set corresponding prompt information for different working states of the device.

It can be seen that, in this embodiment, after it is determined that a cluster failure occurs in the distributed storage system, corresponding prompt information is sent, so as to timely and effectively prompt the operator of the failure of the distributed storage system, thereby further improving the user experience.

The embodiments of a fault detection method for a distributed storage system provided by the present invention are described in detail above, and the present invention also provides a fault detection device, device, and computer for a distributed storage system corresponding to the method. For the readable storage medium, since the embodiments of the apparatus, device, and computer-readable storage medium part correspond to the embodiments of the method part, please refer to the embodiments of the method part for the embodiments of the apparatus, device, and computer-readable storage medium part. description, which will not be repeated here.

FIG. 2 is a structural diagram of a fault detection apparatus of a distributed storage system provided by an embodiment of the present invention. As shown in FIG. 2 , a fault detection apparatus of a distributed storage system includes:

The threshold value determination module 21 is used to determine the fault threshold value by using the corresponding calculation rule according to the storage pool type of the distributed storage system;

The obtaining module 22 is used to obtain the number of reports that each node in the distributed storage system is reported as an abnormal state;

The fault determination module 23 is configured to determine the fault condition of the distributed storage system according to the number of reporting times and the fault threshold value.

The fault detection device of the distributed storage system provided by the embodiment of the present invention has the beneficial effects of the above-mentioned fault detection method of the distributed storage system.

As a preferred embodiment, when there are multiple storage pools on the first node, it further includes:

a first obtaining module, configured to obtain respective fault thresholds corresponding to the plurality of storage pools on the first node;

The setting module is configured to set the maximum value of the multiple fault threshold values as the fault threshold value of the first node.

As a preferred embodiment, it further includes:

The display module is used for displaying the number of nodes currently judged as faulty nodes by using a display device.

As a preferred embodiment, it further includes:

The prompt module is used to send out the corresponding prompt information.

FIG. 3 is a structural diagram of a fault detection device of a distributed storage system according to an embodiment of the present invention. As shown in FIG. 3 , a fault detection device of a distributed storage system includes:

memory 31 for storing computer programs;

The processor 2 is configured to implement the steps of the above-mentioned fault detection method of the distributed storage system when executing the computer program.

The fault detection device of the distributed storage system provided by the embodiment of the present invention has the beneficial effects of the above-mentioned fault detection method of the distributed storage system.

To solve the above technical problems, the present invention also provides a computer-readable storage medium, where a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned fault detection method for a distributed storage system are implemented.

The computer-readable storage medium provided by the embodiment of the present invention has the beneficial effects of the above-mentioned fault detection method of a distributed storage system.

The fault detection method, apparatus, device and computer-readable storage medium of the distributed storage system provided by the present invention have been described in detail above. The principles and implementations of the present invention are described herein by using specific embodiments, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

Claims (8)

1. a fault detection method of a distributed storage system, is characterized in that, comprises: According to the storage pool type of the distributed storage system, use the corresponding calculation rule to determine the failure threshold value; Obtain the number of reports that each node in the distributed storage system is reported as an abnormal state; Determine the fault condition of the distributed storage system according to each of the reported times and the fault threshold; The process of determining the fault threshold value by using the corresponding calculation rule according to the storage pool type of the distributed storage system specifically includes: If the storage pool of the distributed storage system is of the copy type, obtaining the first number of hard disks in the storage pool that belong to the same placement group; setting a value greater than half of the first number as the fault threshold value; The process of determining the fault threshold value by using the corresponding calculation rule according to the storage pool type of the distributed storage system specifically includes: If the storage pool of the distributed storage system is of the erasure type, acquiring the second quantity of redundant data in the storage pool calculated according to the data blocks; A value greater than the second number is set as the fault threshold value. 2. The method according to claim 1, wherein when there are multiple storage pools on the first node, the method further comprises: respectively acquiring fault thresholds corresponding to a plurality of the storage pools on the first node; A maximum value among a plurality of the fault threshold values is set as the fault threshold value of the first node. 3. The method according to claim 1, wherein the process of acquiring the number of reports that each node in the distributed storage system is reported as an abnormal state is specifically: The number of times that each node in the distributed storage system is reported as an abnormal state is acquired according to a preset time period. 4 . The method according to claim 3 , wherein after determining the fault condition of the distributed storage system according to each of the reporting times and the fault threshold value, the method further comprises: 5 . The number of nodes currently judged to be faulty nodes is displayed using a display device. 5. The method according to any one of claims 1 to 4, characterized in that, after determining the fault condition of the distributed storage system according to each of the reporting times and the fault threshold value, further comprising: : Send the corresponding prompt message. 6. A fault detection device for a distributed storage system, comprising: The threshold value determination module is used to determine the fault threshold value by using the corresponding calculation rule according to the storage pool type of the distributed storage system; an acquisition module, configured to acquire the number of reports that each node in the distributed storage system is reported as an abnormal state; a fault determination module, configured to determine the fault condition of the distributed storage system according to the number of times of reporting and the fault threshold; The threshold value determination module is specifically configured to obtain the first number of hard disks belonging to the same placement group in the storage pool if the storage pool of the distributed storage system is a copy type; The value of half of the quantity is set as the failure threshold value; if the storage pool of the distributed storage system is of the erasure type, obtain the second quantity of redundant data in the storage pool calculated according to the data block; A value greater than the second number is set as the fault threshold value. 7. A fault detection device for a distributed storage system, comprising: memory for storing computer programs; The processor is configured to implement the steps of the fault detection method of the distributed storage system according to any one of claims 1 to 5 when executing the computer program. 8. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the distribution according to any one of claims 1 to 5 is realized The steps of the fault detection method of the storage system.

Images