CN115801553B - An improved consensus method based on Raft and related equipment - Google Patents
An improved consensus method based on Raft and related equipment Download PDFInfo
- Publication number
- CN115801553B CN115801553B CN202211509442.XA CN202211509442A CN115801553B CN 115801553 B CN115801553 B CN 115801553B CN 202211509442 A CN202211509442 A CN 202211509442A CN 115801553 B CN115801553 B CN 115801553B
- Authority
- CN
- China
- Prior art keywords
- node
- raft
- consensus
- nodes
- slave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0894—Policy-based network configuration management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明公开了一种基于Raft改进的共识方法及相关设备，所述方法包括：当Raft共识系统启动时所有节点均为从节点，自动进行主节点选举，若主节点选举成功，则统计节点信息并进行处理得到节点排序序列；根据自适应策略计算副节点个数，根据所述节点排序序列生成副节点，根据分片策略生成网络拓扑结构，将网络拓扑结构广播并对除开主节点外的所有节点进行分片；执行共识业务，若主节点发生故障，则通过崩溃恢复机制使得所述节点排序序列中排序顺序第一位的副节点继承为主节点，以使所述Raft共识系统恢复正常服务。本发明通过预排序方法与崩溃恢复机制提高了共识系统的可用性，通过自适应网络分片提高了共识系统的吞吐性能以及可扩展性。
The present invention discloses a consensus method and related equipment based on Raft improvement, the method comprising: when the Raft consensus system is started, all nodes are slave nodes, and the master node election is automatically performed. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence; the number of slave nodes is calculated according to an adaptive strategy, and slave nodes are generated according to the node sorting sequence, and a network topology is generated according to a sharding strategy, and the network topology is broadcast and all nodes except the master node are sharded; the consensus service is executed, and if the master node fails, the slave node with the first order in the node sorting sequence is inherited as the master node through a crash recovery mechanism, so that the Raft consensus system can resume normal service. The present invention improves the availability of the consensus system through a pre-sorting method and a crash recovery mechanism, and improves the throughput performance and scalability of the consensus system through adaptive network sharding.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a Raft improved consensus method, system, terminal, and computer readable storage medium.
Background
Blockchain (Blockchain) technology is a decentralised architecture and computing paradigm that uses a blockchain data structure to validate and store data, a distributed node consensus algorithm to generate and update data, cryptography to secure data transfer and access, and intelligent contracts composed of automated script code to program and manipulate data.
Consensus (Consensus) is the idea that several server nodes agree on a certain transaction, even in case of partial node failure, network delay, network split.
RAFT Raft is a consensus algorithm intended to replace Paxos, which makes it easier to understand by logical separation, but which has also proved to be formally safe and provides some additional functionality. Raft provides a general method of distributing state machines in a cluster of computing systems, ensuring that each node in the cluster agrees to a series of identical state transitions. Consensus algorithms are the cornerstone of modern distributed systems and blockchain technology development, and discrete server nodes achieve consistency of service provision by means of consensus.
The master node (Leader) provides services such as log read-write access to clients outside the consensus system, requests logs synchronously to the inward direction Follower, tells Follower to submit the logs after synchronizing the logs to nodes exceeding 1/2, and keeps the log states of all the nodes consistent with the master node as much as possible. It is also because all content within the consensus system remains consistent with the master node, which can cause significant problems when the master node is lost. Even if the secondary node role is added, the primary node is only helped to share the network pressure of sending the log to the secondary node, and the decision right of the log content is still in the primary node.
When the system does not have the master node (or is initialized), the slave node considers that the master node is lost because the heartbeat message sent by the master node is not received, so that the master node election is entered, at the moment, the slave node which is overtime (considered that the master node is lost) firstly is converted into the candidate node, voting requests are sent to other slave nodes, and a voting link in the Raft consensus algorithm is carried out. This section is not mentioned in this document, as no content modification is involved. Election, voting, is a business function inherent to Raft consensus algorithm. Temporary roles in the Leader election process.
From node Follower, the Leader synchronized log is accepted and persisted, and after the Leader tells that the log can be committed, the log is committed, which is the node that completes the consensus process. Only the request is received and responded to, and no active transmission is performed. (Raft consensus process: the master node sends the log to all the slave nodes, the log on the slave node is kept as consistent as possible with the master node, this log replication process is Raft consensus process, i.e. how all nodes are replicated to keep the same log, and the same state is reached as much as possible, i.e. agreement is reached).
The auxiliary node (Vice-Leader) improves the added node identity, is used for realizing the network fragmentation of the technical point 2, is a slave node with the first k strategies of strong communication capacity or high hardware configuration and the like selected in a self-adaptive mode from a pre-sequencing sequence, and is then converted into the auxiliary node. The role of the method is to share the communication pressure of the master node, the master node sends the log to the slave node, and the slave node forwards the log to the slave node corresponding to the sharding.
Raft as a common knowledge mechanism which is applied today more often, it makes common knowledge of nodes in all traffic areas by using a powerful master Node (master Leader Node). In Raft examples with multiple member nodes, when a main node breaks down, the main node is reselected and the problems of failure in election and the like can cause continuous service unavailability, and secondly, as the number of the nodes increases, the throughput of the whole service can be influenced by the bandwidth of the main node due to the fact that a single main node becomes a communication bottleneck, and simultaneously the expandability of a consensus system is restricted.
For the system availability decline caused by the breakdown of the master node in the Raft cluster, the more common improvement technical scheme is real-time monitoring, namely, the availability of the consensus system is judged in real time by additionally arranging a node for cyclic monitoring, and when the master node breaks down, a new master node is designated through a series of operations, so that the cluster system is recovered to be normal.
For the throughput bottleneck of the master node caused by the increase of the number of nodes in Raft, a more common improvement technical scheme is network fragmentation, namely, a plurality of Raft examples are established by grouping and combining the nodes in the original Raft examples, the number of the nodes of each Raft example is limited, and the master nodes of the plurality of consensus examples are reestablished into a consensus cluster by using Raft or other consensus protocols, so that the problem of the throughput bottleneck of the master node caused by the increase of the number of the nodes in Raft is relieved.
The real-time monitoring scheme is realized based on a method of additionally arranging monitoring nodes, and the availability of the consensus system is judged in real time by circularly monitoring all nodes in Raft clusters of the additionally arranged monitoring nodes. Because the monitoring node needs to judge whether the node fault occurs or not and whether the main node needs to be replaced or not according to the availability of the consensus system, the monitoring node needs to continuously communicate with the system node to monitor. However, in practice the frequency of occurrence of a primary node crash event is less and the time to failure is unpredictable. If the monitoring node detects with the frequency of too large a timing interval, the system fault time may be longer, if the monitoring node monitors with the frequency of too small a timing interval, the network communication is performed too frequently, and additional network overhead is caused, if the monitoring node uses a feedback adjustment mechanism, namely, the monitoring node obtains the message of the downtime of the main node from other nodes and reassigns the main node after checking, the situation that the main node election mechanism of the monitoring node is successful once is compared with Raft consensus itself. Since Raft consensus algorithm provides a recovery mechanism when the main node crashes, when the failure rate of the system node is extremely low and the benefit is very little, the setting of the monitoring node is no longer worth.
The network slicing scheme is realized based on a grouping method of system nodes, and the Raft network is divided into one or more parts by changing the network architecture of Raft, so that the number of slave nodes responsible for communication of a master node is reduced, the bandwidth pressure of the network communication of the master node is reduced, and the throughput of the system is improved. A single simple network slice is only a rough structuring of the network into two parts, one secondary Node (Vice-Leader Node) is designated in the other part to communicate with the primary Node, and then broadcast to the general network nodes that are responsible for itself. Because the network fragments only depend on the connection between the main node and the auxiliary node, when the main node or the auxiliary node has a breakdown error, part of network nodes are isolated, so that network isolation occurs, and the usability of the system is greatly reduced. The network slicing scheme of using a plurality of Raft groups uniformly divides all nodes into a plurality of groups, each group is internally provided with a Raft consensus example, and main nodes of all groups reach consensus by means of methods such as Raft consensus or route broadcasting among the groups. Since the system network is abstracted into two or more layers by means of Raft packets, the communication delay increases. In addition, if an independent routing node is set, the routing node may become a system performance bottleneck, and the number Raft of packets needs to configure the node in advance, which is not beneficial to the automatic smooth expansion of the system node.
Accordingly, the prior art is still in need of improvement and development.
Disclosure of Invention
The invention mainly aims to provide a Raft-based improved consensus method, a Raft-based improved consensus system, a Raft-based improved consensus terminal and a Raft-based improved consensus storage medium, and aims to solve the problems that in the prior art, due to the increase of the number of nodes, a single main node becomes a communication bottleneck, the throughput of the whole service is influenced by the bandwidth of the main node, and the expandability of the consensus system is restricted.
To achieve the above object, the present invention provides a common identification method based on Raft improvement, the common identification method based on Raft improvement includes the following steps:
When Raft the consensus system is started, all nodes are slave nodes, the Raft consensus system automatically performs master node election, if the master node election is successful, node information is counted and processed to obtain a node ordering sequence;
after the node ordering sequence is obtained, the number of auxiliary nodes is calculated according to the self-adaptive strategy, the auxiliary nodes are generated according to the node ordering sequence, a network topology structure is generated according to the slicing strategy, and all nodes except the main node are sliced by broadcasting the network topology structure;
And after the self-adaptive network slicing is completed, executing the consensus service of the Raft consensus system, and if the main node fails, inheriting the auxiliary node with the first order in the node ordering sequence as the main node through a crash recovery mechanism so as to enable the Raft consensus system to recover normal service.
Optionally, the common knowledge method based on Raft improvement, wherein the statistics of node information and processing are performed to obtain a node ordering sequence, specifically includes:
The master node collects and counts related information of all slave nodes in the Raft consensus system, wherein the related information comprises log indexes, hardware performance and network states;
Judging whether the log index changes or not;
if the log index changes, updating log index information of the slave node according to the statistical result;
if the log index is unchanged, judging whether the hardware performance is changed or not;
If the hardware performance changes, updating the hardware performance information of the slave node according to the statistical result;
if the hardware performance is not changed, judging whether the network state is changed or not;
If the network state changes, updating the network state information of the slave node according to the statistical result;
if the network state is unchanged, ordering the nodes according to the statistical information and the ordering strategy to obtain a node ordering sequence;
and updating the node ordering sequence in the Raft consensus system.
Optionally, the method for consensus based on Raft improvement, wherein the step of inheriting, by a crash recovery mechanism, a secondary node in a first order of the node ordering sequence as a primary node, so that the Raft consensus system resumes normal service, specifically includes:
Judging whether the master node is lost or not;
if the master node is lost, searching a first slave node according to the node ordering sequence;
Other auxiliary nodes in the node ordering sequence send a main node losing message to a first auxiliary node in the inheritance order for requiring the first auxiliary node to become a new main node;
judging whether the first auxiliary node receives auxiliary node information exceeding 1/2;
if the first auxiliary node receives the auxiliary node information exceeding 1/2, generating a new main node, adding the wilting period number of the main node and broadcasting the information, so that the whole Raft consensus system enters the next wilting period;
If the first auxiliary node does not receive the auxiliary node information exceeding 1/2, the first auxiliary node in the inheritance order is considered to be unavailable, and the next auxiliary node is searched for trying again;
if the main node is not lost, judging whether the auxiliary node is lost or not;
if the auxiliary node is lost, the main node searches a first auxiliary node according to the segmented node sequencing sequence;
The master node sends a slave node loss message to the node in the inheritance order, and the slave node loss message is used for requesting the node in the inheritance order to become a new slave node;
after the new secondary node is generated, the network fragments of the original secondary node are replaced.
Optionally, in the common knowledge method based on Raft improvement, the calculating the number of the secondary nodes according to the adaptive policy, generating the secondary nodes according to the node ordering sequence, generating a network topology structure according to the slicing policy, broadcasting the network topology structure, and slicing all nodes except the primary node specifically includes:
the master node calculates the number of auxiliary nodes required by network fragmentation according to the set self-adaptive strategy;
Determining the number of the auxiliary node from front to back in sequence according to the ordering sequence of the node ordering sequence;
Dividing the rest slave nodes according to the slicing strategy by using the counted node information, and generating a network topological structure according to the rule one-to-one correspondence with the slave nodes;
the master node sends the network topology structure to the slave node through heartbeat messages, and the slave node automatically adjusts a message receiving and transmitting object or converts the message receiving and transmitting object into the slave node after receiving the message.
Optionally, the common knowledge method based on Raft improvement, wherein the slicing strategy comprises selecting and dividing according to node number hash mapping, network conditions and hardware configuration to generate a network topology structure.
Optionally, the common knowledge method based on Raft improvement, wherein the secondary node message represents a primary node lost message.
Optionally, the consensus method based on Raft improvement, wherein the consensus method based on Raft improvement further comprises:
When election is carried out, the candidate nodes which are not selected become slave nodes after the generation of the master node or after the next period is entered;
when the election is carried out, the election is carried out again until the election timer is overtime and the election is not successful;
when the master node is changed, the rest of the slave nodes are changed into slave nodes, and the slave nodes wait for re-screening.
In addition, to achieve the above object, the present invention further provides a consensus system based on Raft improvement, wherein the consensus system based on Raft improvement includes:
the node pre-sequencing module is used for enabling all nodes to be slave nodes when the Raft consensus system is started, automatically performing master node election by the Raft consensus system, and if the master node election is successful, counting node information and processing to obtain a node sequencing sequence;
The self-adaptive network segmentation module is used for calculating the number of the auxiliary nodes according to the self-adaptive strategy after the node ordering sequence is obtained, generating the auxiliary nodes according to the node ordering sequence, generating a network topology structure according to the segmentation strategy, broadcasting the network topology structure and segmenting all nodes except the main node;
and the crash recovery module is used for executing the consensus service of the Raft consensus system after the self-adaptive network slicing is completed, and if the main node fails, the auxiliary node with the first order in the node ordering sequence is inherited as the main node through a crash recovery mechanism so as to enable the Raft consensus system to recover normal service.
In addition, in order to achieve the aim, the invention also provides a terminal, wherein the terminal comprises a memory, a processor and a common identification program which is stored in the memory and can run on the processor and is based on Raft, and the common identification program based on Raft is executed by the processor to realize the steps of the common identification method based on Raft.
In addition, to achieve the above object, the present invention also provides a computer-readable storage medium storing a consensus program based on Raft improvement, which when executed by a processor, implements the steps of the consensus method based on Raft improvement as described above.
In the invention, when Raft consensus system is started, all nodes are slave nodes, the Raft consensus system automatically performs master node election, if the master node election is successful, node information is counted and processed to obtain a node sorting sequence, after the node sorting sequence is obtained, the number of the slave nodes is calculated according to an adaptive strategy, the slave nodes are generated according to the node sorting sequence, a network topology structure is generated according to a slicing strategy, the network topology structure is broadcasted and all nodes except the master node are sliced, after the adaptive network slicing is completed, the consensus service of the Raft consensus system is executed, if the master node fails, the slave nodes with the first order in the node sorting sequence are inherited as the master node through a crash recovery mechanism, so that the Raft consensus system is recovered to normal service. The invention improves the common-knowledge algorithm based on Raft, improves the availability of the common-knowledge system through a pre-sequencing method and a crash recovery mechanism, improves the throughput performance and the expandability of the common-knowledge system through the self-adaptive network slicing, and realizes the great improvement of the availability, the throughput performance and the expandability of the Raft common-knowledge system.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 2 is a flow chart of the overall implementation of the improved consensus method based on Raft in the preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 3 is a schematic diagram of node states in a preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 4 is a flow chart of a node pre-ordering method in a preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 5 is a flow chart of a crash recovery mechanism in a preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 6 is a schematic diagram of an adaptive network allocation flow in a preferred embodiment of the improved consensus method based on Raft of the present invention;
FIG. 7 is a schematic diagram of a preferred embodiment of the present invention based on Raft improved consensus system;
FIG. 8 is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention improves the common-knowledge algorithm based on Raft, improves the availability of the common-knowledge system through a pre-sequencing method and a crash recovery mechanism, improves the throughput performance and expandability of the common-knowledge system through the self-adaptive network slicing, realizes the great improvement of the availability, throughput performance and expandability of the Raft common-knowledge system, and the technical breakthrough of the invention is helpful for the further development of the performances of the traditional distributed system and the blockchain system.
Aiming at the improvement of Raft consensus availability, the invention provides a pre-ordering method, which utilizes the information of log synchronous progress, network condition, node configuration and the like of all nodes to generate a node ordering sequence, namely a collapse inheritance sequence, and realizes automatic and rapid recovery when a main node in Raft has a collapse error through a collapse recovery mechanism.
Aiming at the improvement of Raft common knowledge in terms of throughput and expandability, the invention provides a self-adaptive network slicing method, which calculates the number of auxiliary nodes according to a self-adaptive strategy and generates the auxiliary nodes, and generates a network topology structure according to a slicing strategy, thereby reducing the network communication pressure of a main node and enabling a common knowledge system to automatically perform network slicing.
The improved consensus method based on Raft according to the preferred embodiment of the present invention, as shown in fig. 1, comprises the following steps:
Step S10, when Raft the consensus system is started, all nodes are slave nodes, the Raft consensus system automatically performs master node election, if the master node election is successful, node information is counted and processed to obtain a node sequencing sequence;
step S20, after the node ordering sequence is obtained, calculating the number of auxiliary nodes according to the self-adaptive strategy, generating auxiliary nodes according to the node ordering sequence, generating a network topology structure according to the slicing strategy, broadcasting the network topology structure and slicing all nodes except the main node;
and step S30, after the self-adaptive network slicing is completed, executing the consensus service of the Raft consensus system, and if the main node fails, inheriting the auxiliary node with the first order in the node ordering sequence as the main node through a crash recovery mechanism so as to enable the Raft consensus system to recover normal service.
Specifically, the invention adds two innovation points of a pre-sequencing method, a collapse recovery mechanism and an adaptive network slicing on the basis of Raft common to the original functions, enables the common system to realize the function of quickly recovering the faults of the main nodes through the pre-sequencing method and the collapse recovery mechanism, improves the usability of the system, enables the common system to automatically perform network slicing on all nodes according to the adaptive strategy through the adaptive network slicing, reduces the network communication pressure of the main nodes, and improves the throughput capacity and the expandability of the common system. As shown in fig. 2, the detailed operation steps of the Raft improved consensus system are as follows:
Starting;
Step S1, master node election, wherein all nodes are slave nodes when a Raft consensus system is started, and the Raft consensus system automatically performs master node election;
Step S2, judging whether the master node is selected successfully or not, if yes, entering step S3, executing node pre-sequencing, if no, entering step S1, and re-selecting the master node;
step S3, node pre-sequencing, namely if the main node election is successful, node pre-sequencing is carried out, and the node pre-sequencing is carried out by counting and processing node information to obtain a node sequencing sequence;
S4, self-adaptive network slicing, namely calculating the number of auxiliary nodes according to the self-adaptive strategy, generating the auxiliary nodes according to the node ordering sequence, generating a network topology structure according to the slicing strategy, broadcasting the network topology structure and slicing all nodes except the main node;
S5, executing other part of services of the Raft consensus system until the Raft consensus system exits;
step S6, judging whether the main node has faults, if yes, entering step S7, executing a crash recovery mechanism, if no, entering step S5, and continuing to execute the consensus service;
and S7, a crash recovery mechanism, wherein if the main node fails, the Raft consensus system is recovered to normal service through the crash recovery mechanism.
And (5) ending.
The node states are shown in fig. 3, one Raft consensus system is composed of a plurality of consensus nodes, and the nodes in the Raft consensus system have three states, namely a slave node, a candidate node and a master node, and auxiliary nodes are added in the invention to realize subsequent functions. As shown in fig. 3, the detailed change situation of the node state is as follows:
case 1, lift consensus System initialization, all nodes are slave nodes.
And 2, converting the slave node into a candidate node due to the fact that the slave node does not receive the master node message after overtime, and starting to take the election.
And 3, the candidate nodes acquire more than half votes and become master nodes, and start to perform consensus service.
Case 4, master node discovers node in higher tenure period and becomes slave node.
Case 5 when election is held, the candidate node that is not selected will become the slave node after the master node has generated or entered the next period.
And 6, when the election is carried out, the election is carried out again until the election timer is overtime and the election is not successful.
And 7, after the primary node is changed or the secondary node is lost, carrying out node pre-sequencing and self-adaptive network slicing flow according to a preset strategy, and selecting the secondary node from the secondary node.
Case 8. When the master fails, the slave node that is first in the ordering order will inherit the master node.
And 9, after the main node is changed, the rest auxiliary nodes are changed into auxiliary nodes to wait for re-screening.
Specifically, the node pre-ordering method evaluates log indexes, hardware performance, network states or other custom strategy conditions of the nodes by counting node information, orders all slave nodes, and finally obtains an ordered node sequence. As shown in fig. 4, the detailed workflow of node pre-ordering is as follows:
Starting;
s101, counting slave node information, namely collecting and counting related information of all slave nodes in the Raft consensus system by a master node, wherein the related information comprises a log index, hardware performance and network state;
s102, judging whether the log index changes or not, if not, executing a step S104, and if so, executing a step S103 to update the node log index information;
S103, updating the log index information of the slave node, wherein if the log index changes, the log index information of the slave node is updated according to the statistical result;
s104, if the log index is not changed, judging whether the hardware performance is changed, if not, executing step S106, if so, executing step S105, and updating node hardware performance information;
s105, updating the hardware performance information of the slave node, wherein if the hardware performance changes, the hardware performance information of the slave node is updated according to the statistical result;
s106, if the hardware performance is not changed, judging whether the network state is changed, if not, executing step S108, and if so, executing step S107 to update the node network state information;
S107, updating the network state information of the slave node, wherein if the network state changes, the network state information of the slave node is updated according to the statistical result;
S108, pre-ordering the slave nodes, namely, if the network state is not changed, ordering the nodes according to the statistical information and the ordering strategy to obtain a node ordering sequence;
S109, updating the node ordering sequence in the Raft consensus system;
and (5) ending.
Specifically, through the existing slave node sequencing sequence, the probability sequence of the node which can be selected by the Raft consensus system if reelect is carried out can be obtained equivalently, so that the crash recovery mechanism uses the sequence as the inheritance sequence when the master node fails, thereby rapidly recovering the function of the consensus system. Similarly, the node ordering sequence after the adaptive network slicing is also effective, so that the crash recovery mechanism can also process the secondary node loss fault. As shown in fig. 5, the detailed workflow of the crash recovery mechanism is as follows:
Starting;
S301, judging whether the main node is lost, if yes, executing a step S302, and if no, executing a step S307;
S302, searching a first auxiliary node in the sequence, namely searching the first auxiliary node according to the node ordering sequence if the main node is lost, and regarding an ordering strategy, the first node in the node ordering sequence is a node with the optimal ordering result and sequentially goes backwards;
S303, other auxiliary nodes send main node loss information to the auxiliary nodes, wherein the other auxiliary nodes in the node sequencing sequence send the main node loss information to the first auxiliary node in the inheritance order for requiring the first auxiliary node to be a new main node;
s304, judging whether the first auxiliary node receives auxiliary node information exceeding 1/2, if yes, executing step S305, and if no, executing step S306;
S305, generating a new main node and entering a next period, namely if a first auxiliary node receives auxiliary node information exceeding 1/2, generating the new main node, adding the period number of the main node and carrying out information broadcasting so that the whole Raft consensus system enters the next period;
s306, if the first auxiliary node does not receive the auxiliary node message exceeding 1/2, the first auxiliary node in the inheritance order is considered to be unavailable, and the next auxiliary node is searched for trying again, namely when the first auxiliary node in the node ordering sequence does not receive the main node loss message of the auxiliary node which is equal to or greater than 1/2 in the appointed time, the first auxiliary node in the inheritance order is considered to be unavailable, and then the next auxiliary node is searched for trying again;
S307, if the main node is not lost, judging whether the auxiliary node is lost, if yes, executing step S308, and if no, executing step S311;
S308, searching a first slave node in the slicing, namely searching the first slave node according to the sliced node sequencing sequence by the master node if the slave node is lost, wherein the sliced node sequencing sequence is still effective in terms of slicing strategies, and the first node is the node with the optimal sequencing result and sequentially goes backwards;
S309, the master node sends a slave node loss message to the master node, wherein the master node sends the slave node loss message to the node in the inheritance order for requiring the node in the inheritance order to become a new slave node;
S310, generating a new auxiliary node and replacing the original auxiliary node, namely after generating the new auxiliary node, replacing the network fragment of the original auxiliary node;
S311, the processing service is lost by the slave node, wherein when the slave node is lost, the processing service is executed according to the slave node loss in Raft consensus system;
and (5) ending.
Specifically, by means of the pre-ordering method of the nodes, a set of node ordering sequences according to a certain priority rule can be obtained, where the node ordering sequences are considered to meet the priority condition of the current service logic. And the self-adaptive network slicing is used for generating a secondary node sequence according to the existing node sequencing sequence and combining self-adaptive strategy calculation, generating a topological network structure according to the slicing strategy, and completing network slicing through broadcasting. As shown in fig. 6, the detailed workflow of adaptive network slicing is as follows:
Starting;
s201, calculating the number of auxiliary nodes according to the self-adaptive strategy, wherein the main node calculates the number of auxiliary nodes required by network fragmentation according to the set self-adaptive strategy;
s202, generating auxiliary nodes according to the node pre-ordering sequence, wherein the auxiliary node numbers are determined from front to back in sequence according to the ordering sequence of the node ordering sequence;
S203, generating a network topology structure according to a slicing strategy, namely dividing the rest slave nodes according to the slicing strategy by using the counted node information and generating the network topology structure in a one-to-one correspondence with the slave nodes according to rules, wherein the slicing strategy is not unique and can be selected and divided according to node number hash mapping, network conditions and hardware configuration to generate the network topology structure;
and S204, fragmenting the nodes, namely, the master node sends the network topology structure to the slave nodes through heartbeat messages, and the slave nodes automatically adjust the message receiving and transmitting objects or convert the message receiving and transmitting objects into the slave nodes after receiving the message receiving and transmitting objects.
And (5) ending.
(1) The invention provides a node pre-ordering and crash inheritance mechanism, which can greatly improve the usability of Raft consensus system when crash errors occur under the condition of not increasing excessive system overhead. The present invention may be adapted to other consensus algorithms and Raft improvement techniques and methods, such as multiple Raft groupings, without regard to the specific consensus scheme and form.
(2) The invention provides a self-adaptive network slicing solution, which determines the number of auxiliary nodes through a self-adaptive algorithm, and then performs network slicing through condition information such as number Hash, network condition, geographical position and the like, thereby relieving network communication pressure brought by a main node along with the increase of nodes and improving system throughput and expandability.
The evaluation strategies used in the two techniques can be recombined or changed according to specific use conditions and requirements, and the flow steps can be customized and combined.
Compared with a common identification system of common Raft, the common identification method based on Raft improvement provided by the invention has the following advantages:
(1) Compared with a real-time monitoring scheme, the inheritance sequence when the crash error occurs is obtained through the pre-sequencing method, so that the main node can be quickly recovered when the crash error occurs, the log safety is ensured as much as possible, and the usability of the system is improved.
(2) Compared with the network slicing scheme, the adaptive network slicing is adopted, the number of auxiliary nodes is determined in an adaptive mode without pre-configuring a consensus system, slicing of network nodes is completed automatically, system delay and throughput are balanced well, and system performance and expandability are improved.
In conclusion, the availability and throughput capacity of the consensus system are greatly improved on the basis of the prior art. And the invention has been carried out many times of simulation tests, the experiment proves that the invention can meet the expected design index.
Further, as shown in fig. 7, based on the above-mentioned consensus method based on Raft improvement, the present invention also correspondingly provides a consensus system based on Raft improvement, wherein the consensus system based on Raft improvement includes:
The node pre-ordering module 51 is configured to, when the Raft consensus system is started, automatically perform master node election by the Raft consensus system, and if the master node election is successful, count node information and process the node information to obtain a node ordering sequence;
The adaptive network slicing module 52 is configured to calculate the number of secondary nodes according to an adaptive policy after the node ordering sequence is obtained, generate secondary nodes according to the node ordering sequence, generate a network topology according to a slicing policy, broadcast the network topology, and slice all nodes except the primary node;
and the crash recovery module 53 is configured to execute the consensus service of the Raft consensus system after the adaptive network slicing is completed, and if the master node fails, inherit the slave node with the first order in the node ordering sequence as the master node through a crash recovery mechanism, so that the Raft consensus system resumes normal service.
Further, as shown in fig. 8, based on the above-mentioned common recognition method and system based on Raft improvement, the invention also correspondingly provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Fig. 8 shows only some of the components of the terminal, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead.
The memory 20 may in some embodiments be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 20 may in other embodiments also be an external storage device of the terminal, such as a plug-in hard disk provided on the terminal, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. Further, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used for storing application software installed in the terminal and various data, such as program codes of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 has stored thereon a Raft-based improved consensus program 40, the Raft-based improved consensus program 40 being executable by the processor 10 to implement the Raft-based improved consensus method of the present application.
The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, such as performing the Raft-based improved consensus method or the like.
The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 30 is used for displaying information at the terminal and for displaying a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.
In an embodiment, the steps of the improved consensus method based on Raft as described above are implemented when the processor 10 executes the consensus program 40 based on Raft improvement in the memory 20.
The present invention also provides a computer-readable storage medium storing a Raft-based improved consensus program, the Raft-based improved consensus program implementing the steps of the Raft-based improved consensus method as follows when executed by a processor:
When Raft the consensus system is started, all nodes are slave nodes, the Raft consensus system automatically performs master node election, if the master node election is successful, node information is counted and processed to obtain a node ordering sequence;
after the node ordering sequence is obtained, the number of auxiliary nodes is calculated according to the self-adaptive strategy, the auxiliary nodes are generated according to the node ordering sequence, a network topology structure is generated according to the slicing strategy, and all nodes except the main node are sliced by broadcasting the network topology structure;
And after the self-adaptive network slicing is completed, executing the consensus service of the Raft consensus system, and if the main node fails, inheriting the auxiliary node with the first order in the node ordering sequence as the main node through a crash recovery mechanism so as to enable the Raft consensus system to recover normal service.
The node information is counted and processed to obtain a node ordering sequence, which specifically comprises the following steps:
The master node collects and counts related information of all slave nodes in the Raft consensus system, wherein the related information comprises log indexes, hardware performance and network states;
Judging whether the log index changes or not;
if the log index changes, updating log index information of the slave node according to the statistical result;
if the log index is unchanged, judging whether the hardware performance is changed or not;
If the hardware performance changes, updating the hardware performance information of the slave node according to the statistical result;
if the hardware performance is not changed, judging whether the network state is changed or not;
If the network state changes, updating the network state information of the slave node according to the statistical result;
if the network state is unchanged, ordering the nodes according to the statistical information and the ordering strategy to obtain a node ordering sequence;
and updating the node ordering sequence in the Raft consensus system.
The method for recovering the node sequence comprises the steps of enabling the secondary node with the first order in the node sequence to be inherited as a primary node through a crash recovery mechanism so as to enable the Raft consensus system to recover normal service, and specifically comprises the following steps:
Judging whether the master node is lost or not;
if the master node is lost, searching a first slave node according to the node ordering sequence;
Other auxiliary nodes in the node ordering sequence send a main node losing message to a first auxiliary node in the inheritance order for requiring the first auxiliary node to become a new main node;
judging whether the first auxiliary node receives auxiliary node information exceeding 1/2;
if the first auxiliary node receives the auxiliary node information exceeding 1/2, generating a new main node, adding the wilting period number of the main node and broadcasting the information, so that the whole Raft consensus system enters the next wilting period;
If the first auxiliary node does not receive the auxiliary node information exceeding 1/2, the first auxiliary node in the inheritance order is considered to be unavailable, and the next auxiliary node is searched for trying again;
if the main node is not lost, judging whether the auxiliary node is lost or not;
if the auxiliary node is lost, the main node searches a first auxiliary node according to the segmented node sequencing sequence;
The master node sends a slave node loss message to the node in the inheritance order, and the slave node loss message is used for requesting the node in the inheritance order to become a new slave node;
after the new secondary node is generated, the network fragments of the original secondary node are replaced.
The method specifically comprises the steps of generating auxiliary nodes according to the node ordering sequence, generating a network topology structure according to a slicing strategy, broadcasting the network topology structure and slicing all nodes except a main node, wherein the number of the auxiliary nodes is calculated according to the self-adaptive strategy, and the method specifically comprises the following steps:
the master node calculates the number of auxiliary nodes required by network fragmentation according to the set self-adaptive strategy;
Determining the number of the auxiliary node from front to back in sequence according to the ordering sequence of the node ordering sequence;
Dividing the rest slave nodes according to the slicing strategy by using the counted node information, and generating a network topological structure according to the rule one-to-one correspondence with the slave nodes;
the master node sends the network topology structure to the slave node through heartbeat messages, and the slave node automatically adjusts a message receiving and transmitting object or converts the message receiving and transmitting object into the slave node after receiving the message.
The slicing strategy comprises selecting and dividing according to node number hash mapping, network conditions and hardware configuration to generate a network topology structure.
Wherein the secondary node message represents a primary node lost message.
Wherein, the common identification method based on Raft improvement further comprises the following steps:
When election is carried out, the candidate nodes which are not selected become slave nodes after the generation of the master node or after the next period is entered;
when the election is carried out, the election is carried out again until the election timer is overtime and the election is not successful;
when the master node is changed, the rest of the slave nodes are changed into slave nodes, and the slave nodes wait for re-screening.
The present invention also provides a computer readable storage medium storing a consensus program based on Raft, which when executed by a processor implements the steps of the consensus method based on Raft improvement as described above.
In summary, the invention provides a common identification method based on Raft improvement and related equipment, the method comprises the steps that when a Raft common identification system is started, all nodes are slave nodes, the Raft common identification system automatically performs master node election, if the master node election is successful, node information is counted and processed to obtain a node ordering sequence, after the node ordering sequence is obtained, the number of auxiliary nodes is calculated according to an adaptive strategy, the auxiliary nodes are generated according to the node ordering sequence, a network topology is generated according to a slicing strategy, the network topology is broadcasted and all nodes except the master node are sliced, after the adaptive network slicing is completed, common identification service of the Raft common identification system is executed, and if the master node fails, the auxiliary node with the first ordering sequence in the node ordering sequence is inherited as the master node through a crash recovery mechanism, so that the Raft common identification system recovers normal service. The invention improves the common-knowledge algorithm based on Raft, improves the availability of the common-knowledge system through a pre-sequencing method and a crash recovery mechanism, improves the throughput performance and the expandability of the common-knowledge system through the self-adaptive network slicing, and realizes the great improvement of the availability, the throughput performance and the expandability of the Raft common-knowledge system.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
Of course, those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by a computer program for instructing relevant hardware (e.g., processor, controller, etc.), the program may be stored on a computer readable storage medium, and the program may include the above described methods when executed. The computer readable storage medium may be a memory, a magnetic disk, an optical disk, etc.
It is to be understood that the invention is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (9)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211509442.XA CN115801553B (en) | 2022-11-29 | 2022-11-29 | An improved consensus method based on Raft and related equipment |
| PCT/CN2022/136054 WO2024113334A1 (en) | 2022-11-29 | 2022-12-02 | Improved consensus method based on raft and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211509442.XA CN115801553B (en) | 2022-11-29 | 2022-11-29 | An improved consensus method based on Raft and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115801553A CN115801553A (en) | 2023-03-14 |
| CN115801553B true CN115801553B (en) | 2025-05-09 |
Family
ID=85442949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211509442.XA Active CN115801553B (en) | 2022-11-29 | 2022-11-29 | An improved consensus method based on Raft and related equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115801553B (en) |
| WO (1) | WO2024113334A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118332617B (en) * | 2024-06-12 | 2024-08-23 | 中国民航大学 | A traceability method and system for trusted storage of aircraft engine maintenance data |
| CN119094521B (en) * | 2024-08-19 | 2025-09-26 | 浙江大学 | A serverless blockchain consensus optimization system |
| CN118796566B (en) * | 2024-09-10 | 2025-02-18 | 阿里云计算有限公司 | Distributed service system, instance splitting method, device, medium and program product |
| CN119402238B (en) * | 2024-10-23 | 2025-09-19 | 扬州大学 | GRAFT-EWMA consensus algorithm-based distributed e-commerce order processing method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113676541A (en) * | 2021-08-23 | 2021-11-19 | 南昌航空大学 | An Improved PBFT Consensus Method |
| CN114928446A (en) * | 2022-06-10 | 2022-08-19 | 中国电子科技集团公司第五十四研究所 | Efficient consensus method for large-scale nodes |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019213867A1 (en) * | 2018-05-09 | 2019-11-14 | 合肥达朴汇联科技有限公司 | Method and device for reaching consensus in blockchain |
| CN111464349A (en) * | 2020-03-30 | 2020-07-28 | 南京中诚区块链研究院有限公司 | Mixed consensus network algorithm and system of block chain Raft + PBFT |
| CN114490125B (en) * | 2022-01-19 | 2025-04-15 | 上海沄熹科技有限公司 | An optimization method for the pre-election process in the Raft consensus algorithm |
| CN114499890B (en) * | 2022-03-15 | 2023-09-15 | 南京信息工程大学 | Raft PBFT two-stage consensus method based on node grouping in alliance chain |
| CN115022022B (en) * | 2022-05-31 | 2023-07-18 | 南京邮电大学 | An Improved Raft Consensus Mechanism Based on Past Behavior Analysis of Nodes |
| CN115102967B (en) * | 2022-06-15 | 2024-08-27 | 西安工业大学 | Consensus method with high consensus efficiency and distributed system |
-
2022
- 2022-11-29 CN CN202211509442.XA patent/CN115801553B/en active Active
- 2022-12-02 WO PCT/CN2022/136054 patent/WO2024113334A1/en not_active Ceased
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113676541A (en) * | 2021-08-23 | 2021-11-19 | 南昌航空大学 | An Improved PBFT Consensus Method |
| CN114928446A (en) * | 2022-06-10 | 2022-08-19 | 中国电子科技集团公司第五十四研究所 | Efficient consensus method for large-scale nodes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115801553A (en) | 2023-03-14 |
| WO2024113334A1 (en) | 2024-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115801553B (en) | An improved consensus method based on Raft and related equipment | |
| US11924044B2 (en) | Organizing execution of distributed operating systems for network devices | |
| US10887173B2 (en) | Communicating state information in distributed operating systems | |
| US11316775B2 (en) | Maintaining coherency in distributed operating systems for network devices | |
| US9325757B2 (en) | Methods and systems for fault-tolerant distributed stream processing | |
| KR101887581B1 (en) | Flow-based packet transport device and packet management method thereof | |
| EP3688934B1 (en) | Migrating services in data communication networks | |
| US8923113B2 (en) | Optimizations in multi-destination tree calculations for layer 2 link state protocols | |
| US11595294B1 (en) | Satisfying demands in data communication networks | |
| CN113760468A (en) | Distributed election method, device, system and medium | |
| CN115277379A (en) | Distributed lock disaster tolerance processing method and device, electronic equipment and storage medium | |
| CN113254511B (en) | Distributed vector retrieval system and method | |
| Fang et al. | A fast and load-aware controller failover mechanism for software-defined networks | |
| CN112866833A (en) | Service recovery method and device, electronic equipment and storage medium | |
| CN109189403B (en) | Operating system OS batch installation method and device and network equipment | |
| CN114448868B (en) | Path scheduling method, device and equipment based on segmented routing strategy | |
| CN110798492B (en) | Data storage method and device and data processing system | |
| US11695856B2 (en) | Scheduling solution configuration method and apparatus, computer readable storage medium thereof, and computer device | |
| CN110704245A (en) | A service cluster implementation method, apparatus, device and readable storage medium | |
| CN117978851B (en) | Session connection method, interaction method, device, equipment and medium | |
| WO2022156354A1 (en) | Version configuration method, apparatus, device, system, and storage medium | |
| US10165086B2 (en) | Information processing system, server apparatus, information processing method, and computer program product | |
| JP4818379B2 (en) | Distributed system having failover function and failover method in the same system | |
| US11671323B1 (en) | Preplan assignment generation | |
| CN106533936A (en) | Management method and device for graceful restart of router |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |