CN100530106C - Realization Method of Multi-machine Fault-Tolerant System Kernel - Google Patents

Abstract

The present invention discloses a method for realizing a multi-machine fault-tolerant system kernel, wherein a middleware is formed between an application program and an operating system, and system resources and application programs are managed in a multi-tasking manner, so that the input and output of the application program are all completed through the middleware. Synchronization, data exchange, and output voting between operation units are realized in the middleware. An interface with the application program is realized by establishing and managing a buffer in the middleware, synchronization between operation units is realized by multi-tasking management and data exchange, and output voting is realized by output data comparison. The present invention realizes a fault-tolerant kernel by adding a software middleware between the application program and the operating system, without the need for complex hardware circuits; a higher synchronization density is realized by multi-tasking management and data exchange; and output voting is realized by output data comparison, without the need for adding an output module. Various different forms of fault-tolerant structures can also be realized by configuring the middleware.

Description

Realization Method of Multi-machine Fault-Tolerant System Kernel

Technical field

The invention relates to a method for realizing the kernel of a multi-machine fault-tolerant system in a computer automatic control system.

Background technique

In the field of automatic control systems involving major personal and equipment safety, the system is required to have extremely high reliability, and the system is required not only to ensure system safety during normal operation, but also to ensure that the system is guided to safety by failure when a failure occurs. Therefore, a high-reliability, high-security multi-machine fault-tolerant system based on fault-tolerant technology came into being. Taking two out of three computer systems as an example, three computers are set up as three computing units, and only when two or more of them have the same computing results, the computing results are considered correct and output.

The core technology to realize multi-computer fault tolerance is synchronization. Only under the premise of synchronization can multiple computing units output computing results in the same state for voting. There are two ways of synchronization, one is tight synchronization: that is to force multiple computing units to run synchronously in strict accordance with the coordinated beat through the hardware synchronization device. However, this method needs to increase the synchronization hardware equipment, and the technical requirements for realizing this function are relatively high. Another way is loose synchronization: that is, through software methods, multiple computing units can be coordinated to an approximately synchronized state through software under their respective clocks. Most loose synchronization mechanisms use timing synchronization and output synchronization, that is, after the system starts running, it sends synchronization information regularly for timing synchronization, and waits for the output results of multiple computing units through a device at the output end, and when the output results arrive Then vote on the output. Chinese patent 00109094 proposes to use a synchronization frame timing sending program for timing synchronization, and connect a plurality of independent output units OCM with solidified execution programs in series on the field data line, so that the three computing units can realize synchronization and voting. This synchronization method is mainly realized by running software, which saves complex hardware circuits for the entire system. However, since voting is only performed on the external output unit, and the three computing units do not exchange data during application execution, the synchronization accuracy of the system is reduced. And need to increase the output unit.

Contents of the invention

The purpose of the present invention is to overcome the technical defects in the prior art and provide a method for realizing the kernel of a multi-computer fault-tolerant system through software. It can provide high system synchronization accuracy, so that system faults can be found and solved as soon as possible. At the same time, the method is completely implemented in software, based on a real-time multitasking system, and uses standard posix and operating system interfaces. There is no need to add any hardware devices such as input and output units, and it has strong portability.

The basic principle of the present invention to realize the purpose of the invention: on the basis of the real-time multitasking operating system, a middleware is formed between the application program and the operating system, and the system resources, application programs, etc. are managed in a multi-task manner, so that the input and output of the application program Both are completed through the middleware, the interface between the middleware and the application program is realized through buffer management, and the interface between the middleware and the operating system is realized through the standard posix and operating system interface. In the middleware, the synchronization and data exchange and output voting between the computing units (also known as computing units) are realized. In the middleware, the interface with the application program is realized by establishing and managing the buffer zone, the synchronization between the computing units is realized through multi-task management and data exchange, and the output voting is realized through output data comparison.

The present invention is based on the above-mentioned principles, and the technical solution adopted for realizing the purpose of the invention is: the realization method of the kernel of the multi-machine fault-tolerant system, forming a software middleware between the application program and the operating system, and adopting a multi-task mode to manage system resources, application Program; it is specifically divided into application tasks, main scheduling tasks, computing unit status monitoring tasks, and communication channel management tasks; among them:

Application task: application execution task, created by the main scheduling task, ends by itself or destroyed by the main scheduling task;

The main scheduling task (the scheduling task of the main channel, also known as the scheduling task of the computing unit of the main channel): responsible for the synchronization of the computing units of the system, data distribution, data comparison, and operation scheduling;

Computing unit status monitoring task: responsible for monitoring the communication status of the current computing unit and partner computing units, and judging the status of the entire system;

Communication channel management task: responsible for monitoring and managing communication channels, receiving and sending data;

System kernel process: a process synchronization is performed at the beginning of the entire operation cycle, so that multiple computing units start the operation of the cycle at approximately the same time; after the process synchronization, the data input process is performed, and multiple computing units obtain data from the outside respectively, and then Just perform data synchronization once (data synchronization 1), so that the data acquired by multiple computing units is consistent, and at the same time, the application program starts to execute at approximately the same time; after the input data is synchronized, the application task is started to execute the application program; the waiting application program After completion, the calculation result is obtained, and the calculation result is distributed to the partner computing unit, after which the second data synchronization (data synchronization 2) is performed to ensure that each computing unit has obtained the data required for comparison, and at the same time makes the result The comparison process starts to execute at approximately the same time; then the result comparison process is performed, and the third data synchronization (data synchronization 3) is performed after the comparison result is obtained to ensure that the comparison result, that is, the data to be output is consistent, and The output of the result data is performed at approximately the same time; next, a single or multiple computing units can be selected to output at the same time.

Process synchronization process: At the beginning of a computing cycle, the scheduling task of the computing unit as the main channel first issues a process synchronization command, and then the scheduling task is suspended to wait for the reply from the channel; the scheduling task from the channel first suspends itself and waits for the response from the main channel Process synchronization command; when the communication channel management task receives the process synchronization command sent by the main channel, it responds immediately; after the communication channel management task of the main channel receives the process synchronous response (or timeout) of all slave channels, it sends a message to all slave channels Issue a process execution instruction, and at the same time give a signal to continue the execution of the local scheduling task, so that the scheduling task of the main channel continues to execute; after the slave channel obtains the process execution instruction sent by the main channel, it gives a signal to continue the execution of the local scheduling task Signal, so that the scheduling tasks of the slave channel continue to execute; in this way, the difference between multiple computing units is only one transmission delay time, which can be approximately regarded as three computing units starting to run at the same time.

The selection of the main channel can be realized in various ways such as rotation or preemption.

Data synchronization process: the scheduling task distributes the data that needs to be synchronized, such as input data, to the partner computing channel, and then sends a data synchronization command, indicating that the data needs to be synchronized after sending, and then the scheduling task suspends itself, waiting for the data transmission of other computing channels to complete ( data synchronization command); next, after the communication management task receives the data synchronization command (or timeout) sent by all partner computing units, it indicates that the data of all computing channels has been received, and then gives the local scheduling task to continue execution Signal, so that the scheduling tasks of the local machine continue to execute; since each channel waits until the data of all other channels has been sent, the multiple computing units at this time can be approximately regarded as steps after data synchronization is performed at the same time.

The present invention is based on the real-time multitasking operating system, realizes the fault-tolerant kernel by adding middleware (software) between the application program and the operating system, and does not need complex hardware circuits; realizes the interface between the middleware and the application program through buffer management, The interface between the middleware and the operating system is realized through the standard posix and operating system interface, which is extremely portable; high synchronization density is achieved through multi-task management and data exchange; output voting is realized through output data comparison without adding an output module . And can realize many different forms of fault-tolerant structures through the configuration of the middleware.

Description of drawings

Fig. 1 is the computing cycle process diagram (take two out of three fault-tolerant systems) of the multi-machine fault-tolerant system kernel implementation method of the present invention;

Fig. 2 is the process synchronous process diagram (take two out of three fault-tolerant systems) of the multi-machine fault-tolerant system kernel implementation method of the present invention;

Fig. 3 is the data synchronous process diagram (take two out of three fault-tolerant systems) of the multi-computer fault-tolerant system kernel implementation method of the present invention;

Fig. 4 is a program block diagram of the method for realizing the kernel of the multi-machine fault-tolerant system of the present invention (the selection of the main channel adopts a rotation mode)

Fig. 5 is a program block in which the selection of the main channel in the multi-computer fault-tolerant system kernel implementation method of the present invention is implemented in a preemptive manner.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

Embodiment: Take two out of three fault-tolerant systems as an example. The multi-computer fault-tolerant system kernel implementation method proposed by the invention forms a software middleware between the application program and the operating system, and manages system resources, application programs, etc. in a multi-task manner. Specifically, it can be divided into application tasks, main scheduling tasks, computing unit status monitoring tasks, and communication channel management tasks, among which:

Application task: the application execution task, created by the main scheduling task, ends by itself or is destroyed by the main scheduling task.

Main scheduling task: responsible for the synchronization and operation scheduling of the computing units of the two-out-of-three system.

Computing unit status monitoring task: responsible for monitoring the communication status of the current computing unit and partner computing units, and judging the status of the entire system.

Communication channel management task: responsible for monitoring and managing communication channels, receiving and sending data.

The system operation cycle process is shown in Figure 1: a process synchronization is performed at the beginning of the entire operation cycle, so that the three computing units start the operation of the cycle at approximately the same time. After the process is synchronized, the data input process is performed. The three computing units obtain data from the outside respectively, and then perform a data synchronization (data synchronization 1), so that the data obtained by the three computing units are consistent, and at the same time, the application program starts to execute at approximately the same time. The start of the application task execution application starts after the input data is synchronized. After the application program is completed, the calculation results are obtained, and the calculation results are distributed to the partner computing units. After that, the second data synchronization (data synchronization 2) is performed to ensure that each computing unit has obtained the data required for comparison. At the same time, the result comparison process is started at approximately the same time. Next, the result comparison process is performed. After the comparison result is obtained, the third data synchronization (data synchronization 3) is performed to ensure that the comparison result, that is, the data to be output is consistent, and the result data output is performed at approximately the same time. . Next, you can choose single, double or three arithmetic units to output simultaneously.

The process synchronization process is shown in Figure 2: at the beginning of a computing cycle, the scheduling task of the computing unit as the master channel first issues a process synchronization command, and then the scheduling task is suspended and waits for the slave channel to respond. The slave channel scheduling task first suspends itself and waits for the process synchronization command of the master channel. When the communication channel management task receives the process synchronization instruction sent by the main channel, it responds immediately. After the communication channel management task of the main channel receives the process synchronous response (or timeout) of all the slave channels, it sends process execution instructions to all the slave channels, and at the same time gives a signal to continue the execution of the local scheduling task, so that the scheduling task of the main channel continues to execute . After the slave channel receives the process execution instruction sent by the master channel, it gives a signal to continue the execution of the local scheduling task, so that the scheduling task of the slave channel continues to execute. In this way, the difference between the three computing units is only one transmission delay time, and it can be approximately regarded as that the three computing units start running at the same time.

In this embodiment, the selection of the main channel adopts a rotation method.

The data synchronization process is shown in Figure 3: the scheduling task distributes the data to be synchronized, such as input data, to the partner computing channel, and then sends a data synchronization command, indicating that the data needs to be synchronized after sending, and then the scheduling task suspends itself and waits for other computing channels The data transmission is completed (data synchronization command). Next, the communication management task indicates that the data of all computing channels have been received after receiving the data synchronization command (or timeout) sent by all the partner computing units. Then, a signal to continue execution of the scheduled task of the local machine is given, so that the scheduled task of the local machine continues to execute. Since each channel has waited until the data of all other channels have been sent, the three computing units at this time can be approximately regarded as a step after performing data synchronization at the same time.

Accompanying drawing 4 provided is the program block diagram of the software middleware of this embodiment (the selection of the main channel adopts the mode of rotation). The main channel is rotated once per computing cycle. The three computing units take turns as the main unit. Manage the entire system together.

When the computing unit re-joins the system, host identification conflicts may occur, so conflict avoidance strategies need to be set. In this embodiment, when host conflicts occur, the pre-set post-order method is used to avoid collisions.

Embodiment 2 is basically the same as the embodiment, except that the selection of the main channel is implemented in a preemptive manner.

Accompanying drawing 5 has given the program block diagram that the selection of main channel adopts preemptive way to realize in this embodiment. In this way, the computing unit status monitoring task sends a status query command to the partner channel in each channel status detection cycle. When there is a return, check the communication status record table, and when the following conditions are met, set the machine as the master.

Condition 1: When only one partner channel establishes a connection with itself, judge whether the other partner channel is really offline (the self-judgment is consistent with the status from the partner channel); the status of the host that responds to the partner channel is non-host; the status of its own host is off-host. At this time, set the host status of itself as the host.

Condition 2: When the two partner channels are connected to themselves, judge whether the status of the two partner channels is consistent (the self-judgment is consistent with the status from the partner channel); the host status of the two partner channels is non-host; the host status of itself for non-host. At this time, set the host status of itself as the host.

Since the three hosts are not fully time-synchronized, there is a certain error in the host identification between channels, which may cause multiple hosts to be judged as hosts at the same time. Once the state monitoring task detects that the state of its own host conflicts with the state of the partner channel host, it will clear the state of its own host and wait for the next judgment to avoid conflicts.

Claims (4)

1, a kind of implementation method of multi-machine fault-tolerance system kermel forms a software middleware between application program and operating system, adopt mode management of system resource, the application program of multitask; Specifically be divided into application task, master scheduler task, computing unit status surveillance task and communication channel management role; Wherein:

Application task: promptly application program is executed the task, and is created by master scheduler task, finishes voluntarily or is destroyed by master scheduler task;

Master scheduler task: be responsible for synchronous and data distribution, data comparison, the computing scheduling of system-computed unit;

Computing unit status surveillance task: be responsible for monitoring current computing unit and partner's computing unit communication state, and carry out the judgement of total system state;

Communication channel management role: be responsible for the monitor management communication channel, receive and send data;

The system kernel process: it is synchronous to carry out a process at the beginning of whole execution cycle, makes a plurality of computing units be in approximate synchronization and begins the computing in this cycle; Process is carried out the data input process synchronously later, a plurality of computing units respectively from outside obtain data, next just carry out the data sync first time, make the data that a plurality of computing unit obtained be consistent, make application program begin to carry out simultaneously at approximate synchronization; Promptly begin to start the application task executive utility after the input data sync; Treat to draw result of calculation after application program is finished, and result of calculation is distributed to partner's computing unit, carry out the data sync second time after this, all obtained more required data, make that simultaneously comparison procedure begins to carry out at approximate synchronization as a result to guarantee each computing unit; Next carry out comparison procedure as a result, carry out data sync for the third time after having drawn comparative result, to guarantee comparative result, the data of promptly preparing output are consistent, and make result data output carry out at approximate synchronization; Next, can select single or multiple arithmetic elements to export simultaneously.

2, according to the implementation method of the described multi-machine fault-tolerance system kermel of claim 1, it is characterized in that: described process synchronizing process is: at the beginning of an execution cycle, scheduler task as the computing unit of main channel is at first sent the process synchronic command, and scheduler task is suspended wait and replys from passage then; Hang up at first voluntarily from the channel scheduling task, wait for the process synchronic command of main channel; Receive when the communication channel management role to make immediately behind the process synchronic command of sending the main channel and reply; The communication channel management role of main channel receives that all send process execution command to all from passage from the process syn ack of passage or overtime back, provides this machine scheduler task simultaneously and continues the signal carried out, makes the scheduler task of main channel continue to carry out; , after the process execution command that has obtained to send the main channel, provide this machine scheduler task and continue the signal carried out from passage, make and continue to carry out from the scheduler task of passage; A plurality of like this arithmetic elements only differ the time of a propagation delay time, can regard three arithmetic elements approx as and bring into operation at synchronization.

3, according to the implementation method of the described multi-machine fault-tolerance system kermel of claim 2, it is characterized in that: the selection of described main channel adopts alternating mode or the mode of trying to be the first to realize.

4, according to the implementation method of the described multi-machine fault-tolerance system kermel of claim 1, it is characterized in that: described data synchronization process is: scheduler task need data in synchronization be distributed to the partner and calculates passage, send the data sync instruction subsequently, it is synchronous to show that data send the needs that finish, scheduler task is hung up voluntarily subsequently, wait for the Data Transfer Done of other arithmetic channels, i.e. data sync instruction; Next the communication channel management role receive the instruction of data sync that all partner's arithmetic elements are sent here or overtime after, the data that show all arithmetic channels are all received, then provide the signal that this machine scheduler task continues execution, make the scheduler task of this machine continue to carry out; Because each passage has all been waited until the data transmission of other all passages and has finished that therefore, a plurality of arithmetic elements of this moment can be regarded the step after synchronization is carried out data sync approx as.

Images