JPH07183891A - Computer system - Google Patents

Abstract

(57) [Summary] [Purpose] In computer systems, the objective is to improve operating rates and reliability by developing less hardware. [Structure] A computer system composed of one or a plurality of computers connected to a plurality of networks,
The programs 112, 113 and 114 to be inspected are the computer 103
Fault detection program 108, 107 or 106 while changing the transmission interval of the detection signal according to the load condition of
Send to. On the other hand, the fault detection program is also the computer 103
The load condition is checked to see if the sent detection signal has timed out to determine whether a failure has occurred.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fault tolerant computer system, and more particularly to a computer system connected to a network.
The present invention relates to a computer system including a plurality of computers.

[0002]

2. Description of the Related Art In recent years, interest in the reliability of computer systems has increased. As the background, in the conventional computer system, the function that was on the general-purpose computer is moved to the terminal side, which is called downsizing, and the low cost by realizing the center function itself on the microprocessor-based computer. There are many attempts to achieve this. However, while a general-purpose machine system has a means for coping with hardware failures, malfunctions, software failures caused by these, and failures of the software itself, while achieving higher operating rates and reliability, In a microprocessor-based computer system, failure coping measures are poor, and it has been difficult to run an application that requires high availability and reliability.

By the way, in downsizing,
Characteristically, there is a change in the connection form of computers.
A system centered on conventional general-purpose machines, with its terminal group,
In contrast to systems that were cluster-connected by telephone lines or serial lines, individual computers are often connected to high-speed local area networks in systems that consist mainly of microprocessor-based computers that are almost equivalent. It was

In order to realize a high operating rate and high reliability of a computer system, which is the subject of the present invention, there have been commercialized products in the past by Tandem, Stratus, Sequoia Systems and the like. These systems
The hardware components are made redundant so that a single point of failure, that is, the entire system does not stop due to the failure of one component. However, hardware redundancy makes these systems expensive.

In recent years, due to the improvement in reliability of hardware and the required operation rate and reliability level, redundancy of all components cannot be required in many cases in view of cost. The present invention is applied to such a system to improve the operating rate and reliability, and minimizes the hardware mechanism, and is mainly implemented by software. A high-speed local area network is the key to achieving these tasks in a computer group with low hardware redundancy. In other words, the lack of hardware redundancy can be covered by complementing multiple computers through the network.
Some products based on this approach are already on the market, such as DEC's VAX cluster and IBM's HAN.
There is FS etc. The product called VAX cluster of DEC is composed of multiple computers that share a dual port disk and are connected by a network. When a failure occurs in the primary computer, the secondary computer takes over the work. To do. IBM's HANFS is running on a computer that is also composed of multiple computers connected to a network that shares disk devices, and when a program that performs a file sharing service between multiple computers fails. The service program is started on another computer sharing the disk device.

Japanese Unexamined Patent Publication No. 4-230538 discloses a method of detecting a faulty software component in which whether or not a response is received within a certain time interval is used as one of the judgments of the fault occurrence, and Japanese Unexamined Patent Publication No. 4-340649 discloses a detection signal. A method of detecting a software failure that does not depend on the transmission of a message is described. JP-A-1-61855 describes a method of activating a backup processor in a multiprocessor system.

Further, as a precedent example for speeding up the dumping of data on the main memory when a failure occurs, Japanese Patent Laid-Open No. Hei 3 (1998)
-2111638 describes a method of compressing core data and then saving it in secondary storage. Also, as a prior patent that was conceived to ensure consistency of disk data,
Japanese Patent Application Laid-Open No. 1-277372 describes a method of returning the written data contents to the host system when an error occurs.

[0008]

SUMMARY OF THE INVENTION An object of the present invention is to provide a basic element for constructing a computer system that provides a high operating rate and reliability with less investment of hardware. .

In the conventional system, when the software failure detection mechanism uses the time-out as a criterion for judging whether or not the failure has occurred, depending on the load state of the computer, the time longer than the time determined for transmission may be exceeded. However, there was a problem that it was timed out, it was judged that a failure occurred, and an accurate judgment could not be made.

Further, when a program for which a failure is to be detected is made up of services of several programs or mutually made up of services, it is possible to detect a failure of a target program. There was a problem that it was not enough to do it, and an accurate judgment could not be made.

The conventional failure detection method has a problem that it cannot cope with a single point failure of the detection mechanism itself.

Further, in the conventional fault detection and restoration method, the restoration means is uniform, and there is a problem that in many cases there is no choice but to take unnecessary measures against the fault.

Further, when a software failure occurs and it is necessary to transfer the software to another computer in the network for recovery, if the transfer destination computer is fixedly determined, the load condition of the transfer destination may be fixed. However, there is a problem that it may not always be desirable as a migration destination depending on the resource situation.

In constructing a system having a higher operating rate and a higher reliability, it is also a problem to collect main memory data at high speed, which is indispensable when a failure occurs. The operating rate can be improved by shortening the repair period. Therefore,
When the system is restarted, shortening the dump time of the main memory data contributes to improving the operating rate.

It is another object to maintain the consistency of the disk data cached in the main memory, so that when the system is restarted, the consistency of the file system constructed on the disk can be recovered. The time required can be kept to a minimum.

A function to be used for a particularly reliable application to be guaranteed, such as a database program. When a write operation to all disks is successful or unsuccessful, after updating and before updating. By guaranteeing the state without fail, the task of maintaining data consistency, which was conventionally performed in application programs, can be simplified, and data consistency can be guaranteed at the hardware level to speed up the system. is there.

The present invention solves the above problems,
It was made in order to achieve the task, and aims to realize higher computer system operation rate and reliability with less hardware development. It also aims to improve the operating rate by shortening the computer startup time during failure recovery. It is also an object of the present invention to speed up the system by guaranteeing the consistency of the disk data and simplifying the data consistency maintaining operation performed in the application program.

[0018]

According to a first aspect of the present invention, there is provided a computer system which: (a) executes a predetermined process, detects the presence or absence of a failure during the process, and transmits the detected signal as a detection signal. (B) In a computer system having failure detection software for receiving a detection signal from the software to be tested and detecting a failure in the software to be tested, the software to be tested and the software for failure detection each have a load on the system. It is characterized by having a load detection unit for detecting the frequency and a frequency table for adjusting the transmission / reception interval of the detection signal according to the load of the system.

A computer system according to the second invention is
(A) A software under test that executes a predetermined process and detects whether or not a failure has occurred during the execution of the process and transmits it as a detection signal; (b) receives a detection signal from the software under test, In a computer system having a failure detection software for detecting a failure of the software to be tested, the software to be tested includes the information of the transmission interval of the detection signal in the detection signal and notifies the failure detection software, and the failure detection software is notified. The failure of the software under test is detected based on the transmission interval of the detected signal.

A computer system according to a third aspect of the present invention is the computer system according to the second aspect of the present invention, wherein the software to be tested includes a load detecting section for detecting the load of the system, and the load of the system detected by the load detecting section. It is characterized by having a frequency table for adjusting the transmission interval of the detection signal according to.

In the computer system according to a fourth aspect of the present invention, the software to be tested includes a plurality of programs, each program sends a detection signal, and the fault detection software is related to the plurality of programs. It is characterized in that management information indicating a program is stored, and a detection signal from a plurality of related programs is received based on the management information to detect a failure of the plurality of related programs.

In the computer system according to the fifth aspect of the present invention, the fault detection software comprises first fault detection software and second fault detection software,
It is characterized in that one fault detection software detects a fault of the other fault detection software.

In the computer system according to the sixth aspect of the present invention, the software under test includes a message instructing an operation procedure for the fault detection software in a detection signal, and the fault detection software includes a procedure corresponding to the message. It is characterized by having the written procedure information and executing the procedure instructed by the message and the procedure information.

A computer system according to a seventh aspect of the present invention comprises a plurality of computers, each computer comprising an investigation program for investigating the status of each computer and outputting it to the fault detection software. It is characterized in that the software under test in which the failure is detected is executed by another computer in consideration of the investigation result from the program and the execution condition of the software under test.

A computer system according to an eighth aspect of the present invention is a computer system in which a plurality of computers having a main storage device and a secondary storage device are connected to each other, and the contents of the main storage device are divided into secondary storages of the plurality of computers. It is characterized in that the apparatus is provided with a split retraction means.

A computer system according to a ninth invention is characterized in that, in the computer system according to the eighth invention, a management table in which a division save destination is set in advance is provided.

A computer system according to a tenth invention is the computer system according to the eighth invention, further comprising:
A plurality of computers having a secondary storage device are connected to the network, and the division saving means saves data to the plurality of secondary storage devices using the network.

A computer system according to the eleventh invention is characterized by having the following elements. (A) Secondary storage device for storing data, (b) Main storage device for temporarily storing the data, (c) Presence / absence of write-back of data stored in the main storage device to the secondary storage device And storage means for storing data for judging the validity of the data as management information, (d) of the data stored in the main storage device by the management information stored by the storage means when a failure occurs, 2 A write-back means for determining data that has not been written back to the next storage device, checking its validity, and writing back the data.

A computer system according to a twelfth aspect of the present invention is the computer system according to the eleventh aspect of the present invention, which is a multiple system including a plurality of secondary storage devices, wherein the write-back means is a plurality of secondary storage devices. The feature is that the data is written back to the device.

A computer system according to a thirteenth invention is characterized by having the following elements. (A) a secondary storage device for storing data, (b) a writing means for writing data to the secondary storage device, and (c) a non-volatile storage of the original data when updating by the writing means. Saving means for saving to the medium and marking the start of updating and the mark of completion of updating in the saved data after completion of updating,
(D) When a failure occurs, the data saved by the saving means is examined, and it is considered that an error has occurred during the update of data that has a mark indicating the start of update and no mark indicating the completion of update, and the saved data is used. Restoring means for restoring the data in the secondary storage device, and (e) releasing means for releasing the saved data when the update is successful.

[0031]

The computer system according to the first aspect of the present invention detects the occurrence of a failure during execution of predetermined processing by the software under test,
It transmits as a detection signal, and the failure detection software receives the detection signal and detects the failure of the software under test. By using two independent programs in this way, if the detection signal does not arrive at a predetermined interval, the fault detection software side considers that a fault in which the detection signal cannot be transmitted on the software side to be detected has occurred and the time-out fault occurs. It can be one of the judgments as to whether or not it has occurred. At this time, the time required for the detection signal to arrive may be delayed depending on the system load. Therefore, both software should have a load detection unit that detects the system load, and the transmission / reception interval of the detection signal should be adjusted according to the load. To do.

In the computer system according to the second aspect of the present invention, since the software under test notifies the fault detection software of the transmission interval of the detection signal, the fault detection software can omit the procedure for checking the transmission interval.

In the computer system according to the third aspect of the present invention, the software to be tested has a load detecting section and a frequency table, and the sending interval of the detection signal is adjusted according to the load. Further, since the information of the transmission interval is also included in the detection signal and notified to the fault detection software side, it is not necessary for the fault detection software side to perform processing by the load detection unit or the frequency table.

In the computer system of the fourth invention,
The software to be tested includes a plurality of programs, and each program sends a detection signal. The failure detection software stores management information indicating a related program among the plurality of programs, receives a detection signal from the plurality of related programs based on the management information, and detects a failure of the plurality of related programs. Can be detected.

The computer system according to the fifth aspect of the present invention can avoid the single point fail of the fault detection mechanism itself by providing the fault detection software for detecting the fault.

In the computer system according to the sixth aspect of the present invention, the software under test includes a message instructing an operation procedure for the fault detection software in the detection signal. The fault detection software has procedure information describing the procedure corresponding to the message, and executes the procedure corresponding to the message according to the procedure information, so that wasteful processing can be omitted.

In the computer system of the seventh invention,
The computer system includes a plurality of computers, and each computer includes an investigation program that investigates the state of each computer and outputs the investigation status to the failure detection software. The failure detection program is based on the load status and resource status obtained from the investigation program in consideration of the execution condition of the software to be tested, and the failure is detected, and the software to be tested needs to be migrated to another computer. Find the best calculator for the transition,
Migrate and execute.

In the computer system according to the eighth aspect, when a failure occurs, the contents of the main storage device can be divided and saved in the secondary storage devices of a plurality of other connected computers.

Since the computer system in the ninth aspect of the present invention has the management table in which the division save destination is set, the data collection destination of the main storage device can be promptly determined.

A computer system according to the tenth invention is
The work of saving the divided data via the network can be divided into other computers.

A computer system according to the eleventh invention is
It has, as management information, whether or not the data cached in the main storage device is written back to the secondary storage device and the data for judging the validity of the data. Thus, when a failure occurs, the data stored in the main storage device, which has not been written back to the secondary storage device, is determined based on this management information, and whether the data is affected by the failure is valid. Check the sex again and write the data back.

A computer system according to the twelfth invention is
Also in the multiple system, similar to the eleventh invention,
It is possible to keep the consistency between the data read and read on the main memory and the disk data when a failure occurs.

A computer system according to the thirteenth invention is
At the time of writing to the disk device, after marking the start of update to the data before being updated and saving that data,
Writing to the disk device is started, and after the update is completed, the saved data is marked as completed. When the write command to the disk ends abnormally, the saved data without the update completion mark is written back in the saved data, so that the data state before the write command is issued can be guaranteed. When the update is successful, the saved data is released.

[0044]

【Example】

Example 1. FIG. 1 shows an example of a computer system in this embodiment.
Will be explained. Independent computers 103 and 104 are connected to the networks 101 and 102, respectively. Only two computers are shown in the figure,
There is no limit to the number of units. When the function redundancy is realized in the main / sub mode, the disk device 110 that can be accessed from each of the main and sub computers enables the transfer of consistent data between the two computers. Program group 112, 113, 1 in the primary server 111
Reference numerals 14 correspond to application programs having respective dependency relationships, that is, test programs. 106,1
Reference numerals 07 and 108 denote failure detection programs, which are executed on the same computer on which the test program is executed and on different computers in the network. Reference numeral 109 is a check agent program that performs services such as start / end of the section of the failure detection mechanism.

The failure detection mechanism is divided into a part processed in the application program (test program) and a part processed in the failure detection program. FIG. 2 shows the software configuration of the failure detection mechanism according to this embodiment. The test program 201 includes a load detection unit 202 that investigates the load status of a computer and a frequency table 2 that indicates the failure detection frequency for the load whose contents are shown in FIG.
03, and a transmitter 204 for sending a failure detection signal to the failure detection program. On the other hand, the fault detection program includes a frequency table 207 equal to the load detection units 206 and 203 equivalent to 202, and a reception unit 208 that receives a fault detection signal from the test program. Test program 20
1 is transmitted from the transmission unit 204 at a rate of several times during a certain period.
I have sent a message "I am normal" to the failure detection program 205. The reception unit 208 of the failure detection program 205 receives the message, and the program under test is normal depending on the content or whether the message arrives. It is determined whether or not

Therefore, the software failure detection mechanism uses the timeout (the time until it is judged to be abnormal when the next message does not arrive at the time when it should arrive) as a criterion for judging whether or not a failure has occurred. However, depending on the load condition of the computer, there is a problem that an accurate judgment cannot be made with a uniform judgment standard. Therefore, both programs periodically collect the load state of the computer and set the failure detection frequency according to the frequency table from the value. For example,
In the case where the content of the frequency table is as shown in FIG. 3, the frequency is 10 when the load is 0, so that both programs have 1 during a certain period.
Messages will be exchanged for 0 times of failure detection. The load is determined by the operating rate of the computer and the length of the job queue.

The reason why both programs have the load detecting section and the frequency table will be described. Since the program to be tested changes the transmission frequency depending on the load state of the computer, it is necessary to change the timeout value on the fault detection program side. Therefore, the failure detection program side also performs the same processing as that of the test program, and sets a new timeout value.

The load detection section in the test program and the failure detection program uses the inquiry means of the operating system to obtain the load according to the length of the job queue and the operating rate of the computer. However, since the load condition of the computer may change abruptly, the load detection unit should have a sufficiently long sampling period in order to prevent a difference between the load detection values of the two. Since the load status is constantly changing, it is more practical to set the load as a rough trend value in that time zone rather than detecting a fine value for each moment.

Further, the reason why the program to be inspected and the failure detection program are divided is that it depends on the creativity of the program, and if the program becomes independent and the detection signal cannot be transmitted, This is because the failure detection program side uses the timeout as one of the judgments as to whether or not a failure has occurred.

As described above, in this embodiment, a computer system characterized by detecting a fault occurring in software running on the system and recovering from the fault state, which is used for detecting the fault, is used. The frequency of transmission of the detection signal transmitted by the inspection software itself is adjusted according to the load status of the system.

For this purpose, the load condition detecting means of the computer system, the load condition of the computer system, and the frequency table of the fault detection frequency are provided on the detection signal transmitting side, that is, the test program and the detection mechanism, respectively. The failure detection frequency is adjusted according to the system load status.

Example 2. In this embodiment, transmission interval information (also referred to as interval information) of the failure detection signal is sent in the failure detection signal on the side of the program to be tested, and based on that,
An example in which the failure detection program sets the timeout time will be described.

FIG. 4 shows the software configuration of the fault detection mechanism of this embodiment. The test program 401 shows a load detection unit 402 for investigating the load status of the computer, the frequency of fault detection for the load whose contents are shown in FIG. 5, and fault detection interval information transmitted to the fault detection program. It includes a frequency table 403 and a transmission unit 405 that sends a failure detection signal to the failure detection program. On the other hand, the failure detection program 406 includes a reception unit 407 that receives a failure detection signal from the program under test. The test program periodically collects the load state of the computer, sets the failure detection frequency from the value according to the frequency table, and sets the transmission information to be sent as a part of the failure detection signal from the frequency table. For example, in 501 of FIG. 5, when the load is 0, the test program transmits a failure detection signal including the transmission information 1 to the failure detection program at a frequency of 10 times in a certain period. The failure detection program sets a timeout value in accordance with the transmission information and determines whether or not there is a failure.

This is because whether or not the test program is normal is also determined by the failure detection program by whether or not the next message arrives within a certain period. Also, since the program to be detected changes the transmission interval of the detection signal by referring to the frequency table depending on the load condition, the value is notified to the failure detection program as transmission information. As a result, the failure detection program side can change the timeout value according to the change on the tested program side by the transmission information.

As described above, this embodiment is characterized in that the information for adjusting the detection frequency is added to the fault detection signal.

Example 3. When the program that is trying to detect a failure consists of using the services of several programs, or when the programs are using each other's services, it is not enough to detect the failure of the target program. I can't make an accurate decision. Therefore, in this embodiment, by providing the failure detection mechanism with a table indicating the program to be monitored and the dependency relationship between the program to be monitored and the program using the service, the above dependency can be realized. This section describes an example that enables monitoring of the programs that it has.

The test program and the fault detection program are
It has the functions of the first or second embodiment. FIG. 6 is a diagram showing the execution status of a program at a certain moment on the computer system. Application program A (60
2), the application program B (603), the application program C (601), and the failure detection program 604 are executed. Fault detection program 60
4 includes a dependency relationship table 605 between application programs, the details of which are shown in FIG. 7, and a reception unit 606 that receives a failure detection signal from each application.

The dependency table 605 is set as follows. For example, when creating the program A, the dependent programs are known to be the programs B and C. Program B depends on Program A and also depends on Program C through Program A. Program C does not depend on any program. In this way, since the dependency relationship between each program is known, this is provided as the dependency relationship table 605 when the failure detection program is created.

FIG. 7 is a dependency relationship table showing the dependency relationship between application programs. For example, A depends on C and B (701), B depends on A, and A depends on C (( 702) and C indicate that the service is provided but does not depend on any program (703). The failure detection program 604 also refers to this dependency table, and when monitoring the program A, for example, also detects failures in the programs C and B.

As described above, by having the dependency relationship table between the application programs, it becomes possible to comprehensively monitor the program to be monitored and the program used by this program.

As described above, this embodiment has the management information of a plurality of software for the purpose of detecting a failure occurring in a plurality of software which mutually operate on the system and recovering from the failure state. The computer system characterized by performing fault detection and fault recovery for all software described in the management information has been described.

Example 4. Until now, failure detection methods have not been able to deal with single point failures of the detection mechanism itself. In this embodiment, an example of avoiding a system failure due to a failure of the failure detection mechanism itself by duplicating the failure detection mechanism will be described.

FIG. 8 is a diagram showing the software configuration of the test program and the failure detection program. The test program 801 sends the failure detection signal to the main failure detection program 80.
2 and the secondary fault detection program 803.
FIG. 9 shows the operation of the sub-fault detection program. If a failure is detected (901), the secondary failure detection program checks the status of the primary failure detection program (902),
If sound, do nothing. If not healthy, the fault detection program is restored (903). The failure detection program recovery means that the sub failure detection program stops the main failure detection program and the sub failure detection program deals with the failure of the test program instead of the main failure detection program. Also, at this time, the secondary failure detection program makes a copy of itself and makes it monitor itself thereafter. In the present embodiment, the main failure detection program is unaware of the existence of the sub failure detection program.

In this embodiment, a computer system characterized by detecting a fault occurring in software running on the system and recovering from the fault state,
The computer system characterized by avoiding the system failure due to the failure of the failure detection mechanism itself has been described by duplicating the failure detection mechanism.

Example 5. The above embodiment is only one example, and there is also a method in which the sub failure detection program monitors only the main failure detection program. FIG. 10 is a diagram showing the relationship between the test program and the failure detection program of this embodiment.

Example 6. In the conventional fault detection and restoration methods, the restoration means are uniform, and in many cases, the fault must be taken more than necessary. In this embodiment, it has a failure detection mechanism that classifies failure types by detection signals sent by the software under test, and has procedure information describing failure recovery procedures according to failure types, and
An example of providing a failure recovery means according to the failure type by having a procedure information setting means will be described.

This embodiment is applied to the failure detection mechanism in the first to fifth embodiments, and the failure recovery or the service is performed according to the failure detection signal. FIG. 11 shows a correspondence table showing correspondence between the failure detection signal and the service procedure started by the failure detection program. The fault detection program
If this correspondence table is included in the program and a failure detection signal is received, the corresponding procedure is executed. FIG. 11 will be described. As long as it receives the normal signal, the failure detection program does nothing (1001). When the stop signal is received, the failure detection program delays the timeout (1002). When receiving the start signal, the failure detection program starts monitoring the test program (1003). When the end signal is received, the monitoring of the test program is ended or the end processing is performed (100
4). When the fault 1 signal is received, the same processing is retried three times (1005). When the fault 2 signal is received, the disk data is restored (1006). When the fault 3 signal is received, it is re-executed by another computer (100
7).

As described above, in this embodiment, the computer system is characterized by detecting a fault occurring in software running on the system and recovering from the fault state. We have described a computer system that has a failure detection mechanism that classifies failure types by signals and that has procedure information that describes a failure recovery procedure depending on the failure type.

Example 7. When a software failure occurs and it is necessary to migrate to another computer in the network when recovering from it, the fixed migration destination may not always be the migration destination depending on the load status and resource status of the migration destination. In some cases it was not desirable. Therefore, in this embodiment, a system for deciding which computer in the network will continue the service when a computer on which a certain service is executed fails. That is, the load status of each computer in the network,
A table indicating the resource status, its updating means, a program to be started, a load status, and a table showing the correspondence with the resource status, and the comparison result of the load and the resource status are used to start the specified program. To be achieved.

An example of a computer system to which this embodiment is applied will be described with reference to FIG. Independent computers 103 and 104 are connected to the networks 101 and 102, respectively. Although only two computers are shown in the figure, the number of computers is not limited. When the function redundancy is realized in the main / sub mode, the disk device 110 that can be accessed from each of the main and sub computers enables the transfer of consistent data between the two computers. Program group 112, 1 in the primary server 111
Reference numerals 13 and 114 respectively correspond to the application programs having a dependency relationship, that is, the test programs described in the above-described embodiments. Reference numerals 106, 107, and 108 are failure detection programs, which are executed on the same computer on which the test program is executed and on different computers in the network. A check agent program 109 performs services such as starting / ending a session of these failure detection mechanisms.

If a failure occurs in the program to be tested, or if the computer 103 itself becomes inoperable, the application program group of the standby server 105 is either the disk device 110 or the disk device 115 through the network. The data is taken over from the disk device 116 to which the data is copied and is activated. At this time, it is the fault detection program 106 on the computer 104 that performs these takeover processes.

This embodiment is applied when a computer on which a certain application program is executed becomes inoperable and is re-executed by one of the computers, and FIG. 12 shows its software configuration. . The fault detection program 1203 receives a fault detection signal from the test program 1201 running on a certain computer,
In addition, a status report is regularly received from the program 1202 that investigates the load and resource status of each computer connected to the network. The fault detection program 1203 includes the execution condition data of the test program 1201 in the program.

The program 1202 for investigating the load and resource status of each computer uses the inquiry means of the operating system to investigate the load and resource status of each computer. The load and resource status of the computer may have changed at the time of receiving the report, so we think that it is sufficient if we can understand the general trend, and provide a sufficiently long sampling period.

FIG. 13 shows an example of the execution condition data. Program A has a computer load of 1 or less and an I / O frequency of 1
The condition for the execution is that the main memory remaining is 00 or less and 2 or more (1101). Program B has a computer load of 4 or less, an I / O frequency of 1000 or less, and a main memory remaining of 0.
One or more is a condition for the execution (110
2). On the other hand, the failure detection program 1203 periodically receives the information as shown in FIG. 14 from the program 1202 which investigates the load and resource status. The load on computer A is 0.
1, the I / O frequency is 10 and the remaining main memory is 100 (13
01). The computer B has a load of 2, an I / O frequency of 50, and a main memory remaining of 10 (1302). Computer C has a load of 1
The I / O frequency is 1000 and the main memory remaining is 50 (1
303). After comparing these pieces of information, the fault detection program determines on which computer the faulty test program is to be restarted. For example, in FIG.
If the program A has a failure in the data shown in FIG. 14, the program A is restarted on the computer A.

As described above, in this embodiment, in a computer system which is composed of a plurality of computers connected to one or a plurality of networks and mutually cooperates, the software running on the system causes a failure. The software is restarted on another healthy computer in the network link, which is monitored by the detection means, and when the software is in a failure state and there is a failure in the running computer itself. . At that time, when the software is to be restarted on another computer, it has information on which computer in the network link should be started, and also has means for generating the information. .

Example 8. This embodiment is an example for speeding up the dumping of data on the main memory when a failure occurs. The main memory is divided and dumped onto the secondary memory of a plurality of computers at the same time through a plurality of network links to speed up the processing. This is achieved by registering a network link, a dump-destination computer, and a secondary storage area for data dump on that computer in the divided main storage areas.

This embodiment is applied on a plurality of computers connected by a plurality of networks as shown in FIG. 15, and the high speed main memory data dump is realized as follows. Each computer has a main memory management table as shown in FIG. FIG. 16 shows a management table for the computer 0. The main storage areas from 0 to 11 are managed separately in three areas, and the main storage data in the areas from 0 to 3 are stored in the computer 0.
To the disk device 0 of (1401) and the areas 4 to 7 of the main storage data through the network 0, and to the disk device 0 of the computer 1 (1402) and data of the areas 8 to 11 of the main storage area through the network 1 of the computer 2 Is associated with the disk device 0 of (1403). When a failure occurs in the computer, the main storage data is dumped according to the management table when the computer is started up after the reset. The system mechanism is shown in FIG.
The system shown in FIG.

In this way, main memory data, which is indispensable when a failure occurs, can be collected at high speed. In addition, the operating rate can be improved by shortening the failure repair period. Therefore, when the system is restarted, shortening the dump time of the main memory data contributes to improving the operating rate.

As described above, in this embodiment, in a computer system which is composed of a plurality of computers connected to a plurality of network links and mutually operates, when a failure occurs in a computer in the system, The main storage contents of the computer are divided by the core dump information set in advance by the information setting means, and sent out to the computers in the network link including itself through the predetermined network link to save. The computer system was explained.

Example 9. This example is aimed at maintaining the consistency of the disk data cached in the buffer on the main memory, and in the case of a system restart, it is necessary to recover the consistency of the file system built on the disk. This makes it possible to minimize the time required for doing so. That is, it is realized by verifying the disk data on the main memory by the checksum, and writing to the disk device on the own computer and the disk device of another computer having the copied data through the network.

This embodiment is applied to the computer system as shown in FIG. 1, and the disk device having the relationship of 115 and 116 is the object of this embodiment. That is, the data in the disk device 115 is copied to the disk device 116 every time writing is performed in the disk device 115. When the data of the disk device 115 read on the computer 103 is updated, checksum data is obtained for each block.

FIG. 18 shows management information of disk data on a computer according to this embodiment. The items in the drive sector indicate the computer and drive number of the primary / secondary disk device, the flag indicates the state of each buffer, and the checksum data contains the checksum for each buffer data update. Data is stored.
FIG. 19 is a diagram showing types of states. FIG.
Then, it means that the disk data managed in 1501 is the disk device 0 of the computer A, the copy destination of the data is the disk device 0 of the computer B, and the data of the 0th sector is stored. The state of this buffer is BU
In SY, the CPU is in the process of updating data reference, calculating checksum data, or writing. The buffer at 1502 is unused. The buffer of 1503 is the disk device 0 of the computer A, the copy destination of the data is the disk device 0 of the computer B, and the data of the second sector is stored. The state of this buffer is DIRTY, which is data that needs to be written back to the disk device. The buffer at 1504 is unused.

In this state, once the computer corresponding to the computer 103 has failed and becomes unusable, after resetting the computer and restarting it, BUSY, or Checksum calculation is performed on the contents of the buffer that is DIRTY, and if the checksum obtained from the contents of the buffer and the checksum data in the management table match, the contents of the buffer are regarded as correct and both the primary and secondary disks are confirmed. Write the contents of the buffer. In this way, the data read and modified in the buffer on the main memory at the time of failure is also checked by the checksum.
Once you know that the data has not been corrupted, you can write it back to disk to keep the disk data consistent.

As described above, this embodiment is a computer system which is composed of computers connected to one or a plurality of network links and mutually operates. In the computer system, the data in the disk storage device is stored in the disk storage devices in different computers in a multiplex manner, and a file system or the like is constructed as the data. When the disk data is used, the data is stored in the main storage. The data in the main memory buffer is subjected to checksum at the time of updating. At that time, the computer that is using the data is in a failure state, and when the computer is stopped and restarted, the buffer in the main memory is the disk data that has been expanded and has been updated. Characteristic is that data that has not been written back to the disk device and is justified is written back to the disk device of the computer and the disk device of another computer where the data is multiplexed. It is what

Example 10. This embodiment guarantees the data state before writing even if an error occurs during writing to the disk device. The cause of the error may be a power failure or damage to the disk head. The disk device to which this embodiment is applied writes data according to the procedure shown in the flowchart of FIG. If there is a write request, the data in the write destination sector is first saved (1701). It is possible to use a disk area as a save destination or another non-volatile storage. The sector data is saved as shown in FIG. That is, the write destination sector number 1601 and the BEGIN mark 160 indicating that the write operation has started
2, and the data 1603 is saved. When the data saving is completed in this way, data writing is started (1702), and if the data updating is successful, 160
Enter the COMMIT mark of 4 (1703). Even if a failure occurs during the data saving, the disk data is retained. Further, when an error occurs during data updating, it is possible to find that the data without the COMMIT mark is being updated by examining the saved data, and thus the saved data makes it possible to restore the original data. Also,
The saved data is released when it is no longer needed.

The function described above is a function that should be used for a particularly reliable application that should be guaranteed, such as a database program, and is updated when a write operation to all disks is successful or unsuccessful. This is a disk device that guarantees the state before and after the update. This simplifies the data consistency maintaining operation conventionally performed in the application program and guarantees the data consistency at the hardware level, so that it is expected to speed up the system.

As described above, in this embodiment, in a disk storage device used for a program such as a database program that strictly requires data consistency, two-phase commitment is performed when updating data. The example has been described in which the data before the update operation is guaranteed even if a failure occurs in the disk storage device when updating the data.

[0088]

According to the first aspect of the present invention, the software to be detected and the fault detection software detect the load on the system and adjust the transmission / reception interval of the detection signal according to the load. More accurate judgment of occurrence. In addition, when a failure occurs in the software under test for adjusting the transmission interval by performing load detection by both softwares, the failure detection software can detect the failure. Further, since the function relating to the fault detection is realized by software instead of hardware, the cost required for fault detection can be reduced.

According to the second invention, the software under test notifies the fault detection software of the transmission interval of the detection signal. Therefore, it is possible to omit the processing for checking the transmission interval on the fault detection software side. Thereby, the load on the system can be reduced.

According to the third aspect of the invention, the software to be tested is provided with a load detecting section, and the transmission interval of the detection signal is adjusted by the load. Further, since the information on the transmission interval is also included in the detection signal and notified to the fault detection software side, the fault detection software side does not need to have a load detection unit and does not need to perform the processing. Therefore, the number of program steps of the fault detection software can be reduced and the processing can be omitted, so that the load on the system can be reduced.

According to the fourth aspect of the present invention, when the software under test is made up of the services of several programs, or when the software is made up of mutually using services, the relationship between the programs is By having the management information to be shown, it is possible to monitor all related programs, and it is possible to detect a fault more accurately.

According to the fifth invention, by duplicating the failure detection mechanism, it is possible to avoid a system failure due to a failure of the failure detection mechanism itself.

According to the sixth aspect of the invention, the detection signal includes a message instructing an operation procedure to be executed by the fault detection software, and the fault detection software has procedure information for the message. For this reason, unnecessary processing can be omitted on the fault detection software side, and accurate processing can be performed.

According to the seventh aspect of the present invention, it has a check program for checking the state of each computer. As a result, when a failure is detected in the software to be inspected and it is necessary to transfer to another computer, the optimum transfer destination can be selected in consideration of the load status and resource status of each computer.

According to the eighth invention, when a failure occurs, the contents of the main storage device can be divided and saved in the secondary storage devices of a plurality of other computers. Therefore, the data in the main memory can be collected at high speed, and the dump time of the data in the main memory can be shortened. Therefore, the repair period can be shortened, which contributes to the improvement of the operating rate.

According to the ninth invention, since the management table in which the divided save destination is set is provided, it is possible to promptly determine the data collection destination of the main memory device, and to dump the data of the main memory device at high speed. There is an effect that it contributes to the improvement of operating rate.

According to the tenth aspect of the present invention, the work of saving the divided data via the network can be divided into other computers, so that the speed of dumping the data in the main storage device can be increased. It can contribute to the improvement of operating rate.

According to the eleventh aspect, even the data read and modified in the main storage device can be checked for correctness and written back to the secondary storage device at the time of recovery after a failure occurs. Therefore, the consistency of the disk data can be maintained, and the time required to recover the consistency of the file system built on the disk can be minimized when the system is restarted, improving the operating rate. Contribute.

According to the twelfth invention, the same effect as that of the eleventh invention can be obtained even in the multiplex system.

In the thirteenth aspect of the present invention, when the write command to the disk is abnormally terminated because the data before the update is saved when the data is written to the disk device, the data before the write command is issued You can always guarantee the condition. As described above, the present invention is a function that should be used for a particularly reliable application that should be guaranteed, such as a database program, and after a write operation to all disks succeeds or fails and is updated. It is a disk device that guarantees the state before update. This simplifies the operation of maintaining data consistency conventionally performed in an application program and guarantees data consistency at the hardware level, so that the speed of the system can be expected.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a computer system to which the present invention is applied.

FIG. 2 is a diagram showing a software configuration example to which the present invention is applied.

FIG. 3 is a diagram showing an example of a frequency table according to the present invention.

FIG. 4 is a diagram showing a software configuration example to which the present invention is applied.

FIG. 5 is a diagram showing an example of a frequency table according to the present invention.

FIG. 6 is a diagram showing a software configuration example to which the present invention is applied.

FIG. 7 is a diagram showing an example of a dependency relationship table according to the present invention.

FIG. 8 is a diagram showing a software configuration example to which the present invention is applied.

FIG. 9 is a diagram showing an operation algorithm of a sub-fault detection program according to the present invention.

FIG. 10 is a diagram showing a software configuration example to which the present invention is applied.

FIG. 11 is a diagram showing an example of a correspondence table regarding a procedure for a failure in the present invention.

FIG. 12 is a diagram showing a software configuration example according to the present invention.

FIG. 13 is a diagram showing an example of execution condition data of a test program according to the present invention.

FIG. 14 is a diagram showing an example of the result of a survey on the load and resource status of a computer according to the present invention.

FIG. 15 is a diagram showing an example of a computer system to which the present invention is applied.

FIG. 16 is a diagram showing a management table of a main memory dump destination for the computer 0 according to the present invention.

FIG. 17 is a diagram showing an example of a computer system to which the present invention is applied.

FIG. 18 is a diagram showing management information of disk data on a computer according to the present invention.

FIG. 19 is a diagram showing types of buffer states in the present invention.

FIG. 20 is a diagram showing a disc data writing algorithm according to the present invention.

FIG. 21 is a diagram showing an example of a saved data form in the present invention.

[Explanation of symbols]

 103 computer (main) 104 computer (secondary) 105 standby server 106 failure detection program 107 failure detection program (secondary) 108 failure detection program (main) 109 check agent program 110, 115, 116 disk device 111 primary server 112, 113, 114 test program 202, 206 load detection unit 203, 207 frequency table 204 transmission unit 208 reception unit

Claims (13)

[Claims] 1. (a) performing predetermined processing,
Software under test that detects whether or not a failure has occurred during the execution of the processing and transmits it as a detection signal, (b) failure detection software that receives a detection signal from the software under test and detects a failure in the software under test In the computer system having, the test software and the failure detection software each include a load detection unit that detects the load of the system, and adjust the transmission / reception interval of the detection signal according to the load of the system detected by the load detection unit. A computer system characterized by. 2. (a) performing predetermined processing,
Software under test that detects whether or not a failure has occurred during the execution of the processing and transmits it as a detection signal, (b) failure detection software that receives a detection signal from the software under test and detects a failure in the software under test In the computer system having the: A computer system characterized by. 3. The software under test includes a load detection unit for detecting a load of the system, and adjusts a detection signal transmission interval according to the load of the system detected by the load detection unit. The computer system described in 2. 4. The software under test comprises a plurality of programs, each program transmitting a detection signal, and the fault detection software stores management information indicating a related program among the plurality of programs. 5. The computer system according to claim 1, further comprising: receiving a detection signal from a plurality of related programs based on the management information to detect a failure of the plurality of related programs. 5. The fault detection software comprises first fault detection software and second fault detection software, and one fault detection software detects a fault of the other fault detection software. The computer system according to claim 1, 2, or 3. 6. The detection software includes a message instructing an operation procedure for the fault detection software in a detection signal, and the fault detection software has procedure information, and the procedure instructed by the message and the procedure information. 5. The computer system according to claim 1, wherein the computer system executes. 7. The computer system comprises a plurality of computers, each computer is provided with an investigation program for investigating the status of each computer and outputting it to the failure detection software, and the failure detection software is provided from the investigation program. 4. The computer system according to claim 1, wherein the software under test in which the failure is detected is executed by another computer based on the investigation result. 8. In a computer system to which a plurality of computers having a main storage device and a secondary storage device are connected, a division saving means for dividing the contents of the main storage device and saving the contents in the secondary storage devices of the plurality of computers is provided. A computer system characterized by having. 9. The computer system according to claim 8, wherein the computer system is provided with a management table in which a division save destination is set in advance. 10. The computer system is further connected to a plurality of computers having a secondary storage device on a network, and the partition saving means saves data to the plurality of secondary storage devices using the network. The computer system according to claim 8, which is characterized in that: 11. A computer system having the following elements: (a) a secondary storage device for storing data, (b) a main storage device for temporarily storing the data, and (c) a main storage device. Storage means for storing, as management information, data for judging whether the data is written back to the secondary storage device and the validity of the data, and (d) the management information stored by the storage means when a failure occurs. A write-back unit that determines, out of the data stored in the main storage device, data that has not been written back to the secondary storage device, checks its validity, and writes back the data. 12. The computer system is a multiple system including a plurality of secondary storage devices, and the write-back means writes back data to the plurality of secondary storage devices. A computer system according to item 11. 13. A computer system having the following elements: (a) a secondary storage device for storing data, (b) a writing means for writing data to the secondary storage device, and (c) an update by the writing means. When trying to save the original data to a non-volatile storage medium, a mark indicating the start of update,
Save means for marking the save data with a mark indicating update completion after update completion, (d) when a failure occurs, the data saved by the save means is checked, and there is a mark indicating update start,
Resuming means for recovering the data in the secondary storage device by using the saved data by regarding the data without the update completion mark as an error during the data update, and (e) releasing the saved data when the update is successful. means.