JPH04181461A - How to manage distributed files - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a distributed file management method, and particularly to a management method suitable for online recovery of distributed files multiplexed on multiple computers.

[Conventional technology]

Conventional techniques include a technique in which when an attempt is made to access a file and the file cannot be accessed because it is being recovered, the access requesting computer immediately resends the access request.

Incidentally, examples related to this type of technology include, for example, Japanese Unexamined Patent Publication No. 102342/1983.

[Problem to be solved by the invention]

The above conventional technology does not take into account the time required for recovery, so the access requesting computer has to send the file access request many times until it can access the file, resulting in a lot of wasteful sending and receiving. There is a problem.

Also, since the access status of the file is not taken into consideration, 1
There is a high possibility that the part of the divided file that is being recovered will be accessed, and requests to access the file must be sent many times until the file can be accessed, resulting in a lot of wasteful sending and receiving. .

An object of the present invention is to provide a distributed file management method that can prevent the waste of sending and receiving access requests and access result messages many times during file recovery.

[Means to solve the problem]

In order to achieve the above purpose, if access is not possible, the time required to recover the file is calculated from the length of the entire file and the length of the recovered part, and an access request is sent to the transmission line based on the calculation result. resend to.

Furthermore, when file recovery is performed for each divided file, the size of the file recovery unit is changed depending on the file access status.

[Effect]

The time required to recover the file is calculated from the length of the entire file and the length of the recovered portion, etc., and the access request is re-sent to the transmission path based on the calculation results, thereby avoiding unnecessary transmission and reception. You don't have to do it.

In addition, in a distributed file system where a file is divided into parts and recovered part by part, and parts other than the part being recovered can be accessed, the size of the file recovery unit is changed depending on the access status of the file. The possibility of not being able to access the file is reduced, and if you try to access the part that is being recovered, the time required to recover the file is calculated based on the size of the part that is being recovered, and an access request is sent based on the calculation result. Since the data is retransmitted to the network, there is no need to repeatedly send and receive data.

〔Example〕

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 4 shows an example of a multi-computer system, in which a plurality of computers 1, 2, 3, etc. are connected to a transmission line 7.
External storage devices connected to each other to accumulate data! 4. , 5.6...

Files A8, 10. File B9゜File C1l, 12. File D13... is defined, and multiplexed files exist (File B and File D are not shown in the figure, and are connected to the external storage device of another computer via transmission path 7. ).

FIG. 1 shows an example of the configuration of a computer according to an embodiment of the present invention. The computers 1, 2, 3, . . . consist of a transmission control device 101 and an information processing device 104.

The transmission control device 101 has a selective reception management unit 102 and a selective reception table 103, and the logical file names of files on the external storage device are registered in the selective reception table 103.

The information processing device W104 includes reception buffers 105, 107,
It has transmission buffers 106 and 108, a client management unit 110, a server management unit 114, a distributed file management table 119, and an external storage management unit 120.

The client management unit 110 that interfaces with the user program includes an access request transmission section 111;
It consists of an access result receiving section 112 and a timer setting section 113.

Further, the server management unit 114 that actually accesses the distributed file includes an access request receiving section 115, an access result transmitting section 116, and a file recovery data transmitting section 11.
7. It consists of a file recovery data receiving section 118.

As shown in FIG. 2, the distributed file management table 119 has buffer areas divided by file names of distributed files. Each buffer area has a logical file name 21, a physical file name 22, and a recovery flag 23.
, the number of recovered records is 24.

FIG. 3 shows the format of the message 30 flowing through the transmission path 7, where 31 is the message length, 32 is the logical file name, 33 is the address of the message source computer, 34 is the message type, and 35 is the message serial number. , 36 is information.

The message type 34 indicates whether the message 30 is an access request, an access result, or file recovery data.

The information 36 consists of a file client list 37 and data 38 when the message type 34 is an access request or an access result. In the case of file recovery data, it consists of data 38. The file client list 37 includes information necessary for file access, such as record length, block length, number of records, and access status.

If a failure occurs in file A on computer 1 (if the contents are inconsistent), the distributed file management table 119
The file recovery data sending unit 117 of the computer 2 sets the message 30 in the sending buffer 108 in units of records of the distributed file, and sends the message 30 to the transmission line 7 through the selection management unit 102. do.

Since the logical file name 32 of the message 30 flowing on the transmission path 7 is registered in the logical file name of the selective reception table 103, the selective reception management unit 102 of the computer 1 classifies the message 30 as message type 3.
4 is the file recovery data, so the receive buffer 10
Receive to 7.

When the server management unit 114 of the computer 1 receives the message 30 in the reception buffer 107, it distributes processing according to the message type 34.

Since message type 34 is file recovery data,
In the file recovery data receiving unit 118, the distributed file management table 11 is
9 is searched, its recovery flag 23 is set to II II II, and file copy is performed based on the information in the information 37. File copying is performed record by record and counted in the number of recovered records 24 in the distributed file management table 119. When the file copy is completed, the recovery flag 23 is set to "O'".

When the computer 3 accesses the file B that is not being recovered, the access request sending unit 111 sets a message in the sending buffer 106 in response to the access request from the user program, and sends the selected reception management unit to the transmission line 7 through the selected reception management unit 102. Send.

Since the logical file name 32 of the message 30 flowing on the transmission path 7 is registered in the logical file name of the selective reception table 103, the selective reception management unit 102 of the computer 1 receives the message.

The selective reception management unit 102 looks at the message type 34 and switches the buffer to be received. Since it is an access request, it is received in the reception buffer 107.

Since the message type 34 of the message 30 received in the reception buffer 107 is an access request, the file access request receiving unit 115 searches the distributed file management table 119 using the logical file name 32 of the message 30 and checks that the recovery flag 23 is ○'', the file is accessed based on the information in the file client list 37, and the file access result transmitter 116 creates a message 30 representing the access result and sends it to the access requesting computer 3.

In the client management unit 110 of the computer 3, the access result receiving unit 112 receives the message 30 whose message type 34 is the access result into the receiving buffer 105.

When the computer 3 accesses the file A that is being recovered, the access request sending unit 111 sends a message to the sending buffer 106 in response to the access request from the user program.
, and transmits it to the transmission path 11 through the selective reception management unit 102.

Logical file name 3 of message 30 flowing on transmission path 7
2 is registered in the logical file name of the selective reception table 103, the selective reception management unit 102 of the computer 1 receives the message.

The selective reception management unit 102 looks at the message type 34 and switches the buffer to be received. Since it is an access request, it is received in the reception buffer 107.

Since the message type 34 of the message 30 received in the reception buffer 107 is an access request, the file access request receiving unit 115 searches the distributed file management table 119 using the logical file name 32 of the message 30 and checks that the recovery flag 23 is Since it is "1", the file recovery status and the number of recovered records are set in the access status of the file client list 37 and sent to the access request source computer 3.

In the client management unit 110 of the computer 3, the access result receiving unit 112 receives the message 3o whose message type 34 is the access result into the receiving buffer 35. Since the access status of the file client list 37 is the file recovery status, the timer setting unit 113 calculates the length of the unrecovered portion of the file from the total number of records, the number of recovered records, and the record length of the file. The time to output a certain length of data to disk can be obtained as an actual performance value, so
From these, the time required to recover the unrecovered portion is calculated, and the access is set to be performed again after the time required for recovery has elapsed.

Note that the time required for the recovery process referred to here is a theoretical value. This does not take into account the load on the CPU or line, the processing time of the CPU, the time it takes from the time the server management unit sends the access result to the time it receives the access result, so try again after the time required for recovery processing has elapsed. Even if you make an access request, there are cases where the recovery is still in progress and the timer needs to be set again, or where the recovery has already finished.

Therefore, the time you had to wait after making an access request again after the time required for recovery processing had elapsed,
Set the correction value based on the time you did not have to wait,
Increase or decrease the time required to recover the theoretical value.

When the time required for the recovery process has elapsed according to the timer, the access request sending unit 111 sends the access request message again.

If the result of sending the access request message again is that it is still being recovered, the time required to recover the unrecovered portion is calculated again, and the access is set to be performed again after the time required for recovery has elapsed. do.

Furthermore, an example in which the distributed file management table 119 is configured as shown in FIG. 5 and the size of the file recovery unit is made variable will be described. Distributed file management table 11
9 consists of a logical file name 21, a physical file name 22, a recovery flag 23, a recovery first record number 25, a recovery last record number 26, and the number of access users 27.

A request for access to the distributed file A from the user program of the computer 3 is sent to the client management unit 11.
This is done by transmitting a message 30 whose message type 34 is an access request to the server management unit 114, where 0 is the access request transmitter 111. At this time, the message serial number 34 of message 3° starts with ○'' for each user program, and is 1'' 112 II, 11
3 Set the value as P+... When the user program finishes accessing the distributed file, it sets the message serial number 34 to 71111 and sends the message.

When the server management unit 114 of the computer 2 receives a message 30 whose message type 34 is an access request in the reception buffer 107, the file access request receiving unit 115 stores the logical file name 32 of the message 30.
to search the distributed file management table 119. If the message serial number 35 is "no", it means that this is the first access request from the user program, so 1 is added to the corresponding access user number 27 in the distributed file management table 119. If it is P+11'', it means that the access request from the user program has ended.

Subtract 1 from the corresponding number of access users 27 in the distributed file management table 119. Through the above processing and by referring to the distributed file management table 119, the number of users accessing the distributed file A can be determined.

When a user makes a file recovery request to recover file A on computer 1, server management unit 114 on computer 2 sends file recovery data transmitter 117 to
, searches the distributed file management table 119,
The size of the recovery unit is determined according to the number of access users 27. If the number of access users 27 is O, file recovery is performed in large units in order to complete it quickly, and if it is not O, file recovery is performed in small units so that access requests are not made to wait as much as possible.

The size of a message that can actually be sent at one time is up to the size of the sending buffer, so if the number of access users 27 is 0, the file is divided by the size of the sending buffer 108,
Each time the number of access users 27 increases by 1, division is performed, for example, by half the size.

It is desirable to set this to an optimal value depending on the characteristics of the computer system. However, division below the record length is not performed. The first record number and the last record number of the first divided part of the divided file are set in the distributed file management table 119 as the first record number 25 during recovery and the last record number 26 during recovery, and then the recovery data is transmitted. And once the part you sent is recovered,
Set the record number next to the last record number 26 during recovery that was sent last time as the first record number 25 during recovery, set the last record number 26 during recovery so that it is the same as the number of records sent last time, and then send the recovery data. .

When the server management unit 114 of the computer 2 receives the message 30 requesting access to the file being recovered, it searches the distributed file management table 119,
If the access request relates to a record between the first record number under recovery 25 and the last record number under recovery 26, the access status of the file client list 37 will include the status ``file recovery'' and the first record number 25 under recovery and the record number 26. Set the intermediate and final record number 26, and the client management unit 1 of the access request source
Return to 10. The access result receiving unit 112 receives it, and the timer setting unit 113 calculates the time required for recovery based on the first record number 25 during recovery, the last record number 26 during recovery, etc., and after the time necessary for recovery has elapsed, The access request is made again in the same way as in the case described above.

If the access request does not relate to a record between the top record number 25 during recovery and the last record number 26 during recovery, access is performed. What can be accessed is not only the portion of the file that has been recovered, but also includes portions of the file that have not yet been recovered but are not currently being recovered. There is no problem even if the data in the unrecovered part is updated, as it will be the same once that part is recovered.

In this embodiment, the size of the recovery unit is changed depending on the number of accessing users at the start of one recovery process, but it may be made variable depending on the number of accessing users during one recovery.

According to this embodiment, since the file is divided into several parts,
Since parts of the file other than the part that is being recovered can be accessed, the waiting time for a user program to complete the recovery process when accessing the distributed file that is being recovered can be shortened.

〔Effect of the invention〕

According to the present invention, by making an access request again after the time required for file recovery has elapsed, it is possible to prevent the waste of sending and receiving access requests and access result messages many times during file recovery. be.

Furthermore, by changing the size of the recovery unit depending on the number of accessing users, it is possible to recover as quickly as possible while reducing the probability of accessing the portion of the file that is being recovered.

[Brief explanation of drawings]

FIG. 1 shows a computer according to an embodiment of the present invention. 2 is a diagram showing the format of a distributed file management table, FIG. 3 is a diagram showing a message format, and FIG. 4 is a multicomputer system with multiplexed files according to an embodiment of the present invention. FIG. 5 is a diagram showing another format of the distributed file management table. 1.2.3...Computer, 4.5.6...External storage device, 7...Transmission line, 8.1o...File A, 9...File B, 11.12... File C1 13...File D, 21...Logical file name. 22...Physical file name. 23...Recovery flag, 24...Recovered record number, 25...Recovery first record number, 26...Recovery last record number, 27...Number of access users, 30...Message , 31... Message length, 32... Logical file name, 33... Source computer address, 34...
- Message type, 35 - Message serial number, 36... Information, 37... File client list. 38... Data, 101... Transmission control device, 102... Selected reception management unit, 103... Selected reception table, 104... Information processing device, 105.107... Reception buffer, 106. 108・Transmission buffer, 110... Client management unit, 111...
Access request transmitting section, 112... Access result receiving section, 113... Timer setting section, 114... Server management unit, 115... Access request receiving section, 116... Access result transmitting section, 117. ...File recovery data transmission section, 118...
File recovery data receiving unit, 119... Distributed file management table, 120... External storage management unit.

Claims (1)

[Claims] 1. In a distributed file system that is composed of multiple computers and in which multiple files storing the same data exist distributed among the multiple computers, the access requesting computer attempts to access the file system. 1. A distributed file management method, comprising: requesting file access again to the file after a time period required for recovery of the file has elapsed in response to the fact that the file is being recovered. 2. A distributed file system consisting of multiple computers, where multiple files containing the same data are distributed across multiple computers, and file recovery is performed for each subdivided file part. A distributed file management method characterized in that the size of a file recovery unit is changed depending on the file access status. 3. The access request source computer is characterized in that, in response to the fact that the file to be accessed is being recovered, the access requesting computer issues another file access request for the file after the time required to recover the file has elapsed. The distributed file management method according to claim 2.