JP2020003945A - Computer system and data management method - Google Patents

Abstract

An object of the present invention is to realize a system that suppresses an increase in the amount of I / O and data transfer, and that efficiently executes processes using the same data in parallel with easy control.
A computer system in which a plurality of processes that execute processing using collected data acquired from an external device operate in parallel, wherein the computer system includes a computer having a data management unit, When the data is received, the identification information different from the identification information of the collected data, the access identification information used by the process is generated, the correspondence between the identification information of the collected data and the plurality of access identification information is managed, and When the collected data is stored in the area and a read request including the first access identification information for accessing the first collected data from the first process is received, the first collected data is stored in the storage area based on the first access identification information. And sends it to the first process.
[Selection diagram] Fig. 1

Description

  The present invention relates to a computer system and a data management method in which a process for processing a large amount of data acquired from outside operates.

  With the development of IoT (Internet of Things), services that provide new value by analyzing and utilizing data collected from devices such as sensors have appeared.

  In the system that provides the service, processes (applications) that perform various processes such as data collection, data analysis, data utilization, and utilization of analysis results operate. The system temporarily stores data in a predetermined storage area in order to process a large amount of data. The process executes processing using the data stored in the storage area, and after the processing is completed, deletes the data from the storage area.

  Techniques described in Patent Literature 1 and Patent Literature 2 are known as techniques for processing a large amount of data.

  Patent Literature 1 states that “a data processing system includes a first storage device that stores, as input data, a plurality of divided data obtained by dividing the same type of data into predetermined units, and a first storage device that stores each of the plurality of divided data. A child job generation unit that generates a child job based on a parent job that executes processing on each divided data in response to storage in the device, and a child job activation that starts a child job generated by the child job generation unit And a second storage device for storing output data corresponding to each divided data when the child job is executed. " Patent Document 2 discloses that “workflow system 1 includes a terminal device 2 that instructs execution of a workflow with respect to document data, and a job control device that controls processes defined in the workflow to be executed in a predetermined order. And a printing device 4 for performing a printing process, which is a type of process.The control unit 30 of the job control device 3 includes a setting unit 300 for setting a review process in the workflow management table 312A among processes included in the workflow. And a job control unit 302 that controls the execution of a workflow so that a plurality of exclusive processes are executed in accordance with the input order of a plurality of document data. "

WO 2015/125225 JP 2010-9200 A

  When processes using the same data are executed in parallel, there is a possibility that data may be deleted by another process, even though a certain process is executing the process, due to the characteristics of the processes described above. Therefore, the following method is considered as a method of realizing a process in which a plurality of processes handle the same data.

  The first method is a method in which the processing of each process is controlled so as to be sequential. The second method is to generate a copy of data for each process and process the copied data.

  In the first method, in order to prevent processing inconsistency, it is necessary to perform scheduling in consideration of synchronization between processes. Therefore, there is a problem that the processing performance is reduced because the control of the processing becomes complicated. In the second method, there is a problem that the processing performance decreases because the I / O and the data transfer amount increase in accordance with the duplication of data and the like.

  SUMMARY OF THE INVENTION It is an object of the present invention to realize a system and method capable of suppressing an increase in I / O and data transfer amount, and efficiently executing processes using the same data in parallel with easy control.

  A typical example of the invention disclosed in the present application is as follows. That is, a computer system in which a plurality of processes that execute processing using collected data obtained from an external device operate in parallel, wherein each of the plurality of processes executes different processing using the same collected data. The computer system further includes a computer having a processor, a storage device connected to the processor, and an external interface connected to the processor, wherein the computer manages the collected data and controls access to the collected data. A data management unit for controlling, wherein the data management unit, when receiving the collected data, has identification information different from identification information of the collected data, and each of the plurality of processes accesses the collected data. Generating access identification information to be used when performing the A process for managing the correspondence with the access identification information assigned to each of the processes, storing the collected data in a storage area, and accessing the first collected data from a first process included in the plurality of processes. When a read request including access identification information is received, the first collected data is read from the storage area based on the first access identification information, and the first collected data is transmitted to the first process.

  According to the present invention, it is possible to suppress an increase in the amount of I / O and data transfer, and to execute processes using the same data efficiently in parallel with easy control.

FIG. 2 is a diagram illustrating a configuration example of a computer system according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of metadata added to sensor data according to the first embodiment. 9 is a flowchart illustrating an example of a process performed when the data management unit according to the first embodiment receives sensor data. 9 is a flowchart illustrating an example of a process performed when the data management unit according to the first embodiment receives an access request. FIG. 9 illustrates a configuration example of a computer system according to a second embodiment. FIG. 14 is a diagram illustrating a configuration example of a computer system according to a third embodiment. FIG. 14 illustrates an example of a data structure of data management information according to a third embodiment. FIG. 14 illustrates an example of a data structure of data management information according to a fourth embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not construed as being limited to the description of the embodiments below. It is easily understood by those skilled in the art that the specific configuration can be changed without departing from the spirit or spirit of the present invention.

  In the structures of the invention described below, the same or similar structures or functions are denoted by the same reference numerals, and redundant description will be omitted.

  FIG. 1 is a diagram illustrating a configuration example of a computer system according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of metadata added to the sensor data according to the first embodiment.

  The computer system includes a server 100 and a plurality of sensors 101. In the example shown in FIG. 1, the computer system includes M sensors. The server 100 and the plurality of sensors 101 are connected to each other directly or via a network. The network is a WAN (Wide Area Network), a LAN (Local Area Network), or the like. The network connection method may be either wireless or wired.

  The sensor 101 measures a value indicating a state or the like of a control target. For example, the sensor 101 measures temperature, humidity, acceleration, angle, and the like. The sensor 101 transmits sensor data including a measurement value to the server 100. Note that the sensor data includes identification information of the sensor 101, a measurement time, a data type, and the like.

  The server 100 collects sensor data, stores the collected data in a storage area, and executes a process using the collected sensor data. When storing the collected data in the storage area, the server 100 converts the sensor data into a predetermined data format such as a file format. At this time, the metadata 200 is added to the sensor data.

  Here, the metadata 200 added to the sensor data will be described with reference to FIG. The metadata 200 according to the first embodiment includes data identification information 201, date 202, and access identification information 203. Note that the metadata 200 may include data size, data owner, group, permissions, and the like.

  The data identification information 201 is a field for storing identification information of sensor data. The data identification information 201 stores a name such as a path where sensor data is stored.

  The date 202 is a field for storing the date when the sensor data was generated. Instead of the date when the sensor data was generated, the date when the value included in the sensor data was measured, the date when the server 100 received the sensor data, and the like may be stored in the date 202.

  The access identification information 203 is a field for storing the access identification information. As will be described later, access identification information is set in one piece of metadata 200 by the number of processes 150.

  Here, the access identification information is identification information used when each of the plurality of processes 150 accesses the sensor data. One access identification information is assigned to one process 150. Each access identification information is generated so as not to overlap with the identification information of the sensor data. Each access identification information is generated so as not to overlap each other. The above is the description of the metadata 200. Returning to the description of FIG.

  The server 100 has a processor 110, a memory 111, a hard disk drive (HDD) 112, and an external interface 113 as hardware. Each hardware is connected to each other via an internal bus. The server 100 may include an input device such as a keyboard, a mouse, and a touch panel, and an output device such as a display and a printer.

  The processor 110 is an arithmetic device having at least one arithmetic core, and executes a program stored in the memory 111. The processor 110 operates as a functional unit (module) that implements a specific function by executing processing according to a program. In the following description, when processing is described with the functional unit as the subject, it indicates that the processor 110 is executing a program that implements the module.

  The memory 111 stores a program executed by the processor 110 and information used by the program. Further, the memory 111 includes a work area used by the program. The memory 111 may be either a volatile memory or a nonvolatile memory. The programs and information stored in the memory 111 will be described later.

  The HDD 112 is a large-capacity storage device, and stores data permanently. The programs and information stored in the memory 111 may be stored in the HDD 112. In this case, the processor 110 reads a program and information from the HDD 112, loads the program and information into the memory 111, and executes the loaded program.

  Instead of the HDD 112, another storage device such as an SSD (Solid State Drive) may be used. The sensor data received from the sensor 101 is temporarily stored in the HDD 112 according to the first embodiment.

  Here, programs and information stored in the memory 111 will be described. The memory 111 stores a program that implements the data receiving unit 120, the data managing unit 121, and the plurality of processes 150. In the example shown in FIG. 1, M processes 150 are operating.

  The process 150 performs a predetermined process using the sensor data. For example, the process 150 performs a process of storing data in an external database, a statistical process, an analysis process, and the like. It is assumed that the process 150 of the first embodiment does not execute a writing process such as modification and processing of the sensor data itself stored as source data in the HDD 112. In addition, when the process is completed, the process 150 according to the first embodiment transmits a request to delete sensor data or a request to close sensor data.

  The data receiving unit 120 receives the sensor data received via the external interface 113, and transmits the data to the data management unit 121. Note that the data receiving unit 120 may perform conversion of the data format of the sensor data and the like as necessary.

  The data management unit 121 manages sensor data and controls access to the sensor data. Specifically, the following processing is executed.

  When receiving the sensor data A transmitted by the sensor 101-1 from the data receiving unit 120, the data management unit 121 converts the sensor data A into data in a file format to which the metadata 200 is added, and stores the sensor data A in the HDD 112. A is stored.

  At this time, the data management unit 121 generates access identification information to be assigned to each process 150, and sets the access identification information in the metadata 200. Each process 150 accesses the sensor data A using the assigned access identification information. In the data management unit 121, information on the number of running processes 150 (process number setting information) and information on a method of generating access identification information are set in advance.

  When the data management unit 121 receives an access request for any sensor data from the process 150, the data management unit 121 refers to the metadata 200, and performs access to the sensor data to which the metadata 200 including the access identification information included in the access request is added. Control. For example, when receiving a read request including the access identification information “sensor data A1” from the process 150-1, the data management unit 121 reads the sensor data A from the HDD 112 and transmits the sensor data A to the requesting process 150.

  Note that the configuration of the computer system is an example and is not limited to this. For example, the computer system may include a computer that executes each process 150. Further, a virtual machine having the data management unit 121 and a virtual machine on which the process 150 operates may be set on the server 100.

  The data processed by the server 100 may be data other than sensor data, such as operation information of devices such as air conditioners, images captured by an imaging device, and data collected from an external device such as sound collected by a sound collection device. Should be fine.

  Note that each functional unit of the server 100 may combine a plurality of functional units into one module, or may divide one functional unit into a plurality of functional units for each function.

  Next, details of processing executed by the data management unit 121 will be described with reference to FIGS.

  FIG. 3 is a flowchart illustrating an example of a process performed when the data management unit 121 according to the first embodiment receives sensor data.

  When receiving the sensor data from the data receiving unit 120 (step S101), the data management unit 121 generates the sensor data to which the metadata 200 is added and stores the sensor data in the HDD 112 (step S102). At this time, the access identification information 203 is not set in the metadata 200. When the metadata 200 is set in the sensor data received from the data receiving unit 120, the data management unit 121 stores the received sensor data in the HDD 112 as it is.

  Next, the data management unit 121 generates access identification information to be assigned to each process 150, and sets the generated access identification information in the metadata 200 (Step S103).

  Specifically, the data management unit 121 generates access identification information to be assigned to each process 150 based on the process number setting information. Further, the data management unit 121 sets the generated access identification information in the access identification information 203 of the metadata 200. For example, the data management unit 121 generates access identification information by combining the identification information of the received sensor data and the identification information of the process 150. The present invention is not limited to a method for generating access identification information.

  Note that the order of the processes in step S102 and step S103 may be reversed. That is, after generating the access identification information, the data management unit 121 stores, in the HDD 112, the sensor data to which the metadata 200 in which the access identification information is set is added.

  Note that the data management unit 121 may transmit a notification of reception of sensor data including access identification information to each process 150 as necessary. When the process 150 receives the reception notification, the process 150 accesses the sensor data by transmitting an access request including the access identification information at an arbitrary timing.

  FIG. 4 is a flowchart illustrating an example of a process performed when the data management unit 121 according to the first embodiment receives an access request.

  When receiving an access request from an arbitrary process 150 (step S201), the data management unit 121 determines whether the access request is a read request (step S202).

  When it is determined that the received access request is a read request, the data management unit 121 acquires sensor data from the HDD 112 by referring to the metadata 200 based on the access request (step S203).

  Specifically, the data management unit 121 searches the metadata 200 in which the access identification information 203 includes the access identification information included in the read request. The data management unit 121 acquires sensor data from the HDD 112 based on the data identification information 201 of the searched metadata 200. The above-described processing can be speeded up by storing the cache data of the metadata 200 in the memory 111 in advance.

  Next, the data management unit 121 transmits the sensor data obtained from the HDD 112 to the request source process 150 (Step S204), and thereafter ends the processing.

  At this time, the data management unit 121 may replace the value of the data identification information 201 with the access identification information included in the read request.

  If it is determined in step S202 that the received access request is not a read request, that is, the received access request is a request for instructing deletion of sensor data, the data management unit 121 updates the metadata 200 (step S202). S205). The request to instruct to delete the sensor data is a request transmitted when the process 150 completes the process using the predetermined sensor data, and is, for example, a request to delete the sensor data or a request to close the sensor data. Here, it is assumed that a deletion request has been received.

  Specifically, the data management unit 121 searches the metadata 200 in which the access identification information 203 includes the access identification information included in the deletion request. The data management unit 121 deletes the access identification information included in the deletion request from the access identification information 203 of the searched metadata 200. Further, the data management unit 121 notifies the process 150 that the deletion of the sensor data has been completed. As described above, when the data management unit 121 detects that the process 150 has completed the process using the predetermined sensor data, the data management unit 121 deletes the access identification information from the metadata 200.

  The above-described processing can be speeded up by storing the cache data of the metadata 200 in the memory 111 in advance.

  Next, the data management unit 121 determines whether the sensor data to be deleted can be deleted from the HDD 112 (Step S206).

  Specifically, the data management unit 121 determines whether the access identification information 203 of the metadata 200 specified in step S205 is empty. If the access identification information 203 is empty, the data management unit 121 determines that the sensor data to be deleted can be deleted from the HDD 112.

  If it is determined that the sensor data to be deleted cannot be deleted from the HDD 112, the data management unit 121 ends the processing.

  If it is determined that the sensor data to be deleted can be deleted from the HDD 112, the data management unit 121 deletes the sensor data from the HDD 112 (Step S207). After that, the data management unit 121 ends the processing. As described above, the deletion of the sensor data from the HDD 112 is performed independently of the process of the process 150.

  When the process 150 is added or deleted, the process number setting information set in the data management unit 121 may be updated. Since there is no need to set the workflow of the process 150 in consideration of the exclusive control as in the related art, it is possible to flexibly cope with an increase or decrease in the number of processes 150.

  As described above, the first embodiment has the following features.

  First, the data management unit 121 manages identification information of sensor data and access identification information assigned to the process 150 in association with each other. This produces the following effects. The data management unit 121 can respond to the read request from each process 150 with the same sensor data. Since the sensor data used by each process 150 is not copied, an increase in I / O and data transfer amount due to data copying can be suppressed. In addition, since each process 150 recognizes the sensor data as different sensor data, it is not necessary to perform synchronous control of the processing executed by each process 150. Therefore, a plurality of processes 150 can be executed in parallel without performing complicated control.

  Second, the data management unit 121 can delete sensor data based on the presence or absence of access identification information in the access control processing. As a result, it is not necessary to separately execute a process for deleting sensor data, so that there is an effect that the consumption of computer resources and the processing load can be reduced. Also, there is an effect that the development cost of the system can be reduced.

  The second embodiment differs from the first embodiment in the mechanism by which the data management unit 121 deletes sensor data. Hereinafter, the second embodiment will be described focusing on differences from the first embodiment.

  FIG. 5 is a diagram illustrating a configuration example of a computer system according to the second embodiment.

  The computer system according to the second embodiment includes a server 100 and a plurality of sensors 101, similarly to the computer system according to the first embodiment. The hardware configuration of the server 100 according to the second embodiment is the same as that of the server 100 according to the first embodiment.

  The software configuration of the server 100 according to the second embodiment is partially different from the server 100 according to the first embodiment. Specifically, the memory 111 stores a counter 130. The counter 130 has only the number of sensor data stored in the HDD 112.

  The counter 130 stores the number of processes 150 that have not been completed among the processes 150 that execute the process using the sensor data.

  In the second embodiment, the processing executed by the data management unit 121 is partially different. First, a process when sensor data is received will be described. This process differs from the process in step S103.

  In step S103, the data management unit 121 sets access identification information in the metadata 200 and generates a counter 130 in which the number of processes 150 is set. The counter 130 is provided with identification information of the sensor data.

  Next, a process performed when an access request for sensor data is received will be described. In this process, the processes of step S205, step S206, and step S207 are different.

  In step S205, the data management unit 121 searches for the metadata 200 in which the access identification information 203 includes the access identification information included in the deletion request. The data management unit 121 subtracts “1” from the value of the counter 130 to which the identification information stored in the data identification information 201 of the metadata 200 has been added. The above-described processing can be speeded up by storing the cache data of the metadata 200 in the memory 111 in advance.

  In step S206, the data management unit 121 determines whether the value of the counter 130 specified in step S205 is “0”. When the value of the counter 130 is “0”, the data management unit 121 determines that the sensor data to be deleted can be deleted.

  In step S207, the data management unit 121 deletes the specified counter 130 together with the sensor data. Other processes in the second embodiment are the same as those in the first embodiment.

  According to the second embodiment, the same effects as those of the first embodiment can be obtained.

  The third embodiment differs from the first embodiment in the management method of the access identification information and the like. Hereinafter, the third embodiment will be described focusing on differences from the first embodiment.

  FIG. 6 is a diagram illustrating a configuration example of a computer system according to the third embodiment. FIG. 7 is a diagram illustrating an example of a data structure of the data management information 131 according to the third embodiment.

  The computer system according to the third embodiment includes a server 100 and a plurality of sensors 101, similarly to the computer system according to the first embodiment. The hardware configuration of the server 100 according to the third embodiment is the same as that of the server 100 according to the first embodiment.

  The software configuration of the server 100 according to the third embodiment is partially different from the server 100 according to the first embodiment. Specifically, the memory 111 stores the data management information 131.

  The data management information 131 is information for controlling access to sensor data. Here, an example of the data structure of the data management information 131 will be described with reference to FIG. The data management information 131 is information in a table format that stores an entry composed of data identification information 701, date 702, and access identification information 703.

  One entry corresponds to one sensor data. The access identification information 703 of one entry includes the same number of rows as the number of processes 150.

  The data identification information 701 and the date 702 are fields for storing some values included in the metadata 200. The data identification information 701 and date 702 are the same fields as the data identification information 201 and date 202. Note that the entry may include a field for storing a value of an attribute other than the data identification information 701 and the date 702.

  The access identification information 703 is a field for storing a value for controlling access to sensor data. The access identification information 703 is the same field as the access identification information 203.

  In the third embodiment, the processing executed by the data management unit 121 is different. First, a process when sensor data is received will be described. This process differs from the process in step S103.

  In step S103, the data management unit 121 generates access identification information. Further, the data management unit 121 adds an entry to the data management information 131, sets a value included in the metadata 200 to the data identification information 701 and date 702 of the added entry, and sets the access identification information of the added entry. The same number of rows as the number of processes 150 are added to 703, and the generated access identification information is set in the access identification information 703 of each row. The metadata 200 according to the third embodiment is general metadata in which no access identification information is set.

  Processing executed when a request for access to sensor data is received will be described. In this process, the processes in step S203, step S205, step S206, and step S207 are different.

  In step S203, the data management unit 121 acquires sensor data from the HDD 112 with reference to the data management information 131 based on the read request.

  Specifically, the data management unit 121 refers to the data management information 131 to search for an entry in which the access identification information 703 is set with the access identification information included in the read request. The data management unit 121 acquires sensor data from the HDD 112 based on the identification information set in the data identification information 701 of the searched entry.

  In step S205, the data management unit 121 searches for an entry in which the access identification information included in the deletion request is set in the access identification information 703. The data management unit 121 deletes the access identification information included in the deletion request from the access identification information 703 of the searched entry.

  In step S206, the data management unit 121 determines whether the access identification information 703 of the entry searched in step S205 is empty. When the access identification information 703 is empty, the data management unit 121 determines that the sensor data to be deleted can be deleted from the HDD 112.

  In step S207, the data management unit 121 deletes the sensor data from the HDD 112, and deletes the entry searched in step S205 from the data management information 131.

  According to the third embodiment, the same effects as those of the first embodiment can be obtained. In the third embodiment, the data management unit 121 can specify the sensor data and determine whether to delete the sensor data without accessing the HDD 112, so that the processing delay can be suppressed.

  The fourth embodiment differs from the first embodiment in the management method of the access identification information and the like. Hereinafter, the fourth embodiment will be described focusing on differences from the first embodiment.

  The configuration of the computer system of the fourth embodiment is different from that of the third embodiment in that the computer system holds data management information 131. FIG. 8 is a diagram illustrating an example of a data structure of the data management information 131 according to the fourth embodiment.

  The data management information 131 is information in a table format that stores an entry including an i-node number 801 and data identification information 802.

  One entry corresponds to one sensor data. Further, the data identification information 802 of one entry includes as many rows as the number obtained by adding 1 to the number of processes 150.

  The inode number 801 is a field for storing information for accessing the metadata 200. In the present embodiment, an i-node number which is an identification number of the i-node is stored. The data identification information 802 is the same field as the data identification information 201. However, in the fourth embodiment, the access identification information is stored in the data identification information 201.

  In the fourth embodiment, the processing executed by the data management unit 121 is different. First, a process when sensor data is received will be described. This process differs from the process in step S103.

  In step S103, the data management unit 121 generates access identification information. The data management unit 121 adds an entry to the data management information 131, and sets an i-node number in the i-node number 801 of the added entry. In addition, the data management unit 121 adds rows to the data identification information 802 of the added entry by the number obtained by adding 1 to the number of processes 150, sets data identification information in one row, and generates data in another row. Set access identification information. The metadata 200 according to the fourth embodiment is general metadata in which no access identification information is set.

  Processing executed when a request for access to sensor data is received will be described. In this process, the processes of step S203, step S205, step S206, and step S207 are different.

  In step S203, the data management unit 121 acquires sensor data from the HDD 112 with reference to the data management information 131 based on the read request.

  Specifically, the data management unit 121 searches the data management information 131 for an entry in which the data identification information 802 is set with the access identification information included in the read request. The data management unit 121 acquires sensor data from the HDD 112 based on the inode number set in the inode number 801 of the searched entry.

  In step S205, the data management unit 121 searches for an entry in which the access identification information included in the deletion request is set in the data identification information 802. The data management unit 121 deletes the access identification information included in the deletion request from the data identification information 802 of the searched entry.

  In step S206, the data management unit 121 determines whether or not the identification information stored in the data identification information 802 of the entry searched in step S205 is one. That is, it is determined whether or not only the data identification information originally given to the sensor data is stored in the data identification information 802. If there is one identification information stored in the data identification information 802, the data management unit 121 determines that the sensor data to be deleted can be deleted from the HDD 112.

  In step S207, the data management unit 121 deletes the sensor data from the HDD 112, and deletes the entry searched in step S205 from the data management information 131.

  According to the fourth embodiment, the same effects as those of the first embodiment can be obtained. In the fourth embodiment, the data management unit 121 can specify the sensor data and determine whether to delete the sensor data without accessing the HDD 112, so that the processing delay can be suppressed.

  Note that the present invention is not limited to the above-described embodiment, and includes various modifications. Further, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described above. In addition, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of one embodiment can be added to the configuration of another embodiment. Further, for a part of the configuration of each embodiment, it is possible to add, delete, or replace another configuration.

  Further, each of the above-described configurations, functions, processing units, processing means, and the like may be partially or entirely realized by hardware, for example, by designing an integrated circuit. In addition, the above-described configurations, functions, and the like may be implemented by software by a processor interpreting and executing a program that implements each function. Information such as a program, a table, and a file for realizing each function may be stored in a storage device such as a memory, an HDD and an SSD, or a recording medium such as an IC card, an SD card, and an optical disk. In addition, control lines and information lines indicate those which are considered necessary for explanation, and do not necessarily indicate all control lines and information lines on a product. For example, it may be considered that almost all components are actually connected to each other.

100 server 101 sensor 110 processor 111 memory 112 HDD
113 External interface 120 Data reception unit 121 Data management unit 130 Counter 131 Data management information 150 Process 200 Metadata

Claims (12)

A computer system in which a plurality of processes that execute processing using collected data obtained from an external device operate in parallel,
Each of the plurality of processes performs a different process using the same collected data,
The computer system includes a processor having a processor, a storage device connected to the processor, and an external interface connected to the processor.
The computer has a data management unit that manages the collected data and controls access to the collected data,
The data management unit includes:
When receiving the collected data, the identification information is different from the identification information of the collected data, and generates access identification information used when each of the plurality of processes accesses the collected data,
Managing the correspondence between the identification information of the collected data and the access identification information assigned to each of the plurality of processes,
Storing the collected data in a storage area;
When receiving a read request including first access identification information for accessing the first collected data from the first process included in the plurality of processes, the first collected data is based on the first access identification information. Read from the storage area,
A computer system for transmitting the first collected data to the first process. The computer system according to claim 1, wherein
The data management unit includes:
If the first process detects that the process using the first collected data is completed, delete the first access identification information associated with the identification information of the first collected data,
A computer system, wherein, when the access identification information associated with the first collected data does not exist, the first collected data is deleted from the storage area. The computer system according to claim 1, wherein
The data management unit includes:
When generating the access identification information, the identification information of the collected data is given, and generates a counter in which the number of the plurality of processes is set,
When the first process detects that the process using the first collected data is completed, subtracts 1 from the value of the counter to which the identification information of the first collected data is added,
Determine whether the value of the counter to which the identification information of the first collected data is added is 0,
A computer system, wherein, when it is determined that the value of the counter to which the identification information of the first collected data is added is 0, the first collected data is deleted from the storage area. The computer system according to claim 2 or 3, wherein:
The data management unit includes:
When receiving the collected data, set the access identification information in the metadata of the collected data,
When receiving the read request of the first collected data from the first process, based on the first access identification information, by referring to the metadata of the collected data stored in the storage area, from the storage area A computer system, wherein the first collected data is specified as collected data to be read. The computer system according to claim 2 or 3, wherein:
The data management unit includes:
Holding data management information for controlling access to the collected data,
When receiving the collected data, register an entry in which the identification information of the collected data and the access identification information are associated with the data management information,
When receiving a read request for the first collected data from the first process, by referring to the data management information based on the first access identification information, the first collected data as collected data to be read from the storage area A computer system characterized in that: The computer system according to claim 2 or 3, wherein:
When the collected data is stored in the storage area, the data management unit transmits a data reception notification including the access identification information assigned to each of the plurality of processes, to the plurality of processes, Computer system. A data management method in a computer system in which a plurality of processes that execute processing using collected data acquired from an external device operate in parallel,
Each of the plurality of processes performs a different process using the same collected data,
The computer system includes a processor having a processor, a storage device connected to the processor, and an external interface connected to the processor.
The computer has a data management unit that manages the collected data and controls access to the collected data,
The data management method includes:
When the data management unit receives the collected data, the identification information is different from the identification information of the collected data, and the access identification information used when each of the plurality of processes accesses the collected data. The first step of generating,
The second step of managing the correspondence between the identification information of the collected data and the access identification information assigned to each of the plurality of processes, and storing the collected data in a storage area,
When the data management unit receives a read request including first access identification information for accessing the first collected data from the first process included in the plurality of processes, the data management unit performs the processing based on the first access identification information. A third step of reading the first collected data from the storage area,
A fourth step in which the data management unit transmits the first collected data to the first process,
A data management method comprising: The data management method according to claim 7, wherein
When the data management unit detects that the first process has completed processing using the first collected data, the data management unit deletes the first access identification information associated with the identification information of the first collected data. Steps and
When the data management unit does not have the access identification information associated with the first collected data, deleting the first collected data from the storage area,
A data management method comprising: The data management method according to claim 7, wherein
The first step, the data management unit is provided with identification information of the collected data, and includes a step of generating a counter in which the number of the plurality of processes is set,
The data management method includes:
When the data management unit detects that the first process has completed the process using the first collected data, subtracting 1 from the value of the counter to which the identification information of the first collected data is added When,
A step in which the data management unit determines whether or not the value of the counter to which the identification information of the first collected data is assigned is 0;
When it is determined that the value of the counter to which the identification information of the first collected data is given is 0, the data management unit deletes the first collected data from the storage area,
A data management method comprising: A data management method according to claim 8 or claim 9, wherein
The first step, when the data management unit receives the collected data, including the step of setting the access identification information in the metadata of the collected data,
The third step is that, based on the first access identification information, the data management unit refers to the metadata of the collected data stored in the storage area, and as the collected data read from the storage area, A data management method, comprising a step of specifying first collected data. A data management method according to claim 8 or claim 9, wherein
The data management unit holds data management information for controlling access to the collected data,
The first step, the data management unit includes a step of registering in the data management information an entry associated with the identification information of the collected data and the access identification information,
The third step is a step in which the data management unit refers to the data management information based on the first access identification information to specify the first collected data as collected data to be read from the storage area. A data management method comprising: A data management method according to claim 8 or claim 9, wherein
The data management unit includes a step of transmitting, to the plurality of processes, a data reception notification including the access identification information assigned to each of the plurality of processes, when the collected data is stored in the storage area. A data management method characterized by the following.

Images