JP7662054B2 - Data processing device, data processing method, and program - Google Patents

Description

The present invention relates to a data processing device, a data processing method, and a program.

In order to grasp the communication status in a network, it is common to acquire packets flowing through the network and their information. Conventionally, packets on a network generally had a unique field structure in the design concept, but as network virtualization progresses and multiple networks are configured on a single physical configuration, the field structure of packets on a network is no longer uniquely determined. In such a situation, a communication data identification device is required to be able to flexibly select an identification target for the field structure of a packet. To solve this problem, a comparison and matching method using a TCAM (Ternary Content Addressable Memory) is generally used (for example, Non-Patent Document 1). However, since TCAM is implemented as a dedicated chip, there is a problem that it can only be introduced by an expensive dedicated device. In contrast, for hardware and software that are not dedicated chips, a design method using a high-level language such as C or JAVA, which is easy to adapt to new fields with few man-hours, is used.

Catalyst - CAM / TCAM [Retrieved December 1, 2021], Internet: https://www.infraexpert.com/study/catalyst21.html

A high-level language is a procedural description language, and describes the processing that is performed sequentially. Figure 9 shows a data processing device 910 as a comparative example that can be considered as a device that employs sequential processing. In this comparative example, the output data from the receiving unit 911, the time stamp unit 912, and the packet identification unit 913 are all stored in the buffer 919 before processing by the processing execution unit 914 begins. Therefore, the processing execution unit 914 must wait until all the output data (processing results) from each unit 911 to 913 are available, which results in poor operating efficiency. Such issues can also be said to be true of data processing devices in general other than the data processing device 910.

The present invention aims to improve the operating efficiency of the processing execution unit.

In order to solve the above problem, the data processing device of the present invention comprises a receiving unit that executes a receiving process to receive and output data, a first processing unit that executes a first process on the data output from the receiving unit and outputs a first processing result of the first process, a second processing unit that executes a second process on the data after the first process and outputs a second processing result of the second process, and a processing execution unit that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result, wherein the processing time for at least the first processing and the second processing of the receiving process, the first processing, and the second processing is constant regardless of the content of the data to be processed, and the processing execution unit starts processing using at least one of the data and the first processing result for the data before receiving the second processing result.

In order to solve the above problem, the data processing method of the present invention includes a receiving step of executing a receiving process to receive and output data, a first processing step of executing a first process on the data output in the receiving step and outputting a first processing result of the first process, a second processing step of executing a second process on the data after the first process and outputting a second processing result of the second process, and a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result, wherein the processing time of at least the first processing and the second processing among the receiving process, the first processing, and the second processing is constant regardless of the content of the data to be processed, and in the processing execution step, processing using at least one of the data and the first processing result for the data is started before the second processing result is received.

In order to solve the above problem, the computer of the present invention causes the computer to execute a reception process that receives and outputs data, a first processing step that executes a first process on the data output in the reception step and outputs a first processing result of the first process, a second processing step that executes a second process on the data after the first process and outputs a second processing result of the second process, and a processing execution step that receives the data output in the reception step, receives the first processing result output in the first processing step, receives the second processing result output in the second processing step, and executes processing based on the received data, the first processing result, and the second processing result, wherein the processing time for at least the first processing and the second processing among the reception process, the first processing, and the second processing is constant regardless of the content of the data to be processed, and in the processing execution step, processing using at least one of the data and the first processing result for the data is started before the second processing result is received.

According to the present invention, the operating efficiency of the processing execution unit is improved.

FIG. 1 is a diagram showing the configuration of a data processing device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a data structure of a packet. FIG. 3 is a diagram showing an example of the structure of the rule table. FIG. 4 is a diagram for explaining a method for making the processing time constant. FIG. 5 is a diagram for explaining a method for making the processing time constant. FIG. 6 is a timing chart showing the operation of the data processing device. FIG. 7 is a timing chart showing the operation of the data processing device. FIG. 8 is a diagram showing the configuration of the data processing device of FIG. 1 implemented by a computer. FIG. 1 is a configuration diagram of a data processing device according to a comparative example.

Below, an embodiment of the present invention is explained with reference to the drawings.

1, the data processing device 10 according to the present embodiment includes a receiving unit 11, a time stamp unit 12 configured as a first processing unit, a packet identification unit 13 configured as a second processing unit, a processing execution unit 14, an output unit 15, and a storage unit 19. The data processing device 10 thus configured is configured as a traffic monitoring device that monitors traffic on a communication network NW for each flow.

At least a part of each of the above units 11 to 14 is composed of a logic circuit written in, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. At least a part of each of the units 11 to 14 may be composed of a processor such as a CPU (Central Processing Unit) that executes a program. The memory unit 19 is composed of an appropriate non-volatile storage device such as a flash memory or an SSD (Solid State Drive). The memory unit 19 may be composed of at least a part of a memory such as an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. Each of the units 11 to 14 may be composed of a memory that temporarily holds data being processed, etc.

The receiver 11 receives packets (more specifically, mirroring packets) flowing through the communication network NW and executes a receiving process to output the packets to the time stamp unit 12 and the process execution unit 14. As shown in FIG. 2, a packet is composed of data (also called a payload) and a header placed before the data. Each piece of data constituting the header is assigned a meaning for each number of bits. This bit-separated data with a meaning is called a header field value, and a group of packets with matching header field values is called a flow. Therefore, the header field value is also information that identifies a flow. The header field value may include each IP (Internet Protocol) address indicating the source and destination of the L3 header. A flow may be, for example, a set of packets that have a common header field value indicating at least one of the destination, source, and communication protocol.

Returning to FIG. 1, the time stamp unit 12 executes time stamp processing by referring to a real time clock or the like, generating a time stamp for the packet output from the receiving unit 11, and outputting the time stamp to the processing execution unit 14. The time stamp unit 12 outputs the packet from the receiving unit 11 to the packet identification unit 13. At this time, the packet may have been assigned a time stamp.

The packet identification unit 13 executes a packet identification process to identify the flow to which the packet from the timestamp unit 12, in which the timestamp was generated, belongs, and to output a flow ID, which is flow information specifying the identified flow. The packet identification unit 13 comprises a header analysis unit 13A, a hash calculation unit 13B, and a rule matching unit 13C, and each of these units 13A to 13B executes the following processes to execute the packet identification process.

A packet input from the time stamp unit 12 to the packet identification unit 13 is first input to the header analysis unit 13A. The header analysis unit 13A executes an analysis process to analyze the header of the input packet and extract a header field value from the header. The extracted header field value is input to the hash calculation unit 13B.

The hash calculation unit 13B executes a hash calculation process to hash the input header field value. The hash calculation unit 13B outputs the hashed header field value together with the timestamp to the rule matching unit 13C.

The rule matching unit 13C executes a matching process to compare the input header field value (hashed header field value) with a predetermined rule that specifies packets belonging to the flow to be monitored and is registered in the rule table 19A stored in the storage unit 19. As shown in FIG. 3, the rules registered in the rule table 19A are registered for each flow, and one rule includes the header field value (hashed header field value) of the flow and a flow ID associated with the header field value. The flow ID may be a rule ID that identifies the rule. In the matching process, the rule matching unit 13C refers to each rule in the rule table 19A and obtains a flow ID corresponding to the input header field value. The rule matching unit 13C outputs the obtained flow ID to the processing execution unit 14.

The processing execution unit 14 receives packets from the receiving unit 11, receives timestamps from the timestamp unit 12, receives a flow ID from the packet identification unit 13, and executes processing (here, processing required for traffic monitoring) based on the received packets, timestamps, and flow ID. The processing execution unit 14 includes a calculation unit 14A, a packet capture unit 14B, and an anomaly detection unit 14C as calculation units that execute the processing.

The counting unit 14A counts the number of times a flow ID is input from the packet identification unit 13 for each flow ID, i.e., for each flow. The counted number of receptions can also be said to be the number of packets received by the receiving unit 11, i.e., the number of packets. The counting unit 14A measures the amount of data based on the packets from the receiving unit 11 and accumulates the measured amount of data for each flow. For each fixed period of time, the counting unit 14A outputs the number of packets and the amount of data counted or accumulated for each flow as the accumulation result to the output unit 15. The processing result may be either the number of packets or the amount of data.

The packet capture unit 14B performs packet capture to store packets from the receiving unit 11 in the memory unit 19. Furthermore, the packet capture unit 14B assigns a timestamp from the timestamp unit 12.

The anomaly detection unit 14C detects a traffic anomaly (e.g., a microburst) for each flow based on some discriminant, such as packets stored in the memory unit 19, the counting results by the counting unit 14A, and the timestamps from the timestamp unit 12. When the anomaly detection unit 14C detects a traffic anomaly, it outputs to the output unit 15 anomaly occurrence information indicating that a traffic anomaly has occurred. The anomaly occurrence information may include information on the type of traffic anomaly that has occurred.

The output unit 15 outputs the processing results by the processing execution unit 14, here the counting results from the counting unit 14A and the abnormality occurrence information from the abnormality detection unit 14C. The output unit 15 displays the counting results and/or the abnormality occurrence information on, for example, a display unit provided in the data processing device 10. The output unit 15 may, for example, output the abnormality occurrence information to a predetermined processing unit provided inside or outside the data processing device 10, and the processing unit may execute processing to cut off communication for a flow in which a traffic abnormality has been detected based on the abnormality occurrence information. The output unit 15 may acquire packets captured by the packet capture unit 14B and output the acquired packets to a device external to the data processing device 10.

In this embodiment, the receiving unit 11 is configured so that the processing time of the receiving process (hereinafter also referred to as processing time A) is constant regardless of the data content of the packet input to the receiving unit 11. The time stamp unit 12 is also configured so that the processing time of the time stamp process (hereinafter also referred to as processing time B) is constant regardless of the data content of the packet input to the time stamp unit 12. The packet identification unit 13 is also configured so that the processing time of the packet identification process (hereinafter also referred to as processing time C) is constant regardless of the data content of the packet input to the packet identification unit 13. In other words, each of the units 11 to 13 completes the necessary processing for any input data (data to be processed) in a constant processing time. The start timing of each of the above processing times A to C is, for example, the timing when the data starts to be input. The end timing of each of the above processing times A to C is, for example, the timing when the data (processing result, etc.) starts to be output. The processing times A to C may be the same processing time as each other, or may be different from each other.

The fact that the processing time A is the same for any packet means that the processing is completed in a constant processing time regardless of which processing branch is taken in the processing in the receiving unit 11. For example, as shown in the upper diagram of FIG. 4, if the processing time of processing A after branching is three clock cycles and the processing time of processing B is two clock cycles, as shown in the lower diagram, a delay circuit for one clock cycle is inserted after processing B. This makes the processing time after branching constant. Alternatively, as shown in FIG. 5, a part of processing A after branching is parallelized to reduce the processing time to two clock cycles. This makes the processing time after branching constant. The same applies to the time stamp unit 12 and the packet identification unit 13. Note that the adjustment of the processing time is not limited to this. Each unit 11 to 13 may measure the time from the start of processing using a timer or the like, and even if the processing is completed, the data output of the processing result may be waited until the time from the start of processing measured by the timer or the like reaches the processing times A to C.

As shown in FIG. 6, the receiving unit 11 executes the receiving process at processing time A. The packet received in the receiving process is input to the time stamping unit 12 and the processing execution unit 14 (timing T2). The time stamping unit 12 executes the time stamping process at processing time B, triggered by the input of the packet from the receiving unit 11. The time stamp added in the time stamping process is input to the processing execution unit 14, and the packet to be timestamped is input to the packet identification unit 13 (timing T3). The packet identification unit 13 executes the packet identification process at processing time C, triggered by the input of the packet from the time stamping unit 12. The flow ID and the like obtained in the packet identification process are input to the processing execution unit 14 (timing T4).

The processing execution unit 14 can identify which data has been input to it in relation to the processing times A to C. This is because the processing times A to C are constant as described above. The receiving unit 11 sends a packet reception notification to the processing execution unit 14, for example, when the packet reception processing starts (timing T1).

After the reception notification is sent, the packet is always input to the processing execution unit 14 after the processing time A has elapsed (timing T2). Therefore, the processing execution unit 14 recognizes the data input after the processing time A has elapsed after receiving the reception notification as a packet. The processing execution unit 14 performs packet capture, for example, by using the packet capture unit 14B to store the packet in the memory unit 19.

A timestamp is always input to the processing execution unit 14 after the processing time B has elapsed since the input of the packet (timing T3). Therefore, the processing execution unit 14 recognizes the data input after the processing time B has elapsed since the input of the packet as the timestamp assigned to the packet. The processing execution unit 14 associates the timestamp input from the timestamp unit 12 with the packet and stores the timestamp in the storage unit 19, for example, so that the timestamp is assigned to the packet saved in the storage unit 19 by the packet capture unit 14B. Note that the processing execution unit 14 may hold the packet, and assign a timestamp to the packet upon input of the timestamp, and store the packet in the storage unit 19.

After the timestamp is input, the flow ID and other information obtained by the packet identification process are always input to the processing execution unit 14 after the processing time C has elapsed (timing T4). Therefore, the processing execution unit 14 recognizes the data input after the processing time C has elapsed after the timestamp is input as the flow ID of the flow to which the current packet belongs. The processing execution unit 14 executes the processing of the counting unit 14A and the processing of the anomaly detection unit 14C based on the flow ID and other information.

The processing execution unit 14 can determine which data arrived at which timing based on the processing times A to C of the designed units 11 to 13 for each piece of data input at different times. . The processing execution unit 14 can then start processing using at least one of a packet and its timestamp (for example, packet capture and timestamp assignment in FIG. 6) before a flow ID or other information about a packet is input. This improves the efficiency of operation because the processing execution unit 14 does not have to wait until a flow ID or other information is input before starting processing.

Furthermore, after receiving a packet from the receiving unit 11, the processing execution unit 14 processes the multiple pieces of data input at each fixed processing time of the timestamp processing and the packet identification processing as a timestamp for the packet and a flow ID for the packet. This prevents the processing execution unit 14 from mistaking the timestamp or flow ID for one packet for the timestamp or flow ID of another packet.

Furthermore, each of the sections 11 to 13 preceding the processing execution section 14 can sequentially accept subsequent input data, starting from the section that has completed data output. As shown in Figure 7, for example, the receiving section 11 is configured to be able to receive a second packet (see "receiving process" in dotted lines) after outputting a first packet (see "receiving process" in solid lines) before the packet identification process by the packet identification section 13 is completed. This speeds up data processing.

As shown by the dotted lines in Figure 1, at least one of the header analysis unit 13A and the hash calculation unit 13B of the packet identification unit 13 may output processing result data to the processing execution unit 14. The processing execution unit 14 may perform a predetermined process based on the input processing result data. In such a case, it is preferable that the processing time of each of the units 13A to 13C of the packet identification unit 13 is constant regardless of the content of the data to be processed. The processing execution unit 14 may detect the input of a packet based on the data content, etc., and in this case, which data is the timestamp and flow ID can be identified based on the processing times B and C after the input of the packet.

Figure 8 shows a hardware configuration diagram of the data processing device 10 when configured by a computer. The data processing device 10 comprises a processor 101 such as a CPU, a main memory 102 of the processor 101, and a non-volatile storage device 103 that stores programs and various data and constitutes the storage unit 19 of Figure 1. The data processing device 10 further comprises a NIC (Network Interface Card) 104 that is connected to the communication network NW and relays packets. The processor 101 executes or uses the programs and data stored in the storage device 103 and read into the main memory 102 to operate as the above-mentioned units 11 to 15. The receiving unit 11 and the output unit 15 may be realized by a combination of the processor 101 that executes programs and the NIC 104.

The data processing device 10 is not limited to the above configuration. The data processing device 10 may include, for example, a receiving unit (receiving unit 11 in the above configuration) that performs a receiving process to receive and output data (packets in the above configuration), and a first processing unit (timestamp unit 12 in the above configuration) that performs a first process (timestamp process in the above configuration) on the data output from the receiving unit and outputs a first processing result of the first process (timestamp in the above configuration). The data processing device 10 may include, for example, a second processing unit (packet identification unit 13 in the above configuration) that performs a second process (packet identification process in the above configuration) on the data after the first process and outputs a second processing result of the second process, and a processing execution unit (processing execution unit 14 in the above) that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result (processing by the aggregation unit 14A, packet capture unit 14B, and anomaly detection unit 14C in the above). The processing time of at least the first processing and the second processing among the reception processing, the first processing, and the second processing may be constant regardless of the contents of the data to be processed, and the processing execution unit may start processing using at least one of the data and the first processing result for the data (packet capture and time stamp assignment in FIG. 6 in the above) before receiving the second processing result. With this configuration, the processing execution unit does not wait to start processing until the second processing result is input as in the comparative example, and the operation efficiency of the processing execution unit is improved. Note that specific examples of each of the above processing are not limited to the above embodiment. The data may be image data, the first processing may be a binarization process of an image, and the second processing may be image recognition of the binarized image. The processing execution unit may execute a process of storing the image data, the binarized image, and the result of the image recognition in a storage unit.

The processing execution unit may execute the processing on a plurality of data input at fixed processing times for each of the first processing and the second processing after receiving the data from the receiving unit, as the first processing result and the second processing result for the data, respectively, thereby preventing the first processing result and the second processing result for some data from being confused as the processing result for other data.

The receiving unit is configured to receive second data as the other data after outputting the first data as the data and before the second processing is completed. This improves the processing speed.

The present invention is not limited to the above-mentioned embodiments and modifications. For example, the present invention includes various modifications to the above-mentioned embodiments and modifications that can be understood by a person skilled in the art within the scope of the technical concept of the present invention. The configurations listed in the above-mentioned embodiments and modifications can be appropriately combined to the extent that there is no contradiction. It is also possible to delete any of the above-mentioned configurations. The above program is not limited to the non-volatile storage device 103, and may be stored in a non-transitory computer-readable storage medium. The "device" and "part" may be an entity in which the configuration that realizes the operation is contained in one housing, or an entity (system) in which the configuration that realizes the operation is distributed and contained in multiple housings.

10...data processing device, 11...receiving unit, 12...time stamp unit, 13...packet identification unit, 13A...header analysis unit, 13B...hash calculation unit, 13C...rule matching unit, 14...processing execution unit, 14A...counting unit, 14B...packet capture unit, 14C...anomaly detection unit, 15...output unit, 19...memory unit, 19A...rule table, 101...processor, 102...main memory, 103...memory device, NW...communication network.

Claims (7)

A receiving unit that executes a receiving process for receiving and outputting data;
a first processing unit that executes a first processing on the data output from the receiving unit and outputs a first processing result of the first processing;
a second processing unit that performs a second processing on the data after the first processing and outputs a second processing result of the second processing;
a processing execution unit that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result,
a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
the processing execution unit starts processing using at least one of the data and the first processing result for the data before receiving the second processing result.
Data processing device. The data is a packet transmitted through a communication network,
the first processing unit is a time stamp unit that executes, as the first processing, a time stamp process of generating a time stamp for the packet output from the receiving unit and outputting the generated time stamp as a result of the first processing,
the second processing unit is a packet identification unit that executes, as the second processing, a packet identification process of identifying a flow to which the packet for which the timestamp is generated belongs, and outputting flow information specifying the identified flow as a result of the second processing.
2. A data processing apparatus according to claim 1. the processing execution unit executes the processing for a plurality of data input at each fixed processing time of the first processing and the second processing after receiving the data from the receiving unit, as the first processing result and the second processing result for the data, respectively.
3. A data processing device according to claim 1 or 2. the receiving unit is configured to be able to receive second data as the other data before the second process is completed after outputting first data as the data,
A data processing device according to any one of claims 1 to 3. The data is a packet transmitted through a communication network,
the second processing unit includes a header analysis unit that analyzes a header of the packet to extract a header field value, a hash calculation unit that hashes the extracted header field value, and a rule matching unit that compares the hashed header field value with a predetermined rule to obtain flow information that identifies a flow to which the packet belongs.
A data processing device according to any one of claims 1 to 4. a receiving step of executing a receiving process for receiving and outputting data;
a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing;
a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing;
a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result,
a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
In the processing execution step, processing using at least one of the data and the first processing result on the data is started before the second processing result is received.
Data processing methods. On the computer,
a receiving step of executing a receiving process for receiving and outputting data;
a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing;
a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing;
a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result,
a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
In the processing execution step, processing using at least one of the data and the first processing result on the data is started before the second processing result is received.
program.

Images