CN115604203B - Incomplete matching data stream processing method and system - Google Patents

Abstract

The application provides a data flow processing method and a system for incomplete comparison, which are implemented in a network device, wherein a memory is provided with a data flow table and a data flow filter, and a data flow analysis module is implemented to analyze and classify the packets of an input data flow and identify the application program category to which the input data flow belongs. In the method, a data flow table is queried according to the result of analyzing an input data flow so as to judge whether the input data flow accords with any data flow entry in the data flow table, and when the input data flow does not accord with the data flow entry in the data flow table, a data flow filter is queried again so as to judge whether the characteristic value of the input data flow corresponds to the condition therein, and accordingly, the input data flow is correspondingly processed, and not all the data flows which do not accord with the data flow entry are copied into the data flow table.

Description

Incomplete comparison data stream processing method and system

Technical Field

The present invention relates to a data stream processing technique, and more particularly, to a data stream processing method and system for reducing processing load by adopting an incomplete alignment mechanism.

Background

Identifying network traffic types, and thus improving quality of service (QoS, quality of Service), or improving network security, is a differentiated appeal of network devices of considerable importance in recent years. For example, if the network switch can recognize two different traffic types, namely video conference (video conference) and data transmission (FILE TRANSFER), it is suitable for improving quality of service (QoS), that is, it can preferentially serve the traffic of the video conference, so that the user experience can be improved. As another example, if the network switch can identify malicious application traffic behavior, such as traffic behavior of Trojan programs, the network switch can also block the occurrence of security holes at the first time.

However, identifying the type of network traffic has been a critical issue, and most often, the network administrator is given priority to the port numbers of various network protocols, such as whether TCP (Transmission Control Protocol) or UDP (User Datagram Protocol) port numbers are high priority or low priority. However, in addition to the inconvenience to the user and threshold, there are more and more applications using dynamic TCP or UDP port numbers, and more applications are hidden under known TCP or UDP port numbers (e.g., port 80), and more applications are encrypted and transmitted, which makes traffic identification difficult.

In order to solve the above-mentioned problems, in the prior art, there is a traffic feature-based identification method, in which the type of traffic is identified according to the header (header) and the statistics of the first (N) packets of each data flow (flow). The statistical characteristics include unidirectional or bidirectional packet length, packet interval, packet length average value, packet length variation, packet interval average value, packet interval variation, etc., so that the prior art adopts machine learning (MACHINE LEARNING) or deep learning (DEEP LEARNING) technology to classify traffic types according to the characteristics of the first few packets of each data stream.

For the purpose of inspecting the first few packets of each flow, a prior art schematic diagram of processing data flows within a network switch is shown with reference to fig. 1. The network switch receives an incoming data stream 10, which is parsed by the processor within the network switch to form a forwarding table 12, which shows the forwarding data as the port number (port=y) of the destination Media Access Control (MAC). A flow table (flow table) is provided in the network switch for recording each data flow flowing through the network switch, and as shown in the figure, a header of a data flow can be generally expressed by a 5-tuple (5-tuple) 14, and the 5-tuple (DIP, SIP, SP, DP, prot (Protocol) is shown in this example) including a Destination network address (Destination IP, DIP), a Source network address (Source IP, SIP), a Destination Layer 4port (Destination Layer port, dp), a Source Layer 4port (sp), and a Protocol (Protocol), which shows that two data flow headers with different states are recorded in the 5-tuple 14.

When the network switch receives a packet, it queries the data flow table, e.g., 5-tuple 14, and if it finds that the data flow entry (flow entry) corresponding to the data flow does not exist, it indicates that the data flow is a new data flow, i.e., it copies the packet to the data flow analysis module 18 (data flow direction 101), and the data flow analysis module 18 is a software module that performs analysis and classification on the packet to identify the application class to which the data flow belongs. After the data flow analysis module 18 receives the first packets of the data flow, it starts to execute the flow identification algorithm, and finally inserts the identified data flow into the data flow table (data flow direction 103) of the network switch, and marks the classification result on the data flow entry, and the data flow continues to reenter the packets of the switch, so that the data flow table (i.e. the 5-tuple 14 in this example) can be queried, and the data flow is not required to be processed by the data flow analysis module 18. Finally, the output data stream 16 is formed according to the destination output recorded in the packet in the data stream.

However, according to the above prior art, a specific processing is required for each data stream, all data streams need to be recorded in the data stream table, and new data streams need to be copied to the data stream analysis module 18 and then inserted into the data stream table, wherein a disadvantage is that the processing circuit (such as an ASIC) in the network switch needs to have enough memory to store a large number of data stream records. Generally, the required capacity is on the order of 100K, which is quite large compared to the number of application data streams that the user is really careful about. Further, because packets of a new data stream are copied to the data stream analysis module 18, additional processing requirements are created, and although the data stream analysis module 18 only needs the first few packets of the data stream, it may not be possible to send the sorted data stream back to the data stream table in time due to the time difference in processing, resulting in packets exceeding the amount of packets originally needed being copied to the data stream analysis module 18, creating additional processing burden.

Disclosure of Invention

In view of the problems of the prior art that the processing circuit of the network switch needs a large storage space to store the data stream record and causes extra processing load when processing each packet, the disclosure proposes a data stream processing method and system with incomplete comparison, which reduces the storage capacity requirement of the data stream table and the load of analyzing the data stream by adopting a method of incomplete comparison table look-up (incomplete comparison table) through a packet screening mechanism.

According to an embodiment, the data stream processing system for executing the incompletely aligned data stream processing method is arranged in a network device, and comprises a memory, a data stream table, a data stream filter and a data stream analysis module which is implemented by software or a circuit, wherein the data stream analysis module is used for analyzing and classifying packets of an input data stream and identifying the application program category to which the input data stream belongs.

The data stream processing method of incomplete comparison executed by the system comprises the steps of receiving an input data stream, analyzing the input data stream, inquiring a data stream table according to the result of analyzing the input data stream to judge whether the input data stream accords with any data stream entry in the data stream table, and inquiring a data stream filter to judge whether the input data stream accords with any filtering condition in the data stream filter when the input data stream does not accord with any data stream entry in the data stream table.

According to the result of inquiring the data flow table and the data flow filter, one of the following steps is executed, namely when the input data flow accords with any data flow item in the data flow table, a corresponding processing strategy is applied, when the input data flow does not accord with any data flow item in the data flow table, the data flow filter is inquired again to judge whether the input data flow accords with any filtering condition, when the input data flow accords with any filtering condition in the data flow filter, the input data flow is a data flow which exists and has transmitted more than a plurality of packets, namely, an action is executed according to a flow set by a network device, and when the input data flow does not accord with any filtering condition in the data flow filter, the input data flow does not accord with the conditions of the items in the data flow table and the data flow filter, and the system guides the input data flow to the data flow analysis module to process the input data flow.

Preferably, the data stream processing system implements data processing circuitry within a network switch for processing data streams to and from the network switch.

Further, the data flow table is applicable to all forms of data flows, and when the input data flow accords with any data flow entry in the data flow table, one of the following processing strategies is executed, namely, setting the input data flow to be in high priority order, in another embodiment, setting the priority to be in low priority order, and forwarding the input data flow to a destination transmission port, discarding the input data flow, copying the input data flow to a data flow analysis module, and forwarding the input data flow to the destination transmission port.

Further, the data flow table includes 5-tuple data in the header of the data flow, which may include a destination network address, a source network address, a destination layer 4 port, a source layer 4 port, and a communication protocol, and includes processing policies according to each data flow entry.

Preferably, the data stream filter implements a look-up table of incomplete alignments with a bloom filter and is used to query wire-guided data streams. The bloom filter obtains a hash value after carrying out hash operation on the input data flow k times, and judges whether the hash value corresponds to k 1-bit entries in the bloom filter.

Further, when the input data stream is the first packet of the connection oriented data stream, after the data stream table is queried, it is determined that any data stream entry in the data stream table is not met, that is, the first packet is oriented to the data stream analysis module to analyze, classify and identify the application class to which the first packet belongs, and then the first packet is forwarded according to the destination information recorded in the header of the first packet according to the flow set by the network device. When the input data stream is judged to generate distortion in the data stream filter, the data stream analysis module places the input data stream into the data stream table in advance, and sets a corresponding processing strategy according to the application program category to which the input data stream belongs.

And when the input data stream is judged to be in accordance with the filtering condition of the data stream filter, filling the input data stream in accordance with the filtering condition into the data stream filter, and removing the input data stream which is prevented from being distorted in the data stream filter and is put into the data stream table in advance.

For a further understanding of the nature and the technical aspects of the present invention, reference should be made to the following detailed description of the invention and the accompanying drawings, which are provided for purposes of reference only and are not intended to limit the invention.

Drawings

FIG. 1 shows a prior art schematic diagram of processing data flows within a network switch;

FIG. 2 is a schematic diagram of an exemplary bloom filter;

FIG. 3 is a schematic diagram of an exemplary bloom filter employing parallel k hash tables;

FIG. 4 is a diagram of an embodiment of a architecture of a data stream processing system employing a data stream filter;

FIG. 5 shows a flow chart of an embodiment of a data stream processing method employing incomplete alignment of data stream filters;

FIG. 6 is a flow chart of an embodiment of processing a first packet of a data stream;

FIG. 7 shows a flow of an embodiment of a method of processing a distorted data stream in a data stream;

FIG. 8 is a flow chart showing an embodiment of the operation of the data stream analysis module in the data stream processing method and

FIG. 9 is a flow chart diagram showing an embodiment of the operation of the data stream analysis module in the data stream processing method.

Symbol description

10 Input data stream

12 Transfer table

14:5 Tuple

16 Output data stream

18 Data stream analysis module

101,103 Data flow direction

20 Bit array

30:5 Tuple

40 Input data stream

42 Transfer table

44 Data flow table

45 Data stream Filter

46 Output data stream

48 Data flow analysis Module

401-413 Data flow direction

400 Memory

Data flow processing flow for incomplete comparison of S501-S517

S601-S607 process flow for processing first packet of data flow

S701-S705 distorted data stream processing flow

S801-S809 data flow processing flow

S901-S907 data flow processing flow

Detailed Description

The following embodiments of the present invention are described in terms of specific examples, and those skilled in the art will appreciate the advantages and effects of the present invention from the disclosure herein. The invention is capable of other and different embodiments and its several details are capable of modifications and various other uses and applications, all of which are obvious from the description, without departing from the spirit of the invention. The drawings of the present invention are merely schematic illustrations, and are not intended to be drawn to actual dimensions. The following embodiments will further illustrate the related art content of the present invention in detail, but the disclosure is not intended to limit the scope of the present invention.

It will be understood that, although the terms "first," "second," "third," etc. may be used herein to describe various elements or signals, these elements or signals should not be limited by these terms. These terms are used primarily to distinguish one element from another element or signal from another signal. In addition, the term "or" as used herein shall include any one or combination of more of the associated listed items as the case may be.

The disclosure relates to a method and a system for processing an incompletely aligned data stream, wherein the proposed method for processing a data stream is a lookup table (incomplete comparison table) method based on incomplete alignment, and the embodiments show that the method adopts a data stream filter (flow filter) such as a bloom filter (halation filter), etc., where the method has the advantage that the problem that a large capacity memory and a processing load are required for processing each data stream can be avoided. According to one embodiment, the bloom filter is a probabilistic data structure (probabilistic data structure) that can quickly verify whether each data stream exists in the data stream table and uses relatively little memory space.

In the method for processing incompletely aligned data streams, the concept of one embodiment is to use incompletely aligned table lookup and table lookup (such as data stream table) requiring complete alignment in the prior art, and query the data stream header of the 5-tuple 14 as shown in fig. 1, thereby reducing the size of the completely aligned table lookup and further reducing the overall system cost.

Further, the incompletely aligned lookup table refers to that when a record of a data stream is stored in the incompletely aligned lookup table, only a characteristic value of the data stream is stored, where the characteristic value may be compressed information, digest (digest), or hash value (hash value). When the characteristic values of the data streams are recorded in this incompletely aligned table, the capacity of such a table may be significantly smaller than the recorded content of the completely aligned table, since the complete data stream is not stored in the table. However, using an incompletely aligned lookup table still requires that distortion (aliasing) conditions occur, e.g., there is no data stream within the lookup table, but the false determination is made at the time of the lookup.

Thus, the disclosure proposes a data stream processing method employing an incomplete alignment of a data stream filter, through which a look-up table for the incomplete alignment is implemented. The data stream filter may be the bloom filter described above, which has a very compact (compact) k-time hash table structure, wherein the principle is that when an element is added to a set (i.e. a look-up table is implemented), the element is mapped to k points in a bit array by k hash functions, and these (k) points are set to 1. When filtering (checking) the data stream, it can be determined whether the data stream is contained in the set, as long as it is checked whether the positions of the points mapped are all 1. If any of the mapped points is 0, the filtered data stream is not in the set, and if the mapped points are all 1, the filtered data stream is covered in the set.

According to the data stream processing method using the bloom filter, when a data stream is inserted into the bloom filter, the position (data stream entry) with 1bit width) to which k hash corresponding to the data stream is addressed is set to 1, when a bloom filter is searched by a certain data stream to determine whether the data stream is inserted into the bloom filter, if k 1bit entries (1-bit entry) corresponding to the data stream after being hashed k times are all 1, namely, the data stream conforming to the bloom filter is filtered, the data stream is judged to be inserted into the bloom filter before being filtered out, otherwise, the data stream is not filtered out by the bloom filter.

For example, taking the exemplary bloom filter embodiment shown in fig. 2 as an example, a bit array (bit array) 20 is shown, taking the example that the data sets { x, y, z } form a filter, under the example of k=3, 3 hash operations are performed on each data element (element) in the data sets { x, y, z } as the characteristic value of each data element, in this example, the location of addressing each data element is set to 1, x is shown to have three lines representing 3 1's (3 bits are set to 1) addressed to the bit array 20 respectively, y is shown to have three lines representing 3 1's addressed to the bit array 20 respectively, and z has three lines representing 3 1's addressed to the bit array 20 respectively, so as to form a bloom filter.

Taking the input w data as an example, the eigenvalues of the w data are calculated, and the bit array 20 is compared (mapping) to find that the eigenvalues of the w data have positions (not all 1 s) corresponding to 0s, which means that the eigenvalues are not in the data set { x, y, z }. This is an example of a data stream processing method that performs incomplete alignment using bloom filters.

Fig. 3 again illustrates a bloom filter employing k hash tables in parallel. In this example, the bloom filter performs hash operation for the 5-tuple 30 of the input data stream k times to obtain k (k=4) hash tables, and compares hash 0, hash 1, hash 2 and hash 3 respectively, so as to filter k eigenvalues in the data stream, where a data stream entry (flow entry) width of a location where the k-time hash is addressed is shown as 1 bit and is set to 1. When the method is applied, the characteristic value of the 5-tuple algorithm of one input data stream, namely k times of hash algorithm, is compared with the bloom filter adopting parallel k hash tables aiming at different hash values, and whether the input data stream exists in the incompletely compared lookup table can be judged.

FIG. 4 is a diagram showing an embodiment of a data stream processing system using a data stream filter according to the disclosure, where the data stream processing system implements data processing circuitry in a network device, such as a network switch, to perform an incompletely aligned data stream processing method for processing data streams transmitted from and to the network switch.

In the embodiment of the data stream processing system architecture shown in the figure, when the system receives the incoming data stream 40, the data stream is parsed and forwarded to a forwarding table 42, where the forwarding table 42 is used to record the media access Control address (MEDIA ACCESS Control, MAC ADDRESS) of the second layer (L2) or the network address (IP address) of the third layer (L3) in the network protocol of the data stream. The forwarding table 42 records the destination media access control address (DMAC) and destination Port number (port=y) in the input data stream 40.

After parsing the input data stream 40, the data of the data stream is submitted to both a data stream table (flow table) 44 and a data stream filter 45 implemented in a memory 400 in the system (data processing circuit). According to an embodiment, the data flow table 44 may record 5-tuple data obtained from the headers of multiple data flows, such as destination network address (DIP), source network address (SIP), destination layer 4 port (DP), source layer 4 port (SP), and communication protocol (Prot (Protocol)), and record processing policies conforming to each data flow entry (entry), such as setting priority of the data flow, discarding the data flow (drop), copying the data flow (copy) to the data flow analysis module, and so on. The data flow filter 45 is a bloom filter as described above to achieve a look-up table of incomplete alignments.

The data stream processing system may implement a data stream analysis module 48 in software or circuitry in addition to the data stream table 44 and the data stream filter 45 in the memory 400. When the input data stream 40 does not match any of the data stream entries in the data stream table 44, a new data stream is determined and copied to the data stream analysis module 48. In the data flow analysis module 48, the program can analyze and classify the packet, identify the application class to which the data flow belongs, and forward the packet according to the destination information recorded in the packet header according to the flow set by the switch or the network device of the application, for example, forward to the destination transport port number Y (port=y).

The data stream processing system may implement processing circuitry within a network switch in which an embodiment of a data stream processing method employing incomplete alignment of bloom filters may be run as described with simultaneous reference to fig. 5.

The data stream processing system receives the input data stream 40, for example, a packet formed by transmitting a port number X (port=x) from a source (fig. 4, data stream direction 401, step S501), analyzes the input data stream 40 to obtain the tag information therein (step S503), records a destination transmission port number Y (port=y) in a forwarding table 42 in a legend, then performs a procedure of querying the data stream table 44 and the data stream filter 45, and performs one of the following steps according to the results of querying the data stream table 44 and the data stream filter 45. The data flow table 44 is applicable to all types of data flows, and the data flow filter 45 is a connection-oriented data flow (connection-oriented flow) for querying the connection.

Next, the data flow table 44 is queried based on the result of analyzing the input data flow (data flow direction 403, step S505), and at the time of the query process, it is determined whether the characteristics of the input data flow 40 match any data flow entry (flow entry) in the data flow table (step S507), and if the input data flow 40 matches any data flow entry in the data flow table 44 (yes), the corresponding processing policy is described therein (step S509). For example, when the input data stream 40 conforms to any data stream entry in the data stream table 44, one of the following processing strategies may be executed, i.e. setting the input data stream to be in high priority (in another embodiment, setting the priority to be low priority is allowed) and simultaneously performing related actions according to the original flow in the network device (such as a network switch) to which the data stream processing system is applied, such as forwarding the packet, so that the flow chart may execute the forwarding data stream described in step S515 to the destination transmission port, such as the above example, transmitting the data stream to the destination transmission port number Y (port=y) (fig. 4, data stream direction 413). The incoming data stream 40 may be discarded or copied to the data stream analysis module 48 according to the processing policy (step S517), and the packet may be forwarded in addition to the analysis, classification and identification (step S515).

However, if the input data stream 40 does not meet any of the data stream entries in the data stream table 44 (no), the data stream filter 45 is queried according to the result of parsing the input data stream (data stream direction 405, step S511), and it is determined whether the input data stream 40 meets any of the filtering conditions. Taking the bloom filter as an example, the hash value is obtained by hashing the elements in the input data stream k times, and the hash value is used as the characteristic value of the input data stream, so as to determine whether k 1-bit entries in the bloom filter are corresponding (step S513). If the query results in compliance with one of the filtering conditions (yes), it means that the input data stream 40 is a data stream that already exists and has been transmitted over a plurality of packets, but no additional action is required, as long as an action is performed directly according to the originally set flow of the network device (e.g., network switch) to which the data stream processing system is applied, for example, forwarding the input data stream to the destination transmission port number Y (port=y) according to the setting of the network device (step S515), to form the output data stream 46 in the illustration. However, if the query result indicates that the input data stream 40 does not meet a filtering condition in the data stream filter 45 (no), it indicates that the input data stream 40 does not meet any information in the data stream table and the data stream filter, i.e. the input data stream is directed to the data stream analysis module to process the data stream (data stream direction 407, step S517), and the input data stream 40 may be forwarded to the destination transmission port according to an original flow in a network device (such as a network switch) applied by the data stream processing system to form the output data stream 46.

FIG. 6 is a flow chart of an embodiment of processing a first packet of a data stream, the flow running in a data stream filter. The flow of this embodiment is directed specifically to a connection-oriented flow (TCP protocol) where a communication session (communication session) is established between data transfers, and conversely, the proposed flow filter does not handle non-connection-oriented flows, such as UDP protocol flows.

In this embodiment, the first packet of the connection oriented data stream is received from the network device to which the method applies (step S601). Taking the TCP protocol as an example, the first packet in the input data stream may be determined based on the header record content of the first packet, in which the SYN flag (flag) is set to 1 and the ACK flag is set to 0, and if the first packet is a new data stream, it is not in accordance with any data stream entry in the data stream table, i.e., it is directly directed to the data stream analysis module (data stream direction 407, step S603).

At this time, the data flow analysis module analyzes, classifies and identifies the application class to which the input data flow belongs (step S605), and then the data flow analysis module records the analysis and forwards the packet to the destination transport port number Y according to the destination information recorded in the packet header according to the flow set by the network device (step S607, fig. 4, data flow direction 413). It is mentioned that the data flow analysis module is operative to obtain the packets which do not meet the data flow entries therein according to the result of the data flow table query comparison, and also receive the data flows which do not meet any filtering condition in the data flow filter, and analyze, classify and identify the application program category to which the data flows belong.

Continuing with the flow of fig. 6, fig. 7 shows a flow of an embodiment of a method of processing a distorted data stream in a data stream.

When the first packet of the connection-oriented data stream is directed to the data stream analysis module, the data stream analysis module can determine whether the new data stream will cause distortion (aliasing) if it is placed in the data stream filter, i.e., determine whether it will collide with any filtering condition (e.g., any data stream entry) already existing in the data stream filter. Taking the data stream under the TCP protocol as an example, if the first packet is received, it is found that the values of the k data stream entries associated with the data stream are all not equal to 0, which indicates that there is a distortion.

Thus, when it is determined that the input data stream is a new data stream and distortion is actually generated in the data stream filter (step S701), the data stream analysis module is previously placed in the data stream table (fig. 4, data stream direction 409, step S703), and sets a corresponding processing policy (step S705). For example, in the data flow table, the processing policy of the new data flow is set to be copied to the data flow analysis module, so that the other (2 nd to N th) packets except the first packet of the data flow can be copied to the data flow analysis module for analysis by the processing policy of the data flow table, and the data flow is not directly transferred to the output port of the network device due to the fact that the data flow meets the data flow filter by mistake.

Fig. 8 is a flowchart illustrating an embodiment of the operation of the data flow analysis module in the data flow processing method, in this example, after the first N packets of the received input data flow are analyzed by the data flow analysis module, the data flow analysis module determines the application class of the input data flow (step S801), and provides a corresponding processing policy (step S803), for example, the first N packets may be set to a high priority order, or the packets may be discarded. Then, the process determines whether the input data stream is already stored in the data stream table (step S805), if the data stream is not stored in the data stream table (no), the data stream that does not conform to the data stream entry defined in the data stream filter may be inserted into the data stream table (step S807), and if the distortion factor of the data stream due to the collision of the data stream filter (e.g. step S701) has been pre-placed in the data stream table, the data stream analysis module may directly copy the processing policy of the data stream entry (flow entry) described in the data stream table from the original copy to the data stream analysis module to a desired final policy (step S809).

Another embodiment of the operation of the data stream analysis module in the data stream processing method may refer to the flowchart shown in fig. 9.

When an input data stream is received, the data stream is copied to the data stream analysis module, the data stream analysis module analyzes the first N packets, determines the application type of the input data stream (step S901), obtains a corresponding processing policy from the application type (step S903), and after comparing the filtering conditions of the data stream filter, can directly fill the data stream meeting the filtering conditions into the data stream filter (step S905), thereby forming a data stream entry therein.

At this time, referring to step S701 of fig. 7, in order to avoid distortion of the input data stream due to collision in the data stream filter, the input data stream is put into the data stream table in advance, and the processing strategy is to copy the data stream to the data stream table. After step S905, if the N packets are analyzed to meet the filtering conditions in the data stream filter, the data stream analysis module removes the data stream from the data stream table and fills the data stream into the data stream filter (fig. 4, data stream direction 411, step S907).

In summary, according to the above-described method and system for processing incompletely aligned data streams according to the above embodiments, the incompletely aligned data stream filters (which may be implemented by bloom filters) and the completely aligned data stream tables are adopted at the same time, wherein the data streams meeting the filtering conditions in the data stream filters are put into the data stream filters, and if the data streams not meeting the filtering conditions are received, the data streams are copied to the data stream analysis module, so that the requirement of the existing data stream tables is reduced, so that the data stream tables only record the data streams (such as the data streams with high priority order or malicious data streams) which need special processing and the non-connection-oriented data streams (connection-less flows) instead of storing all the data streams, which are needed by the existing table lookup method only adopting complete alignment can be reduced, and therefore the overall system cost and the additional burden can be effectively reduced.

The above disclosure is only a preferred embodiment of the present invention and is not intended to limit the claims of the present invention, so that all equivalent technical changes made by the application of the specification and the drawings of the present invention are included in the claims of the present invention.

Claims (9)

1. A data stream processing method of incomplete comparison is applied to a network device, and comprises the following steps:

receiving an input data stream and analyzing the input data stream;

Inquiring a data flow table according to the result of analyzing the input data flow so as to judge whether the input data flow accords with any data flow item in the data flow table;

Inquiring a data stream filter according to the result of analyzing the input data stream to judge whether the input data stream meets any filtering condition in the data stream;

Wherein, according to the results of querying the data flow table and the data flow filter, one of the following steps is executed:

When the input data stream accords with any data stream entry in the data stream table, a corresponding processing strategy is applied;

When the input data stream does not accord with any data stream item in the data stream table, inquiring the data stream filter, and judging whether the input data stream accords with any filtering condition;

When the input data stream meets any filtering condition in the data stream, the input data stream is a data stream which exists and has been transmitted for more than a plurality of packets, namely, an action is executed according to the flow set by the network device, and

When the input data stream does not meet any filtering condition in the data stream filter, the input data stream is guided to a data stream analysis module to process the input data stream, wherein the data stream realizes a lookup table of incomplete comparison by a bloom filter and is used for querying the data stream guided by a connecting line.

2. The incomplete alignment data stream processing method as recited in claim 1, wherein the data stream table is adapted to all types of data streams, and when the input data stream matches any one of the data stream entries in the data stream table, one of the following processing strategies is executed:

Setting the input data stream as a high priority order, and forwarding the input data stream to a destination transmission port;

discarding the incoming data stream, and

The input data stream is copied to the data stream analysis module and then forwarded to the destination transport port.

3. The method of claim 2, wherein the data flow table includes 5-tuple data in a header of the data flow, including a destination network address, a source network address, a destination layer 4 port, a source layer 4 port, and a communication protocol, and includes processing policies according to each data flow entry.

4. The method of claim 1, wherein the bloom filter hashes the input data stream k times to obtain a hash value, and determines whether the k 1-bit entries in the bloom filter are corresponding to each other.

5. The incomplete comparison data stream processing method of claim 4, wherein the bloom filter performs hash operations k times on the 5-tuple of the input data stream to obtain k hash values.

6. The method of claim 1-5, wherein the data flow analysis module is configured to analyze and classify packets of the input data flow and identify an application class to which the input data flow belongs.

7. The method of claim 6, wherein when the received incoming data stream is a first packet of a wire-oriented data stream, the first packet is directed to the data stream analysis module for analysis, classification and identification of the application class to which the first packet belongs after the data stream table is queried to determine that any data stream entry in the data stream table is not met, and the first packet is forwarded according to destination information recorded in a header of the first packet according to a flow set by the network device.

8. The incomplete comparison data stream processing method as claimed in claim 6, wherein the data stream analysis module places the input data stream into the data stream table in advance and sets a corresponding processing policy according to the application class to which the input data stream belongs when it is determined that the input data stream is distorted in the data stream filter.

9. A data stream processing system, provided in a network device, comprising:

a memory having a data flow table and a data flow filter therein, and

A data flow analysis module for analyzing and classifying the packets of an input data flow and identifying the application program category to which the input data flow belongs;

wherein the data stream processing system performs a data stream processing method of incomplete alignment, comprising:

receiving the input data stream and analyzing the input data stream;

Inquiring the data flow table according to the result of analyzing the input data flow so as to judge whether the input data flow accords with any data flow item in the data flow table;

Inquiring the data stream filter according to the result of analyzing the input data stream to judge whether the input data stream meets any filtering condition in the data stream;

Wherein, according to the results of querying the data flow table and the data flow filter, one of the following steps is executed:

When the input data stream accords with any data stream entry in the data stream table, a corresponding processing strategy is applied;

When the input data stream does not accord with any data stream item in the data stream table, inquiring the data stream filter, and judging whether the input data stream accords with any filtering condition;

When the input data stream meets any filtering condition in the data stream, the input data stream is a data stream which exists and has been transmitted for more than a plurality of packets, namely, an action is executed according to the flow set by the network device, and

When the input data stream does not meet any filtering condition in the data stream filter, the input data stream is guided to the data stream analysis module to process the input data stream, wherein the data stream realizes a lookup table of incomplete comparison by a bloom filter and is used for querying the data stream guided by a connecting line.