CN118869333A

CN118869333A - Traffic data processing method and network element equipment

Info

Publication number: CN118869333A
Application number: CN202411142843.5A
Authority: CN
Inventors: 熊逸风; 武利明; 葛增冉; 段威
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2024-08-20
Filing date: 2024-08-20
Publication date: 2024-10-29

Abstract

The application provides a processing method of flow data and network element equipment. The method is applied to network element equipment and comprises the following steps: collecting flow data locally at the network element equipment; acquiring a pre-trained AI model; according to the flow data and the AI model, carrying out local network reasoning; and carrying out network intelligent decision according to the reasoning result. The method has the advantages that the method can collect, analyze and manage the flow data locally in the network element equipment without transmitting the flow data to a central control system, so that the method has good instantaneity, can save network bandwidth, reduce communication cost and is not limited by application scenes; because the local network reasoning can be carried out on the flow data according to the AI model, and the network intelligent decision is carried out according to the reasoning result, the accuracy of the network reasoning can be improved, the local real-time intelligent application of the network flow can be realized, complex and changeable flow scenes can be dealt with, and the application and management requirements of the network flow in different scenes can be met.

Description

Traffic data processing method and network element equipment

Technical Field

The present application relates to the field of network monitoring and management technologies, and in particular, to a method for processing traffic data and a network element device.

Background

A network typically includes a plurality of network element devices (e.g., routers, switches, etc.) that can receive traffic from upstream devices and forward the traffic. With the proliferation of traffic data and the increasing complexity of network environments, in some scenarios, it is required to monitor network traffic and manage the network according to the monitoring result, such as defending against network attacks, quality of service (Quality of Service, qoS) control, and the like.

In the related art, when monitoring and managing network traffic, traffic data of each network element device in a network is generally collected and transmitted to a central control system, and the central control system analyzes the traffic data of a plurality of network element devices and manages the network according to the analysis result. However, although the solution can intensively process and analyze large-scale traffic data, the timeliness is insufficient, the communication cost is high, and the application scenario is limited, for example, in an edge network or a network with wide geographical distribution, some network element devices may not be connected to a central control system, so that traffic monitoring and management cannot be performed on the network element devices.

Disclosure of Invention

The application provides a processing method of flow data and network element equipment, which are used for solving the problems of insufficient timeliness, high communication cost and limited application scene when monitoring and managing network flow in the related technology.

In order to solve the technical problems, the application is realized as follows:

In a first aspect, a method for processing traffic data is provided, which is applied to a network element device, and includes:

Collecting flow data locally at the network element device;

acquiring a pre-trained artificial intelligence AI model;

according to the flow data and the AI model, carrying out local network reasoning;

And carrying out network intelligent decision according to the reasoning result.

In a second aspect, a network element device is provided, including a traffic collection module and a network intelligent application module, where:

the flow acquisition module is used for locally acquiring flow data at the network element equipment;

The network intelligent application module is used for acquiring a pre-trained AI model; according to the flow data and the AI model, carrying out local network reasoning; and carrying out network intelligent decision according to the reasoning result.

In a third aspect, there is provided an electronic device comprising:

A processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method according to the first aspect.

In a fourth aspect, there is provided a computer readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the method of the first aspect.

In a fifth aspect, there is provided a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of the method according to the first aspect.

In the embodiment of the application, the flow data can be acquired locally at the network element equipment and analyzed and managed locally at the network element equipment, so that the real-time performance is better, and the real-time performance requirement for analyzing and managing the network flow can be met; the collection, analysis and management of the flow data can be realized at the local of the network element equipment without transmitting the flow data to a central control system, so that the network bandwidth can be saved, and the communication cost can be reduced; after the network element equipment locally collects the flow data, local network reasoning can be carried out on the flow data according to the AI model, and network intelligent decision can be carried out according to the reasoning result, so that the accuracy of network reasoning can be improved, the real-time intelligent application of the network flow in the local can be realized, complex and changeable flow scenes can be dealt with, and the application and management requirements of the network flow in different scenes can be met; because the collection, network reasoning and intelligent decision of the flow data can be carried out locally on the network element equipment, the network element equipment in the network can monitor and manage the network flow locally for the edge network or the network with wide geographical distribution, thereby being not limited by application scenes and having better universality.

Drawings

In order to more clearly illustrate the application or the technical solutions of the prior art, the drawings that are used in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.

FIG. 1 is a flow chart of a method of processing traffic data according to an embodiment of the present application;

FIG. 2 is a schematic diagram of AI model training and local network reasoning based on the AI model in accordance with one embodiment of the application;

FIG. 3 is a schematic diagram of local network reasoning according to the AI model in the DDOS attack detection scenario according to an embodiment of the present application;

FIG. 4 is a schematic diagram of local network reasoning according to the AI model in a service type classification scenario in accordance with an embodiment of the application;

FIG. 5 is a schematic diagram of the architecture of an electronic device according to one embodiment of the application;

fig. 6 is a schematic structural diagram of a network element device according to an embodiment of the present application;

Fig. 7 is a schematic structural diagram of a network element device according to yet another embodiment of the present application;

fig. 8 is a schematic structural diagram of a network element device according to still another embodiment of the present application;

Fig. 9 is a schematic diagram of flow data processing performed by the network element device according to an embodiment of the present application.

Detailed Description

In the related art, when monitoring and managing network traffic, traffic data of each network element device in a network is generally collected and transmitted to a central control system (hereinafter referred to as a control system), and the control system analyzes the traffic data of a plurality of network element devices and manages the network according to the analysis result. However, in practical applications, this approach of centralized monitoring management by the control system has at least the following drawbacks:

(1) The timeliness is not enough: because the flow data needs to be collected from the network element equipment and transmitted to the control system, the flow information cannot be updated in time at the control system side, so that the application requiring instant decision cannot be supported. For example, the intelligent QoS decision needs to dynamically adjust network resources based on real-time traffic data, so as to ensure quality of service, and for example, in a real-time defending network attack scenario, security defense needs to be performed according to the real-time traffic data, and if the real-time performance of the traffic data is insufficient, a delay response may cause serious network service interruption.

(2) The communication cost is higher: when traffic data of a plurality of network element devices is transmitted to a control system, a large amount of network bandwidth is required for data transmission, resulting in high communication cost. In addition, the control system also consumes a large amount of computing resources when processing traffic data for a plurality of network element devices, and also requires high-performance hardware to process the traffic data, which increases the overall cost and complexity of the system.

(3) The application scenario is limited: some network element devices in a network may not have the ability to connect to a control system, particularly in an edge network or a geographically widely distributed network. This limits the coverage and effectiveness of the centralized monitoring management scheme, resulting in monitoring and management blind spots.

The embodiment of the application provides a processing method of flow data and network element equipment, wherein when monitoring and managing network flow, the network element equipment can locally acquire the flow data, then acquire a pre-trained AI model, locally perform network reasoning according to the AI model and the locally acquired flow data, and perform local network intelligent decision according to a reasoning result. Therefore, the method can collect the flow data locally at the network element equipment and analyze and manage the flow data locally at the network element equipment, so that the method has good instantaneity and can meet the requirement of the network flow analysis and management on instantaneity; the collection, analysis and management of the flow data can be realized at the local of the network element equipment without transmitting the flow data to a central control system, so that the network bandwidth can be saved, and the communication cost can be reduced; after the network element equipment locally collects the flow data, local network reasoning can be carried out on the flow data according to the AI model, and network intelligent decision can be carried out according to the reasoning result, so that the accuracy of network reasoning can be improved, the real-time intelligent application of the network flow in the local can be realized, complex and changeable flow scenes can be dealt with, and the application and management requirements of the network flow in different scenes can be met; because the collection, network reasoning and intelligent decision of the flow data can be carried out locally on the network element equipment, the network element equipment in the network can monitor and manage the network flow locally for the edge network or the network with wide geographical distribution, thereby being not limited by application scenes and having better universality.

In order that those skilled in the art will better understand the present application, a more particular and complete description of the same will be rendered by reference to the appended drawings, wherein it is to be understood that the present application is illustrated in one or more embodiments thereof. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, shall fall within the scope of the application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate, such that the application may be practiced otherwise than as specifically illustrated or described herein. Furthermore, in the present application and in the claims, "and/or" means at least one of the connected objects, and the character "/" generally means that the associated object is an or relationship.

It should be noted that, the network element device in the embodiment of the present application may be a device capable of forwarding traffic, including, but not limited to, a router, a switch, and the like. The network to which the technical scheme provided by the embodiment of the application can be applied includes, but is not limited to, an operator backbone network, a data center, a cloud service provider and a large enterprise network, and the environments need an efficient data processing scheme to ensure the security and stability of the network and optimize the use of resources.

The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

FIG. 1 is a flow chart of a method for processing traffic data according to an embodiment of the present application. The processing method of the traffic data shown in fig. 1 may be applied to a network element device, that is, the processing method of the traffic data shown in fig. 1 may be performed by the network element device, in other words, the processing method of the traffic data shown in fig. 1 may be performed by software or hardware installed in the network element device, and the processing method of the traffic data includes the following steps.

S102: traffic data is collected locally at the network element device.

When monitoring and managing network traffic, the network element device can locally collect traffic data.

In some embodiments, when the network element device collects traffic data locally, the network element device may collect raw traffic data specified via an access control list (Access Control List, ACL) on the line card (forwarding plane). When the original flow data is collected, the original flow data can be collected in real time, or can be collected according to a sampling period (which is specific according to actual requirements and is not specifically limited herein), for example, the sampling period is 5s, then the flow data in the 5s can be aggregated (or counted) every 5s, and the aggregation result is taken as the collected flow data. The specific collection mode of the flow data can be a hardware flow table, a counter combined with a sketch algorithm, a NetFlow flow table and other modes.

S104: a pre-trained artificial intelligence AI model is obtained.

The artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) model can be trained in advance and used for carrying out network reasoning according to the flow data of the network element equipment, and the reasoning result can be used for network intelligent decision. The pre-trained AI model may be obtained while monitoring and managing network traffic.

In this embodiment, the AI model may include various types, such as a machine learning model (e.g., SVM, XGBoost), a deep learning model (e.g., CNN, LSTM), etc., which are not particularly limited herein. The AI model may be obtained by training the sample flow data and a flow type label corresponding to the sample flow data (supervised model training), or may be obtained by training the sample flow data (unsupervised model training), which is not particularly limited herein. The sample traffic data used in the model training may be historical traffic data collected locally at the network element device, or may also be analog traffic data obtained in an analog manner, where the source of the sample traffic data is not specifically limited.

In some embodiments, the AI model may be trained by a remote server (e.g., a central control system), from which the network element device may obtain the AI model when obtaining the AI model. In other embodiments, if the network element device has a relatively strong processing capability and is capable of supporting the network element device to perform training of the AI model locally, the AI model may also be obtained by the network element device by performing local training, and the network element device may obtain the AI model locally when obtaining the AI model. The apparatus for training the AI model is not particularly limited here.

S106: and carrying out network reasoning locally according to the traffic data and the AI model.

After the network element equipment locally collects the flow data and acquires the pre-trained AI model, network reasoning can be locally performed according to the flow data and the AI model.

When network reasoning is locally performed according to the traffic data and the AI model, in some embodiments, the traffic data may be directly used as input of the AI model, and output of the AI model is a reasoning result. In other embodiments, the flow data may be preprocessed, and then the preprocessed flow data is used as an input of an AI model, and the output of the AI model is the reasoning result. In the preprocessing of the traffic data, for example, determining the traffic characteristics according to the traffic data, screening the traffic characteristics to obtain key characteristics for performing network reasoning, and the like, the preprocessing operation of the traffic data is not specifically limited.

S108: and carrying out network intelligent decision according to the reasoning result.

After the network element equipment obtains the reasoning result, network intelligent decision can be locally made according to the reasoning result, so that the network can be optimized. For example, attack defense, intelligent QoS control and the like can be performed according to the reasoning result, so that the defending capability of the network is enhanced, and the distribution and management of network resources are optimized.

The method can collect the flow data locally at the network element equipment and analyze and manage the flow data locally at the network element equipment, so that the method has good instantaneity and can meet the instantaneity requirement of analyzing and managing the network flow; the collection, analysis and management of the flow data can be realized at the local of the network element equipment without transmitting the flow data to a central control system, so that the network bandwidth can be saved, and the communication cost can be reduced; after the network element equipment locally collects the flow data, local network reasoning can be carried out on the flow data according to the AI model, and network intelligent decision can be carried out according to the reasoning result, so that the accuracy of network reasoning can be improved, the real-time intelligent application of the network flow in the local can be realized, complex and changeable flow scenes can be dealt with, and the application and management requirements of the network flow in different scenes can be met; because the collection, network reasoning and intelligent decision of the flow data can be carried out locally on the network element equipment, the network element equipment in the network can monitor and manage the network flow locally for the edge network or the network with wide geographical distribution, thereby being not limited by application scenes and having better universality.

In some embodiments, the flow data collected in S102 may include flow data of multiple dimensions. After the network element device locally collects the traffic data, the method may include the following steps:

and analyzing the flow data according to the multiple dimensions to obtain multi-dimensional flow characteristics of the flow data.

Thus, in S106, the network reasoning is performed locally according to the traffic data and the AI model, and may include the following steps:

And carrying out network reasoning locally according to the multidimensional flow characteristics of the flow data and the AI model.

When the flow data is collected, the flow data of multiple dimensions can be collected, and then the multidimensional flow characteristics are determined according to the flow data of the multiple dimensions, so that the richer flow characteristics can be obtained, the local flow condition of the network element equipment can be reflected more accurately, and when the network reasoning is carried out, the network reasoning is carried out according to the multidimensional flow characteristics and the AI model, so that a more accurate reasoning result can be obtained.

In some embodiments, the analyzing the flow data according to the multiple dimensions to obtain the multi-dimensional flow characteristic of the flow data may include the following steps:

Aggregating the flow data in the form of a detection window to obtain a flow aggregation result;

and analyzing the flow aggregation result according to the multiple dimensions to obtain the multidimensional flow characteristics of the flow data.

The detection window is a time window, and may be specifically determined according to practical requirements, which is not specifically limited herein. If the flow data is collected locally according to a sampling period, the detection window may be N times the sampling period, where N is an integer greater than 1.

When analyzing the locally collected flow data with multiple dimensions, the flow data can be gathered (or counted) in the form of a detection window to obtain a flow gathering result, and then the flow data is subjected to multi-dimensional analysis according to the multiple dimensions of the flow data to obtain the multi-dimensional flow characteristics of the flow data. The multidimensional flow characteristic may include flow characteristics of multiple dimensions, the flow characteristic of each dimension may be obtained by analyzing flow data according to one dimension of the flow data, and the flow characteristic of each dimension may include multiple characteristics.

The flow data can be subjected to multidimensional analysis after being aggregated, so that the data quantity to be analyzed can be reduced, the analysis efficiency is improved, and the analysis result, namely the multidimensional flow characteristics, can reflect the aggregation characteristics of the flow data under a time window.

In some embodiments, the multidimensional traffic feature may include at least two of a five-tuple traffic feature, a destination IP feature, a service feature, a source IP feature, and a global feature, and features including which dimensions specifically may depend on an actual traffic scenario, which is not specifically limited herein. Each dimensional feature will be explained below.

The data stream generally has quintuple information, and the quintuple flow characteristic is a characteristic obtained by analyzing flow data according to the dimension of the quintuple information of the flow. Five-tuple flow characteristics can include, but are not limited to, flow base information, flow cumulative characteristics, and flow baseline characteristics (baseline: a baseline value reflecting a dimension characteristic at normal operation). The flow basic information may include key information of the flow, such as source IP, destination IP, source port, destination port, protocol type, etc. The flow accumulation features can be, for example, the number of packets, the total number of bytes of the packet, the flow statistics time, etc., and these features can be obtained by continuous monitoring and data accumulation calculation of the data flow. The baseline characteristic may be, for example, a single-cycle flow packet number baseline (i.e., a reference value for the number of packets included in the data flow in one cycle), a single-cycle flow byte number baseline (i.e., a reference value for the number of bytes included in the data flow in one cycle), etc., and these characteristics may be updated in a sliding manner based on historical data for monitoring and comparing the current flow to a historical standard.

The destination IP characteristic is obtained by analyzing the flow data according to the dimension of the destination IP address of the data flow. For example, according to the destination IP, traffic data having the same destination IP may be statistically analyzed, and the destination IP feature may be obtained according to the result of the statistical analysis. The destination IP characteristics may include, but are not limited to, destination IP basic information, destination IP cumulative characteristics, and destination IP baseline characteristics. The destination IP basic information may be, for example, an aggregation destination IP (obtained by aggregating traffic based on a destination IP), an earliest time of occurrence of the destination IP in the network, a latest time of occurrence of the destination IP in the network, and the like. The destination IP accumulation feature may be, for example, the number of packets and the number of streams aggregated according to the destination IP, the total statistical time, the average length of the packets, the ratio of the number of abnormal streams to the number of abnormal streams, and the like. The destination IP baseline characteristic may be, for example, a flow rate baseline, a flow number baseline, a packet number baseline, an entropy value baseline, and the like of the flows aggregated according to the destination IP. Alternatively, destination IP baseline characteristics may be used to monitor and evaluate the behavior patterns of the destination IP.

The destination IP + destination port + protocol of the data stream may be referred to as a service (server). The service feature is a feature obtained by analyzing the flow data according to the dimension of the service of the data flow. For example, according to the service, traffic data with the same destination IP, destination port and protocol may be statistically analyzed, and the service feature may be obtained according to the result of the statistical analysis. The service features may include, but are not limited to, service base information, service accumulation features, and service baseline features. The service basic information may be, for example, a service, an earliest time of occurrence of the service in the network, a latest time of occurrence of the service in the network, and the like. The service accumulation feature may be, for example, the number of packets and the number of streams aggregated according to the service, the total statistical time, the average length of the messages, the ratio of the abnormal number of streams to the abnormal number of streams, and the like. The service baseline characteristic may be, for example, a flow rate baseline, a flow number baseline, a packet number baseline, an entropy value baseline, and the like of flows aggregated according to the service. Alternatively, the service baseline characteristics may be used to monitor and evaluate the behavior patterns of the service.

The source IP feature is a feature obtained by analyzing the traffic data according to the dimension of the source IP address of the data stream. For example, according to the source IP, traffic data with the same source IP may be statistically analyzed, and the source IP feature may be obtained according to the result of the statistical analysis. The source IP characteristics include, but are not limited to, source IP basic information, source IP cumulative characteristics, and source IP baseline characteristics. The source IP basic information may be, for example, aggregate source IP, earliest time of occurrence of source IP in the network, latest time of occurrence of source IP in the network, and the like. The source IP accumulation feature may be, for example, the number of packets and the number of streams aggregated according to the source IP, the total statistical time, the average length of the packets, the ratio of the number of abnormal streams to the number of abnormal streams, and the like. The source IP baseline characteristic may be, for example, a flow rate baseline, a flow number baseline, a packet number baseline, an entropy value baseline, and the like of the flows aggregated according to the source IP. Alternatively, the source IP baseline characteristics may be used to monitor and evaluate the behavior patterns of the source IP.

The global feature is a feature obtained by analyzing the flow data from the global dimension. Global features include, but are not limited to, time window cumulative features and global baseline features. The time window accumulation features may be, for example, flow rate, source IP base, destination IP base, total number of source IP and source port, total number of destination IP and destination port, flow number, etc. in the time window, and these features may be obtained by statistical analysis according to flow data in multiple detection windows. The global baseline features, such as a flow rate baseline, a source IP cardinal number baseline, a destination IP cardinal number baseline, and the like, can be used to define and update a standard value for global monitoring to help detect abnormal behavior of the network.

After multi-dimensional analysis of the flow data and obtaining multi-dimensional flow characteristics of the flow data, in some embodiments, the method may further include the steps of:

And updating a local flow database of the network element equipment according to the multidimensional flow characteristics of the flow data.

The local traffic database of the network element device may include a plurality of data tables, where the plurality of data tables are used to store traffic characteristics of the traffic data. Specifically, the local traffic database of the network element device may include at least two of a five-tuple flow table (FlowTable), a destination IP table (DipTable), a service table (ServerTable), a source IP table (SipTable), and a global feature table (GlobalTable). The five-tuple flow table is used for storing five-tuple characteristics of the flow data, the destination IP table is used for storing destination IP characteristics of the flow data, the service table is used for storing service characteristics of the flow data, the source IP table is used for storing source IP characteristics of the flow data, and the global characteristic table is used for storing global characteristics of the flow data. After the multi-dimensional flow characteristics of the flow data are determined, a plurality of data tables in the local flow database may be updated according to the multi-dimensional flow characteristics. For example, the five-tuple flow table may be updated according to the five-tuple characteristics of the traffic data, the target IP table may be updated according to the destination IP characteristics of the traffic data, the service table may be updated according to the service characteristics of the traffic data, the source IP table may be updated according to the source IP characteristics of the traffic data, and the global characteristic table may be updated according to the global characteristics of the traffic data.

For ease of understanding, a more specific embodiment will be described below as to how to perform multidimensional analysis on traffic data and update a local traffic database based on the analysis results.

Under the condition that the local flow database comprises a quintuple flow table, and the quintuple flow table comprises flow basic information, flow accumulation characteristics and flow baseline characteristics, analyzing flow data to obtain quintuple characteristics and updating the quintuple flow table according to the quintuple characteristics, the method can comprise the following steps:

in each detection window, aggregating the flow data according to the quintuple information to obtain a first aggregation result;

Updating the flow basic information and the flow accumulation characteristics in the five-tuple flow table according to the first aggregation result;

And updating the flow baseline characteristic in the five-tuple flow table according to the first aggregation result and the flow accumulation characteristic in the five-tuple flow table.

Specifically, in each detection window, the flow data may be first aggregated according to five-tuple information of the flow data to obtain a first aggregation result, that is, the flow data is statistically calculated according to the detection window to obtain a statistical value, where the detection window is a basic unit time of the aggregation result (or the statistical value). After the first aggregation result is obtained, the flow basic information and the flow accumulation feature in the five-tuple flow table may be updated according to the first aggregation result, that is, the flow basic information and the flow accumulation feature are determined according to the first aggregation result, and then the flow basic information and the flow accumulation feature are written into the five-tuple flow table. Then, the flow baseline characteristic in the five-tuple flow table may be updated according to the first aggregation result and the flow accumulation characteristic, i.e. the flow baseline characteristic is determined according to the first aggregation result and the flow accumulation characteristic, and then the flow baseline characteristic is written into the five-tuple flow table. Thus, updating of the five-tuple flow table can be achieved.

Under the condition that the local flow database comprises a destination IP table, the destination IP table comprises destination IP basic information, destination IP accumulation characteristics and destination IP baseline characteristics, analyzing flow data to obtain destination IP characteristics and updating the destination IP table according to the destination IP characteristics, the method can comprise the following steps:

in each detection window, aggregating the flow data according to the destination IP address to obtain a second aggregation result;

Updating the destination IP basic information and the destination IP accumulation characteristics in the destination IP table according to the second aggregation result;

and updating the destination IP baseline characteristic in the destination IP table according to the second aggregation result and the destination IP accumulation characteristic in the destination IP table.

Specifically, in each detection window, the traffic data may be first aggregated according to the destination IP address of the traffic data to obtain a second aggregation result, that is, flows with the same destination IP are aggregated according to the detection window, where the detection window is the basic unit time of the aggregation result (or statistics value). After the second aggregation result is obtained, the destination IP basic information and the destination IP accumulation feature in the destination IP table may be updated according to the second aggregation result, that is, the destination IP basic information and the destination IP accumulation feature are determined according to the second aggregation result, and then the destination IP basic information and the destination IP accumulation feature are written into the destination IP table. And then, updating the destination IP baseline characteristic in the destination IP table according to the second aggregation result and the destination IP accumulation characteristic, namely determining the destination IP baseline characteristic according to the second aggregation result and the destination IP accumulation characteristic, and writing the destination IP baseline characteristic into the destination IP table. Thus, the update of the destination IP table can be realized.

In the case that the local traffic database includes a service table, the service table includes service basic information, service accumulation features and service baseline features, the analyzing the traffic data to obtain service features and updating the service table according to the service features may include the following steps:

In each detection window, aggregating the flow data according to the service to obtain a third aggregation result;

Updating the service basic information and the service accumulation characteristics in the service table according to the third aggregation result;

and updating the service baseline characteristic in the service table according to the third aggregation result and the service accumulation characteristic in the service table.

Specifically, in each detection window, the traffic data may be first aggregated according to the service of the traffic data to obtain a third aggregation result, that is, flows with the same destination IP, destination port, and protocol are aggregated according to the detection window, where the detection window is the basic unit time of the aggregation result (or statistics value). After the third aggregation result is obtained, the service basic information and the service accumulation feature in the service table may be updated according to the third aggregation result, that is, the service basic information and the service accumulation feature are determined according to the third aggregation result, and then the service basic information and the service accumulation feature are written into the service table. Thereafter, the service baseline characteristic in the service table may be updated according to the third aggregation result and the service accumulation characteristic, i.e. the service baseline characteristic is determined according to the third aggregation result and the service accumulation characteristic, and then the service baseline characteristic is written into the service table. Thereby, updating of the service table can be achieved.

Under the condition that the local flow database comprises a source IP table, the source IP table comprises source IP basic information, source IP accumulated characteristics and source IP baseline characteristics, analyzing flow data to obtain source IP characteristics and updating the source IP table according to the source IP characteristics, the method can comprise the following steps:

In each detection window, aggregating the flow data according to the source IP address to obtain a fourth aggregation result;

Updating the source IP basic information and the source IP accumulation characteristics in the source IP table according to the fourth aggregation result;

and updating the source IP baseline characteristic in the source IP table according to the fourth aggregation result and the source IP accumulation characteristic in the source IP table.

Specifically, in each detection window, the traffic data may be first aggregated according to the source IP address of the traffic data to obtain a fourth aggregation result, that is, flows with the same source IP are aggregated according to the detection window, where the detection window is the basic unit time of the aggregation result. After the fourth aggregation result is obtained, the source IP basic information and the source IP cumulative feature in the source IP table may be updated according to the fourth aggregation result, that is, the source IP basic information and the source IP cumulative feature are determined according to the fourth aggregation result, and then the source IP basic information and the source IP cumulative feature are written into the source IP table. Then, the source IP baseline characteristic in the source IP table may be updated according to the fourth aggregation result and the source IP cumulative characteristic, that is, the source IP baseline characteristic is determined according to the fourth aggregation result and the source IP cumulative characteristic, and then the source IP baseline characteristic is written into the source IP table. Thereby, updating of the source IP table can be achieved.

Under the condition that the local flow database comprises a global feature table, and the global feature table comprises a time window accumulated feature and a global baseline feature, analyzing the flow data to obtain global features and updating the global feature table according to the global features, the method can comprise the following steps:

In each detection window, updating the time window accumulation feature in the global feature table according to at least one of the flow accumulation feature in the five-tuple flow table, the destination IP accumulation feature in the destination IP table, the service accumulation feature in the service table and the source IP accumulation feature in the source IP table and the flow data;

And updating the global baseline characteristic in the global characteristic table according to the flow data and the time window accumulated characteristic in the global characteristic table.

Specifically, in each detection window, the time window accumulation feature in the global feature table may be updated according to the accumulation features in the flow data and other data tables, where the other data tables may be at least one of the five-tuple flow table, the destination IP table, the service table, and the source IP table. That is, the time window accumulation feature may be determined according to the flow accumulation feature in the five-tuple flow table, the destination IP accumulation feature in the destination IP table, at least one of the service accumulation feature in the service table and the source IP accumulation feature in the source IP table, and the flow data, and then the time window accumulation feature is written into the global feature table, so as to implement updating of the time window accumulation feature in the global feature table. And then, updating the global baseline characteristic in the global characteristic table according to the flow data and the time window accumulated characteristic, namely determining the global baseline characteristic according to the flow data and the time window accumulated characteristic, and writing the global baseline characteristic into the global characteristic table to realize the updating of the global baseline characteristic in the global characteristic table.

After updating the plurality of data tables in the local flow database based on the method, when the local network reasoning is performed subsequently, the multidimensional flow characteristics can be obtained from the local flow database, and the network reasoning is performed locally according to the multidimensional flow characteristics and the AI model.

In some embodiments, for the local traffic database, the network element device also provides the basic functionality of the database. Specifically, at least one of the following may be included:

adding, deleting and checking flow characteristics of a plurality of data tables in a local flow database according to the operation instruction; the adding, deleting and modifying functions can cover automatic updating, aging and inquiring functions of the data table, such as automatically updating, newly building or refreshing data in the data table, actively or automatically triggering the aging and deleting functions, supporting inquiring and outputting table item contents according to key values and the like;

outputting a plurality of data tables; for example, the information in all data tables can be output and displayed to reflect the current network state;

generating a white list according to flow characteristics in the data tables; for example, a network white list can be automatically generated according to the history rule of the source IP/destination IP list;

Analyzing the network state according to the flow characteristics in the data tables; for example, the network state may be scored based on a set threshold, traffic characteristics exceeding the threshold may increase the network anomaly score, and identify network state anomalies when the anomaly score exceeds a baseline or threshold;

Performing relevant configuration on a local flow database; such as database collection parameter modification and template selection functions.

By providing the above-described functionality of the database, it is possible to facilitate the management of the local traffic database, while better supporting subsequent network reasoning and network intelligence decisions.

In some embodiments, the acquiring the pre-trained AI model in S104 may include:

And acquiring a pre-trained target AI model corresponding to the target service scene according to the target service scene.

In monitoring and managing network traffic, a number of different traffic scenarios may be involved, such as a defensive attack scenario, a service type classification scenario, an intelligent QoS control scenario, etc. In order to adapt to different service scenes so as to perform network reasoning and network intelligent decision conforming to the current service scene under the different service scenes, when the AI model is acquired, a target AI model corresponding to the target service scene can be acquired according to the actual target service scene, and the target AI model is used for performing network reasoning under the target service scene according to flow data.

It should be noted that, when the AI model is pre-trained, different AI models can be obtained according to different service scenarios, and a corresponding relationship between the service scenarios and the AI models is established, so that when the AI model is obtained, a target AI model corresponding to the target service scenario can be obtained according to the target service scenario and the corresponding relationship. That is, there may be a plurality of pre-trained AI models, different AI models may correspond to different service scenarios, and network reasoning may be performed by acquiring corresponding AI models according to an actual service scenario. The types of different AI models may be different, for example, the AI model corresponding to the service scenario 1 is a machine learning model, and the AI model corresponding to the service scenario 2 is a deep learning model. Furthermore, different AI models may correspond to different training samples. For example, AI model 1 is obtained from sample flow data 1 and a flow type label corresponding to sample flow data 1 (supervised model training), and AI model 2 is obtained from sample flow data 2 (unsupervised model training).

Considering that certain traffic features may be key features in some traffic scenarios and may be non-key features in other traffic scenarios, i.e. different traffic scenarios may correspond to different key features, in order to adapt different traffic scenarios so that in different traffic scenarios accurate network reasoning may be performed using key features of the current traffic scenario, in some embodiments, after acquiring the target AI model according to the target traffic scenario, network reasoning may be performed locally according to the traffic data and the AI model, which may include:

selecting a target flow characteristic corresponding to a target service scene from the multidimensional flow characteristics of the flow data;

and carrying out network reasoning under the target service scene locally according to the target flow characteristics and the target AI model.

The multidimensional traffic feature may be at least two of the five-tuple traffic feature, destination IP feature, service feature, source IP feature, and global feature described above. When selecting the target traffic feature, a selection may be made from a five-tuple traffic feature, a destination IP feature, a service feature, a source IP feature, and a global feature.

In some implementations, the target traffic scenario may be a network attack detection scenario. In a network attack detection scenario, when a target traffic feature is selected from the multi-dimensional traffic features, the selected target traffic feature may include a traffic feature of traffic data in a first dimension, where the traffic feature in the first dimension may include a five-tuple traffic feature, a destination IP feature, a source IP feature, and a global feature.

In some implementations, the target business scenario can be a service type classification scenario. In a service type classification scenario, when a target traffic feature is selected from the multi-dimensional traffic features, the selected target traffic feature may include a traffic feature of the traffic data in a second dimension, where the traffic feature in the second dimension includes a five-tuple traffic feature and a global feature, and optionally may further include a destination IP feature and a source IP feature.

The above description of the target traffic scenario as the network attack detection scenario or the service type classification scenario illustrates which features the target traffic features may be, and in other traffic scenarios, features suitable for the service scenario may be selected as the target traffic features according to actual requirements, which is not illustrated one by one.

Optionally, since each dimension may include multiple features, after a feature of a dimension is selected, a further selection may be made from among the multiple features of that dimension. For example, when selecting the target traffic feature, the five-tuple traffic feature and the destination IP feature may be selected first, then the flow accumulation feature and the flow baseline feature may be selected from the five-tuple traffic feature, the destination IP basic information and the destination IP accumulation feature may be selected from the destination IP features, and the finally selected target traffic feature includes the flow accumulation feature, the flow baseline feature, the IP basic information and the destination IP accumulation feature.

After the target flow characteristics are selected, when local network reasoning is performed, the network reasoning under the target service scene can be performed locally according to the target flow characteristics and the target AI model. Because the target flow characteristics and the target AI model are key characteristics and AI models corresponding to (or adapting to) the target service scene, the obtained reasoning result can be more in line with the target service scene, and the accuracy is higher.

In locally performing network reasoning in the target traffic scenario according to the target traffic characteristics and the target AI model, in some embodiments, it may include:

Normalizing the target flow characteristics;

splicing the target flow characteristics after normalization treatment to obtain characteristic vectors;

and taking the characteristic vector as the input of the target AI model, and determining the output of the target AI model as a network reasoning result.

In the process of pre-training an AI model, in order to facilitate model training, flow characteristics are usually subjected to preprocessing such as normalization and characteristic stitching to obtain feature vectors, and then the feature vectors are used as model inputs for model training. Therefore, when local network reasoning is performed according to the target flow characteristics and the target AI model, in order to adapt to the model input of the target AI model, the target flow characteristics are required to be converted into feature vectors, namely, the target flow characteristics are subjected to normalization processing, and then the feature vectors are obtained by splicing the feature after the normalization processing. After the feature vector is obtained, the feature vector is input into a target AI model, and the target AI model can output a corresponding network reasoning result.

After the network reasoning result in the target service scene is obtained, network intelligent decision in the target service scene can be locally carried out according to the network reasoning result.

In order to facilitate understanding of how network reasoning is performed locally according to the AI model, the embodiments shown in fig. 2 to 4 will be described below as examples.

The AI model needs to be pre-trained before reasoning from the AI model. Taking supervised model training as an example, fig. 2 may include the following steps when training an AI model:

Step 21: training samples are generated.

Specifically, flow data can be obtained by simulating or collecting historical flow under a group network state, then the flow data is passed through a network element local flow database, multi-dimensional flow characteristics in the network element local flow database are obtained in a time window form to be used as training samples, and meanwhile, the flow types corresponding to the training samples are used as training labels. Different training samples and corresponding traffic type labels can be generated by different service scenarios, and the specific conditions can be determined according to the actual service scenario, which is not limited herein. For example, taking a network attack detection scenario as an example, the training samples may include a training sample when a network attack exists and a training sample when a network attack does not exist normally, and the traffic type may include a network attack exists and a network attack does not exist normally.

Step 22: feature selection and data preprocessing.

And selecting the characteristics of the multidimensional flow characteristics in the training sample, selecting the characteristics favorable for algorithm classification as the input of the AI model, carrying out data preprocessing operations such as normalization or data enhancement and the like on the characteristics, and finally splicing the characteristics into characteristic vectors as the input of the model. Different traffic characteristics may be selected in different traffic scenarios, and may be specifically determined according to actual traffic requirements, which is not specifically limited herein. For example, taking a network attack detection scenario as an example, the selected traffic characteristics may include five-tuple traffic characteristics, destination IP characteristics, source IP characteristics, and global characteristics.

Step 23: and (5) AI model training.

Specifically, the spliced feature vector in step 22 may be used as a model input, and the flow type label in step 21 may be used as a model output to perform model training.

Note that, the attack detection AI model may be a machine learning model (such as SVM, XGBoost) or a deep learning model (such as CNN, LSTM), which is not limited herein.

After the AI model is obtained through training, under the condition that the network equipment locally collects the flow data, local network reasoning can be carried out according to the multidimensional flow characteristics of the flow data and the AI model, and network intelligent decision can be carried out according to the reasoning result. Specifically, as shown in fig. 2, the following steps may be included:

step 24: and recording a network element local flow database.

Specifically, after the flow data is locally collected, the collected flow data can be subjected to multidimensional analysis to obtain multidimensional flow characteristics, and then a network element local flow database is updated according to the multidimensional flow characteristics and is continuously recorded in a time window mode. The multidimensional traffic features herein may include five-tuple traffic features, destination IP features, service features, source IP features, and global features.

Step 25: feature derivation and data preprocessing.

And (3) deriving the multidimensional flow characteristics of the current time window (the combination of the multidimensional flow characteristics selected in the training stage, namely the combination of the multidimensional flow characteristics matched with the actual service scene) from the local flow database, preprocessing corresponding data, splicing the data into characteristic vectors, and sending the characteristic vectors into an AI model.

Step 26: AI model reasoning.

And (5) taking the feature vector obtained by preprocessing in the step (25) as the input of an AI model, and obtaining a corresponding reasoning result.

Step 27: intelligent decision and report display.

And (5) carrying out corresponding intelligent decision according to the reasoning result in the step (26), and simultaneously generating a result report by combining the basic information of the network element local flow database.

In order to facilitate understanding the above specific implementation manner of network reasoning according to the AI model, a description will be given below of how network reasoning is performed according to the AI model, taking a distributed denial of service (Distributed Denial of Service, DDOS) attack detection scenario and a service type classification scenario as an example.

DDOS attack detection scenario:

When DDOS attacks occur, the flow distribution passing through the network element equipment can change greatly, such as flow number increase, the characteristics of destination IP, source IP, entropy, base number and the like can change obviously, the changes are captured and counted by a network element local flow database in time, and corresponding AI models can be trained by utilizing the changes and then loaded into the network element equipment. And extracting key features of the current network element flow database in each detection period, and sending the key features into a trained AI model to perform real-time detection of DDOS attack. Please refer to fig. 3 in detail.

In fig. 3, when DDOS attack detection is performed, the following steps may be included:

Step 31: feature selection and preprocessing.

Here, the flow characteristics may be selected from the five-tuple flow table, the destination IP table, the source IP table, and the global characteristics table as key characteristics. When selecting the features from the five-tuple flow table, the flow features of M before the five-tuple flow table (M is a super parameter, the features in the five-tuple flow table may be ordered in the order of from big to small flow (may also be ordered according to other orders, here by default according to the flow size), and then the first M features are selected), and the flow features of 25, 50 and 75 bits of the flow features are taken as the flow features. Similarly, when selecting the features from the destination IP table and the source IP table, the features M before the destination IP table and the source IP table, and the features 25, 50, and 75 minutes thereof may be selected as the destination IP features and the source IP features. When selecting the feature from the global feature table, the global feature of the last 10 time windows (or other number of time windows) can be selected as the global feature.

After the features are selected, the features can be spliced to form feature vectors, and the feature vectors are preprocessed by a feature normalization method.

Step 32: AI model selection.

The AI model for DDOS attack detection may be acquired from a DDOS attack detection scenario. The classification can be performed using One-class svm as the network element base model (One-class svm is suitable for two classifications).

Step 33: intelligent decision and report display.

If the detection result is that the DDOS occurs, the detection result can be reported to the main control of the network element equipment (namely, the control plane CPU of the network element equipment) for flow drainage, and the current abnormal characteristic report and result analysis is output. If the detection result is Normal, the state of the flow network is good, and the monitoring can be continued.

Service type classification scenario:

The AI explicit congestion notification (Explicit Congestion Notification, ECN) determines the current ECN configuration according to the traffic model, and selects an appropriate AI-ECN model according to different service scenarios or adopts a heuristic algorithm to search and formulate ECN waterline. For the flow model, the existing scheme realizes the identification of the flow model based on the size flow identification service and the micro emergency. The embodiment of the application classifies the service types through the locally collected flow characteristics, and then selects the corresponding AI ECN configuration according to the service scene. See fig. 4 in detail.

In fig. 4, when classifying service types, the following steps may be included:

Step 41: feature selection and preprocessing.

Here, the flow characteristics may be selected from the five-tuple flow table and the global characteristics table as key characteristics. When the feature is selected from the five-tuple flow table, the flow feature of M in front of the five-tuple flow table and the flow features of 25, 50 and 75 minutes of the five-tuple flow table can be selected as the flow features. When selecting the feature from the global feature table, the global feature of the last 10 time windows (or other number of time windows) can be selected as the global feature.

After the features are selected, the features can be spliced to form feature vectors, and the feature vectors are preprocessed by a data enhancement method.

Step 42: AI model selection.

AI models for service type classification may be obtained from a service type classification scenario. XGBoost may be employed here as a network element basis model for classification (XGBoost is suitable for multi-classification).

Step 43: intelligent decision and report display.

Judging according to the reasoning result, switching the AI-ECN to be a throughput type large-flow model if the current service type is graph vector training or large-model training, switching to be a delay sensitive AI-ECN model if the current service type is a database, a service calling or recommending system, and regulating the AI-ECN to perform heuristic search if the current service type is other (unknown/mixed as shown in fig. 4).

According to the embodiment of the application, advanced data acquisition, analysis and intelligent processing functions are integrated on the network element equipment, flow analysis management is carried out on the network element equipment side, the network element equipment is supported to realize finer flow control and quicker safety response, and the flexibility and efficiency of network management can be improved. Meanwhile, the local flow analysis paradigm can also obviously reduce the dependence on a central control server, optimize the network response speed and lighten the network core burden. Based on the processing method of the flow data provided by the embodiment of the application, at least the following technical effects can be realized:

Real-time and accuracy are improved: by directly monitoring and collecting traffic data at the forwarding plane of the network element device, delay of the data in the transmission process can be reduced, and data update with almost zero delay is realized. This method of collecting data directly from the source significantly improves the real-time and accuracy of the data processing, enabling network administrators to react quickly based on the latest data. This can more effectively support network applications requiring immediate decisions, such as real-time network attack defense and intelligent QoS adjustments, than traditional centralized data collection methods.

Communication resource consumption decreases: because the data processing and management are completed on the local network element equipment, the transmission requirement of the data in the network can be greatly reduced, and the consumption of network bandwidth and the load of a central processing server are reduced. The reduction of the resource consumption not only can optimize the running cost of the whole network, but also can improve the expandability and the reliability of the system.

Network security and management capability enhancement: the technical scheme of the embodiment of the application allows the network traffic to be monitored and analyzed in a fine granularity, and enhances the defending capability of the network, especially in the aspect of security threats such as DDOS attack and the like. Meanwhile, the data table and related baseline characteristics in the local flow database are dynamically updated, so that the method can be more flexibly adapted to the change of network behaviors, and flow abnormality detection and response can be effectively carried out.

Flexibility of operation and management: all operations can be performed through a Command Line Interface (CLI), and thus, flexibility of system management can be improved. This design allows a network administrator to quickly adjust the monitoring policies and response parameters to accommodate changing network environments and demands.

The flow data processing method provided by the embodiment of the application can be applied to the following environments:

1. large enterprise network environment:

In large enterprises, the complexity of the network and the data traffic are huge, and the enterprises can realize high-efficiency traffic monitoring and management by deploying a local traffic database on network element equipment of an enterprise network entrance, including real-time monitoring of network use condition, bandwidth allocation optimization and timely traffic abnormality identification. The localized data processing can reduce the delay and cost of data transmission, so that enterprises can quickly respond to network changes, and the service continuity and the reliability of the network are improved.

2. Cloud service provider:

for operators providing a broad range of cloud services, it can be ensured that traffic management inside and across data centers is critical. The technical scheme provided by the application can allow the cloud service provider to analyze and manage the local flow data on the network element equipment of the data center, so that the resource allocation can be optimized, and the service quality can be improved. In addition, such local traffic management can also support automated traffic regulation policies that address sudden traffic growth or demand for distributed services.

3. Network service provider:

Network service providers are faced with continuing network expansion and upgrade challenges, and the present application may enable service providers to implement traffic monitoring and management on network element devices of various access points. By analyzing the traffic data in real time, the service provider can better understand and predict the user behavior, thereby optimizing the network service and improving the customer satisfaction. In addition, through the local traffic database, the service provider is able to provide customized web services such as bandwidth adjustment and priority setting based on user behavior.

The foregoing describes certain embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Fig. 5 is a schematic structural view of an electronic device according to an embodiment of the present application. Referring to fig. 5, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 5, but not only one bus or type of bus.

And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.

The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the processing device of the flow data on the logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:

Collecting flow data locally at the network element equipment;

acquiring a pre-trained artificial intelligence AI model;

The method performed by the flow data processing apparatus disclosed in the embodiment of fig. 5 of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components. The methods, steps and logic blocks disclosed in the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present application may be embodied directly in a hardware decoding processor for execution, or in a combination of hardware and software modules in a decoding processor for execution. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

The electronic device may also execute the method of fig. 1 and implement the function of driving the access device in the embodiment shown in fig. 1, which is not described herein.

Of course, other implementations, such as a logic device or a combination of hardware and software, are not excluded from the electronic device of the present application, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or a logic device.

The present application also proposes a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of application programs, enable the portable electronic device to perform the method of the embodiment shown in fig. 1, and in particular to perform the operations of:

Collecting flow data locally at the network element equipment;

acquiring a pre-trained artificial intelligence AI model;

The application also provides a network element device, please refer to fig. 6. Fig. 6 is a schematic structural diagram of a network element device 60 according to an embodiment of the present application. As shown in fig. 6, in one embodiment, the network element device 60 may include a traffic collection module 61 and a network intelligence application module 62, where:

a flow collection module 61, configured to collect flow data locally at a network element device;

A network intelligence application module 62 for acquiring a pre-trained AI model; according to the flow data and the AI model, carrying out local network reasoning; and carrying out network intelligent decision according to the reasoning result.

In some embodiments, the traffic collection module 61 may be located in a forwarding plane of the network element device 60, and the network intelligent application module 62 may be located in a control plane of the network element device 60. That is, when collecting traffic data, the traffic data may be collected at the forwarding plane of the network element device 60, and then network reasoning and intelligent decision may be performed at the control plane of the network element device 60 according to the collected traffic data.

In some embodiments, the network element device 60 may also include a traffic database maintenance module. Alternatively, the traffic data maintenance module may be located in a forwarding plane or a control plane of the network element device 60.

In the case where the network element device 60 includes a traffic data maintenance module:

The flow collection module 61 is further configured to send the flow data to the flow database maintenance module in a form of a detection window;

and the flow database maintenance module is used for analyzing the flow data according to multiple dimensions of the flow data to obtain multi-dimensional flow characteristics of the flow data, and updating the local flow database of the network element equipment according to the multi-dimensional flow characteristics of the flow data.

The detection window is a time window, after the traffic data is locally collected by the network element device, the traffic collection module 61 may collect (or count) the traffic data in the time period at intervals (i.e. the detection window), and send the collection result to the traffic database maintenance module. The flow data collected by the flow collection module 61 may include multiple dimensions, and the flow data maintenance module may analyze the flow data according to the multiple dimensions of the flow data after receiving the flow data, obtain multi-dimensional flow characteristics of the flow data according to the analysis result, and update a local flow database of the network element device according to the multi-dimensional flow characteristics of the flow data.

The multidimensional traffic feature may include at least two of a five-tuple traffic feature, a destination IP feature, a service feature, a source IP feature, and a global feature. The local traffic database may include at least two of a five-tuple flow table, a destination IP table, a service table, a source IP table, and a global feature table. The five-tuple flow table is used for storing five-tuple flow characteristics, the destination IP table is used for storing destination IP characteristics, the service table is used for storing service characteristics, the source IP table is used for storing source IP characteristics, and the global characteristic table is used for storing global characteristics.

In some implementations, the network element device 60 may also include a database-based application module. The database application module may be used for at least one of:

Alternatively, the database application module may be located in the control plane of the network element device.

In some embodiments, the network intelligence application module 62, based on the traffic data and the AI model, performs network reasoning locally, may include:

In some embodiments, the network intelligence application module 62, acquiring the pre-trained AI model, may include:

acquiring a pre-trained target AI model corresponding to a target service scene according to the target service scene;

the network intelligence application module 62, based on the traffic data and the AI model, performs network reasoning locally, and may include:

In some embodiments, the AI model may include a plurality of AI models, different AI models corresponding to different traffic scenarios, different AI models being trained from different sample traffic data and traffic type labels.

In some embodiments, the target traffic scenario may include a network attack detection scenario, the target traffic characteristics may include traffic characteristics of the traffic data in a first dimension, and the traffic characteristics in the first dimension may include five-tuple traffic characteristics, destination IP characteristics, source IP characteristics, and global characteristics.

In some implementations, the target traffic scenario may include a service type classification scenario, the target traffic feature may include a traffic feature of the traffic data in a second dimension, and the traffic feature in the second dimension may include a five-tuple traffic feature and a global feature.

In some embodiments, the network intelligent application module 62 locally performs network reasoning in the target traffic scenario according to the target traffic characteristics and the target AI model, and may include:

Normalizing the target flow characteristics;

The network element device 60 provided by the present application may also execute the method of fig. 1 and implement the functions of the embodiment of the network element device 60 shown in fig. 1, which are not described herein.

Fig. 7 is a schematic structural diagram of a network element device according to an embodiment of the present application.

The network element device shown in fig. 7 includes a control plane and a forwarding plane, where a traffic collection module and a decision control module are disposed in the forwarding plane, and a traffic database maintenance module, a database base application module, a network intelligent application module and a decision control module are disposed on the control plane. The flow collection module can collect flow data on the forwarding plane and then send the flow data to the flow database maintenance module of the control plane. And the flow database maintenance module performs multidimensional analysis according to the flow data, and performs update management and aging elimination on the local flow database according to the analysis result. The database basic application module is responsible for basic function interfaces of the database, including operations of configuration management, report summary, active query and the like, so as to support active call of other devices (such as a management plane/terminal/management control shown in fig. 7). In addition, the feature analysis function provided by the database basic application module can also provide input features for the AI model in the network intelligent application module so that the network intelligent module can conduct network reasoning. After the network intelligent application module performs network reasoning, the decision control module can be called according to the reasoning result so as to realize intelligent decision of the whole network element equipment.

The function of the modules shown in fig. 7 can be seen from the function of the corresponding modules in the embodiment shown in fig. 6, and will not be described in detail here.

Fig. 8 is a schematic structural diagram of a network element device according to an embodiment of the present application.

The network element device shown in fig. 8 includes a control plane and a forwarding plane, wherein a flow acquisition module, a flow database maintenance module and a decision control module are disposed in the forwarding plane, and a database base application module, a network intelligent application module and a decision control module are disposed on the control plane. The flow collection module can collect flow data on the forwarding plane and then send the flow data to the flow database maintenance module of the forwarding plane. And the flow database maintenance module performs multidimensional analysis according to the flow data, and performs update management and aging elimination on the local flow database according to the analysis result. The database basic application module is responsible for basic function interfaces of the database, including operations of configuration management, report summary, active query and the like, so as to support active call of other devices (such as a management plane/terminal/management control shown in fig. 8). In addition, the feature analysis function provided by the database basic application module can also provide input features for the AI model in the network intelligent application module so that the network intelligent module can conduct network reasoning. After the network intelligent application module performs network reasoning, the decision control module can be called according to the reasoning result so as to realize intelligent decision of the whole network element equipment.

The function of the modules shown in fig. 8 can be seen from the function of the corresponding modules in the embodiment shown in fig. 6, and will not be described in detail here.

Fig. 9 is a schematic diagram of processing traffic data by a network element device according to an embodiment of the present application.

In fig. 9, when the network element device processes the traffic data, a specific flow is as follows:

(1) Collection procedure (this procedure may be performed by a traffic collection module in the network element device):

The original features of the ACL-specified flows are collected on the line cards (forwarding planes) and the collected flow original features are sent to the corresponding CPUs (the CPUs of the forwarding planes or the CPUs of the control planes). The specific collection method can adopt a plurality of modes such as a hardware flow table, a counter, a joint algorithm, a NetFlow flow table and the like, and the specific sending method is not limited and comprises NetFlow encapsulation sending or shared memory implementation.

(2) Database maintenance procedure (this procedure may be performed by a traffic database maintenance module in the network element device):

step 1: traffic is aggregated according to a computation period.

Taking NetFlow sampling as an example, the collected original characteristics of NetFlow are accumulated and recorded, aggregation is carried out according to time slices, a plurality of sampling periods (or detection periods) form a calculation period, and the calculation period is the basic unit time for updating the statistic value.

Step 2: the basic features (including basic information and accumulated features) in the plurality of data tables are updated.

Each time one calculation cycle has elapsed, the flow characteristics defined in the flow table (FlowTable), including the flow base information and the flow accumulation characteristics, are updated based on the flow information. And then the original stream information is aggregated according to the destination IP, the accumulated characteristics of each destination IP are calculated, and then the destination IP basic information and the destination IP accumulated characteristics in a destination IP table (DipTable) are updated. The source IP and the accumulated characteristics of the service are calculated in the same way, and the source IP basic information and the accumulated characteristics of the source IP in the source IP table (SipTable) and the service basic information and the accumulated characteristics of the service in the service table (ServerTable) are updated. Finally, calculating the global time window accumulation feature according to the data in the plurality of data tables and the flow data in the step 1, and updating the time window accumulation feature in the global feature table (GlobalTable).

Step 3: the baseline characteristics in the plurality of data tables are updated.

The baseline characteristic in each data table is comprehensively calculated according to the flow aggregation characteristic in the step 1 and the accumulation characteristic (FlowTable) of each data table in the step 2. After the baseline characteristic is obtained, the accumulated characteristic of the current window and the baseline characteristic can be compared to judge whether the flow is abnormal or not (the judgment rule can be combined with the actual scene to determine). The baseline characteristic may be calculated in a number of ways, such as a running average, a maximum, a minimum, etc.

(3) Network intelligence application (this flow may be performed by a network intelligence application module in the network element device):

The process is divided into two stages, namely an offline training stage and an online reasoning stage. During the offline training phase, the AI model may be trained by a remote server or network element device. In the online reasoning stage, local network reasoning can be performed according to multidimensional flow characteristics and an AI model of the flow data, corresponding intelligent decisions can be performed according to reasoning results, and specific implementation modes can refer to specific implementation of corresponding steps in the embodiment shown in FIG. 1, and detailed description is omitted here.

In addition to the collection procedure, the data table maintenance procedure, and the network intelligent application described above, the network element device may also provide database basic application functions, including, but not limited to, deletion and correction of information in the data table, visual report output of the data table, white list generation, network state analysis, database configuration, and the like.

Based on the embodiment shown in fig. 9, the application provides a system framework of a network element local traffic database and an intelligent application framework based on the network element local traffic database. Wherein:

System framework of network element local traffic database: the method comprises capturing traffic on a forwarding plane chip of network element equipment, setting timing aging, uploading in a detection window form, and uploading a forwarding plane CPU/control plane CPU to update, manage and apply the traffic. The local flow database of the network element equipment can comprise a five-tuple flow table (FlowTable), a destination IP table (DipTable), a service table (ServerTable), a source IP table (SipTable), a global feature table (GlobalTable) and other tables, wherein each table records the accumulation feature, the real-time feature and the dynamic baseline feature of the flow/destination IP/service/source IP/time window, and meanwhile, the local flow database has corresponding interfaces for updating, deleting and searching, and management operation can be performed manually according to clients.

Intelligent application framework based on network element local flow database: an algorithm framework for realizing intelligent network element application based on a network element local flow database is provided. The framework takes multidimensional flow characteristics maintained by a local flow database as input samples, performs characteristic engineering and preprocessing on the characteristics, and trains corresponding AI models. When intelligent application is performed, the AI model can be deployed into the network element equipment (such as the main control of the network element equipment), then the current network flow characteristics are derived in real time, local network reasoning is performed according to the AI model, and finally intelligent decision is performed according to the reasoning result so as to optimize the network. Based on the framework, various intelligent applications such as DDOS attack defense and service type classification scenes can be realized, network resource allocation and management can be optimized, and the defense capacity of the network is enhanced.

Therefore, the collection, update management and application of the flow data are all integrated on the network element equipment, a brand new and efficient technical scheme can be provided for network management and safety protection, and therefore the autonomous management capacity of the network and the instant response capacity to abnormal states can be remarkably enhanced, and the effective monitoring and management of the network data flow are realized.

The present application also proposes a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of the above-described method embodiments of processing flow data.

In summary, the foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The embodiments of the present application are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

Claims

1. A method for processing flow data, applied to a network element device, comprising:

Collecting traffic data locally on the network element device;

Get pre-trained artificial intelligence AI models;

Performing network reasoning locally according to the traffic data and the AI model;

Make intelligent network decisions based on the inference results.

2. The method according to claim 1, wherein the traffic data includes traffic data of multiple dimensions; after the network element device collects the traffic data locally, the method further comprises:

Analyze the traffic data according to the multiple dimensions to obtain multi-dimensional traffic characteristics of the traffic data;

The locally performing network reasoning according to the traffic data and the AI model includes:

Network inference is performed locally based on the multi-dimensional traffic characteristics of the traffic data and the AI model.

3. The method according to claim 2, wherein analyzing the traffic data according to the multiple dimensions to obtain the multi-dimensional traffic characteristics of the traffic data comprises:

Aggregating the traffic data in the form of a detection window to obtain a traffic aggregation result;

The traffic aggregation results are analyzed according to the multiple dimensions to obtain multi-dimensional traffic characteristics of the traffic data.

4. According to the method described in claim 2 or 3, the multi-dimensional traffic characteristics of the traffic data include at least two of the five-tuple traffic characteristics, destination IP characteristics, service characteristics, source IP characteristics and global characteristics.

5. The method of claim 4, further comprising:

Updating a local traffic database of the network element device according to the multi-dimensional traffic characteristics of the traffic data;

Among them, the local traffic database includes at least two items of a five-tuple flow table, a destination IP table, a service table, a source IP table and a global feature table. The five-tuple flow table is used to store the five-tuple traffic features, the destination IP table is used to store the destination IP features, the service table is used to store the service features, the source IP table is used to store the source IP features, and the global feature table is used to store the global features.

6. The method of claim 5, further comprising at least one of the following:

Add, delete, modify and query the traffic characteristics of multiple data tables in the local traffic database according to the operation instructions;

outputting the plurality of data tables;

Generate a whitelist according to the traffic characteristics in the multiple data tables;

Performing network status analysis according to traffic characteristics in the multiple data tables;

The local traffic database is configured accordingly.

7. The method according to claim 1 or 2, wherein obtaining a pre-trained AI model comprises:

According to the target business scenario, obtain a pre-trained target AI model corresponding to the target business scenario;

Selecting a target traffic feature corresponding to the target business scenario from the multi-dimensional traffic features of the traffic data;

Based on the target traffic characteristics and the target AI model, network inference is performed locally in the target business scenario.

8. According to the method described in claim 7, the AI model includes multiple AI models, different AI models correspond to different business scenarios, and different AI models are trained based on different sample traffic data and traffic type labels.

9. According to the method described in claim 7, the target business scenario includes a network attack detection scenario, the target traffic characteristics include the traffic characteristics of the traffic data in a first dimension, and the traffic characteristics in the first dimension include five-tuple traffic characteristics, destination IP characteristics, source IP characteristics and global characteristics.

10. According to the method as claimed in claim 7, the target business scenario includes a service type classification scenario, the target traffic characteristics include traffic characteristics of the traffic data in a second dimension, and the traffic characteristics in the second dimension include five-tuple traffic characteristics and global characteristics.

11. The method of claim 7, wherein the network reasoning in the target business scenario is performed locally according to the target traffic characteristics and the target AI model, comprising:

Normalizing the target flow characteristics;

The target flow characteristics after normalization are concatenated to obtain a feature vector;

The feature vector is used as the input of the target AI model, and the output of the target AI model is determined as the network reasoning result.

12. A network element device, comprising a traffic collection module and a network intelligent application module, wherein:

The flow collection module is used to collect flow data locally on the network element device;

The network intelligent application module is used to obtain a pre-trained AI model; perform network reasoning locally based on the traffic data and the AI model; and make network intelligent decisions based on the reasoning results.

13. The network element device according to claim 12, further comprising a traffic database maintenance module;

The flow collection module is further used to send the flow data to the flow database maintenance module in the form of a detection window;

The traffic database maintenance module is used to analyze the traffic data according to multiple dimensions of the traffic data to obtain multi-dimensional traffic characteristics of the traffic data, and update the local traffic database of the network element device according to the multi-dimensional traffic characteristics of the traffic data;

Among them, the local traffic database includes at least two items of a five-tuple flow table, a destination IP table, a service table, a source IP table and a global feature table. The five-tuple flow table is used to store five-tuple traffic features, the destination IP table is used to store destination IP features, the service table is used to store service features, the source IP table is used to store source IP features, and the global feature table is used to store global features.

14. The network element device according to claim 13, further comprising a database basic application module;

The database basic application module is used for at least one of the following:

outputting the plurality of data tables;

The local traffic database is configured accordingly.

15. An electronic device, comprising:

processor;

a memory for storing instructions executable by the processor;

The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 11.

16 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to claim 1 .

17. A computer program product, comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps in the method according to any one of claims 1 to 11.