CN117708654A

CN117708654A - Anomaly detection model training method and device

Info

Publication number: CN117708654A
Application number: CN202311718757.XA
Authority: CN
Inventors: 何冬悦
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2023-12-13
Filing date: 2023-12-13
Publication date: 2024-03-15
Also published as: WO2025123983A1

Abstract

The embodiment of the specification provides an anomaly detection model training method and device, wherein the anomaly detection model training method comprises the following steps: after the first mechanism description data of model training under the preset mechanism classification is obtained, selecting an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, constructing a model to be trained based on the abnormality detection unit, carrying out model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, and carrying out abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model.

Description

Anomaly detection model training method and device

Technical Field

The present document relates to the field of data processing technologies, and in particular, to a training method and device for an anomaly detection model.

Background

With the continuous development of internet technology, more and more institutions have a need of showing out the label data of the institutions to users, such as showing out the label data of the institutions to users when the institutions have the recruitment requirement of employees, so that the users can know the institutions in multiple directions, and the users can conveniently select intention institutions to further know in depth.

The organization tag data represents organization images, and the attribute characteristics of the organization are described by using short tag data, so that the importance of the organization tag data is self-evident, and in the process, higher requirements are put forward to an organization tag data management party for management and perfection of the organization tag data.

Disclosure of Invention

One or more embodiments of the present specification provide an anomaly detection model training method, including: first institution description data for model training under a preset institution classification is obtained. And selecting an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit. And carrying out model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to carry out abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model. Wherein, each abnormal detection unit in the abnormal detection unit set is extracted and obtained in a pre-training detection model.

One or more embodiments of the present disclosure provide an anomaly detection processing method, including: and acquiring the mechanism description data to be detected. And inputting the mechanism description data into a mechanism classification model of the mechanism description data and an abnormality detection model with at least one of data types matched with description types for abnormality detection processing, so as to obtain an abnormality detection result of the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

One or more embodiments of the present specification provide an anomaly detection model training device including: the data acquisition module is configured to acquire first institution description data for model training under a preset institution classification. The model construction module is configured to select an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, and construct a model to be trained based on the abnormality detection unit. The model training module is configured to perform model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so that abnormality detection processing is performed on the second mechanism description data under the preset mechanism classification through the abnormality detection model. Wherein, each abnormal detection unit in the abnormal detection unit set is extracted and obtained in a pre-training detection model.

One or more embodiments of the present specification provide an abnormality detection processing apparatus including: and the description data acquisition module is configured to acquire the mechanism description data to be detected. And the abnormality processing module is configured to input the mechanism description data into a mechanism classification of the mechanism description data and perform abnormality detection processing on an abnormality detection model with at least one of data types matched with description types, so as to obtain an abnormality detection result of the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

One or more embodiments of the present specification provide an anomaly detection model training device comprising: a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to: first institution description data for model training under a preset institution classification is obtained. And selecting an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit. And carrying out model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to carry out abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model. Wherein, each abnormal detection unit in the abnormal detection unit set is extracted and obtained in a pre-training detection model.

One or more embodiments of the present specification provide an abnormality detection processing apparatus including: a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to: and acquiring the mechanism description data to be detected. And inputting the mechanism description data into a mechanism classification model of the mechanism description data and an abnormality detection model with at least one of data types matched with description types for abnormality detection processing, so as to obtain an abnormality detection result of the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

One or more embodiments of the present specification provide a storage medium storing computer-executable instructions that, when executed by a processor, implement the following: first institution description data for model training under a preset institution classification is obtained. And selecting an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit. And carrying out model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to carry out abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model. Wherein, each abnormal detection unit in the abnormal detection unit set is extracted and obtained in a pre-training detection model.

One or more embodiments of the present specification provide another storage medium storing computer-executable instructions that, when executed by a processor, implement the following: and acquiring the mechanism description data to be detected. And inputting the mechanism description data into a mechanism classification model of the mechanism description data and an abnormality detection model with at least one of data types matched with description types for abnormality detection processing, so as to obtain an abnormality detection result of the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

Drawings

For a clearer description of one or more embodiments of the present description or of the solutions of the prior art, the drawings that are needed in the description of the embodiments or of the prior art will be briefly described below, it being obvious that the drawings in the description that follow are only some of the embodiments described in the present description, from which other drawings can be obtained, without inventive faculty, for a person skilled in the art;

FIG. 1 is a schematic diagram of an anomaly detection model training method implementation environment provided in one or more embodiments of the present disclosure;

FIG. 2 is a process flow diagram of an anomaly detection model training method provided in one or more embodiments of the present disclosure;

FIG. 3 is a process flow diagram of an anomaly detection model training method for use in model training scenarios in accordance with one or more embodiments of the present disclosure;

FIG. 4 is a process flow diagram of an anomaly detection processing method according to one or more embodiments of the present disclosure;

FIG. 5 is a schematic diagram of an embodiment of an anomaly detection model training device provided in one or more embodiments of the present disclosure;

FIG. 6 is a schematic diagram of an embodiment of an anomaly detection processing apparatus according to one or more embodiments of the present disclosure;

FIG. 7 is a schematic structural diagram of an anomaly detection model training device according to one or more embodiments of the present disclosure;

fig. 8 is a schematic structural diagram of an abnormality detection processing apparatus according to one or more embodiments of the present disclosure.

Detailed Description

In order to enable a person skilled in the art to better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below with reference to the drawings in one or more embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one or more embodiments of the present disclosure without inventive effort, are intended to be within the scope of the present disclosure.

The method for training an anomaly detection model according to one or more embodiments of the present disclosure may be applied to an implementation environment of model training of an anomaly detection model, and referring to fig. 1, the implementation environment of the method at least includes:

a training system 101 for model training of a model to be trained, a pre-training detection model 102 for performing abnormality detection processing; in addition, the implementation environment may further include a set of anomaly detection units 103, a model 104 to be trained.

In the implementation environment, in the process of training an anomaly detection model, acquiring first mechanism description data for model training under a preset mechanism classification, selecting an anomaly detection unit from an anomaly detection unit set 103 according to at least one of the preset mechanism classification and a data type and a description type of the first mechanism description data, constructing a model 104 to be trained based on the selected anomaly detection unit, and then performing model training on the model 104 by using the first mechanism description data as a training sample by a training system 101 to obtain an anomaly detection model so as to perform anomaly detection processing on second mechanism description data under the preset mechanism classification through the anomaly detection model;

wherein, the training system 101 may acquire the first mechanism description data for model training, and the training system 101 may also select an anomaly detection unit from the anomaly detection unit set 103 according to at least one of a preset mechanism classification and a data type and a description type of the first mechanism description data, and construct a model 104 to be trained based on the anomaly detection unit; the training system 101 may also extract and obtain each anomaly detection unit in the anomaly detection unit set from the pre-training detection model 102.

One or more embodiments of an anomaly detection model training method provided in the present specification are as follows:

according to the anomaly detection model training method provided by the embodiment, after the first mechanism description data for model training under the preset mechanism classification is obtained, the anomaly detection unit is selected from the anomaly detection unit set according to at least one of the preset mechanism classification and the data type and description type of the first mechanism description data, the anomaly detection unit to be trained is constructed based on the selected anomaly detection unit, the anomaly detection unit adapting to the first mechanism description data is selected from the plurality of dimensions of the preset mechanism classification, the data type and the description type, the model to be trained is constructed, the comprehensiveness and the flexibility of constructing the model to be trained are improved, the anomaly detection model is further obtained by carrying out model training on the model to be trained according to the first mechanism description data, the anomaly detection processing is carried out on the second mechanism description data under the preset mechanism classification through the anomaly detection model, so that the anomaly detection model obtained through training can carry out anomaly detection on the adapted second mechanism description data, the pertinence and the flexibility of anomaly detection are improved, and the accuracy and the effectiveness of anomaly detection are improved through the more adapted anomaly detection model.

Referring to fig. 2, the training method for the anomaly detection model provided in the present embodiment specifically includes steps S202 to S206.

Step S202, first organization description data of model training under the preset organization classification is obtained.

The organization in this embodiment includes various forms of organizations such as enterprises, institutions, social groups, and the like, and furthermore, the organization includes not only the present-level organization but also an internal organization of the organization, such as an enterprise department or a division.

The preset mechanism classification refers to preset mechanism classification; the preset mechanism classification comprises mechanism classification obtained by dividing the domain in which the mechanism is engaged, for example, the preset mechanism classification can be one of a medical mechanism, a financial mechanism, a catering mechanism and the like; in addition, the preset mechanism classification may further include a mechanism classification obtained by dividing according to a mechanism scale or a mechanism classification obtained by dividing according to a geographical area to which the mechanism belongs, and the preset mechanism classification may also include a mechanism classification obtained by dividing according to a domain in which the mechanism is engaged, a mechanism scale and/or a geographical area to which the mechanism belongs; the facility scale may be a facility personnel scale, a facility asset scale, and/or a facility office scale, among others.

The first organization description data refers to description data describing the organization in one or more description types, such as credit rating and credit score of the organization described by the credit description type; revenue description data of a mechanism described from the revenue description type; discipline description data of a mechanism described from discipline description types, and the like; the first institution description data may be a first institution tag. The first mechanism description data may be mechanism description data of a single mechanism or mechanism description data of a plurality of mechanisms.

The description type of the first mechanism description data comprises a description dimension for describing the mechanism; such as a credit description type, a revenue description type, a punishment description type and/or a scale description type, the description types of the first institution description data are merely illustrative, and the description types of the first institution description data may be one or more; in addition, the description type may further include a statistics type and/or a prediction type, where the statistics type refers to a statistics type for counting data that has occurred at present in an organization, for example, the description type of the first organization description data is a statistics type if the first organization description data is a personnel scale, the prediction type refers to a prediction type for predicting data that has not occurred in an organization, for example, the first organization description data is a prediction type if the first organization description data is a company suitable personnel development, and the description type of the first organization description data is a prediction type.

The data type of the first institution description data includes an image type, a text type, a voice type, and/or a video type.

In specific implementation, in order to promote flexibility of model training, first mechanism description data for model training under preset mechanism classification can be obtained; for example, acquiring revenue description data for model training under financial institutions, punishment description data and/or acquiring revenue description data for model training under dining institutions, credit rating.

It should be noted that, the step S202 may be replaced by obtaining the first mechanism description data for performing the model training, and form a new implementation manner with other processing steps provided in the present embodiment.

Step S204, selecting an anomaly detection unit from the anomaly detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, and constructing a model to be trained based on the anomaly detection unit.

After the first mechanism description data for training the model under the preset mechanism classification is obtained, in this step, an anomaly detection unit is selected from the anomaly detection unit set according to at least one of the preset mechanism classification and the data type and description type of the first mechanism description data, and a model to be trained is constructed based on the anomaly detection unit.

The anomaly detection unit set in this embodiment refers to a set composed of one or more anomaly detection units; the anomaly detection units in the anomaly detection unit set can be anomaly detection units obtained by preliminary training, namely, the anomaly detection units can also perform anomaly detection processing on the mechanism description data.

In order to improve the accuracy of the anomaly detection model obtained by training and reduce the training difficulty of the anomaly detection model, an anomaly detection unit for constructing a model to be trained can be trained in advance; optionally, each anomaly detection unit in the anomaly detection unit set is extracted from a pre-training detection model. The pre-training detection model comprises a pre-training abnormality detection model, namely an abnormality detection model obtained by pre-training; the pre-trained detection model may be a large model LLM (Large Language Model ). The anomaly detection units in the anomaly detection unit set comprise Decision trees (Decision trees), random Forest (Random Forest) and/or Neural networks (Neural networks), and in addition, the anomaly detection units in the anomaly detection unit set can also comprise other types of anomaly detection units; the model structure of the pre-trained detection model may be composed of decision trees, random forests and/or neural networks.

In addition, each anomaly detection unit in the anomaly detection unit set can also be obtained by adopting a mode of performing pre-training independently; the first unit to be trained can be pre-trained to obtain a first abnormal detection unit, the second unit to be trained to obtain a second abnormal detection unit and/or the third unit to be trained can be pre-trained to obtain a third abnormal detection unit, the structure of the first unit to be trained can be a decision tree, the structure of the second unit to be trained can be a random forest, and the structure of the third unit to be trained can be a neural network.

In practical application, since the data types of the mechanism description data are various, the description types of the mechanism description data are also complicated and changeable, and the mechanism classification of the mechanism to which the mechanism description data belongs is also more, if the mechanism description data are subjected to anomaly detection by adopting the same anomaly detection model, the detection precision of the anomaly detection model may not be high, and the training requirement on the anomaly detection model is also high, for this purpose, in order to reduce the training difficulty of the anomaly detection model, the detection precision and the detection flexibility of the anomaly detection model are improved, at least one anomaly detection unit can be selected from the anomaly detection unit set according to at least one of the preset mechanism classification and the data type and the description type of the first mechanism description data, and the model to be trained is constructed based on the at least one anomaly detection unit;

In an optional implementation manner provided in this embodiment, in a process of selecting an anomaly detection unit from the anomaly detection unit set according to at least one of the preset mechanism classification and the data type and description type of the first mechanism description data, and constructing a model to be trained based on the anomaly detection unit, the following operations are performed:

if the preset mechanism is classified into a specific mechanism classification, selecting a first abnormality detection unit from the abnormality detection unit set;

selecting a second abnormality detection unit from the abnormality detection unit set and taking the first abnormality detection unit and the second abnormality detection unit as the abnormality detection units in the case that the description type is a prediction type;

in the case where the description type is not the prediction type, the first abnormality detection unit is taken as the abnormality detection unit.

The specific institution classification refers to a specified institution classification, which may include a medical institution or a financial institution, and may be determined according to an actual application scenario, which is not particularly limited in this embodiment. The first anomaly detection unit may be a decision tree and the second anomaly detection unit may be a random forest. The prediction type refers to that the description type of the first organization description data is the prediction type for predicting the non-occurrence, for example, the first organization description data is suitable for personnel development in the future of the xx company or unsuitable for investment in the future of the xx company, and the description type of the organization description data for predicting the future or the non-occurrence belongs to the prediction type.

In the above-described case where the description type is the prediction type, in an alternative implementation manner provided in this embodiment, on the basis of selecting the second abnormality detection unit from the abnormality detection unit set and taking the first abnormality detection unit and the second abnormality detection unit as the abnormality detection units, selecting the abnormality detection unit from the abnormality detection unit set and constructing the model to be trained based on the abnormality detection unit according to at least one of a preset mechanism classification and a data type and a description type of the first mechanism description data, selecting the second abnormality detection unit from the abnormality detection unit set as the abnormality detection unit if the preset mechanism classification is not the specific mechanism classification and the data type of the first mechanism description data is the text type, and selecting the third abnormality detection unit from the abnormality detection unit set as the abnormality detection unit if the preset mechanism classification is not the specific mechanism classification and the data type of the first mechanism description data is not the text type, specifically may perform the following operations:

if the preset mechanism classification is not the specific mechanism classification, judging whether the data type of the first mechanism description data is a text type or not;

If yes, selecting the second abnormality detection unit from the abnormality detection unit set as the abnormality detection unit;

if not, selecting a third abnormality detection unit from the abnormality detection unit set as the abnormality detection unit.

Wherein the third abnormality detection unit may be a neural network.

Further, the above-described operation may be performed directly in a case where the preset institution classification is not a specific institution classification, instead of selecting the second abnormality detection unit from the set of abnormality detection units and taking the first abnormality detection unit and the second abnormality detection unit as the abnormality detection units in the case where the description type is the prediction type.

In addition, in selecting an abnormality detection unit from the abnormality detection unit set according to at least one of the preset mechanism category and the data type and description type of the first mechanism description data, the following operations may also be performed: if the preset mechanism is classified into a specific mechanism classification, selecting a first abnormality detection unit from the abnormality detection unit set as an abnormality detection unit; if the preset mechanism classification is not the specific mechanism classification, selecting a second abnormality detection unit from the abnormality detection unit set as an abnormality detection unit under the condition that the data type of the first mechanism description data is a text type; if the preset organization classification is not the specific organization classification, selecting a third abnormality detection unit from the abnormality detection unit set as an abnormality detection unit in the case that the data type of the first organization description data is an image type, a voice type and/or a video type.

Note that, since the above-described predetermined mechanism belongs to different execution procedures when the predetermined mechanism is classified into the specific mechanism class and when the predetermined mechanism is not classified into the specific mechanism class, any one or more of the three execution procedures may be selected to select the abnormality detecting means.

For example, if the specific institution is classified as a financial institution or a medical institution, a decision tree is selected from the anomaly detection unit set, and if the description type of the first institution description data is a prediction type, a random forest is selected from the anomaly detection unit set, and the decision tree and the random forest are used as anomaly detection units; if the preset institution type is not a financial institution or a medical institution, selecting a random forest from the anomaly detection unit set as an anomaly detection unit under the condition that the data type of the first institution description data is a text type; if the preset institution type is not a financial institution or a medical institution, a neural network is selected as an abnormality detection unit in the abnormality detection unit set in the case that the data type is an image type, a voice type and/or a video type.

After the abnormality detection unit is selected from the abnormality detection unit set, a model to be trained can be built based on the abnormality detection unit, and in the process of building the model to be trained based on the abnormality detection unit, in order to improve the flexibility of building the model to be trained, if the selected abnormality detection unit is one, the abnormality detection unit is used as the model to be trained; if the number of the selected abnormality detection units is multiple, assembling the selected abnormality detection units to obtain a model to be trained.

In addition, in order to improve the comprehensiveness of the model to be trained and improve the performance of the model to be trained, an input unit and/or an output unit can be added in the process of constructing the model to be trained based on the anomaly detection unit, the model to be trained is constructed based on the input unit, the anomaly detection unit and the output unit, or the model to be trained is constructed based on the input unit and the anomaly detection unit, or the model to be trained is constructed based on the anomaly detection unit and the output unit; the input unit and/or the output unit are added to reduce the execution operation of the abnormality detection unit, and the simple execution operation is given to the input unit and/or the output unit, so that the concentration of the abnormality detection unit on the abnormality detection is improved, and the detection precision of the abnormality detection of the model to be trained obtained through training is improved; the input unit and the output unit may be deployed based on actual needs of building the model to be trained, for example, the input unit may include a feature extraction unit and/or a feature fusion unit, and the output unit may include a data output unit.

It should be noted that, the step S204 may be replaced by at least one of the data type and the description type (at least one of the preset mechanism classification, the data type and the description type) of the first mechanism description data according to the preset mechanism classification, selecting at least one anomaly detection unit from the anomaly detection unit set, and constructing the model to be trained based on the at least one anomaly detection unit;

or, it may be replaced by at least one of classifying the data types of the first mechanism description data according to the preset mechanism (at least one of the two), selecting an anomaly detection unit from a set of anomaly detection units, constructing a model to be trained based on the anomaly detection unit, and forming a new implementation manner with other processing steps provided in the present embodiment; the abnormality detection unit here may be at least one.

Step S206, performing model training on the model to be trained according to the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model.

The method comprises the steps of selecting an abnormality detection unit from an abnormality detection unit set by means of at least one of a preset mechanism classification and a data type and a description type of first mechanism description data, and constructing a model to be trained based on the abnormality detection unit.

The first mechanism description data in this embodiment refers to mechanism description data used as a training sample to perform model training on a model to be trained; the second mechanism description data refers to mechanism description data to be detected after training to obtain an abnormality detection model. The preset organization classification of the first organization description data is the same as the second organization description data, and at least one of the data type and the description type of the first organization description data is the same as the second organization description data.

During the specific implementation, in the model training process of each round, the first mechanism description data can be subjected to anomaly detection processing through the model to be trained to obtain an anomaly detection result of the first mechanism description data, then training loss is calculated based on the anomaly detection result and the label data, and parameter adjustment is performed on the model to be trained based on the training loss; in an optional implementation manner provided in this embodiment, during a model training process of a model to be trained according to the first mechanism description data, the following operations are performed:

inputting the first mechanism description data into the model to be trained for abnormality detection processing to obtain an abnormality detection result of the first mechanism description data;

And calculating training loss based on the abnormality detection result and the label data corresponding to the first mechanism description data, and carrying out parameter adjustment on the model to be trained based on the training loss.

The abnormality detection result may include an abnormality category, a null value index, a quality abnormality index, an invalid flag result and/or an association result of the asset transaction open information of the first institution description data, and may also include other types of detection results. The anomaly category herein refers to a category in which the first mechanism description data has anomalies, for example, the anomaly category includes a prediction direction error (a revenue increase is predicted to decrease), a numerical error, and the like, and the null index includes a null rate, that is, a null proportion, and specifically may represent a proportion in which the first mechanism description data has null, that is, a proportion in which the value of the data in the first mechanism description data is null; the invalid marking result comprises a marking result of the first organization description data belonging to the invalid data; the asset transaction open information includes marketing information.

The tag data corresponding to the first mechanism description data refers to tag data corresponding to the abnormality detection result, for example, if the abnormality detection result is an abnormality type, the tag data corresponding to the first mechanism description data is an abnormality type tag corresponding to the first mechanism description data.

It should be noted that, the above process of model training may represent a process of model training on each round of model to be trained, and in order to improve the model accuracy and the model performance of the anomaly detection model obtained by training, the above process of model training on the model to be trained may be referred to, and the model to be trained may be iteratively trained until the training loss converges, so as to obtain the anomaly detection model.

The above-mentioned abnormality detection results of the first mechanism descriptive data may include a plurality of different abnormality detection results; in practical applications, the first mechanism description data may be one or more, and abnormality detection may be performed on the first mechanism description data from a local angle, for example, abnormality category of each mechanism description data in the first mechanism description data is detected, and abnormality detection is performed on the first mechanism description data from a fine granularity layer; in a first optional implementation manner provided in this embodiment, the model to be trained may determine, according to a description type of the first mechanism description data, an anomaly type of the first mechanism description data as an anomaly detection result, and specifically may perform anomaly detection processing in the following manner:

extracting key data from initial data of a mechanism to which the first mechanism description data belongs according to the description type of the first mechanism description data;

And determining an abnormality category of the first mechanism description data based on the key data as the abnormality detection result.

The initial data may be current organization data of an organization to which the first organization description data belongs, that is, organization data currently related to the organization.

Specifically, according to the description type of the first mechanism description data, key data matching the description type may be extracted from the initial data of the mechanism to which the first mechanism description data belongs, and the abnormality type of the first mechanism description data may be determined as the abnormality detection result based on the key data.

For example, the description type of the first mechanism description data is a prediction type, the first mechanism description data is "punishment data is reduced year by year", the punishment data of the first year is 5 from the initial data of the mechanism to which the first mechanism description data belongs, the punishment data of the second year is 8, the punishment data of the third year is 10 ", the key data is extracted from the punishment data of 5, 8 and 10, and the punishment data of the mechanism is increased year by year, and the abnormal type of the first mechanism description data is a prediction direction error.

In addition to the above-provided manner of detecting the abnormality category of the first mechanism description data, the abnormality detection of the first mechanism description data may be performed from an overall point of view, that is, the abnormality detection of the first mechanism description data is performed from a coarse-grained layer side, such as calculating an abnormality index of the entire first mechanism description data; specifically, a null value index of the first mechanism description data can be calculated, intermediate mechanism description data without null value is screened out from the first mechanism description data, a quality abnormality index of the intermediate mechanism description data is calculated, and the null value index and the quality abnormality index are used as abnormality detection results; specifically, in another optional implementation manner provided in this embodiment, the model to be trained performs abnormality detection processing in the following manner:

Determining that no null value exists in the first mechanism description data;

calculating a null index of the first mechanism description data based on the first mechanism description data and the intermediate mechanism description data;

and calculating a quality abnormality index of the intermediate mechanism description data, and taking the null index and the quality abnormality index as the abnormality detection result.

The null value refers to that a value of data in the first organization description data is blank, for example, a specific numerical value of a personnel scale in the first organization description data is blank, that is, the organization description data with the null value is the personnel scale. The quality anomaly index includes an error rate, i.e., an error ratio.

Further, in an optional implementation manner provided in this embodiment, after calculating the quality anomaly index of the intermediate mechanism description data and executing the null value index and the quality anomaly index as the anomaly detection result, the following operations are further executed:

if the null index is larger than a null index threshold, determining null mechanism description data except the intermediate mechanism description data in the first mechanism description data, performing null filling on the null mechanism description data to obtain target mechanism description data, and storing the target mechanism description data into a training sample set;

And if the quality abnormality index is greater than an abnormality index threshold, performing model training on a data generation model for generating the first mechanism description data to obtain a trained data generation model.

Wherein the null value mechanism description data includes mechanism description data in which a null value exists in the first mechanism description data.

Specifically, under the condition that the null index is larger than the null index threshold, null filling is carried out on null mechanism description data to obtain target mechanism description data, after the target mechanism description data is stored in a training sample set, the mechanism description data can be read from the training sample set to continuously train a model to be trained or carry out model fine adjustment on an abnormal detection model obtained by training; under the condition that the quality anomaly index is larger than the anomaly index threshold, the accuracy of the data generation model representing the generation of the first mechanism description data is lower, so that model training can be carried out on the data generation model generating the first mechanism description data to obtain a trained data generation model, so that mechanism description data generation is carried out through the trained data generation model, virtuous circle is realized, the mechanism description data is more accurate, and the anomaly detection result of the mechanism description data is more accurate.

In addition, the model to be trained can also adopt the following mode to carry out abnormality detection processing: extracting key data from initial data of a mechanism to which the first mechanism description data belongs according to the description type of the first mechanism description data, and determining a detection category of the first mechanism description data as an abnormality detection result based on the key data; wherein the detection category comprises a category that the first mechanism description data is correct or incorrect, the detection category comprises a data exception category and a data normal category, the data exception category represents that the first mechanism description data is incorrect, and the data normal category represents that the first mechanism description data is correct;

alternatively, the model to be trained may further perform anomaly detection processing in the following manner: calculating null indexes of the first mechanism description data as abnormal detection results or calculating quality abnormal indexes of the first mechanism description data as abnormal detection results;

alternatively, the model to be trained may further perform anomaly detection processing in the following manner: under the condition that the logout information of the mechanism to which the first mechanism description data belongs is detected, carrying out invalid marking on the first mechanism description data, and taking an invalid marking result as an abnormality detection result; or under the condition that the asset transaction open information of the mechanism to which the first mechanism description data belongs is detected, carrying out association processing on the first mechanism description data and the asset transaction open information, and taking an association result as an abnormality detection result;

Alternatively, the model to be trained may further perform anomaly detection processing in the following manner: according to the residual mechanism description data except the current mechanism description data in the first mechanism description data, calculating a quality abnormality index of the current mechanism description data, according to the association degree of the current mechanism description data and the residual mechanism description data, carrying out weighted calculation on the quality abnormality index to obtain a target quality abnormality index of the current mechanism description data, and taking the target quality abnormality index of the first mechanism description data as an abnormality detection result.

It should be noted that, the above-mentioned implementation manners of performing the abnormality detection processing on the models to be trained may be combined with each other and referred to each other during the abnormality detection processing on the models to be trained, that is, any one or more of the above-mentioned implementation manners may be adopted during the abnormality detection processing on the models to be trained.

After the model training is performed to obtain the anomaly detection model, the anomaly detection model can be used for performing anomaly detection processing on the second mechanism description data under the preset mechanism classification, wherein the anomaly detection model is used for performing anomaly detection processing on the second mechanism description data under the preset mechanism classification, and the process of performing anomaly detection processing on the first mechanism description data by the model to be trained belongs to the same technical concept, so that the anomaly detection model is used for performing anomaly detection processing on the second mechanism description data by referring to the process of performing anomaly detection processing on the first mechanism description data by the model to be trained.

In an optional implementation manner provided in this embodiment, the abnormality detection model performs abnormality detection processing in the following manner:

under the condition that the logout information of the mechanism to which the second mechanism description data belongs is detected, carrying out invalid marking on the second mechanism description data, and taking an invalid marking result as the abnormality detection result; and/or the number of the groups of groups,

and under the condition that the asset transaction opening information of the mechanism to which the second mechanism description data belongs is detected, carrying out association processing on the second mechanism description data and the asset transaction opening information, and taking an association result as the abnormality detection result.

The logout information of the mechanism in the embodiment includes related information of logout of the mechanism; the asset transaction open message includes a marketing message for an organization.

In addition, in order to promote the detection diversity and comprehensiveness of abnormality detection by the abnormality detection model; in another alternative implementation manner provided in this embodiment, the abnormality detection model performs abnormality detection processing in the following manner:

calculating a quality abnormality index of the current mechanism description data according to the rest mechanism description data except the current mechanism description data in the second mechanism description data;

According to the association degree of the current mechanism description data and the residual mechanism description data, weighting calculation is carried out on the quality abnormality index to obtain a target quality abnormality index of the current mechanism description data; after that, the target quality abnormality index of the second mechanism descriptive data may be taken as an abnormality detection result of the second mechanism descriptive data.

Further, in order to promote convenience of model training of the anomaly detection model; in an optional implementation manner provided in this embodiment, after performing weighted calculation on the quality anomaly index according to the association degree between the current mechanism description data and the remaining mechanism description data to obtain the target quality anomaly index of the current mechanism description data, the following operations are further performed:

if the target quality abnormal index of the second mechanism description data is larger than a preset abnormal index threshold value, calculating a difference value between the target quality abnormal index of the second mechanism description data and the preset abnormal index threshold value;

and judging whether the difference is smaller than a difference threshold, if so, storing the second mechanism description data into a training sample set to perform model training on the anomaly detection model.

Specifically, under the condition that the target quality abnormal index of the second mechanism description data is larger than a preset abnormal index threshold, a difference value between the target quality abnormal index and the preset abnormal index threshold can be calculated, if the difference value is smaller than the difference value threshold, the target quality abnormal index representing the second mechanism description data meets the requirements, the second mechanism description data can be stored in a training sample set, and model training is performed on an abnormal detection model; if the difference value is greater than or equal to the difference value threshold value, the quality abnormality problem of the second mechanism description data may be more, and the model training can be actually performed according to the mechanism description data with moderate target quality abnormality index, so that the model training efficiency is improved, and the processing is not needed.

As described above, the process of performing the abnormality detection processing on the second mechanism description data under the preset mechanism classification by the abnormality detection model and the process of performing the abnormality detection processing on the first mechanism description data by the model to be trained belong to the same technical concept, so that the abnormality detection model may further determine an abnormality class of the second mechanism description data, determine a detection class of the second mechanism description data, calculate a null index and a quality abnormality index of the second mechanism description data, calculate a null index or a quality abnormality index of the second mechanism description data, or any one or more of these.

In the specific execution process, after the model training is performed on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, mechanism description data to be detected can be obtained, and the abnormality detection model matched with at least one of the data type, the mechanism classification and the description type of the mechanism description data is input into the mechanism description data to perform abnormality detection processing, so that an abnormality detection result of the mechanism description data is obtained.

In summary, in the one or more anomaly detection model training methods provided in the present embodiment, first mechanism description data for performing model training under a preset mechanism classification is obtained, if the preset mechanism classification is a specific mechanism classification, a first anomaly detection unit is selected from a set of anomaly detection units, in the case that the description type is a prediction type, a second anomaly detection unit is selected from the set of anomaly detection units, and the first anomaly detection unit and the second anomaly detection unit are used as anomaly detection units, in the case that the description type is not the prediction type, the first anomaly detection unit is used as an anomaly detection unit; secondly, if the preset mechanism classification is not the specific mechanism classification and the data type of the first mechanism description data is the text type, selecting a second abnormality detection unit from the abnormality detection unit set as an abnormality detection unit, and if the preset mechanism classification is not the specific mechanism classification and the data type of the first mechanism description data is not the text type, selecting a third abnormality detection unit from the abnormality detection unit set as an abnormality detection unit; and finally, constructing a model to be trained based on the anomaly detection unit, and carrying out model training on the model to be trained according to the first mechanism description data to obtain an anomaly detection model so as to carry out anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model.

The following further describes the abnormality detection model training method provided in this embodiment by taking the application of the abnormality detection model training method provided in this embodiment to a model training scene as an example, and referring to fig. 3, the abnormality detection model training method applied to the model training scene specifically includes the following steps.

Step S302, first organization description data of model training under the preset organization classification is obtained.

Step S304, if the preset mechanism is classified as a specific mechanism, selecting a first abnormality detection unit from the abnormality detection unit set, selecting a second abnormality detection unit from the abnormality detection unit set if the description type is the prediction type, and taking the first abnormality detection unit and the second abnormality detection unit as the abnormality detection units.

In step S306, in the case where the description type is not the prediction type, the first abnormality detection unit is taken as the abnormality detection unit.

In step S308, if the preset organization classification is not the specific organization classification and the data type of the first organization description data is the text type, selecting the second anomaly detection unit from the anomaly detection unit set as the anomaly detection unit.

In step S310, if the preset organization classification is not the specific organization classification and the data type of the first organization description data is not the text type, a third abnormality detection unit is selected from the set of abnormality detection units as an abnormality detection unit.

Step S312, a model to be trained is built based on the anomaly detection unit.

Step S314, the first mechanism description data is input into the model to be trained for abnormality detection processing, and an abnormality detection result of the first mechanism description data is obtained.

Step S316, training loss is calculated based on the abnormality detection result and the label data corresponding to the first mechanism description data, and parameter adjustment is performed on the model to be trained based on the training loss, so that abnormality detection processing is performed on the second mechanism description data under the preset mechanism classification through the obtained abnormality detection model.

One or more embodiments of an anomaly detection processing method provided in the present specification are as follows:

referring to fig. 4, the abnormality detection processing method provided in the present embodiment specifically includes steps S402 to S404.

Step S402, acquiring mechanism description data to be detected.

The organization description data refers to description data describing the organization in one or more description types, such as credit rating and credit score of the organization described by the credit description type; revenue description data of a mechanism described from the revenue description type; discipline description data of a mechanism described from discipline description types, and the like; the institution description data may be an institution tag. The mechanism description data may be the mechanism description data of a single mechanism or the mechanism description data of a plurality of mechanisms.

And step S404, inputting the mechanism description data into a mechanism classification model of the mechanism description data and an abnormality detection model with at least one of data types matched with description types for abnormality detection processing, and obtaining an abnormality detection result of the mechanism description data.

The description type of the mechanism description data in this embodiment includes a description dimension for describing the mechanism; such as a credit description type, a revenue description type, a punishment description type and/or a scale description type, the description types of the mechanism description data are merely illustrative, and the description types of the mechanism description data can be one or more; in addition, the description type may further include a statistics type and/or a prediction type, where the statistics type refers to a statistics type for counting data that has occurred at present in an organization, for example, the description type of the organization description data is a statistics type if the organization description data is a personnel scale, the prediction type refers to a prediction type for predicting data that has not occurred in an organization, for example, the organization description data is a prediction type if the organization description data is a company suitable personnel development, and the description type of the organization description data is a prediction type.

The data types of the mechanism description data include image types, text types, voice types, and/or video types. The mechanism classification refers to preset mechanism classification; the institution classification comprises a institution classification obtained by dividing the field in which the institution is engaged, for example, the institution classification can be one of a medical institution, a financial institution, a catering institution and the like; in addition, the organization classification may further include an organization classification obtained by dividing according to an organization scale or an organization classification obtained by dividing according to a geographical area to which the organization belongs, and the organization classification may also include an organization classification obtained by dividing according to a field to which the organization is engaged, an organization scale, and/or a geographical area to which the organization belongs; the facility scale may be a facility personnel scale, a facility asset scale, and/or a facility office scale, among others.

Optionally, the anomaly detection model is obtained after selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

In practical application, as the data types of the mechanism description data are various, the description types of the mechanism description data are complex and changeable, and the mechanism classification of the mechanism to which the mechanism description data belongs is more, if the mechanism description data are subjected to anomaly detection by adopting the same anomaly detection model, the detection precision of the anomaly detection model is probably not high, and the training requirement on the anomaly detection model is also higher, aiming at the detection precision and the detection flexibility of the anomaly detection model are improved, the mechanism description data can be input into the mechanism classification of the mechanism description data, and the anomaly detection processing can be carried out on the anomaly detection model with at least one of the data types and the description types, so that the anomaly detection result of the mechanism description data is obtained; specifically, an abnormality detection model in which the mechanism classification and the data type of the mechanism description data are matched with at least one of the description type (at least one of the mechanism classification and the data type of the mechanism description data) may be determined, and then the mechanism description data is input into the abnormality detection model to perform abnormality detection processing, so as to obtain an abnormality detection result of the mechanism description data.

In determining the organization classification of the organization description data and the anomaly detection model for which the data type matches at least one of the description types, the following operations may be performed: if the mechanism is classified as a specific mechanism and the description type is a prediction type, determining that the matched abnormality detection model is a first abnormality detection model, and if the mechanism is classified as a specific mechanism and the description type is not a prediction type, determining that the matched abnormality detection model is a second abnormality detection model; alternatively, the first abnormality detection model is constructed based on the first abnormality detection unit and the second abnormality detection unit, and the second abnormality detection model is constructed based on the first abnormality detection unit. The first anomaly detection unit may be a decision tree and the second anomaly detection unit may be a random forest.

The specific institution classification refers to a specified institution classification, which may include a medical institution or a financial institution, and may be determined according to an actual application scenario, which is not particularly limited in this embodiment. The prediction type refers to that the description type of the organization description data is the prediction type for predicting the non-occurrence, for example, the organization description data is suitable for personnel development in the future of the xx company or unsuitable for investment in the future of the xx company, and the description type of the organization description data for predicting the future or the non-occurrence belongs to the prediction type.

In determining the organization classification of the organization description data and the abnormality detection model whose data type matches at least one of the description types, the following operations may also be performed: and if the mechanism classification is not the specific mechanism classification and the data type of the mechanism description data is the text type, determining that the matched abnormality detection model is a third abnormality detection model, and if the mechanism classification is not the specific mechanism classification and the data type of the mechanism description data is not the text type, determining that the matched abnormality detection model is a fourth abnormality detection model. Optionally, a third anomaly detection model is constructed based on the second anomaly detection unit; the fourth abnormality detection model is constructed based on a third abnormality detection unit, which may be a neural network.

In addition, in determining the mechanism classification of the mechanism description data and the abnormality detection model whose data type matches at least one of the description types, the following operations may also be performed: if the mechanism classification is a specific mechanism classification, determining the matched abnormality detection model as a second abnormality detection model; if the mechanism classification is not the specific mechanism classification, determining that the matched abnormality detection model is a third abnormality detection model under the condition that the data type of the mechanism description data is the text type; if the organization classification is not the specific organization classification, determining the matched abnormality detection model as a fourth abnormality detection model in the case that the data type of the organization description data is an image type, a voice type and/or a video type.

Note that, since the above-described mechanism classification belongs to different execution procedures when the mechanism classification is a specific mechanism classification or when the mechanism classification is not a specific mechanism classification, any one or more of the three execution procedures may be selected to determine the abnormality detection model.

In a specific implementation, in an optional implementation manner provided in this embodiment, the performing an anomaly detection process includes:

extracting key data from initial data of a mechanism to which the mechanism description data belongs according to the description type of the mechanism description data;

and determining an abnormality category of the mechanism description data based on the key data as the abnormality detection result.

On the basis of this, in an optional implementation manner provided in this embodiment, after the performing of the abnormality detection processing on the mechanism classification of inputting the mechanism description data into the mechanism description data and the abnormality detection model with the data type matching at least one of the description types, the method further includes:

determining correction contents for correcting the mechanism description data according to the abnormal category of the mechanism description data;

and carrying out correction processing on the mechanism description data according to the correction content to obtain target mechanism description data.

In another optional implementation manner provided in this embodiment, the performing an anomaly detection process includes:

determining intermediate mechanism description data in which no null value exists in the mechanism description data;

calculating a null index of the mechanism description data based on the mechanism description data and the intermediate mechanism description data;

if the null index is larger than a null index threshold, determining null mechanism description data except the intermediate mechanism description data in the mechanism description data, performing null filling on the null mechanism description data to obtain target mechanism description data, and storing the target mechanism description data into a training sample set;

And if the quality abnormality index is greater than an abnormality index threshold, performing model training on a data generation model for generating the mechanism description data to obtain a trained data generation model.

In addition, the anomaly detection model can also determine any one or more of anomaly category of the mechanism description data, determination of detection category of the mechanism description data, calculation of null index and quality anomaly index of the mechanism description data, calculation of null index or quality anomaly index of the mechanism description data, determination of invalid marking result and determination of association result of the open information of the asset transaction in the anomaly detection process.

The process of performing the anomaly detection processing on the anomaly detection model is similar to the process of performing the anomaly detection processing on the model to be trained, and the anomaly detection processing is only required to be read.

It should be noted that, in this embodiment of the present application, the mechanism classification of inputting the mechanism description data into the mechanism description data and the anomaly detection model matching at least one of the data types and the description types perform anomaly detection processing, so as to obtain an anomaly detection result of the mechanism description data, and the model training is performed on the model to be trained according to the first mechanism description data in the previous embodiment of the present application to obtain an anomaly detection model, which is based on the same inventive concept, so that this embodiment may refer to the implementation of the foregoing anomaly detection model training method, and repeated details are omitted.

The embodiment of the abnormality detection model training device provided in the present specification is as follows:

in the foregoing embodiments, an anomaly detection model training method is provided, and an anomaly detection model training device is provided corresponding to the anomaly detection model training method, which is described below with reference to the accompanying drawings.

Referring to fig. 5, a schematic diagram of an embodiment of an anomaly detection model training device according to the present embodiment is shown.

Since the apparatus embodiments correspond to the method embodiments, the description is relatively simple, and the relevant portions should be referred to the corresponding descriptions of the method embodiments provided above. The device embodiments described below are merely illustrative.

The embodiment provides an anomaly detection model training device, including:

a data acquisition module 502 configured to acquire first institution description data for model training under a preset institution classification;

a model construction module 504 configured to select an abnormality detection unit from a set of abnormality detection units according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, and construct a model to be trained based on the abnormality detection unit;

the model training module 506 is configured to perform model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to perform abnormality detection processing on second mechanism description data under the preset mechanism classification through the abnormality detection model;

Wherein, each abnormal detection unit in the abnormal detection unit set is extracted and obtained in a pre-training detection model.

An embodiment of an abnormality detection processing device provided in the present specification is as follows:

in the above-described embodiments, an abnormality detection processing method is provided, and an abnormality detection processing apparatus is provided corresponding thereto, as described below with reference to the accompanying drawings.

Referring to fig. 6, a schematic diagram of an embodiment of an abnormality detection processing apparatus provided in this embodiment is shown.

The present embodiment provides an abnormality detection processing apparatus including:

a description data acquisition module 602 configured to acquire mechanism description data to be detected;

an anomaly processing module 604 configured to input the mechanism description data into a mechanism classification of the mechanism description data and perform anomaly detection processing on an anomaly detection model with a data type matching at least one of description types, to obtain an anomaly detection result of the mechanism description data;

The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on the mechanism description data sample.

An embodiment of an anomaly detection model training device provided in the present specification is as follows:

corresponding to the above-described method for training an anomaly detection model, one or more embodiments of the present disclosure further provide an anomaly detection model training device for performing the above-provided method for training an anomaly detection model, and fig. 7 is a schematic structural diagram of the anomaly detection model training device provided by the one or more embodiments of the present disclosure, based on the same technical concept.

The embodiment provides an anomaly detection model training device, including:

as shown in FIG. 7, the anomaly detection model training device may vary considerably in configuration or performance, and may include one or more processors 701 and memory 702, where memory 702 may store one or more stored applications or data. Wherein the memory 702 may be transient storage or persistent storage. The application program stored in memory 702 may include one or more modules (not shown in the figures), each of which may include a series of computer-executable instructions in the anomaly detection model training device. Still further, the processor 701 may be configured to communicate with the memory 702 and execute a series of computer executable instructions in the memory 702 on the anomaly detection model training device. The anomaly detection model training device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input/output interfaces 705, one or more keyboards 706, and the like.

In a particular embodiment, the anomaly detection model training device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the anomaly detection model training device, and the execution of the one or more programs by the one or more processors comprises computer-executable instructions for:

acquiring first mechanism description data for model training under preset mechanism classification;

selecting an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit;

performing model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to perform abnormality detection processing on second mechanism description data under the preset mechanism classification through the abnormality detection model;

An embodiment of an abnormality detection processing apparatus provided in the present specification is as follows:

in correspondence to the above-described abnormality detection processing method, one or more embodiments of the present disclosure further provide an abnormality detection processing apparatus for executing the above-provided abnormality detection processing method, and fig. 8 is a schematic structural diagram of the abnormality detection processing apparatus provided by the one or more embodiments of the present disclosure, based on the same technical concept.

The abnormality detection processing apparatus provided in the present embodiment includes:

as shown in fig. 8, the abnormality detection processing device may have a relatively large difference due to different configurations or performances, and may include one or more processors 801 and a memory 802, where one or more storage applications or data may be stored in the memory 802. Wherein the memory 802 may be transient storage or persistent storage. The application programs stored in memory 802 may include one or more modules (not shown in the figures), each of which may include a series of computer-executable instructions in the anomaly detection processing device. Still further, the processor 801 may be configured to communicate with a memory 802 to execute a series of computer executable instructions in the memory 802 on an anomaly detection processing device. The anomaly detection processing device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input/output interfaces 805, one or more keyboards 806, and the like.

In one particular embodiment, an anomaly detection processing apparatus includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the anomaly detection processing apparatus, and execution of the one or more programs by one or more processors comprises computer-executable instructions for:

acquiring mechanism description data to be detected;

inputting the mechanism description data into a mechanism classification model of the mechanism description data and an abnormality detection model with at least one of data types matched with description types for abnormality detection processing to obtain an abnormality detection result of the mechanism description data;

An embodiment of a storage medium provided in the present specification is as follows:

corresponding to the above-described abnormality detection model training method, one or more embodiments of the present disclosure further provide a storage medium based on the same technical concept.

The storage medium provided in this embodiment is configured to store computer executable instructions that, when executed by a processor, implement the following flow:

It should be noted that, in the present specification, an embodiment of a storage medium and an embodiment of an anomaly detection model training method in the present specification are based on the same inventive concept, so that a specific implementation of the embodiment may refer to an implementation of the foregoing corresponding method, and a repetition is omitted.

Another storage medium embodiment provided in this specification is as follows:

in correspondence to the above-described abnormality detection processing method, one or more embodiments of the present specification also provide another storage medium based on the same technical idea.

acquiring mechanism description data to be detected;

It should be noted that, in the present specification, an embodiment of another storage medium and an embodiment of an anomaly detection processing method in the present specification are based on the same inventive concept, so that a specific implementation of the embodiment may refer to an implementation of the foregoing corresponding method, and a repetition is omitted.

In this specification, each embodiment is described in a progressive manner, and the same or similar parts of each embodiment are referred to each other, and each embodiment focuses on the differences from other embodiments, for example, an apparatus embodiment, and a storage medium embodiment, which are all similar to a method embodiment, so that description is relatively simple, and relevant content in reading apparatus embodiments, and storage medium embodiments is referred to the part description of the method embodiment.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

In the 30 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each unit may be implemented in the same piece or pieces of software and/or hardware when implementing the embodiments of the present specification.

One skilled in the relevant art will recognize that one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable anomaly detection model training device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable anomaly detection model training device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable anomaly detection model training device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

One or more embodiments of the present specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is by way of example only and is not intended to limit the present disclosure. Various modifications and changes may occur to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. that fall within the spirit and principles of the present document are intended to be included within the scope of the claims of the present document.

Claims

1. An anomaly detection model training method comprising:

2. The anomaly detection model training method of claim 1, the model training the model to be trained according to the first mechanism description data, comprising:

3. The training method of an anomaly detection model according to claim 2, wherein the model to be trained performs anomaly detection processing by:

4. The training method of an anomaly detection model according to claim 2, wherein the model to be trained performs anomaly detection processing by:

determining that no null value exists in the first mechanism description data;

5. The abnormality detection model training method according to claim 4, said calculating a quality abnormality index of said intermediate mechanism description data, and after executing said null index and said quality abnormality index as said abnormality detection result operation, further comprising:

6. The abnormality detection model training method according to claim 1, wherein the selecting an abnormality detection unit from a set of abnormality detection units according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, comprises:

in the case where the description type is a prediction type, a second abnormality detection unit is selected from the set of abnormality detection units, and the first abnormality detection unit and the second abnormality detection unit are taken as the abnormality detection units.

7. The anomaly detection model training method of claim 6, the selecting anomaly detection units from a set of anomaly detection units according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, further comprising:

8. The anomaly detection model training method of claim 1, the performing anomaly detection processing comprising:

under the condition that the logout information of the mechanism to which the second mechanism description data belongs is detected, carrying out invalid marking on the second mechanism description data, and taking an invalid marking result as the abnormality detection result;

9. The anomaly detection model training method of claim 1, the performing anomaly detection processing comprising:

and carrying out weighted calculation on the quality anomaly index according to the association degree of the current mechanism description data and the residual mechanism description data to obtain the target quality anomaly index of the current mechanism description data.

10. The anomaly detection model training method according to claim 9, wherein the weighting calculation is performed on the quality anomaly index according to the association degree of the current mechanism description data and the remaining mechanism description data, and after the target quality anomaly index operation to obtain the current mechanism description data is performed, further comprising:

11. An anomaly detection processing method, comprising:

acquiring mechanism description data to be detected;

12. The abnormality detection processing method according to claim 11, said performing abnormality detection processing comprising:

13. The abnormality detection processing method according to claim 12, wherein the step of inputting the mechanism description data into the mechanism classification of the mechanism description data and the abnormality detection model having a data type matching at least one of the description types performs abnormality detection processing, and further comprises, after the step of obtaining the abnormality detection result of the mechanism description data is performed:

14. The abnormality detection processing method according to claim 11, said performing abnormality detection processing comprising:

15. The abnormality detection processing method according to claim 14, wherein the step of inputting the mechanism description data into the mechanism classification of the mechanism description data and the abnormality detection model having a data type matching at least one of the description types performs abnormality detection processing, and further comprises, after the step of obtaining the abnormality detection result of the mechanism description data is performed:

16. An anomaly detection model training device comprising:

the data acquisition module is configured to acquire first mechanism description data for model training under preset mechanism classification;

the model construction module is configured to select an abnormality detection unit from an abnormality detection unit set according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, and construct a model to be trained based on the abnormality detection unit;

the model training module is configured to perform model training on the model to be trained according to the first mechanism description data to obtain an abnormality detection model, so as to perform abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model;

17. An abnormality detection processing apparatus comprising:

the description data acquisition module is configured to acquire mechanism description data to be detected;

the abnormality processing module is configured to input the mechanism description data into a mechanism classification of the mechanism description data and perform abnormality detection processing on an abnormality detection model with at least one of data types matched with description types, so as to obtain an abnormality detection result of the mechanism description data;

18. An anomaly detection model training device comprising:

a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to:

19. An abnormality detection processing apparatus comprising:

acquiring mechanism description data to be detected;

20. A storage medium storing computer-executable instructions that when executed by a processor implement the following:

21. A storage medium storing computer-executable instructions that when executed by a processor implement the following:

acquiring mechanism description data to be detected;