CN113095446A - Abnormal behavior sample generation method and system - Google Patents

Abstract

The invention discloses a method and a system for generating an abnormal behavior sample, wherein a sample with the minimum sum of distances with the characteristics of other samples, namely the mass center of the base class is searched from the base class with sufficient samples, and the mass center is used as the center of the base class; calculating Euclidean distances between the current new class sample and centers of all base classes, and selecting k base classes closest to the new class sample as nearest neighbor base classes; calculating the average value of the centers of the k base classes to serve as an approximate class center, and calculating the midpoint between the approximate class center and the new class sample to serve as a final new class distribution center; and constructing a sample generator based on Gaussian distribution by using the final new class distribution so as to randomly generate virtual samples. The method not only estimates the distribution center of the new class more accurately, but also effectively solves the problem of poor classification effect on similar classes in the learning of few samples.

Description

Abnormal behavior sample generation method and system

technical field

The invention relates to the field of deep learning, in particular to a method and system for generating abnormal behavior samples.

Background technique

At present, the issue of campus safety has received more and more attention from all walks of life, and behaviors endangering campus safety such as trampling, bullying, and fighting frequently occur. At present, campus security incidents in our country are frequent, and the difficulty of prevention and control is increasing. Campus security is facing unprecedented severe challenges. However, due to the concealment, suddenness and frequent occurrence of campus security incidents, it is difficult to implement high-efficiency campus security and early-prevention campus security mechanisms. In recent years, the development of computer vision technology has provided the possibility for the safety monitoring and early warning of abnormal behavior on campus. Abnormal behavior identification is highly efficient and accurate, keeping the hidden dangers of campus security incidents in the bud, and actively warning and intervening in advance. Therefore, abnormal behavior recognition and intelligent early warning based on computer vision technology has become an important research problem in the fields of campus security monitoring, anti-terrorism and stability maintenance, and early warning of mass incidents. Previous methods for generating abnormal behavior samples are mainly based on deep learning, which requires the support of large-scale datasets.

The great success of deep learning in the field of image processing relies heavily on the emergence of large-scale labeled datasets, but when the number of samples is limited, deep learning models are prone to overfitting. Therefore, few-shot learning is a very promising and challenging computer vision direction. It imitates the human thinking mode of recognizing new things, that is, it can accurately identify an object that has never been seen through a few examples, which seems to be an effective way to teach machines how to recognize new things like humans. , further narrowing the distance between artificial intelligence and human intelligence ^[1] .

However, how to learn some knowledge from a limited variety and number of samples and deduce it into a new category is very challenging and practical. Most of the research on this problem is still on the classification task ^[2] , and the existing methods are still unsatisfactory, far from reaching the level of industrial application. On the one hand, the lack of samples will inevitably lead to the problem of poor model generalization ability; on the other hand, it is extremely difficult to accurately estimate the class distribution with a small number of samples.

The purpose of few-shot learning is to improve the utilization of samples, and explore how to use a small number of labeled samples to make the model achieve performance comparable to or even better than previous deep learning models. For the problem of the scarcity of labeled samples, there are two ideas. One is to use the labeled part of the samples to label the unlabeled samples when there are a large number of unlabeled samples; the other is to use the labeled samples when the samples are limited. , using labeled samples to generate a large number of virtual samples. The problem of the former is not the amount of data, but the problem of sample labeling, which can be effectively alleviated if combined with relevant experts or algorithms. But the latter is a typical problem with small sample size and is relatively intractable. However, if we assume that the class distribution obeys a Gaussian or a Gaussian-like distribution ^[3] , then we only need to know the center and variation range of the class distribution to estimate the distribution of the entire class, however, it is extremely difficult to estimate the class distribution from a limited sample, And the key is whether the class center can be estimated accurately.

Distribution correction of new classes with the help of base class statistics ^[4] is a very novel and effective few-shot classification method. However, in the process of correcting the distribution of similar classes, there may be a problem that the two corrected distribution centers are too close, resulting in poor classification of similar classes, which in turn affects the final few-sample classification performance.

Taking one-shot as an example, for any shot in each few-shot task, we call it a support image, assuming its feature vector is s ={ s ₁ , s ₂ ,… s _m } , m represents the feature dimension and belongs to the category c. The existing distribution correction method first searches for the k nearest base class centers x _i ={ x _i1 , x _i2 ,… x _im }, ( i = 1, 2, … k ), then average them with the feature vector s of the support image, and finally, use this mean as an estimate of the center of the new class distribution to which s belongs.

Although the effect of the above method is significant, in some special cases, such as when the support sample s deviates greatly from its true distribution, the corrected distribution center may also deviate to varying degrees. We call this phenomenon the center correction offset. (Center Calibration Deviation). Dummy samples generated from off-center distribution centers may lack reliability, and if two corrected distribution centers are too close, the samples generated based on them may overlap, a phenomenon we call Sampling Confusion. The main reason for the above two phenomena is the large deviation in the estimation of the distribution center of the new class.

(1) In summary, the traditional method requires a large amount of labeled data, which not only requires a lot of manpower and material resources, but also due to the scarcity of some abnormal behavior data, the data set is likely to have long-tailed distribution problems.

(2) It is difficult for existing methods to accurately estimate the distribution center of new classes from limited samples.

Existing methods may cross or overlap the distribution estimates of similar abnormal behavior classes, and cannot effectively distinguish similar classes.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a method and system for generating abnormal behavior samples in view of the deficiencies of the prior art, so as to improve the accuracy of estimating the distribution of new classes.

In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is: a method for generating abnormal behavior samples, comprising the following steps:

S1. Search the sample with the smallest sum of distances from the features of other samples from the base class with sufficient samples, that is, the centroid of the base class, and use the centroid as the base class center;

S2. Calculate the Euclidean distance between the current new class sample and the centers of all base classes, and select the k base classes closest to the new class sample as the nearest neighbor base class;

S3. Calculate the average value of the centers of the k base classes as the approximate class center, and calculate the midpoint between the approximate class center and the new class sample as the final new class distribution center;

S4, using the final new class distribution to construct a Gaussian distribution-based sample generator to randomly generate virtual samples.

The great success of deep learning in the field of image processing has benefited from the emergence of large-scale datasets. However, when the sample size is limited, the model can easily overfit, which seriously affects the performance of the abnormal behavior classification model. Due to the particularity and scarcity of abnormal behaviors, it is difficult for us to capture a large number of samples. For such pain points, we propose an abnormal behavior sample generation method. All operations on samples are at the level of extracted features, because the extracted features filter out most of the interference information and improve the reliability of generated samples. Then, for each sample from the new class, the k nearest base class centers are searched, and the average of these base class centers is taken as the approximate class center, because the approximate class is both characteristically and semantically related to this new class. Classes are relatively similar, and their feature changes and distributions may be extremely similar. Next, the midpoint between the approximate class center and the new class sample is calculated as an estimate of the new class center. This is to calibrate the distribution of the new class, so that the estimated new class center is closer to the real position, thereby constructing a Gaussian distribution-based Sample generator to generate a large number of virtual samples of abnormal behavior. The abnormal behavior sample generation method of the present invention is different from the existing data generation method. The present invention is based on data distribution estimation, which is theoretically closer to the real data distribution state, so the generated samples are more reliable. In addition, the method of the present invention can be built on any common feature extractor and sample generator, does not require any additional parameters, only operates on the features of the sample, the algorithm is simple, and the implementation is easy.

The present invention uses ResNet50 to perform feature extraction on samples. Different from ordinary networks, ResNet50 introduces skip connections, improves information flow, and avoids the problem of gradient disappearance caused by the network being too deep.

The generated virtual samples are input into the logistic regression classifier for training, and the final abnormal behavior classifier is obtained.

The present invention selects the logistic regression classifier because of its simple structure, easy implementation and good effect, and is more conducive to verifying the reliability of the samples generated by the present invention.

；利用下式更新当前次迭代的锚点值：

，直至 找出到其他样本点的距离之和最小的锚点，即得到基类中心；其中，x ₀为上一轮迭代后的锚 点，

为更新后的锚点，x _i为同类的其他样本，n为样本数，α为学习率。该过程迭代地搜索与 其他样本点的距离之和最小的样本点，将整个类分布看作一个匀质几何体，这样找到的基 类中心具有一定的数学理论支撑。 In step S1, the calculation process of the base class center includes: using the following formula to calculate the gradient of the current iteration:

; Update the anchor value of the current iteration using the following formula:

, until the anchor point with the smallest sum of distances to other sample points is found, that is, the base class center is obtained; where x ₀ is the anchor point after the previous iteration,

is the updated anchor point, x _i is other samples of the same class, n is the number of samples, and α is the learning rate. This process iteratively searches for the sample point with the smallest sum of distances from other sample points, and regards the entire class distribution as a homogeneous geometry, so that the base class center found has certain mathematical theoretical support.

In step S4, the final new class distribution center is used as the input of the Gaussian distribution function, thereby constructing a sample generator based on the Gaussian distribution, and obtaining a randomly generated virtual sample.

The present invention takes the estimated new class center as the mean value of the Gaussian distribution function, and takes the random value between 0.2 and 0.5 as the variance of the function, because a large number of experimental results show that the random number between 0.2 and 0.5 has the best effect.

After step S4, the method further includes: using the randomly generated virtual samples to train a logistic regression classifier to obtain an abnormal behavior classifier.

The present invention also provides an abnormal behavior sample generation system, comprising a computer device configured or programmed to perform the steps of the above method.

Compared with the prior art, the present invention has the beneficial effects that the present invention not only estimates the distribution center of the new class more accurately, but also effectively solves the problem of poor classification of similar classes in few-sample learning. In addition, the class centroid estimation algorithm proposed by the present invention can more accurately estimate the distribution of the base class by searching the centroid of the base class. Compared with the existing distribution correction methods, the rationality and effectiveness of the method of the present invention can be proved by the error theory, and a relatively accurate base class distribution estimation is beneficial to further improve the reliability of the subsequent estimation of the new class distribution.

Description of drawings

FIG. 1 is a network framework for identifying abnormal behavior based on few samples according to an embodiment of the present invention.

FIG. 2 is a visualization diagram of the principle of a two-order center estimation algorithm according to an embodiment of the present invention.

Fig. 3(a) and Fig. 3(b) are t-SNE visualization diagrams of the real new class distribution according to the embodiment of the present invention and the new class distribution estimated by the present invention, respectively.

Detailed ways

Embodiments of the present invention include two-order center estimation (TCE) and class centroid estimation (CCE). First, ResNet50 is used as a feature extractor to extract feature vectors from the pictures in the dataset; then, the feature vectors are input into the CCE module to estimate the center of the base class, and enough samples are used to estimate the center of the base class, so that the new class ( Then, the output of the CCE module is used as the input of the TCE module to estimate the center of the new class, and TCE is used to solve the problem of calibration bias in order to make the estimated class center closer to the actual location . Based on the estimated class distribution, we construct a Gaussian distribution-based sample generator to generate enough dummy samples for training a logistic regression classifier, resulting in an abnormal behavior classifier. The principle of the above method is shown in Figure 1.

Distribution correction of new classes with the help of base class statistics ^[4] is a very novel and effective few-shot classification method. However, in the process of correcting the distribution of similar classes, there may be a problem that the two corrected distribution centers are too close, resulting in poor classification of similar classes, which in turn affects the final few-sample classification performance.

Taking one-shot as an example, for any shot in each few-shot task, we call it a support image, assuming its feature vector is s ={ s ₁ , s ₂ ,… s _m }, m represents the feature dimension and belongs to the category c. The existing distribution correction method first searches for the k nearest base class centers x _i ={ x _i1 , x _i2 ,… x _im }, ( i =1,2,… k ), then average them with the feature vector s of this support image, and finally, use this mean as an estimate of the center of the new class distribution to which s belongs.

Although the effect of the above method is significant, in some special cases, such as when the support sample s deviates greatly from its true distribution, the corrected distribution center may also deviate to varying degrees. We call this phenomenon the center correction offset. (Center Calibration Deviation). Dummy samples generated from off-center distribution centers may lack reliability, and if two corrected distribution centers are too close, the samples generated based on them may overlap, a phenomenon we call Sampling Confusion. The main reason for the above two phenomena is the large deviation in the estimation of the distribution center of the new class. In response to this problem, the present invention proposes a two-order center method to more accurately estimate the distribution center of the new class. Specifically, in the first stage, the center of the k base class distribution centers closest to the support sample s is estimated; in the second stage, the center between the center of the first stage and the support sample s is estimated, that is, the center of the center , the principle visualization of the method is shown in Figure 2.

To accurately correct the distribution of a new class with only a few samples, it is necessary to ensure that the distribution estimation of the base class is reliable enough, because estimating the distribution center of the new class requires the distribution data of the nearest neighbor base class. However, the number and distribution of base class samples are irregular, which brings great challenges to estimating their distribution centers. In order to accurately estimate the distribution center of the base class, we propose a class centroid estimation algorithm, which can iteratively search for the sample point in a class with the smallest sum of distances to other sample points, that is, the class centroid.

（1）

(1)

（2）

(2)

为新的锚点（anchor point），x _i为同类的其他样本，n为样本数，α为学习率，这里取α=0.03。 Among them, x ₀ is the anchor point after the previous iteration,

is the new anchor point, x _i is other samples of the same type, n is the number of samples, α is the learning rate, here α=0.03.

It is worth noting that the anchor point x ₀ of each iteration comes from the result of the previous gradient descent. In each round of iteration, the gradient is calculated according to formula (1), and then the value of the anchor point is updated according to formula (2), and finally the anchor point with the smallest sum of distances to other sample points is found as the class Estimation of the centroid.

Figure 3(a) is the real new class classification, and Figure 3(b) is the new class distribution estimated by the present invention. Obviously, the real new class distribution crosses or overlaps without any intervention, while the new class distribution estimated by the method of the present invention has greater inter-class spacing and intra-class compactness, Easier to categorize.

The invention firstly uses the class centroid estimation algorithm to estimate the distribution of the base class, then uses the distribution data of the base class to estimate the distribution of the new class, and finally generates a large number of virtual samples based on the estimated result to train an abnormal behavior classifier. The specific process is as follows:

Step 1: Randomly sample 5 categories of image samples from the dataset to form a few-sample task, use ResNet50 as a feature extractor, perform feature extraction operations on the images in the dataset used, and obtain the feature vectors of the corresponding images respectively; use the class centroid The estimation algorithm (CCE) searches for the sample with the smallest sum of distances to other sample features from the sufficient samples of the base class, that is, the centroid of the base class, and uses it as the center estimate of the base class;

Step 2: For each new class image sample in the few-sample task, calculate the Euclidean distance between the new class sample and all the base class centers obtained in the previous step, as the distance measure between it and each base class, and select the distance from it. The nearest k base classes are used as the nearest neighbor base classes;

Step 3: With the help of the distribution centers of the k nearest neighbor base classes, we use the two-order center estimation algorithm (TCE) to estimate the distribution centers of the new class in two stages: In the first stage, calculate the center of all nearest neighbor base classes. The mean value, as the approximate class center for samples of this new class. In the second stage, the midpoint between the approximate class center and the new class sample is calculated as the final estimate of the new class distribution center;

Step 4: Using the estimated new class distribution center, construct a sample generator based on the Gaussian distribution function, randomly generate a large number of virtual samples, and then use these generated virtual samples as the input of the logistic regression classifier to train an abnormal behavior classification device.

references

[1] Wang Y, Yao Q, Kwok J T, et al. Generalizing from a few examples: A survey on few-shot learning[J]. ACM Computing Surveys (CSUR), 2020, 53(3):1-34.

[2]Chen W Y, Liu Y C, Kira Z, et al. A Closer Look at Few-shotClassification[C]//International Conference on Learning Representations.2019.

[3]Hu Y, Gripon V, Pateux S. Leveraging the feature distribution intransfer-based few-shot learning[J]. arXiv preprint arXiv:2006.03806, 2020.

[4] Yang S, Liu L, Xu M. Free Lunch for Few-shot Learning: Distribution Calibration[J]. arXiv preprint arXiv:2101.06395, 2021.

Claims (8)

1. An abnormal behavior sample generation method is characterized by comprising the following steps:

s1, searching a sample with the minimum sum of distances between the sample and the characteristics of other samples from the base class with sufficient samples, namely the centroid of the base class, and taking the centroid as the center of the base class;

s2, calculating Euclidean distances between the current new class sample and centers of all base classes, and selecting k base classes closest to the new class sample as nearest neighbor base classes;

s3, calculating the average value of the centers of the k base classes as an approximate class center, and calculating the midpoint between the approximate class center and the new class sample as a final new class distribution center;

and S4, constructing a sample generator based on Gaussian distribution by using the final new class distribution so as to randomly generate virtual samples.

2. The abnormal behavior pattern generation method according to claim 1, wherein in step S1, the characteristics of the pattern are extracted using ResNet50 as a characteristic extractor.

3. The abnormal behavior sample generation method according to claim 1, further comprising:

and S5, taking the virtual sample as the input of a logistic regression classifier, and training the logistic regression classifier to obtain a final abnormal behavior classifier.

4. The abnormal behavior sample generation method according to claim 1, wherein in step S1, the base class center calculation process includes: calculating the gradient of the current iteration by using the following formula

(ii) a The anchor value for the current iteration is updated using:

until finding out the anchor point with the minimum sum of the distances to other sample points, namely obtaining a base center; wherein,x ₀is the anchor point after the previous round of iteration,

in order to be the anchor point after the update,x _i for the other samples of the same type,nis the number of samples, and α is the learning rate.

5. The abnormal behavior sample generation method according to claim 1, wherein in step S4, a sample generator based on gaussian distribution is constructed by taking the final new class distribution center as a mean value of the gaussian distribution function and taking a random value as a variance of the gaussian distribution function, so as to obtain the randomly generated virtual sample.

6. The abnormal behavior sample generation method according to claim 5, wherein the random value ranges from 0.2 to 0.5.

7. The abnormal behavior sample generation method according to any one of claims 1 to 6, further comprising, after step S4: and training a logistic regression classifier by using the randomly generated virtual samples to obtain an abnormal behavior classifier.

8. An abnormal behavior sample generation system, comprising a computer device; the computer device is configured or programmed for carrying out the steps of the method according to one of claims 1 to 7.

Images