CN110502992B - A fast face recognition method for fixed scene video based on relational graph - Google Patents

Abstract

The invention relates to a method for rapidly identifying a face of a fixed scene video based on a relational graph. The technical scheme is as follows: and (3) intercepting a face image and a non-face image from the fixed scene video, and forming a training data set D2 by the gray level images after gray level transformation. A strong classifier h (x) is trained using all the images in the training data set D2. And processing the human faces in the fixed scene videos with the same n segment lengths by using a graph algorithm to obtain a human relation graph in the fixed scene videos. And (3) carrying out face detection on each frame of image in the fixed scene video to be recognized by using a strong classifier h (x), matching the image with the face image in the face sample library D, and updating the unidentified face matching degree priority queue by using the relation atlas of the person in the fixed scene video. And if the maximum matching degree of the updated matching degree priority queue is greater than the threshold value T1 of the matching degree, the identification is successful, otherwise, the identification is abandoned. The invention has the characteristics of real-time monitoring, small parameter quantity and high accuracy of face recognition.

Description

Relation graph based fast face recognition method for fixed scene video

Technical Field

The invention belongs to the technical field of rapid face recognition. In particular to a method for rapidly identifying human faces of fixed scene videos based on a relational graph.

Technical Field

The fixed scene video device has the advantages of small occupied area, easiness in installation, high precision, low real-time performance, low energy consumption and the like, and becomes a preferred mode for building a security system.

The fixed scene video represents a video obtained by shooting a person or an object appearing in the visual field range of the shooting device under a specific background environment, such as a security video and an attendance system. At present, the monitoring and security system also needs workers to monitor on a video monitoring platform manually, when the workers face a large amount of fixed scene video data, the workers are difficult to have enough energy and time to observe specific contents in monitoring, and observation results are interfered by subjective factors to a great extent, so that the monitoring results are inaccurate, and timely coping strategies cannot be made. It is difficult to identify fast moving objects, such as people running quickly through the surveillance area. And a large amount of manpower and material resources are consumed by visual observation, which causes waste. There are also many face detection algorithms based on deep learning and machine learning, but since they consider many additional factors, such as environmental background, although the algorithms are made robust, the algorithms are easily interfered by irrelevant factors, so that the number of parameters in the algorithms becomes large.

The patent technology of the human face recognition optimization method based on the Adaboost algorithm (CN201510203079) needs to train a large number of human face sample images, although the accuracy is high, the parameter quantity is huge, and the real-time performance is difficult to achieve. The patent of "face recognition method and apparatus" (CN201410602236) is a technology that considers environmental background factors such as clothing features of people and shooting time of photos, increases data volume, introduces irrelevant features, and finally interferes the recognition result of the face. The patent technology of the 'face recognition method' (CN201310748379) uses too many face parameters, so that the algorithm efficiency is greatly reduced, and the instantaneity cannot be achieved.

Object of the Invention

The invention aims to overcome the defects of the prior art and provides a relation graph-based fast face recognition method for a fixed scene video, which has the advantages of real-time monitoring, small parameter quantity and high face recognition accuracy.

In order to achieve the purpose, the technical scheme adopted by the invention comprises the following steps:

step 1, data preprocessing

M face images and L non-face images are intercepted from a fixed scene video to form a training data set D2, wherein one face image represents a face. Firstly, converting N images in a training data set D2 into C multiplied by C gray images one by one, wherein N is M + L; k2 Gabor kernels are formed by K different proportionality coefficients and K1 different rotation angles, the gray level images are decomposed by K2 Gabor kernels, and each image in a training data set D2 is decomposed into K2 matrixes with the size of C multiplied by C; wherein, K2 is K1 × K, C, K, K1 and K2 are natural numbers; and (4) forming a face sample library D by all the face images in the training data set D2, and calibrating all the face images.

Step 2, training process of face detector

Step 2.1, establish training sample (x)^p，y^p)，p＝1，......，N。

x^pRepresenting K2 matrixes with the size of C multiplied by C obtained by Gabor decomposition of the p-th sample in the training data set D2;

y^ppresentation trainingWhether the p-th sample in the data set D2 is a face image;

if x^pIs a face image, then y^p1 is ═ 1; if x^pIs a non-face image, then y^p＝-1。

Step 2.2, initializing the weight of each training sample

Weights of the training samples

Wherein:

p＝1，......，N；

q＝1，......，K2；

n represents the number of sequence numbers of the last image in the training data set D2;

k2 represents the number of sequence numbers of the last feature of each training sample.

Step 2.3, setting

For the jth feature of the ith training sample, the jth feature f of each training sample is used_jFinding the optimal weak classifier h_j(f_j，θ_j，γ_j) Threshold parameter theta of_jClass parameter gamma_jSo that the classification error e_jAnd minimum.

In formulae (1) and (2): theta_jRepresenting a threshold parameter in the jth weak classifier;

γ_jrepresenting the category parameter in the jth weak classifier;

the weight value of each training sample is represented,

representation classifier h_j(f_j；θ_j，γ_j) Whether the jth feature in the ith training sample is classified as erroneous,

in formula (3): y isⁱ＝h_j(f_j；θ_j，γ_j) Indicating that the classification is correct;

yⁱ≠h_j(f_j；θ_j，γ_j) Indicating a classification error.

By using the jth feature f of each training sample_jTraining to obtain the optimal weak classifier h_j(f_j，θ_j，γ_j)：

In formula (4): theta_jRepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A threshold parameter of (d);

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) The category parameter of (2).

Step 2.4, updating the weight of each feature of the next iteration

If the jth feature of the ith training sample is utilized

Make the ith training sample by the jth optimal weak classifier h_j(f_j；θ_j，γ_j) If the identification is correct, the j +1 th feature of the ith training sample of the next iteration is

Weight of (2)

If the jth feature of the ith training sample is utilized

Make the ith training sample unable to be optimized by the jth weak classifier h_j(f_j；θ_j，γ_j) If the identification is correct, the j +1 th feature of the ith training sample of the next iteration is

Weight of (2)

In step 2.4: e.g. of the type_jRepresents h_j(f_j；θ_j，γ_j) The classification error of (2).

And step 2.5, repeating the step 2.3 and the step 2.4 until K2 features are traversed to obtain a final strong classifier h (x):

in formulae (5) and (6): x represents a frame of image in the fixed scene video to be identified;

f_jrepresents the jth feature in the total K2 features after K2 Gabor nuclei are decomposed in x;

θ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A threshold parameter of (d);

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A category parameter of (1);

α_jto represent the jth weak classifier h_j(f_j，θ_j，γ_j) The weight occupied in the final strong classifier h (x);

e_jrepresents h_j(f_j；θ_j，γ_j) The classification error of (2);

representing a symbolic function;

if it is

Then

Is 1;

if it is

Then

Is 0.

Step 3, fast face classification process

And 3.1, performing face detection on one frame of image in the fixed scene video to be recognized by adopting the strong classifier h (x) obtained in the step 2, wherein the image subjected to face detection is a face image set D3 to be recognized.

And 3.2, performing gray level histogram transformation on K2 features of all the face images in the face image set D3 to be recognized to obtain a gray level histogram set X of all the face images in the face image set D3 to be recognized.

And 3.3, performing nearest neighbor matching on the gray level histogram set X and the gray level histograms of K2 features of each face image in the face sample library D by using a graph-based relation atlas fast classification algorithm to obtain the types of all face images to be recognized in the face image set D3 to be recognized.

And 3.4, repeating the steps of 3.1-3.3 until the image of each frame in the fixed scene video to be identified is identified.

The relation map fast classification algorithm based on the graph comprises the following specific steps:

step 1, establishing a relationship map of people in a fixed scene video by using a map algorithm

1.1, establishing a face recognition training data set D1 by using n fixed scene videos with the same length, wherein M kinds of faces are shared in the face recognition training data set D1, and establishing a relation matrix U with the size of M multiplied by M; and initializing a relation matrix U into a matrix with all elements being zero, wherein the rows and the columns of the relation matrix U all represent the types of the human faces in the fixed scene video.

1.2, step by step, taking the J-th section of fixed scene video in a face recognition training data set D1, taking one frame of fixed scene video image every T frames, wherein the J-th section of fixed scene video has T fixed scene video images in total, and forming the T fixed scene video images into a fixed scene video image set Q_J，1≤J≤n。

1.3 step by step, fetch the fixed scene video image set Q_JOne frame of fixed scene video image O in (1)_mM is more than or equal to 1 and less than or equal to T, and the frame of fixed scene video image O is subjected to_mAnd (3) carrying out coordinate system calibration: the left lower corner is marked as the zero point of a coordinate system, the left side edge is the positive direction of the Y axis, and the lower side edge establishes a rectangular coordinate system for the positive direction of the X axis.

1.4, the strong classifier h (x) is used for carrying out substep on the frame of fixed scene video image O_mDetecting a face part, and manually calibrating the detected face part; each face is labeled with a number k, where k is 1, 2, … …, M.

1.5, dividing the frame of fixed scene video image O after the calibration and numbering_mThe center point of the face part in (1) is set as (a)_k，b_k) And k is the number of each face.

1.6, step by step, firstly setting a distance threshold value d, and then calculating the frame of fixed scene video image O_mIf the distance l is larger than a set distance threshold value d, setting the distance l to be infinite; if the distance l is smaller than the set threshold value d, the distance l is not changed.

1.7, dividing the step, if the distance l is infinite, setting 1/l as 0, and adding the reciprocal 1/l of the distance l to the corresponding position of the relation matrix U.

1.8, repeating the 1.6 and 1.7 steps until the frame of the fixed scene video image O_mThe distance l between the center points of any two face portions is calculated.

1.9, stepping and repeating the 1.3 to 1.8 stepping until a fixed scene video image set Q_JThe distance l between the center points of all arbitrary two face portions in each frame of the fixed scene video image is calculated.

1.10, the processing method of n sections of fixed scene videos in the face recognition training data set D1 is the same as that of 1.9, and a relation matrix U with the size of M multiplied by M is finally obtained; and carrying out normalization operation on the values in the relation matrix U, wherein the larger the value in the relation matrix U is, the tighter the human relation represented by the rows and the columns in the relation matrix U is, and the relation matrix U is the obtained relation map.

2, rapidly classifying the human face by using the relation atlas step by step

Step 2.1, carrying out nearest neighbor matching on a gray level histogram set X of the face image to be recognized and gray level histograms of K2 features of each face image in a face sample library D to obtain a matching degree priority queue set P (P) with the matching degree from large to small₁，P₂，……，P_r) And r represents the number of face images in the face image set D3 to be recognized, P_b＝(g_b，1，g_b，2，……，g_b，M) G represents the value of the matching degree, M represents the number of the types of the faces in the face sample library D, b represents the b-th face in the face image set D3 to be recognized, and b is more than or equal to 1 and less than or equal to r.

If the matching degree priority queue P of the b-th face in the face image to be recognized_bMaximum value g of medium matching degree_bmaxIf the matching degree is greater than the set matching degree threshold T1, the maximum value g of the matching degree is identified_bmaxThe corresponding face type; similarly, the maximum value g corresponding to the matching degree of all the faces in the face image to be recognized_1max，g_2max，……，g_rmaxIf the face image is larger than or equal to the set matching degree threshold T1, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_rmaxThe corresponding human face types form a recognized human face data set R1; the remaining face images constitute the candidate face set R2.

2.2, step 2, for the face images in the candidate face set R2, updating all face image matching degree priority queues P in the candidate face set R2 on the basis of the matching degree priority queue set P by adopting the relation atlas U obtained by step 1 and the faces in the identified face data set R1₁，P₂，……，P_cC represents the number of faces in the candidate face set R2, and a matching degree priority queue P is added_cDegree of medium matching g_c，IThe method comprises the following steps:

in formula (7): u represents a relation map obtained in the step 1;

g_c，Irepresenting the degree of matching of the c-th candidate face image in the candidate face set R2 before updating with the I-th personal face image in the recognized face data set R1;

representing the degree of matching between the c-th candidate face image in the updated candidate face set R2 and the I-th personal face image in the recognized face data set R1;

u_Iindicates the degree of matching g_c，IThe face type, I ═ 1, 2, … …, M;

v_zand the type of the face in the face data set R1 is identified, and z is more than or equal to 1 and less than M.

Step 2.3, matching degree priority queue P for face images in the updated candidate face set R2₁，P₂，……，P_cIf the maximum value g corresponding to the matching degrees of all the faces in the candidate face set R2_1max，g_2max，……，g_cmaxIf the face image is larger than or equal to the set matching degree threshold T1, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_cmaxAnd removing the successfully recognized face images from the candidate face set R2 according to the corresponding face types, and adding the recognized face data set R1, wherein the number of the face images to be recognized in the candidate face set R2 is c1, and c1 is more than 0 and less than c.

The step 2.4, the step 2.2 and the step 2.3 are repeated until the face image matching degree priority queue P in the candidate face set R2₁，P₂，……，P_c1No longer updated, if final match priority queue P₁，P₂，……，P_c1Maximum value g of degree of matching_1max，g_2max，……，g_c1maxIf the matching degree is still smaller than the threshold value T1, abandoning the recognition of the residual face images in the candidate face set R2; and finishing the recognition of the face image to be recognized in the frame.

Due to the adoption of the technical scheme, the invention has the following positive effects:

the invention utilizes the relational atlas technology of the graph to model the types of the human faces in the fixed scene video, excavate the internal connection of different human faces, and does not need to carry out special training and learning on different human faces, thereby greatly reducing the calculated amount, complexity and parameter quantity of the recognition algorithm, shortening the processing time, having small parameter quantity and realizing real-time monitoring.

The relation atlas-based fixed scene video face rapid identification method is simple to use, and can rapidly and accurately identify the face in the fixed scene video to be identified, such as a monitoring video, only by requiring a worker to mark different faces in advance.

The method can quickly identify the designated face in the massive offline fixed scene videos, and has the function of re-identifying the historical fixed scene videos.

The method can also quickly judge the faces which never appear in the fixed scene video, gives an alarm in a short time, and can be used for community security and personnel investigation.

The relation map technology established by the invention can dig out the internal connection of different faces through video data, can also establish the social network among people, and can be used for anti-fraud technology. In anti-fraud, the characteristics of group crime often appear, so that the clustering of people on the network can be analyzed through the established social network, and the method has strong guidance for researching commonalities among people and researching the characteristics of people in the social network.

Therefore, the invention has the characteristics of real-time monitoring, small parameter quantity and high accuracy of face recognition.

Detailed Description

The invention is further described with reference to specific embodiments, without limiting its scope.

Example 1

A method for rapidly identifying human faces of fixed scene videos based on a relational graph.

Step 1, data preprocessing

20 face images and 40 non-face images are cut from the fixed scene video to form a training data set D2, wherein one face image represents one face. Firstly, converting 60 images in a training data set D2 into 16 x 16 gray images one by one, wherein 60 is 40+ 20; forming 25 Gabor kernels by using 5 different scale coefficients and 5 different rotation angles, decomposing the gray level images by using the 25 Gabor kernels, and decomposing each image in the training data set D2 into 25 matrixes with the size of 16 multiplied by 16; wherein, 25 is 5 × 5; and (4) forming a face sample library D by all the face images in the training data set D2, and calibrating all the face images.

Step 2, training process of face detector

Step 2.1, establish training sample (x)^p，y^p)，p＝1，……，60。

x^pRepresenting 25 matrixes with the size of 16 multiplied by 16 obtained by Gabor decomposition of the p-th sample in the training data set D2;

y^pindicating whether the p-th sample in the training data set D2 is a face image;

if x^pIs a face image, then y^p1 is ═ 1; if x^pIs a non-face image, then y^p＝-1。

Step 2.2, initializing the weight of each training sample

Weights of the training samples

Wherein:

p＝1，......，60；

q＝1，......，25；

60 represents the number of sequence numbers of the last image in the training data set D2;

and 25 denotes the number of the last feature of each training sample.

Step 2.3, setting

For the 10 th feature of the 30 th training sample, the 10 th feature f of each training sample is used₁₀Finding the optimal weak classifier h₁₀(f₁₀，θ₁₀，γ₁₀) Threshold parameter theta of₁₀Class parameter gamma₁₀So that the classification error e₁₀And minimum.

In formulae (1) and (2): theta₁₀Represents the threshold parameter in the 10 th weak classifier;

γ₁₀representing the category parameter in the 10 th weak classifier;

the weight value of each training sample is represented,

representation classifier h₁₀(f₁₀；θ₁₀，γ₁₀) Whether the 10 th feature in the ith training sample is classified as erroneous,

in formula (3): y isⁱ＝h₁₀(f₁₀；θ₁₀，γ₁₀) Indicating that the classification is correct;

yⁱ≠h₁₀(f₁₀；θ₁₀，γ₁₀) Indicating a classification error.

By using the 10 th feature f of each training sample₁₀Training to obtain the optimal weak classifier h₁₀(f₁₀，θ₁₀，γ₁₀)：

In formula (4): theta₁₀Represents the 10 th best weak classifier h obtained by optimization₁₀(f₁₀；θ₁₀，γ₁₀) A threshold parameter of (d);

γ₁₀represents the 10 th best weak classifier h obtained by optimization₁₀(f₁₀；θ₁₀，γ₁₀) The category parameter of (2).

Step 2.4, updating the weight of each feature of the next iteration

If the 10 th feature of the 30 th training sample is used

Make the 30 th training sample by the 10 th optimal weak classifier h₁₀(f₁₀；θ₁₀，γ₁₀) If the identification is correct, the 10+1 th feature of the 30 th training sample of the next iteration is

Weight of (2)

If the 10 th feature of the 30 th training sample is used

Make the 30 th training sample unable to be selected by the 10 th optimal weak classifier h₁₀(f₁₀；θ₁₀，γ₁₀) If the identification is correct, the 10+1 th feature of the 30 th training sample of the next iteration is

Weight of (2)

In step 2.4: e.g. of the type₁₀Represents h₁₀(f₁₀；θ₁₀，γ₁₀) The classification error of (2).

And 2.5, repeating the steps 2.3 and 2.4 until 25 features are traversed to obtain a final strong classifier h (x):

in formulae (5) and (6): x represents a frame of image in the fixed scene video to be identified;

f_jrepresenting the jth feature in all 25 features after decomposition of 25 Gabor kernels in x;

θ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A threshold parameter of (d);

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A category parameter of (1);

α_jto represent the jth weak classifier h_j(f_j，θ_j，γ_j) The weight occupied in the final strong classifier h (x);

e_jrepresents h_j(f_j；θ_j，γ_j) The classification error of (2);

representing a symbolic function;

if it is

Then

Is 1;

if it is

Then

Is 0.

Step 3, fast face classification process

And 3.1, performing face detection on one frame of image in the fixed scene video to be recognized by adopting the strong classifier h (x) obtained in the step 2, wherein the image subjected to face detection is a face image set D3 to be recognized.

And 3.2, performing gray level histogram transformation on 25 features of all the face images in the face image set D3 to be recognized to obtain a gray level histogram set X of all the face images in the face image set D3 to be recognized.

And 3.3, performing nearest neighbor matching on the gray level histogram set X and the gray level histograms of 25 features of each face image in the face sample library D by using a graph-based relation atlas fast classification algorithm to obtain the types of all face images to be recognized in the face image set D3.

And 3.4, repeating the steps of 3.1-3.3 until the image of each frame in the fixed scene video to be identified is identified.

The relation map fast classification algorithm based on the graph comprises the following specific steps:

step 1, establishing a relationship map of people in a fixed scene video by using a map algorithm

1.1, establishing a face recognition training data set D1 by using 15 fixed scene videos with the same length, wherein 20 faces are shared in the face recognition training data set D1, and establishing a relation matrix U with the size of 20 multiplied by 20; and initializing a relation matrix U into a matrix with all elements being zero, wherein the rows and the columns of the relation matrix U all represent the types of the human faces in the fixed scene video.

1.2, taking the 10 th section of fixed scene video in the face recognition training data set D1, taking one frame of fixed scene video image every 5 frames, wherein the 10 th section of fixed scene video has 80 frames of fixed scene video images, and fixing 80 framesFixed scene video image composition fixed scene video image set Q₁₀。

1.3 step by step, fetch the fixed scene video image set Q₁₀One frame of fixed scene video image O in (1)₄₀For said one frame of fixed scene video image O₄₀And (3) carrying out coordinate system calibration: the left lower corner is marked as the zero point of a coordinate system, the left side edge is the positive direction of the Y axis, and the lower side edge establishes a rectangular coordinate system for the positive direction of the X axis.

1.4, the strong classifier h (x) is used for carrying out substep on the frame of fixed scene video image O₁₀Detecting a face part, and manually calibrating the detected face part; each face is assigned a number k, where k is 1, 2, … …, 20.

1.5, dividing the frame of fixed scene video image O after the calibration and numbering₄₀The center point of the face part in (1) is set as (a)_k，b_k) And k is the number of each face.

1.6, step by step, firstly setting the distance threshold d as 80, and then calculating the frame of fixed scene video image O₄₀Wherein, the distance l between the center points of two face parts is 59, and since the distance l 59 is smaller than the set distance threshold d 80, the distance l 59 is not changed.

Step 1.7, add the reciprocal 1/59 of the distance l 59 to the corresponding position of the relationship matrix U.

1.8, repeating the 1.6, 1.7 steps until the frame of fixed scene video image O₄₀The distance l between the center points of any two face portions is calculated.

1.9, stepping and repeating the 1.3 to 1.8 stepping until a fixed scene video image set Q₁₀The distance l between the center points of all arbitrary two face portions in each frame of the fixed scene video image is calculated.

1.10 substep, the processing method of 15 sections of fixed scene videos in the face recognition training data set D1 is the same as 1.9 substep, and a relationship matrix U with the size of 20 multiplied by 20 is finally obtained; and carrying out normalization operation on the values in the relation matrix U, wherein the larger the value in the relation matrix U is, the tighter the human relation represented by the rows and the columns in the relation matrix U is, and the relation matrix U is the obtained relation map.

2, rapidly classifying the human face by using the relation atlas step by step

Step 2.1, carrying out nearest neighbor matching on a gray level histogram set X of the face image to be recognized and a gray level histogram of 25 features of each face image in a face sample library D to obtain a matching degree priority queue set P (P) with the matching degree from large to small₁，P₂，……，P₃₀) 30 denotes the number of face images in the set of face images D3 to be recognized, P₁₆＝(g_16，1，g_16，2，……，g_16，20) G represents a value of the degree of matching, 20 represents the number of types of faces in the face sample library D, and 16 represents the 16 th face in the set of face images D3 to be recognized.

If the matching degree priority queue P of the 16 th face in the face image to be recognized₁₆Maximum value g of medium matching degree_16maxIf the matching degree is greater than the set matching degree threshold value of 0.85, the maximum value g of the matching degree is identified_16maxThe corresponding face type; similarly, the maximum value g corresponding to the matching degree of all the faces in the face image to be recognized_1max，g_2max，……，g_30maxIf the human face image is more than or equal to the set matching degree threshold value of 0.85, the human face image is identified as the maximum value g of the matching degree_1max，g_2max，……，g_30maxThe corresponding human face types form a recognized human face data set R1; the remaining face images constitute the candidate face set R2.

2.2, step 2, for the face images in the candidate face set R2, updating all face image matching degree priority queues P in the candidate face set R2 on the basis of the matching degree priority queue set P by adopting the relation atlas U obtained by step 1 and the faces in the identified face data set R1₁，P₂，……，P₁₀10 represents the number of faces in the candidate face set R2, and the matching degree priority queue P is increased₁₀Degree of medium matching g_10，5The method comprises the following steps:

in formula (7): u represents a relation map obtained in the step 1;

g_10，5representing the degree of matching of the 10 th candidate face image in the candidate face set R2 before updating with the 5 th personal face image in the recognized face data set R1;

indicating the degree of matching between the 10 th candidate face image in the updated candidate face set R2 and the 5 th personal face image in the recognized face data set R1;

u₅indicates the degree of matching g₅The type of face to which it belongs;

v_zand the type of the face in the face data set R1 is identified, and z is more than or equal to 1 and less than 20.

Step 2.3, matching degree priority queue P for face images in the updated candidate face set R2₁，P₂，……，P₁₀If the maximum value g corresponding to the matching degrees of all the faces in the candidate face set R2_1max，g_2max，……，g_10maxIf the human face image is more than or equal to the set matching degree threshold value of 0.85, the human face image is identified as the maximum value g of the matching degree_1max，g_2max，……，g_10maxAnd removing the successfully recognized face images from the candidate face set R2 according to the corresponding face types, and adding the recognized face data set R1, wherein the number of the face images to be recognized in the candidate face set R2 is 3.

The step 2.4, the step 2.2 and the step 2.3 are repeated until the face image matching degree priority queue P in the candidate face set R2₁，P₂，……，P₃Until no more updates, the final match priority queue P₁，P₂，……，P₃Maximum value g of degree of matching_1max，g_2max，……，g_3maxIf the matching degree is still less than the threshold value of 0.85, abandoning to identify the residual face images in the candidate face set R2; and finishing the recognition of the face image to be recognized in the frame.

Example 2

A method for rapidly identifying human faces of fixed scene videos based on a relational graph. The method for rapidly identifying the human face comprises the following steps:

step 1, data preprocessing

30 face images and 43 non-face images are cut from the fixed scene video to form a training data set D2, wherein one face image represents one face. Firstly, converting 73 images in a training data set D2 into 32 x 32 gray images one by one, wherein 73 is 30+ 43; forming 28 Gabor kernels by using 4 different scale coefficients and 7 different rotation angles, decomposing the gray level images by using the 28 Gabor kernels, and decomposing each image in the training data set D2 into 28 matrixes with the size of 32 multiplied by 32; wherein, 28 is 4 × 7; and (4) forming a face sample library D by all the face images in the training data set D2, and calibrating all the face images.

Step 2, training process of face detector

Step 2.1, establish training sample (x)^p，y^p)，p＝1，……，73。

x^pRepresenting 28 matrixes with the size of 32 multiplied by 32 obtained by Gabor decomposition of the p-th sample in the training data set D2;

y^pindicating whether the p-th sample in the training data set D2 is a face image;

if x^pIs a face image, then y^p1 is ═ 1; if x^pIs a non-face image, then y^p＝-1。

Step 2.2, initializing the weight of each training sample

Weights of the training samples

Wherein:

p＝1，......，73；

q＝1，......，28；

73 denotes the number of sequence numbers of the last image in the training data set D2;

and 28 denotes the number of the last feature of each training sample.

Step 2.3, setting

For the 14 th feature of the 35 th training sample, the 14 th feature f of each training sample is used₁₄Finding the optimal weak classifier h1₄(f₁₄，θ₁₄，γ₁₄) Threshold parameter theta of₁₄Class parameter gamma₁₄So that the classification error e₁₄And minimum.

In formulae (1) and (2): theta₁₄Represents the threshold parameter in the 14 th weak classifier;

γ₁₄representing class parameters in the 14 th weak classifier;

the weight value of each training sample is represented,

representation classifier h₁₄(f₁₄；θ₁₄，γ₁₄) Whether the 14 th feature in the ith training sample is classified as erroneous,

in formula (3): y isⁱ＝h₁₄(f₁₄；θ₁₄，γ1₄) Indicating that the classification is correct;

yⁱ≠h₁₄(f₁₄；θ₁₄，γ₁₄) Indicating a classification error.

By using the 14 th feature f of each training sample₁₄Training to obtain the optimal weak classifier h₁₄(f₁₄，θ₁₄，γ₁₄)：

In formula (4): theta₁₄Represents the 14 th best weak classifier h obtained by optimization₁₄(f₁₄；θ₁₄，γ₁₄) A threshold parameter of (d);

γ₁₄represents the 14 th best weak classifier h obtained by optimization₁₄(f1₄；θ₁₄，γ₁₄) The category parameter of (2).

Step 2.4, updating the weight of each feature of the next iteration

If the 14 th feature of the 35 th training sample is used

Make the 35 th training sample by the 14 th optimal weak classifier h₁₄(f₁₄；θ₁₄，γ₁₄) If the identification is correct, the 14+1 th feature of the 35 th training sample of the next iteration is

Weight of (2)

If the 14 th feature of the 35 th training sample is used

Make the 35 th training sample unable to be optimized by the 14 th weak classifier h₁₄(f₁₄；θ₁₄，γ₁₄) If the identification is correct, the 14+1 th feature of the 35 th training sample of the next iteration is

Weight of (2)

In step 2.4: e.g. of the type₁₄Represents h₁₄(f₁₄；θ₁₄，γ₁₄) The classification error of (2).

And 2.5, repeating the steps 2.3 and 2.4 until 28 features are traversed to obtain a final strong classifier h (x):

in formulae (5) and (6): x represents a frame of image in the fixed scene video to be identified;

f_jrepresenting the jth feature in all 28 features after the decomposition of 28 Gabor kernels in x;

θ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A threshold parameter of (d);

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A category parameter of (1);

α_jto represent the jth weak classifier h_j(f_j，θ_j，γ_j) The weight occupied in the final strong classifier h (x);

e_jrepresents h_j(f_j；θ_j，γ_j) The classification error of (2);

representing a symbolic function;

if it is

Then

Is 1;

if it is

Then

Is 0.

Step 3, fast face classification process

And 3.1, performing face detection on one frame of image in the fixed scene video to be recognized by adopting the strong classifier h (x) obtained in the step 2, wherein the image subjected to face detection is a face image set D3 to be recognized.

And 3.2, performing gray level histogram transformation on 28 features of all the face images in the face image set D3 to be recognized to obtain a gray level histogram set X of all the face images in the face image set D3 to be recognized.

And 3.3, performing nearest neighbor matching on the gray level histogram set X and the gray level histograms of 28 features of each face image in the face sample library D by using a graph-based relation atlas fast classification algorithm to obtain the types of all face images to be recognized in the face image set D3.

And 3.4, repeating the steps of 3.1-3.3 until the image of each frame in the fixed scene video to be identified is identified.

The relation map fast classification algorithm based on the graph comprises the following specific steps:

step 1, establishing a relationship map of people in a fixed scene video by using a map algorithm

1.1, establishing a face recognition training data set D1 by using 40 segments of same fixed scene videos, wherein the face recognition training data set D1 contains 30 faces in total, and establishing a relationship matrix U with the size of 30 multiplied by 30; and initializing a relation matrix U into a matrix with all elements being zero, wherein the rows and the columns of the relation matrix U all represent the types of the human faces in the fixed scene video.

1.2, taking the 20 th section of fixed scene video in a face recognition training data set D1, taking one frame of fixed scene video image every 15 frames, wherein the 20 th section of fixed scene video has 18 fixed scene video images, and forming a fixed scene video image set Q by the 18 fixed scene video images₂₀。

1.3 step by step, fetch the fixed scene video image set Q₂₀One frame of fixed scene video image O in (1)₁₀For said one frame of fixed scene video image O₁₀And (3) carrying out coordinate system calibration: the left lower corner is marked as the zero point of a coordinate system, the left side edge is the positive direction of the Y axis, and the lower side edge establishes a rectangular coordinate system for the positive direction of the X axis.

1.4, the strong classifier h (x) is used for carrying out substep on the frame of fixed scene video image O₁₀Detecting a face part, and manually calibrating the detected face part; each face is assigned a number k, where k is 1, 2, … …, 30.

1.5, dividing the frame of fixed scene video image O after the calibration and numbering₁₀The center point of the face part in (1) is set as (a)_k，b_k) And k is the number of each face.

1.6, step by step, firstly setting the distance threshold value d as 70, and then calculating the frame of fixed scene video image O₁₀The distance l between the center points of any two human face parts is 83; since the distance l is 83 larger than the set distance threshold d is 70, the distance l is set to infinity.

1.7, step by step, setting 1/l as 0 as the distance l is infinite; and adding 1/l to 0 to the corresponding position of the relation matrix U.

1.8, repeating the 1.6 and 1.7 steps until the frame of the fixed scene video image O₁₀The distance l between the center points of any two face portions is calculated.

1.9, stepping and repeating the 1.3 to 1.8 stepping until a fixed scene video image set Q₂₀The distance l between the center points of all arbitrary two face portions in each frame of the fixed scene video image is calculated.

1.10, the processing method of 40 sections of fixed scene videos in the face recognition training data set D1 is the same as that of 1.9, and a relation matrix U with the size of 30 x 30 is finally obtained; and carrying out normalization operation on the values in the relation matrix U, wherein the larger the value in the relation matrix U is, the tighter the human relation represented by the rows and the columns in the relation matrix U is, and the relation matrix U is the obtained relation map.

2, rapidly classifying the human face by using the relation atlas step by step

Step 2.1, carrying out nearest neighbor matching on the gray histogram set X of the face image to be recognized and the gray histograms of the 28 features of each face image in the face sample library D to obtain a matching degree priority queue set P (P) with the matching degree from large to small₁，P₂，……，P₇₀) 70 denotes the number of face images in the set of face images D3 to be recognized, P₅₅＝(g_55，1，g_55，2，……，g_55，30) G represents the value of the matching degree, 30 represents the number of the types of the faces in the face sample library D, and 55 represents the 55 th face in the face image set D3 to be recognized.

If the matching degree priority queue P of the 55 th face in the face image to be recognized₅₅Maximum value g of medium matching degree_55maxIf the matching degree is greater than the set matching degree threshold value of 0.77, the maximum value g of the matching degree is identified_55maxThe corresponding face type; similarly, the maximum value g corresponding to the matching degree of all the faces in the face image to be recognized_1max，g_2max，……，g_70maxIf the face image is more than or equal to the set matching degree threshold value of 0.77, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_70maxThe corresponding human face types form a recognized human face data set R1; the remaining face images constitute the candidate face set R2.

2.2, step 2, for the face images in the candidate face set R2, updating all face image matching degree priority queues P in the candidate face set R2 on the basis of the matching degree priority queue set P by adopting the relation atlas U obtained by step 1 and the faces in the identified face data set R1₁，P₂，……，P₃₈38 represents the number of faces in the candidate face set R2, and adds a matching degree priority queue P₃₈Degree of medium matching g_38，3The method comprises the following steps:

in formula (7): u represents a relation map obtained in the step 1;

g_38，3representing the degree of matching of the 38 th candidate face image in the candidate face set R2 before updating with the 3 rd personal face image in the recognized face data set R1;

indicating the degree of matching between the 38 th candidate face image in the updated candidate face set R2 and the 3 rd personal face image in the recognized face data set R1;

u₃indicates the degree of matching g₃The type of face to which it belongs;

v_zand the type of the face in the face data set R1 is identified, and z is more than or equal to 1 and less than 30.

Step 2.3, matching degree priority queue P for face images in the updated candidate face set R2₁，P₂，……，P₃₈If all the candidate faces in the candidate face set R2Maximum value g corresponding to matching degree of human face_1max，g_2max，……，g_38maxIf the face image is more than or equal to the set matching degree threshold value of 0.77, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_38maxAnd removing the successfully recognized face images from the candidate face set R2 according to the corresponding face types, and adding the recognized face data set R1, wherein the number of the face images to be recognized in the candidate face set R2 is 20.

The step 2.4, the step 2.2 and the step 2.3 are repeated until the face image matching degree priority queue P in the candidate face set R2₁，P₂，……，P₂₀Until no more updates, the final match priority queue P₁，P₂，……，P₂₀Maximum value g of degree of matching_1max，g_2max，……，g_20maxIf the matching degree is greater than the threshold value of 0.77, the maximum value g of the matching degree is identified_1max，g_2max，……，g_20maxAnd the face image to be recognized in the frame is recognized.

The embodiment has the following positive effects:

the specific implementation mode utilizes the relational graph technology of the graph to model the types of the human faces in the fixed scene video, digs the internal connection of different human faces, and does not need to carry out special training and learning on the different human faces, thereby greatly reducing the calculated amount, complexity and parameter quantity of the recognition algorithm, shortening the processing time, having small parameter quantity and realizing real-time monitoring.

The relation-atlas-based method for rapidly identifying the face of the fixed scene video is simple to use, and the face can be rapidly and accurately identified in the fixed scene video to be identified, such as a monitoring video, only by requiring a worker to mark different faces in advance.

The specific implementation method can rapidly identify the designated human face in massive offline fixed scene videos, and has the function of re-identifying the historical fixed scene videos.

The method and the device can quickly judge the face which never appears in the fixed scene video, send out an alarm in a short time, and can be used for community security and personnel investigation.

The relationship graph technology established by the specific implementation mode can be used for mining the internal connection of different faces through video data, establishing a social network among people and being used for anti-fraud technology. In anti-fraud, the characteristics of group crime often appear, so that the clustering of people on the network can be analyzed through the established social network, and the method has strong guidance for researching commonalities among people and researching the characteristics of people in the social network.

Therefore, the specific implementation mode has the characteristics of real-time monitoring, small parameter quantity and high accuracy of face recognition.

Claims (1)

1. A fast face recognition method of a fixed scene video based on a relation atlas is characterized in that the fast face recognition method of the fixed scene video comprises the following specific steps:

step 1, data preprocessing

M face images and L non-face images are intercepted from a fixed scene video to form a training data set D2, wherein one face image represents a face; firstly, converting N images in a training data set D2 into C multiplied by C gray images one by one, wherein N is M + L; k2 Gabor kernels are formed by K different proportionality coefficients and K1 different rotation angles, the gray level images are decomposed by K2 Gabor kernels, and each image in a training data set D2 is decomposed into K2 matrixes with the size of C multiplied by C; wherein, K2 is K1 × K, C, K, K1 and K2 are natural numbers; all the face images in the training data set D2 form a face sample library D, and all the face images are calibrated;

step 2, training process of face detector

Step 2.1, establish training sample (x)^p，y^p)，p＝1，……，N；

x^pRepresents K2C × C matrixes obtained by Gabor decomposition of the p-th sample in the training data set D2,

y^prepresenting the number of exercisesWhether the p-th sample in the set D2 is a face image,

if x^pIs a face image, then y^p1 is ═ 1; if x^pIs a non-face image, then y^p＝-1；

Step 2.2, initializing the weight of each training sample

Weights of the training samples

Wherein:

p＝1，……，N，

q＝1，……，K2，

n denotes the number of sequence numbers of the last image in the training data set D2,

k2 represents the number of the sequence numbers of the last feature of each training sample;

step 2.3, setting f_j ⁱFor the jth feature of the ith training sample, the jth feature f of each training sample is used_jFinding the optimal weak classifier h_j(f_j，θ_j，γ_j) Threshold parameter theta of_jClass parameter gamma_jSo that the classification error e_jMinimum;

in formulae (1) and (2): theta_jRepresenting the threshold parameter in the jth weak classifier,

γ_jindicates the class parameters in the jth weak classifier,

the weight value of each training sample is represented,

representation classifier h_j(f_j；θ_j，γ_j) Whether the jth feature in the ith training sample is classified as erroneous,

in formula (3): y isⁱ＝h_j(f_j；θ_j，γ_j) It is indicated that the classification is correct,

yⁱ≠h_j(f_j；θ_j，γ_j) Indicating a classification error;

by using the jth feature f of each training sample_jTraining to obtain the optimal weak classifier h_j(f_j，θ_j，γ_j)：

In formula (4): theta_jRepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) The threshold value parameter of (a) is,

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) A category parameter of (1);

step 2.4, updating the weight of each feature of the next iteration

If the jth feature f of the ith training sample is utilized_j ⁱSo that the ith training sample is optimized by the jthWeak classifier h of_j(f_j；θ_j，γ_j) If the identification is correct, the j +1 th feature of the ith training sample of the next iteration is

Weight of (2)

If the jth feature f of the ith training sample is utilized_j ⁱMake the ith training sample unable to be optimized by the jth weak classifier h_j(f_j；θ_j，γ_j) If the identification is correct, the j +1 th feature of the ith training sample of the next iteration is

Weight of (2)

In step 2.4: e.g. of the type_jRepresents h_j(f_j；θ_j，γ_j) The classification error of (2);

and step 2.5, repeating the step 2.3 and the step 2.4 until K2 features are traversed to obtain a final strong classifier h (x):

in formulae (5) and (6): the x-table does not apply to one frame of image in the fixed scene video to be recognized,

f_jrepresents the jth feature in the total K2 features decomposed by K2 Gabor nuclei in x,

θ_jrepresentation by optimizationTo the jth optimal weak classifier h_j(f_j；θ_j，γ_j) The threshold value parameter of (a) is,

γ_jrepresents the jth optimal weak classifier h obtained by optimization_j(f_j；θ_j，γ_j) The category parameter of (a) is,

α_jto represent the jth weak classifier h_j(f_j，θ_j，γ_j) The weight occupied in the final strong classifier h (x),

e_jrepresents h_j(f_j；θ_j，γ_j) The error in the classification of (2) is,

the function of the symbol is represented by,

if it is

Then

The number of the carbon atoms is 1,

if it is

Then

Is 0;

step 3, fast face classification process

Step 3.1, performing face detection on one frame of image in the fixed scene video to be recognized by adopting the strong classifier h (x) obtained in the step 2, wherein the image subjected to face detection is a face image set D3 to be recognized;

3.2, performing gray level histogram transformation on K2 features of all the face images in the face image set D3 to be recognized to obtain a gray level histogram set X of all the face images in the face image set D3 to be recognized;

3.3, performing nearest neighbor matching on the gray level histogram set X and the gray level histograms of K2 features of each face image in the face sample library D by using a graph-based relation atlas fast classification algorithm to obtain the types of all face images to be recognized in a face image set D3 to be recognized;

3.4, repeating the steps of 3.1-3.3 until the image of each frame in the fixed scene video to be identified is identified;

the relation map fast classification algorithm based on the graph comprises the following specific steps:

step 1, establishing a relationship map of people in a fixed scene video by using a map algorithm

1.1, establishing a face recognition training data set D1 by using n fixed scene videos with the same length, wherein M kinds of faces are shared in the face recognition training data set D1, and establishing a relation matrix U with the size of M multiplied by M; initializing a relation matrix U into a matrix with all elements being zero, wherein the rows and the columns of the relation matrix U all represent the types of human faces in the fixed scene video;

1.2, step by step, taking the J-th section of fixed scene video in a face recognition training data set D1, taking one frame of fixed scene video image every T frames, wherein the J-th section of fixed scene video has T fixed scene video images in total, and forming the T fixed scene video images into a fixed scene video image set Q_J，1≤J≤n；

1.3 step by step, fetch the fixed scene video image set Q_JOne frame of fixed scene video image O in (1)_mM is more than or equal to 1 and less than or equal to T, and the frame of fixed scene video image O is subjected to_mAnd (3) carrying out coordinate system calibration: the left lower corner is marked as a zero point of a coordinate system, the left side edge is the positive direction of a Y axis, and the lower side edge is the positive direction of an X axis to establish a rectangular coordinate system;

1.4, the strong classifier h (x) is used for carrying out substep on the frame of fixed scene video image O_mDetecting a face part, and manually calibrating the detected face part; each face is marked with a number k, wherein k is 1, 2, … …, M;

1.5, dividing the frame of fixed scene video image O after the calibration and numbering_mPerson in (1)The center point of the face part is set as (a)_k，b_k) Wherein k is the number of each face;

1.6, step by step, firstly setting a distance threshold value d, and then calculating the frame of fixed scene video image O_mIf the distance l is larger than a set distance threshold value d, setting the distance l to be infinite; if the belonging distance l is smaller than a set threshold value d, the distance l is not changed;

1.7, dividing the step, if the distance l is infinite, setting 1/l as 0, and adding the reciprocal 1/l of the distance l to the position corresponding to the relation matrix U;

1.8, repeating the 1.6 and 1.7 steps until the frame of the fixed scene video image O_mThe distances l between the center points of any two human face parts are calculated;

1.9, stepping and repeating the 1.3 to 1.8 stepping until a fixed scene video image set Q_JThe distance l between the center points of any two face parts in each frame of fixed scene video image is calculated;

1.10, the processing method of n sections of fixed scene videos in the face recognition training data set D1 is the same as that of 1.9, and a relation matrix U with the size of M multiplied by M is finally obtained; carrying out normalization operation on the values in the relation matrix U, wherein the larger the value in the relation matrix U is, the tighter the human relation represented by the rows and the columns in the relation matrix U is, and the relation matrix U is the obtained relation map;

2, rapidly classifying the human face by using the relation atlas step by step

Step 2.1, carrying out nearest neighbor matching on a gray level histogram set X of the face image to be recognized and gray level histograms of K2 features of each face image in a face sample library D to obtain a matching degree priority queue set P (P) with the matching degree from large to small₁，P₂，……，P_r) And r represents the number of face images in the face image set D3 to be recognized, P_b＝(g_b，1，g_b，2，……，g_b，M) G represents the value of the degree of matching, M represents the person in the face sample library DB represents the b-th face in the face image set D3 to be recognized, wherein b is more than or equal to 1 and less than or equal to r;

if the matching degree priority queue P of the b-th face in the face image to be recognized_bMaximum value g of medium matching degree_bmaxIf the matching degree is greater than the set matching degree threshold T1, the maximum value g of the matching degree is identified_bmaxThe corresponding face type; similarly, the maximum value g corresponding to the matching degree of all the faces in the face image to be recognized_1max，g_2max，……，g_rmaxIf the face image is larger than or equal to the set matching degree threshold T1, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_rmaxThe corresponding human face types form a recognized human face data set R1; the remaining face images constitute a candidate face set R2;

2.2, step 2, for the face images in the candidate face set R2, updating all face image matching degree priority queues P in the candidate face set R2 on the basis of the matching degree priority queue set P by adopting the relation atlas U obtained by step 1 and the faces in the identified face data set R1₁，P₂，……，P_cC represents the number of faces in the candidate face set R2, and a matching degree priority queue P is added_cDegree of medium matching g_c，IThe method comprises the following steps:

in formula (7): u represents the relationship map obtained in the 1 st substep,

g_c，Iindicating the degree of matching of the c-th candidate face image in the candidate face set R2 before updating with the 1 st individual face image in the recognized face data set R1,

indicating the number of faces identified in the c-th candidate face image in the updated candidate face set R2According to the matching degree of the ith personal face image in the set R1,

u_Iindicates the degree of matching g_c，IThe face type to which the face belongs, I ═ 1, 2, … …, M,

v_zrepresenting the types of the faces in the face data set R1 which are recognized, wherein z is more than or equal to 1 and less than M;

step 2.3, matching degree priority queue P for face images in the updated candidate face set R2₁，P₂，……，P_cIf the maximum value g corresponding to the matching degrees of all the faces in the candidate face set R2_1max，g_2max，……，g_cmaxIf the face image is larger than or equal to the set matching degree threshold T1, the face image is identified as the maximum matching degree value g_1max，g_2max，……，g_cmaxAnd removing the successfully recognized face images from the candidate face set R2 according to the corresponding face types, adding the recognized face data set R1, wherein the number of the face images to be recognized in the candidate face set R2 is c1, and c1 is more than 0 and less than c:

the step 2.4, the step 2.2 and the step 2.3 are repeated until the face image matching degree priority queue P in the candidate face set R2₁，P₂，……，P_c1No longer updated, if final match priority queue P₁，P₂，……，P_c1Maximum value g of degree of matching_1max，g_2max，……，g_c1maxIf the matching degree is still smaller than the threshold value T1, abandoning the recognition of the residual face images in the candidate face set R2; and finishing the recognition of the face image to be recognized in the frame.