CN110472460A - Face image processing process and device - Google Patents

Abstract

The present disclosure provides a face image processing method and device, the method comprising: acquiring a plurality of face images, the plurality of face images including at least one person's face images from different angles; The images are clustered to obtain a plurality of face image groups; and some face images in at least one face image group are removed according to user input and/or preset rules, so that each face image in the at least one face image group Each face image in the group corresponds to the same person. Using the solution provided by the present disclosure, multiple face image groups can be obtained, wherein a plurality of different angle face images in one face image group correspond to a person, so that the training of face image groups can be performed on different angle face images Recognized face recognition model.

Description

Face image processing method and device

technical field

The present disclosure relates to the field of computer technology, and more specifically, to a method and device for processing human face images.

Background technique

In the offline unmanned retail and smart retail scenarios, it is necessary to realize the authentication of customers' facial recognition for payment, and its technical basis is the high-precision recognition and detection of customers' human faces. In the above scenarios, face information is generally collected by surveillance cameras deployed in the store, which is different from the standard face data captured in the experimental environment, such as ID photos, which have the following characteristics: frontal face, no depression angle, single background , clear facial features, no occlusion, etc. However, the face images captured by the surveillance cameras in the above-mentioned open environment are often extremely complex, far below the standard of ID photos.

For existing face recognition algorithms, the training data set is generally a standard face recognition data set. In these data sets, the face shooting scene is too simple compared to the scene shot by the surveillance camera, which makes it impossible to cover the multi-angle face images captured by the surveillance. This results in a very low recognition accuracy for directly using a standard face recognition data set trained model to recognize face images captured by surveillance cameras.

In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the prior art: in scenarios such as smart retail, the face images of users are collected from various angles, and the use of existing standard face recognition The recognition accuracy of the model trained in the dataset to recognize face images captured by surveillance cameras is very low.

Contents of the invention

In view of this, the present disclosure provides a face image processing method and device to solve the problem of recognition accuracy in scenes with multi-angle face image recognition using a model trained on a standard face recognition data set in the prior art. low problem.

One aspect of the present disclosure provides a method for processing human face images, including: first acquiring multiple facial images, in order to obtain multiple facial images of each person from different angles, the multiple facial images are required to include At least one person's face images from different angles; then the obtained multiple face images are clustered to obtain a plurality of face image groups, because each person has its own special facial features, so the clustering result can be After the face images of different angles of each person are clustered, they are gathered in the same face image group, which actually realizes the function of automatically labeling the multiple face images, for example, the face image group 1 and multiple face images from different angles corresponding to face image group 1. In addition, after obtaining the plurality of human face image groups, according to user input and/or preset rules, part of the human face images in at least one human face image group is removed, so that each human face in the at least one human face image group Each face image in the image group corresponds to the same person to correct the wrong clustering results and improve the accuracy of data annotation.

According to an embodiment of the present disclosure, the acquiring multiple face images may include the following operations: acquiring a video, which may be a video captured by a camera in a face recognition environment; and acquiring a human face in the video in units of frames Image, because the person is often active in the video, therefore, the multi-frame images that are captured often contain the face images of the person from different angles. Specifically, the acquisition of the human face image in the video in units of frames may include the following operations: according to experience, experimental results, etc., to acquire video frames of the video frame by frame or set a number of frames at intervals, so that The video frame contains high-quality face images of the same person at different angles; and then the face images of the video frame are acquired to obtain high-quality face images of the same person at different angles.

According to an embodiment of the present disclosure, the method may further include the following operations: after obtaining the plurality of face image groups, respectively obtain the group center of at least one face image group, and then use the group center as the corresponding face image Group identification, so that the group center of the human image group can be used to mark the face image group, that is, the label information of each face image group can be the group center of the face image group, for example, the face image group includes: A group center and multiple face images corresponding to the group center from different angles.

According to an embodiment of the present disclosure, the group center of the at least one human face image group may be obtained through the following scheme, for example, including the following operations: For each human face image group in the at least one human face image group:

Firstly, the distances between each face image in the face image group and other face images are calculated separately, and then the face image with the minimum sum of distances from other face images in the face image group is taken as the group center.

According to an embodiment of the present disclosure, the method may further include the following operations: after obtaining the group center, for the group of human face images, respectively calculate the distance of each face image in the group of human face images relative to the group center Rotation angle; then use the obtained rotation angle to annotate the angle of the corresponding face image. That is to say, the group of face images in this embodiment may also include annotation information of rotation angle, and the annotation information may be applied in a scene where the annotation information of rotation angle is required.

According to an embodiment of the present disclosure, a specific solution for calculating the rotation angle is also provided, specifically, the following operations may be included, for each face image: first obtain a specified number of key points of the face image, and obtain all The specified number of key points of the group center, then obtain the positional relationship between the key points of the face image, and obtain the positional relationship between the key points of the group center, and then according to the key points of the face image and the positional relationship between the key points of the group center to determine the rotation angle of the face image relative to the group center.

According to an embodiment of the present disclosure, the method may further include the following operations: for the group of human face images, performing a rotation on the face images in the group of human face images according to the rotation angle and/or distance between the face images and the center of the group Sorting; and displaying the sorted face images.

According to an embodiment of the present disclosure, the method may further include the following operations: after obtaining a plurality of face image groups, using the plurality of face image groups to train a face recognition model, specifically, after obtaining the plurality of face images After grouping, the face features of the face images from different angles corresponding to the face image group and the face image group to which they belong are input into the first face recognition model for training to obtain model parameter values.

Correspondingly, another aspect of the present disclosure provides a human face image processing device, which may include the following modules: an acquisition module, configured to acquire a plurality of human face images, the plurality of human face images including different angles of at least one person face image;

A clustering module, configured to cluster the multiple face images to obtain multiple face image groups; and

A correction module, configured to remove some face images in at least one face image group according to user input and/or preset rules, so that each face image in each face image group in the at least one face image group correspond to the same person.

According to an embodiment of the present disclosure, the acquisition module may include the following units:

A video acquisition unit, configured to acquire video;

A video frame acquisition unit, configured to acquire video frames of the video frame by frame or at intervals of a set number of frames; and

A human face image acquiring unit, configured to acquire the human face image of the video frame.

According to an embodiment of the present disclosure, the device may further include the following modules: a group center acquisition module connected to the clustering module, and an identification module, wherein the group center acquisition module is used to respectively acquire at least one face image The group center of the group; the identification module is used to use the class group center as the identification of the corresponding face image group.

According to an embodiment of the present disclosure, the group center acquisition module may include the following units: a distance calculation unit and a center acquisition unit, wherein the distance calculation unit is used for each face image in the at least one face image group group, respectively calculate the distance between each face image in the face image group and other face images; the center acquisition unit is used to minimize the sum of distances from other face images in the face image group as the group center.

According to an embodiment of the present disclosure, the device may further include the following modules: a rotation angle calculation module connected to the group center acquisition module, and an angle labeling module. Wherein, the rotation angle calculation module is used to calculate the rotation angle of each face image in the face image group with respect to the group center for the face image group; the angle labeling module is used to use the obtained The rotation angle is used to annotate the angle of the corresponding face image.

According to an embodiment of the present disclosure, the rotation angle calculation module may include the following units: a key point acquisition unit, a positional relationship acquisition unit, and a rotation angle calculation unit, wherein the key point acquisition unit is used for each face image , obtaining the specified number of key points of the face image, and obtaining the specified number of key points of the group center; the positional relationship acquisition unit is used to obtain the positional relationship between the key points of the human face image, And obtain the positional relationship between the key points of the group center; the rotation angle calculation unit is used for according to the positional relationship between the key points of the face image and the positional relationship between the key points of the group center, Determines the rotation angle of this face image relative to the center of the group.

According to an embodiment of the present disclosure, the device may further include the following modules: a sorting module and a display module, wherein the sorting module is used for the face image group, according to the rotation angle and the rotation angle between the face image and the group center Sorting the face images in the face image group by distance; and the display module is used to display the sorted face images.

According to an embodiment of the present disclosure, the device may further include the following module: a face recognition model training module, configured to use the multiple face image groups to train the face recognition model after obtaining the multiple face image groups.

Another aspect of the present disclosure provides a face image processing device for face image processing, the face image processing device may include:

one or more processors; and storage means for storing executable instructions that, when executed by the processors, implement any of the methods described above.

Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions, which are used to implement any one of the above methods when executed.

Another aspect of the present disclosure provides a computer program comprising computer-executable instructions for implementing the method as described above when executed.

According to the embodiment of the present disclosure, since each person has his own special facial features, the clustering result can make the face images of different angles of each person clustered and clustered in the same face image group, so that in fact The function of automatically labeling the multiple face images is realized, for example, face image group 1 and a plurality of face images from different angles corresponding to face image group 1, so that the human face images obtained by clustering can be used The face image group is used for face recognition model training. For example, the face features of one or more multi-angle face images and the face image group to which they belong can be input into a pre-built face recognition model for model training. A model that can meet multi-angle face image recognition can at least partially solve the problem of using the existing standard face recognition data training model for face recognition when performing face recognition on face images captured by surveillance cameras. Face images from different angles lead to the problem of low recognition accuracy, and therefore the technical effect of improving the recognition accuracy of multi-angle face images can be achieved.

According to the embodiment of the present disclosure, since it is further checked whether the multi-angle face image corresponding to each person is correct, if it is not correct, the correctness of the face image in the face image group is improved by removing the incorrect face image, In this way, it is possible to ensure that the face images in the face image group correspond to the same person as much as possible, avoid labeling errors in training data, and achieve the technical effect of improving the recognition accuracy of multi-angle face images.

According to the embodiments of the present disclosure, a specific method for acquiring multiple face images is further provided. The method can simulate the actual face recognition environment, thereby collecting videos that are the same as or similar to the actual face recognition environment, and then can obtain The same face image as the actual face recognition environment helps to improve the similarity between the face images in the face image group with annotation information obtained by using the present disclosure and the face images collected in the actual use environment , so that the recognition accuracy of the model after training the face recognition model by using the face image group provided in this embodiment is higher.

According to the embodiment of the present disclosure, after obtaining the clustering results, the group centers of the face image groups are respectively obtained, and then the group centers can be used as the identification of the corresponding face image groups, which is convenient for checking the clustering results. Whether the face image group is correct.

According to the embodiment of the present disclosure, after the group center of the face image group is obtained, the rotation angle of each face image in the current face image group relative to the group center is also calculated, and then the current face image is calculated using the obtained rotation angle Angle labeling of face images. In this way, the training set obtained in this disclosure also includes the rotation angle information of the face image. Therefore, in a specific scenario, for example, in a specific payment environment, when there is a certain rotation angle between the camera and the payer, it can be obtained from the training Focus on selecting face images with a specific rotation angle to optimize the model, which helps to improve the accuracy of model recognition.

According to the embodiments of the present disclosure, a variety of face recognition model training methods are also provided, the training method can simply and efficiently train the face recognition model, and the trained face recognition model can recognize multi-angle face images , the recognition accuracy is higher.

Description of drawings

The above and other objects, features and advantages of the present disclosure will be more clearly described through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically shows an exemplary system architecture that can be applied to face image processing according to an embodiment of the present disclosure;

FIG. 2 schematically shows face images from different angles according to an embodiment of the present disclosure;

FIG. 3A schematically shows a flow chart of a method for processing a face image according to an embodiment of the present disclosure;

FIG. 3B schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure;

Fig. 3C schematically shows a schematic diagram of a face image group according to an embodiment of the present disclosure;

FIG. 4A schematically shows a flow chart of a method for acquiring multiple face images according to an embodiment of the present disclosure;

FIG. 4B schematically shows a flow chart of a method for acquiring multiple face images according to another embodiment of the present disclosure;

FIG. 4C schematically shows a flow chart of a method for acquiring a face image from a video frame according to an embodiment of the present disclosure;

FIG. 5 schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure;

FIG. 6 schematically shows a flow chart of a method for acquiring a group center of a face image group according to an embodiment of the present disclosure;

FIG. 7 schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure;

FIG. 8 schematically shows a flowchart of a method for calculating a rotation angle according to an embodiment of the present disclosure;

Fig. 9 schematically shows a schematic diagram of a human face image and key points in a video frame according to an embodiment of the present disclosure;

Fig. 10 schematically shows a schematic diagram of key points according to an embodiment of the present disclosure;

Fig. 11 schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure;

Fig. 12 schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure;

Fig. 13 schematically shows a block diagram of a face image processing device according to an embodiment of the present disclosure;

Fig. 14 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 15 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 16 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 17 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 18 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 19 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 20 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 21 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure;

Fig. 22 schematically shows a block diagram of a computer system suitable for implementing the face image processing method according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. It should be understood, however, that these descriptions are exemplary only, and are not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the present disclosure. The terms "comprising", "comprising", etc. used herein indicate the presence of stated features, steps, operations and/or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

All terms (including technical and scientific terms) used herein have the meaning commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted to have a meaning consistent with the context of this specification, and not be interpreted in an idealized or overly rigid manner.

Where expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted as those skilled in the art would normally understand the expression (for example, "having A, B, and C A system of at least one of "shall include, but not be limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ). Where expressions such as "at least one of A, B, or C, etc." are used, they should generally be interpreted as those skilled in the art would normally understand the expression (for example, "having A, B, or C A system of at least one of "shall include, but not be limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ). Those skilled in the art should also understand that virtually any transitional conjunction and/or phrase that represents two or more alternative items, whether in the specification, claims, or drawings, should be understood to include these Possibility of one of the items, either of those items, or both. For example, the phrase "A or B" should be read to include the possibilities of "A" or "B," or "A and B."

Embodiments of the present disclosure provide a method and device for face image processing, which can automatically cluster multiple face images to obtain multiple person image groups corresponding to the multiple face images. Since each person has Unique face features, when the face images of the same person are at different angles, there is still a strong correlation between the face features of the face images of the same person corresponding to different angles, therefore, it can be used for multiple The face images are clustered, so that the clustering result is that multiple face images from different angles corresponding to the same person are clustered in the same face image group. In this way, multiple face images can be automatically marked: which face image group should a face image correspond to, wherein the face images in each face image group correspond to the same person. The obtained face image group can be used to train the face recognition model. Since the face image group contains the face images of the same person from different angles, the recognition accuracy of the face recognition model trained based on this is better than that of The existing face recognition models trained based on standard face images have higher recognition accuracy when applied to scenes with multi-angle face recognition.

FIG. 1 schematically shows an exemplary system architecture 100 to which face image processing can be applied according to an embodiment of the present disclosure. It should be noted that, what is shown in FIG. 1 is only an example of the system architecture to which the embodiments of the present disclosure can be applied, so as to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other device, system, environment or scenario.

As shown in FIG. 1 , a system architecture 100 according to this embodiment may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 . The network 104 is used as a medium for providing communication links between the terminal devices 101 , 102 , 103 and the server 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others. The system architecture 100 can be applied to the company's face attendance check-in system, community management, building access control system, etc. Since it can realize multi-angle face image recognition, it makes it more convenient for users to use without having to face the camera, which can improve user experience. ; In addition, it can also be applied to offline store systems, such as member identification, facial recognition payment, etc., such as when a user enters the store, collect and identify whether he is a member of the store through the surveillance camera, and identify the payer through the surveillance camera to confirm the payer identity information, which facilitates the realization of face-swiping payment methods.

Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages and the like. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as image collection applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social networking platform software, etc.

The terminal devices 101, 102, 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc., wherein the terminal devices 101, 102 , 103 may also include information collection equipment, for example, image collection equipment, sound collection equipment, etc., specifically, the image collection equipment may be an external camera, such as a monitoring camera, or an airborne camera, such as a camera that comes with a laptop computer etc., the sound collection device may be a microphone or the like.

The server 105 may be a server that provides various services, such as clustering the face images collected by the user using the terminal devices 101, 102, 103, processing the collected video into frames, identifying and obtaining human faces from the video frames. Face image, get the group center of the face image group, calculate the rotation angle of each face image in the face image group relative to the group center, build the face recognition model, face recognition model training, and use the trained face The recognition model performs face recognition, etc. The background management server can analyze and process received data such as user requests, and feed back processing results (such as webpages, information, or data obtained or generated according to user requests) to the terminal device.

It should be noted that, generally, the face image processing method provided by the embodiment of the present disclosure may be executed by the server 105 . Correspondingly, the clustering module of the face image processing apparatus provided in the embodiment of the present disclosure can generally be set in the server 105 . The face image processing method provided by the embodiments of the present disclosure may also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101 , 102 , 103 and/or the server 105 . Correspondingly, the face image processing apparatus provided by the embodiments of the present disclosure may also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101 , 102 , 103 and/or the server 105 .

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Fig. 2 schematically shows human face images from different angles according to an embodiment of the present disclosure. It can be seen from the figure that the five face images in Figure 2 correspond to the same person, but the angles of the faces in each picture are different. When performing face recognition, the 5 face images in the picture may be judged as 5 different people. This is because the training data set in the prior art is generally a standard face recognition data set such as: ALFW, CASIA-WebFace, The standard face recognition data set provided by MS-Celeb-1M, etc., and the face images in the standard face recognition data set usually include one or more of the following characteristics: frontal face, no depression angle, single background, facial features Clear or unobstructed, for example, passport photos, etc. The face recognition model trained by using these standard face recognition data sets has a better effect on standard face recognition. However, in the existing technology, the demand for face recognition is not only for standard face images. For example, for offline unmanned retail and smart retail scenarios, the technical basis It is a high-precision recognition and detection of customer faces. In the above scenarios, face information is generally collected through surveillance cameras installed in the store. Different from the standard face images captured in the standard face image collection environment, the face images captured by the surveillance cameras in the above-mentioned open environment often cannot meet the collection requirements of standard face images. For example, in the collected face images The human face may be a frontal face, a side face, a crooked face, etc. In addition, the camera is often placed on the wall to produce a shooting depression angle, etc. In summary, the face recognition models trained by the existing standard-based face recognition datasets cannot meet the face recognition requirements in scenarios such as offline unmanned retail and smart retail.

The embodiments of the present disclosure provide a face image processing method and device. Considering that each person has their own special facial features, even if the face image has a certain angle with respect to the front face image, the face image of the same person at different angles There is still a large correlation between the facial features of the face image. Therefore, the clustering result can make the face images of different angles of each person clustered and clustered in the same face image group, which actually achieves The function of automatically labeling the multiple face images, for example, the face image group 1 and the face images from different angles corresponding to the face image group 1, so that the face images obtained by clustering can be used group for face recognition model training, for example, the face features of one or more people's multi-angle face images and the face image groups to which they belong can be input into a pre-built face recognition model for model training, which can be trained to meet The multi-angle face image recognition model can at least partially solve the problem of using the existing standard face recognition data set for training. Face images lead to the problem of very low recognition accuracy, and therefore the technical effect of improving the recognition accuracy of multi-angle face images can be achieved.

Fig. 3A schematically shows a flowchart of a method for processing a face image according to an embodiment of the present disclosure.

As shown in FIG. 3A , the method includes, in operation S301 , acquiring multiple face images, where the multiple face images include face images of at least one person from different angles.

Specifically, the plurality of face images may be multiple photos of at least one person taken from different angles by a photographing device, or videos of at least one person with different angles recorded by a video recording device, and then obtained from video frames Face images from different angles. In addition, it can also be multiple pictures of at least one person captured from the Internet. For example, first obtain multiple face images and corresponding identity information from the Internet, and then according to the identity information, such as mobile phone number, name, and identification number Select a specified number of identity information as a candidate face image, and then obtain multiple images of at least one person containing different angles through one or more operations in identity information comparison, removing duplicate images, and selecting face images from different angles. A picture of the face image, and then get the face image from the picture.

Then, in operation S302, the multiple face images are clustered to obtain multiple face image groups.

Specifically, since each person has its own special facial features, the facial features of the face images of the same person at different angles are correlated. Therefore, the clustering result can make the face images of each person at different angles clustered, are gathered in the same face image group, so that the function of automatically labeling the multiple face images is actually realized, for example, face image group 1 and a plurality of different angles corresponding to face image group 1 A face image, ... a face image group n and a plurality of face images from different angles corresponding to the face image group n, wherein n is a positive integer ≥ 1. The clustering method can be the same as the prior art, for example, first extract the facial features from each facial image, then cluster the facial features to obtain a plurality of facial image groups, and the corresponding facial images in each facial image group same person.

It should be noted that, as shown in FIG. 3B , it schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure. In this embodiment, the face image processing method may also include the following operations:

In operation S303, part of the face images in at least one face image group are removed according to user input and/or preset rules, so that each face image in each face image group in the at least one face image group corresponds to to the same person.

Because the results obtained by clustering may have errors, for example, the face images of different people are clustered in the same face image group, in order to improve the accuracy rate, the same face image group can be manually and/or based on certain rules Each face image in each face image group is checked and corrected to ensure that each face image in each face image group corresponds to the same person. As shown in FIG. 3C , a schematic diagram of a face image group according to an embodiment of the present disclosure is schematically shown. Specifically, the above-mentioned terminal devices 101, 102, 103, etc. can display each group of face images, wherein the group of face images can be displayed in multiple rows or columns in the candidate area, wherein the identification of the group of face images It can be the name of the face image group, for example, face image group 1, ... face image group n, etc., or a certain face image in the face image group can be used as the face image group identification, for example, the closest to the front face face image, the face image with the minimum sum of distances from other face images in the face image group, etc. In addition, the displayed content can also include an identification result area, which is used to display a specific one or more face image groups, and can also be used to correct and review face images that are automatically labeled incorrectly, so as to better improve the face recognition. The correctness of the image group.

The face image processing method provided by the embodiments of the present disclosure can use the face image group obtained by clustering to perform face recognition model training, for example, the face features and the associated face images of one or more people's multi-angle face images can be The face image group is input with a pre-built face recognition model for model training, and a model that can meet multi-angle face image recognition can be trained, which can at least partially solve the problem of using existing standard face recognition data sets for training. Model, when performing face recognition on the face images captured by the surveillance camera, due to the existence of face images from different angles, the recognition accuracy is very low, and therefore the technical effect of improving the recognition accuracy of multi-angle face images can be achieved.

The method shown in FIG. 3A will be further described below in conjunction with specific embodiments with reference to FIGS. 4A to 4C . FIG. 4A schematically shows a flow chart of a method for acquiring multiple human face images according to an embodiment of the present disclosure.

In this embodiment, the acquisition of multiple face images may include the following operations:

In operation S401, a video is acquired. Among them, the camera can be used to rotate and shoot around someone to obtain a video containing multiple face images of someone from different angles; in addition, it can also be a fixed camera, allowing someone to rotate at a certain angle in front of the camera, so that the camera A video is captured that contains multiple images of a person's face from different angles. In addition, in order to make the face images in the finally obtained face image group more similar to the face images acquired in the actual use environment, when the layout of the specific use environment is known, the same layout as the actual use environment can be used. The arrangement of the camera, or directly using the camera in the actual use environment, and then using a similar flow of people, such as the flow of people paying by swiping the face, to shoot multiple videos from different angles containing at least one person, and the face image group obtained accordingly The face image is more in line with the face image obtained in the actual application scene, and the recognition accuracy of the face recognition model trained based on this is higher.

In operation S402, the face image in the video is acquired in units of frames.

In this embodiment, each frame of at least two frames can be used as an object, and the face image in each frame can be extracted. Since in the video shooting process, different moments correspond to faces from different angles, therefore, in different frames The angles of the face images can be different, and the face images of different angles can be extracted from different frames. In addition, FIG. 4B schematically shows a flowchart of a method for acquiring multiple human face images according to another embodiment of the present disclosure.

In operation S401, video is acquired, which may be as in the previous embodiment, and will not be described in detail here.

In operation S403, video frames of the video are acquired frame by frame or at intervals of a set number of frames. In this embodiment, video frames frame1, frame2, . Since the payer does not appear in the video for a long time in the face-swiping payment situation, in order to obtain as many face images of the payer as possible from different angles, the video frame of the video can be obtained frame by frame. video frame.

In operation S404, a face image of the video frame is acquired. In this embodiment, an existing method for obtaining a face image from an image may be used.

FIG. 4C schematically shows a flow chart of a method for acquiring a face image from a video frame according to an embodiment of the present disclosure. In this embodiment, the following operations may be included:

In operation S4041, a face detection frame is set. Specifically, the size of the face detection frame can be determined according to the distance between the camera and the face, or can be determined according to other rules, so that the window of the face detection frame is a face window if and Only when it just frames a human face, that is, the size of the window is consistent with the size of the human face, and the window basically fits the outer contour of the human face. In addition, when multiple face images appear in the video, and the distance between each person and the camera is different, the size of each face image in the frame image will be different. At this time, the face image cannot be detected with the same size face detection frame , at this time, it is possible to use face detection frames of different sizes to detect faces, or use face detection frames of the same size and scale the frame image to detect each face image in the frame image.

In operation S4042, human face features of the portion of the video frame covered by the human face detection frame are extracted. In this embodiment, face feature extraction is carried out, and the face feature is corresponding to the above-mentioned facial feature, and the face feature is a vector representation of the face feature obtained by feature extraction of the face image by a computer, that is, Feature vector, such as X=(x1, x2,...,xd), d is a positive integer greater than or equal to 1, and each dimension is a value, which is calculated according to the input (image area), such as summing , subtract, compare sizes, etc. The feature extraction process is the process of transforming the original input data (a matrix composed of color values arranged in the image area) to the corresponding feature vector.

In operation S4043, the face features are input into a pre-trained face model to obtain a recognition result. Among them, the face model can be a classifier, a neural network, a random forest model, etc., and a classifier is used as an example for illustration. The classifier uses a feature vector as an input, and uses a category as an output through mathematical calculation. Each category will correspond to a Numerical encoding is called the label corresponding to this category. For example, the category of face window is encoded as 1, and the category of non-face window is encoded as -1; the classifier is a function that transforms the feature vector into a category label , as shown in formula (1):

Among them, the parameter t can be obtained by means of model training, for example, the correct face image is marked as 1, and the incorrect face image is marked as -1, and then the above-mentioned face image with label information is input into the above model , until the consistency between the model output and the labeled information reaches the set threshold.

In operation S4044, a face image is determined according to the recognition result. Specifically, the frame image corresponding to the face detection frame whose output result of the model is a face image is taken as the face image.

In a specific embodiment, the clustering of the plurality of face images includes: clustering the face features of the face images based on K-means or hierarchical clustering methods to obtain at least one face image groups and the face images corresponding to each face image group. In this way, it is possible to cluster multiple face images to obtain at least one face image group.

As shown in FIG. 5 , it schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure. In this embodiment, after obtaining the plurality of face image groups, the method may also include the following operations:

In operation S501, group centers of at least one human face image group are obtained respectively. Specifically, the group center can be the face image with the smallest distance from other face images in the face image group, where the distance can be the distance between the face features (feature vectors) of the two face images Cosine distance, etc., the larger the distance, the greater the difference between the two face images, and the smaller the distance, the more similar the two face images.

In operation S502, the center of the group is used as an identifier of the corresponding face image group.

Since the group center is the face image with the smallest distance from other face images in the face image group, that is to say, the group center can better represent other face images in the face recognition group, therefore, the It is more representative as an identification of a face image group.

As shown in FIG. 6 , it schematically shows a flow chart of a method for obtaining a group center of a face image group according to an embodiment of the present disclosure. In this embodiment, for each face image group in the at least one face image group, the following operations may be included:

In operation S601, the distances between each face image in the face image group and other face images are respectively calculated. Specifically, suppose that there are m face images in a certain face image group, corresponding to m face features: X1, X2, ... Xm, where X1 is the face feature of the first face image, which is n-dimensional Feature vector, X1=(a1, a2,...an), X2 is the face feature of the second face image, X2=(b1, b2,...bn), then the first face image (face 1) and The distance between the second face image (face 2) is shown in formula (2):

Wherein, i is a positive integer ≥ 1.

In operation S602, the face image with the smallest sum of distances from other face images in the face image group is taken as the group center.

First, calculate the average distance from each face image in the face image group to other face images in the face image group; take a face image group with only 3 face images as an example, the above average distance can be obtained by Formula (3) obtains:

Wherein, L_face 1 is the average distance of face image 1 relative to other face images in the place face image group, and L_face 2 is the average distance of face image 2 relative to other face images in the place face image group, L_face3 is the average distance between face image 3 and other face images in the face image group.

Correspondingly, the central face can be determined by formula (4):

Center face = min(L_face1, L_face2, L_face3) (4)

The group center can be obtained through the above operations.

As shown in FIG. 7 , it schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure. In this embodiment, after obtaining the group center, the method may further include the following operations:

In operation S701, for the group of human face images, respectively calculate the rotation angle of each human face image in the group of human face images relative to the center of the group. Specifically, the rotation angle of the face image relative to the group center can be determined according to the position relationship between the key points in the face image and the position relationship between the key points in the group center, for example, according to the distance between the eyes of the group center and the distance between the eyes in the face image to determine the rotation angle of the face image relative to the group center in the direction of the eyes. Of course, the horizontal and vertical rotation angles of the face image relative to the group center can be determined according to the positional relationship among more key points.

In operation S702, use the obtained rotation angle to perform angle labeling on the corresponding face image. Specifically, through the above operation, the rotation angle of each face image in the face image group relative to the group center can be obtained, and then the rotation angle information can be marked for each face image in the face image. The rotation angle can be used as a basis for screening training data in specific application scenarios. For example, in the face-swiping payment scenario, since the camera usually uses a surveillance camera, and the surveillance camera is usually at a certain angle to the payer's frontal face, and the angle is basically within a general range, if people with various angles are used If the face image group of the face image is used as the training data set of the face recognition model, the angle range of the face images that can be recognized by the face recognition model is relatively large. Optimization, the accuracy of face recognition at this specific angle cannot be further improved. In this embodiment, since the rotation angle information of the face image is also marked in the face image group, the face image with a specific rotation angle can be selected from a plurality of face image groups as the training data of the face recognition model This can help improve the recognition accuracy of the face recognition model at a specific rotation angle, or train the face recognition model for a specific rotation angle according to a specific application scenario.

As shown in FIG. 8 , it schematically shows a flowchart of a method for calculating a rotation angle according to an embodiment of the present disclosure. In this embodiment, for each face image, the following operations may be included:

In operation S801, a specified number of key points of the face image and a specified number of key points of the group center are obtained. Wherein, the key points include, but are not limited to: one or more of eye center points, mouth corner points, nose vertices, cheekbone vertices or brow vertices. As shown in FIG. 9 , it schematically shows a schematic diagram of a human face image and key points in a video frame according to an embodiment of the present disclosure, wherein the image in the box is a human face image recognized from the video frame, and in the picture rectangular area. Represented as [x, y, w, h], (x, y) represents the position of the upper left corner of the picture relative to the face image, (w, h) represents the width and height of the box; the bright spot in the face image is Each key point can be expressed as kp_1, kp_2, ..., kp_N, where N represents the number of key points; kp_N is a two-dimensional data, representing the position coordinates relative to the upper left corner of the picture, and each face extracts N key points .

The key point detection technology may be SDM-based face key point detection and/or MTCNN-based face key point detection technology. That is to say, face detection can be done first, and then key point detection can be performed on the detected face detection frame separately, such as using SDM technology; face detection and key point recognition can also be performed simultaneously, such as using MTCNN technology.

For the SDM technology, the following operations may be included: first, modeling is performed, the input of the model is a face image, and the output of the model is the key points in the face image. The model training process may include the following operations: input a human face image with artificially labeled information into the model, and adjust the model parameters to make the feature points output by the model approach the key points of the manually labeled human face, wherein the manually labeled information is Keypoints manually annotated.

For multi-task convolutional neural networks (MTCNN) technology, three kinds of neural networks can be used to form the MTCNN, including: Proposal Network (P-Net), RefineNetwork (R-Net) and Output Network (O -Net). Among them, the network structure of P-Net mainly obtains the regression vector of the candidate window and the bounding box of the face image. And use the bounding box for regression, calibrate the candidate windows, and then merge highly overlapping candidate windows through non-maximum suppression (NMS). The network structure of R-Net still uses bounding box regression and NMS to remove false-positive (false-positive) areas, but the network structure of R-Net is different from that of P-Net, and there is an additional fully connected layer, so it will get Better suppression of false-positive regions. O-Net's network structure has one more convolutional layer than R-Net's network structure, so the processing results will be more refined. The image has undergone more supervision, and at the same time it will output 5 landmarks, that is, key points, as shown in Figure 10, which schematically shows a schematic diagram of key points according to an embodiment of the present disclosure, where each face The image has 5 key points, of course, more or less key points can be set according to actual needs.

In operation S802, the positional relationship between the key points of the face image is acquired, and the positional relationship between the key points of the group center is acquired. The above positional relationship includes but is not limited to: the coordinates of key points relative to the upper left corner of the face image, the relative coordinates between key points of the same type (such as the relative coordinates between the centers of two eyes), and the distance between key points any one or more of these.

In operation S803, a rotation angle of the face image relative to the group center is determined according to a positional relationship between key points of the face image and a positional relationship between key points of the group center. Specifically, a horizontal and/or vertical rotation angle of the face image relative to the group center is obtained through calculation.

As shown in FIG. 11 , it schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure. In this embodiment, the method may also include the following operations:

In operation S111, for a group of human face images, the face images in the group of human face images are sorted according to the rotation angle and/or distance between the face images and the center of the group. Among them, the rotation angle is the rotation angle of each face image in the face image group relative to the group center, and the distance can be a cosine distance, which means the similarity between each face image and the group center. Specifically, it can be the face The cosine distance between the feature and the face feature at the center of the group.

In operation S112, the sorted face images are displayed. In this embodiment, it may be necessary to manually check the face images in the face image groups to ensure that the face images in each face image group correspond to the same person. If there are many face images in the face image group, and some face images have a large rotation angle relative to the group center, it is difficult to directly compare the face image and the group center verification method manually. If according to the similarity and/or Sorting the face images in the face image group, or rotation angle, etc., can show a gradual process. If it is difficult to determine whether the clustering is correct by directly comparing the group center and the face image, you can refer to the sorting and display The gradual change process is used to assist in determining whether or not clustering is correct. This will help to improve the convenience of manual verification, and it will be less likely to cause misjudgments or missed judgments.

As shown in FIG. 12 , it schematically shows a flow chart of a method for processing a face image according to another embodiment of the present disclosure. In this embodiment, the method may also include the following operations:

In operation S121, a face recognition model is trained using a plurality of face image groups. The face recognition model can be a newly constructed model; it can also be a model trained using an existing standard training data set, and then use multiple face image groups to retrain the trained model, that is, use multiple face image groups Fine-tuning the above-mentioned trained model can greatly improve the face recognition accuracy of the face recognition model in the prior art in the scene of multi-angle face recognition.

In a specific embodiment, the first face recognition model is pre-built, and after obtaining the multiple face image groups, the face features of the face images from different angles corresponding to the face image groups, and the corresponding face images The group inputs the first face recognition model for training to obtain model parameter values. The face recognition of multi-angle face images can be realized by using the trained first face recognition model.

In another specific embodiment, for the second face recognition model obtained by pre-training with standard face data, after obtaining the multiple face image groups, the face images of different angles corresponding to the face image groups are The face features and the associated face image group are input into the second face recognition model for training, and adjusted model parameter values are obtained. The face recognition of multi-angle face images can be realized by using the trained second face recognition model, and the recognition accuracy is high.

In yet another specific embodiment, the third face recognition model is pre-built. After obtaining the multiple face image groups and the rotation angles between the face images and the group centers of the face image groups to which they belong, first, obtain the application scene Then, use the face image with angle label information, the face image group to which the face image belongs, and the face rotation angle to obtain the model parameter value of the third face recognition model. It should be noted that the model parameter value of the third face recognition model obtained by using the face image with angle label information, the face image group to which the face image belongs, and the face rotation angle can be in the following two forms: One of the two types is implemented separately, or implemented in combination: the first type, first select the face images with a specific angle from the face image group, and then use the face images with angle annotation information and the face images to which the face images belong The group is input to the third face recognition model for training to obtain model parameters; the second method is to directly input the face image with angle label information, the face image group to which the face image belongs, and the face rotation angle into Training is performed in the third face recognition model to obtain model parameters.

In other specific embodiments, for the fourth face recognition model obtained by using standard face data training in advance, after obtaining the plurality of face image groups, and the rotation angle between the face image and the group center of the face image group to which it belongs Afterwards, first obtain the face rotation angle of the application scene; then, use the face image with angle label information, the face image group to which the face image belongs, and the face rotation angle to obtain the fourth face recognition model adjustment After the model parameter values.

Correspondingly, the present disclosure also provides a face image processing device 1300 corresponding to the above-mentioned face image processing method, and FIG. 13 schematically shows a block diagram of a face image processing device according to an embodiment of the present disclosure.

As shown in FIG. 13 , the face image processing apparatus includes: an acquisition module 131 and a clustering module 132 .

Wherein, the obtaining module 131 is used for obtaining multiple face images, and the multiple face images include face images of at least one person from different angles.

The clustering module 132 is used for clustering the multiple face images to obtain multiple face image groups.

In addition, FIG. 14 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may further include:

The correction module 141 is used to remove some face images in at least one face image group according to user input and/or preset rules, so that each face image in each face image group in the at least one face image group correspond to the same person.

Wherein, the acquisition module 141 includes:

The video acquisition unit is used for acquiring video.

The video frame acquisition unit is used to acquire video frames of the video frame by frame or at intervals of a set number of frames.

A human face image acquiring unit, configured to acquire the human face image of the video frame.

In a specific embodiment, the human face image acquisition unit includes:

The setting subunit is used to set the face detection frame.

The feature extraction subunit is used to extract the face features of the part of the video frame covered by the face detection frame.

The model training subunit is used for inputting the face features into the pre-trained face model to obtain the recognition result.

The image recognition unit is used to determine the face image according to the recognition result.

Wherein, the clustering module 132 is specifically used to cluster the face features of the face images based on K-means or hierarchical clustering methods to obtain at least one face image group and the face corresponding to each face image group. image.

In another embodiment, FIG. 15 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The group center acquisition module 151 connected to the clustering module 132 is configured to respectively acquire the group center of at least one face image group.

The identification module 152 is configured to use the cluster center as an identification of the corresponding face image group.

Wherein, the group center acquisition module 151 includes:

The distance calculating unit is used for calculating, for each face image group in the at least one face image group, the distance between each face image in the face image group and other face images.

The center obtaining unit is used for taking the face image with the smallest sum of distances from other face images in the face image group as the group center.

In yet another embodiment, FIG. 16 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The rotation angle calculation module 161 connected with the group center acquisition module 151 is used for calculating the rotation angle of each face image in the group center relative to the group center for the group of face images.

The angle labeling module 162 is used to use the obtained rotation angle to perform angle labeling on the corresponding face image.

Wherein, the rotation angle calculation module 161 includes:

The key point obtaining unit is used for obtaining a specified number of key points of the human face image and a specified number of key points of the center of the group for each of the face images.

The positional relationship acquiring unit is used to acquire the positional relationship between the key points of the face image, and acquire the positional relationship between the key points of the group centers.

The rotation angle calculation unit is used to determine the rotation angle of the face image relative to the group center according to the positional relationship between the key points of the face image and the positional relationship between the key points of the group center.

Specifically, the key point acquisition unit is specifically configured to obtain the key points based on SDM and/or MTCNN face key point detection.

In other embodiments, FIG. 17 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The sorting module 171 is used for sorting the face images in the face image group according to the rotation angle and/or distance between the face images and the group center.

The display module 172 is used for displaying the sorted face images.

In yet another embodiment, the device may further include the following module: a face recognition model training module, configured to use the multiple face image groups to train the face recognition model after obtaining the multiple face image groups.

In a specific embodiment, FIG. 18 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The first face recognition model construction module 181 is used to pre-build the first face recognition model.

The first face recognition model training module 182 is used for inputting the face features of the face images of different angles corresponding to the face image groups and the corresponding face image groups into the first face image group after obtaining the plurality of face image groups. A face recognition model to obtain model parameter values.

In yet another specific embodiment, FIG. 19 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The second face recognition model construction module 191 is used to pre-build the second face recognition model;

The second face recognition model training module 192 is used to train the pre-built second face recognition model using standard face data to obtain the second face recognition model;

The second face recognition model adjustment module 193 is used to input the face features of the face images from different angles corresponding to the face image groups and the corresponding face image groups into the first face image group after obtaining the plurality of face image groups. The two-person face recognition model obtains the adjusted model parameter values.

In yet another specific embodiment, FIG. 20 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The third face recognition model construction module 201 is used to pre-build the third face recognition model;

Rotation angle obtaining module 202 is used for obtaining the face rotation angle of application scene;

The third face recognition model parameter acquisition module 203 is used to use the face images with angle annotation information, The face image group to which the face image belongs and the face rotation angle obtain the model parameter values of the third face recognition model.

In yet another specific embodiment, FIG. 21 schematically shows a block diagram of a face image processing device according to another embodiment of the present disclosure, and the device may also include the following modules:

The fourth face recognition model construction module 211 is used to pre-build the fourth face recognition model;

The fourth face recognition model training module 212 is used to train the pre-built fourth face recognition model using standard face data to obtain the fourth face recognition model;

Rotation angle obtaining module 213 is used for obtaining the face rotation angle of application scene;

The fourth face recognition model parameter acquisition module 214 is used to use the face images with angle annotation information, The face image group to which the face image belongs and the face rotation angle obtain the adjusted model parameter values of the fourth face recognition model.

Since the face recognition device corresponds to the face recognition method, the face recognition device will not be described in detail. For details, please refer to the relevant parts of the face recognition method.

Modules, sub-modules, units, any multiple of sub-units according to the embodiments of the present disclosure, or at least part of the functions of any multiple of them may be implemented in one module. Any one or more of modules, submodules, units, and subunits according to the embodiments of the present disclosure may be implemented by being divided into multiple modules. Any one or more of modules, submodules, units, and subunits according to embodiments of the present disclosure may be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), system-on-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or hardware or firmware that may be implemented by any other reasonable means of integrating or packaging circuits, or in a combination of software, hardware, and firmware Any one of these implementations or an appropriate combination of any of them. Alternatively, one or more of the modules, submodules, units, and subunits according to the embodiments of the present disclosure may be at least partially implemented as computer program modules, and when the computer program modules are executed, corresponding functions may be performed.

For example, any number of acquisition module 131, clustering module 132, correction module 141, group center acquisition module 151, identification module 152, sorting module 171, and angle labeling module 162 can be combined into one module, or one of them Any module can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 131, the clustering module 132, the correction module 141, the group center acquisition module 151, the identification module 152, the sorting module 171, and the angle labeling module 162 can be at least partially implemented as Hardware circuits such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), System-on-Chip, System-on-Substrate, System-on-Package, Application-Specific Integrated Circuits (ASICs), or circuits that can be integrated or packaged Any other reasonable way, such as hardware or firmware, or any one of the three implementations of software, hardware and firmware, or an appropriate combination of any of them. Or, at least one of the acquisition module 131, the clustering module 132, the correction module 141, the group center acquisition module 151, the identification module 152, the sorting module 171, and the angle labeling module 162 can be at least partially implemented as a computer program module, when When the computer program module is executed, it can perform corresponding functions.

Fig. 22 schematically shows a block diagram of a computer system suitable for implementing the methods described above according to an embodiment of the present disclosure. The computer system shown in FIG. 22 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 22 , a computer system 2200 according to an embodiment of the present disclosure includes a processor 2201 that can be loaded into a random access memory (RAM) 2203 according to a program stored in a read-only memory (ROM) 2202 or from a storage section 1408 Various appropriate actions and processing are performed by the program. The processor 2201 may include, for example, a general-purpose microprocessor (eg, a CPU), an instruction set processor and/or related chipsets and/or a special-purpose microprocessor (eg, an application-specific integrated circuit (ASIC)), and the like. Processor 2201 may also include on-board memory for caching purposes. The processor 2201 may include a single processing unit or a plurality of processing units for executing different actions of the method flow according to the embodiments of the present disclosure.

In the RAM 2203, various programs and data necessary for the operation of the system 2200 are stored, for example, for storing executable instructions. The processor 2201 , the ROM 2202 and the RAM 2203 are connected to each other through the bus 2204 . The processor 2201 executes various operations according to the method flow of the embodiment of the present disclosure by executing programs in the ROM 2202 and/or RAM 2203 . It should be noted that the program may also be stored in one or more memories other than the ROM 2202 and RAM 2203 . The processor 2201 may also perform various operations according to the method flow of the embodiment of the present disclosure by executing the programs stored in the one or more memories, for example, causing the one or more processors to acquire multiple faces image, the plurality of face images includes at least one person’s face images from different angles; the plurality of face images are clustered to obtain a plurality of face image groups, which can also be input and/or preset according to the user Part of the face images in the at least one face image group is removed according to the rule, so that the face images in each face image group in the at least one face image group correspond to the same person.

According to an embodiment of the present disclosure, the system 2200 may also include an input/output (I/O) interface 2205 which is also connected to the bus 2204 . System 2200 may also include one or more of the following components connected to I/O interface 2205: an input section 2206 including a keyboard, mouse, etc.; including components such as a cathode ray tube (CRT), liquid crystal display (LCD), etc.; etc.; a storage section 2208 including a hard disk or the like; and a communication section 2209 including a network interface card such as a LAN card, a modem, or the like. The communication section 2209 performs communication processing via a network such as the Internet. A drive 2210 is also connected to the I/O interface 2205 as needed. A removable medium 2211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 2210 as necessary so that a computer program read therefrom is installed into the storage section 2208 as necessary.

According to the embodiments of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 2209, and/or installed from removable media 2211. When the computer program is executed by the processor 2201, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to the embodiments of the present disclosure, the above-described systems, devices, devices, modules, units, etc. may be implemented by computer program modules.

The present disclosure also provides a computer-readable medium. The computer-readable medium may be included in the device/device/system described in the above embodiments; it may also exist independently without being assembled into the device/device/system. system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed, various methods according to the embodiments of the present disclosure are realized.

According to an embodiment of the present disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, cable, optical cable, radio frequency signal, etc., or any suitable combination of the above.

For example, according to an embodiment of the present disclosure, the computer-readable medium may include one or more memories other than the ROM 2202 and/or the RAM 2203 and/or the ROM 2202 and the RAM 2203 described above.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions. Those skilled in the art can understand that various combinations and/or combinations of the features described in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the various embodiments of the present disclosure and/or the features described in the claims can be combined and/or combined in various ways. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the various embodiments have been described separately above, this does not mean that the measures in the various embodiments cannot be advantageously used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims (18)

1. A face image processing method, comprising: Acquiring multiple face images, the multiple face images include at least one person's face images from different angles; Clustering the multiple face images to obtain multiple face image groups Partial face images in at least one face image group are removed according to user input and/or preset rules, so that each face image in each face image group in the at least one face image group corresponds to the same person. 2. The method according to claim 1, wherein said acquiring a plurality of face images comprises: get video; Acquiring the video frames of the video frame by frame or by setting the number of frames at intervals; Obtain the face image of the video frame. 3. The method according to claim 1, further comprising, after obtaining said plurality of human face image groups: Respectively obtain the group center of at least one face image group; The center of the group is used as the identifier of the corresponding face image group. 4. method according to claim 3, wherein, the group center that described obtaining at least one human face image group respectively comprises, for each human face image group in the described at least one human face image group: Calculate the distance between each face image and other face images in the face image group respectively; The face image with the smallest sum of distances from other face images in the face image group is taken as the group center. 5. The method according to claim 3, further comprising, after obtaining the group center: For the face image group, calculate the rotation angle of each face image in the face image group with respect to the group center; Use the obtained rotation angle to annotate the angle of the corresponding face image. 6. The method according to claim 5, wherein, said calculating the angle of rotation of each face image in the face image group with respect to the group center comprises, for said each face image: Obtaining a specified number of key points of the face image, and obtaining a specified number of key points of the group center; Obtaining the positional relationship between the key points of the face image, and obtaining the positional relationship between the key points of the group center; According to the positional relationship between the key points of the human face image and the positional relationship between the key points of the group center, determine the rotation angle of the human face image relative to the group center. 7. The method of claim 5, further comprising: For this group of human face images, sort the human face images in the group of human face images according to the rotation angle and/or distance between the human face images and the center of the group; and Display the sorted face images. 8. The method according to any one of claims 1-7, further comprising, after obtaining a plurality of groups of human face images, using the plurality of groups of human face images to train a face recognition model. 9. A face image processing device, comprising: An acquisition module, configured to acquire a plurality of face images, the plurality of face images including at least one person's face images from different angles; A clustering module, configured to cluster the multiple face images to obtain multiple face image groups; A correction module, configured to remove some face images in at least one face image group according to user input and/or preset rules, so that each face image in each face image group in the at least one face image group correspond to the same person. 10. The device according to claim 9, wherein the acquiring module comprises: A video acquisition unit, configured to acquire video; A video frame acquisition unit, configured to acquire the video frames of the video frame by frame or at intervals of a set number of frames; A human face image acquiring unit, configured to acquire the human face image of the video frame. 11. The apparatus of claim 9, further comprising: A group center acquisition module connected to the clustering module, used to respectively acquire the group center of at least one face image group; An identification module, configured to use the cluster center as the identification of the corresponding face image group. 12. The device according to claim 11, wherein the group center obtaining module comprises: A distance calculation unit, for each face image group in the at least one face image group, respectively calculate the distance between each face image in the face image group and other face images; The center acquisition unit is configured to use the face image with the smallest sum of distances from other face images in the face image group as the group center. 13. The apparatus of claim 11, further comprising: A rotation angle calculation module connected to the group center acquisition module, for the face image group, respectively calculate the rotation angle of each face image in the face image group relative to the group center; The angle labeling module is used to use the obtained rotation angle to perform angle labeling on the corresponding face image. 14. The device according to claim 13, wherein the rotation angle calculation module comprises: a key point acquisition unit, for each of the face images, to obtain a specified number of key points of the face image, and to obtain a specified number of key points of the group center; a positional relationship acquiring unit, configured to acquire the positional relationship between the key points of the face image, and acquire the positional relationship between the key points of the group center; The rotation angle calculation unit is used to determine the rotation angle of the face image relative to the group center according to the positional relationship between the key points of the face image and the positional relationship between the key points of the group center. 15. The apparatus of claim 13, further comprising: A sorting module for sorting the face images in the face image group according to the rotation angle and/or distance between the face image and the group center for the face image group; and The display module is used to display the sorted face images. 16. The device according to any one of claims 9-15, further comprising: The face recognition model training module is used to use the multiple face image groups to train the face recognition model after obtaining the multiple face image groups. 17. A device for face image processing, comprising: one or more processors; A storage device for storing executable instructions, the executable instructions implement the method according to any one of claims 1-8 when executed by the processor. 18. A computer-readable medium, on which are stored executable instructions, which implement the method according to any one of claims 1-8 when executed by a processor.