CN110991307B - Face recognition methods, devices, equipment and storage media - Google Patents

Abstract

The embodiment of the present invention discloses a face recognition method, device, equipment and storage medium. The method includes: performing face recognition on a video to be detected; inputting the recognized face video into a live body detection model to determine whether the face in the face video is a live face; the live body detection model determines whether the face in the face video is a living face; It is obtained by training on live face videos, and the non-live face videos include videos of printed faces or videos of screen faces. The technical solution of the embodiment of the present invention performs face recognition on the video to be detected, which can prevent recognition errors caused by using photos to disguise faces for recognition when using picture recognition. Inputting the face video into the liveness detection model for live face detection reduces the number of external devices, reduces the difficulty of user cooperation, improves the accuracy of liveness detection, and improves the simplicity of the model.

Description

Face recognition methods, devices, equipment and storage media

Technical field

Embodiments of the present invention relate to the field of image processing, and in particular, to a face recognition method, device, equipment and storage medium.

Background technique

As the level of digitalization in our country's cities becomes higher and higher, face recognition is an important technology in digital cities. In order to prevent users from deceiving the face recognition system to complete verification through photos or videos in some important circumstances, distinguishing between live faces and video faces has gradually become an important technology related to property and personal security.

Existing liveness detection mostly refers to the user taking corresponding actions in accordance with the system instructions. After the user takes the corresponding actions in accordance with the system instructions, the user's completed actions are recognized in the background. However, the existing living body detection models with relatively accurate recognition effects mostly use action-based detection, such as video blink detection or nodding detection, while traditional face detection methods also include assisted detection through infrared equipment and assisted detection through 3D modeling.

However, action-based detection and recognition are difficult and unfriendly to user interactivity. For small and medium-sized project applications, adding infrared detection equipment and 3D modeling operations will greatly increase costs and reduce the competitiveness of products. Therefore, it is necessary to develop a face recognition method that reduces the difficulty of user cooperation and improves the accuracy and convenience of face recognition without increasing the cost of peripherals.

Contents of the invention

The present invention provides a face recognition method, device, equipment and storage medium to achieve the effect of improving the accuracy and convenience of live face recognition by the face recognition device.

In a first aspect, embodiments of the present invention provide a face recognition method, including:

Perform face recognition on the video to be detected;

The recognized face video is input into the liveness detection model to determine whether the face in the face video is a live face. The liveness detection model is trained based on the live face video and the non-living face video. The non-living face video Face videos include videos of printed faces or videos of screen faces.

Furthermore, before performing face recognition on the video to be detected, it also includes:

Live human face videos and non-living human face videos are trained using convolutional neural networks to obtain the live body detection model.

Further, the number of videos of living faces is smaller than the number of videos of non-living faces.

Further, the live face video and the non-live face video are trained using a convolutional neural network to obtain the live detection model, which includes:

Collect living face videos and non-living face videos;

Scale the living face video and the non-living face video to a preset size;

The scaled live face video and the non-live face video are trained using a convolutional neural network to obtain the live detection model.

Further, scaling the living face video and the non-living face video to a preset size includes:

Extract a preset number of pictures from each video at a preset time interval, and scale the extracted pictures to the preset size;

After linearly stretching the dense optical flow direction field in the extraction time period, the direction field picture is scaled to a preset size. The scaled extraction picture and the scaled direction field picture form the training picture of the convolutional neural network.

Further, the step of inputting the recognized face video into the living body detection model and determining whether the face in the face video is a living face includes:

According to the preset number of times, extract a preset number of pictures from each video at a preset time interval, and scale the extracted pictures to the preset size;

If the scaled extracted image is input to the liveness detection model and the number of times it is determined that the face in the face video is a live face reaches the threshold of times, then the face in the face video is a live face.

In a second aspect, embodiments of the present invention also provide a face recognition device. The face recognition device includes:

Video face recognition module, used for face recognition of videos to be detected;

A living face determination module, used to input the recognized face video into a living body detection model to determine whether the face in the human face video is a living human face. The living body detection model is based on the living human face video and the non-living human face video. As a result of training, the non-living face videos include videos of printed faces or videos of screen faces.

Further, the device also includes:

A model training module is used to train live face videos and non-live face videos using convolutional neural networks to obtain the live detection model.

In a third aspect, an embodiment of the present invention further provides a device, which includes:

one or more processors;

A storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the face recognition method as provided in any embodiment of the present invention.

In a fourth aspect, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform face recognition as provided by any embodiment of the present invention. method.

The embodiment of the present invention performs face recognition on the video to be detected; inputs the recognized face video into a living body detection model to determine whether the face in the face video is a living human face. The living body detection model determines whether the face in the face video is a living human face. It is obtained by training on non-living face videos, including videos of printed faces or videos of screen faces. Performing face recognition on the video to be detected can prevent recognition errors caused by using photos to disguise faces for recognition. Inputting the face video into the liveness detection model for live face detection reduces the number of external devices, reduces the difficulty of user cooperation, improves the accuracy of liveness detection, and improves the simplicity of the model.

Description of the drawings

Figure 1 is a flow chart of a face recognition method in Embodiment 1 of the present invention;

Figure 2 is a flow chart of a face recognition method in Embodiment 2 of the present invention;

Figure 3 is a flow chart of the training process of a living body detection model in Embodiment 2 of the present invention;

Figure 4 is a flow chart for controlling zooming of live face video and non-living face video in Embodiment 2 of the present invention;

Figure 5 is a schematic structural diagram of a face recognition device in Embodiment 3 of the present invention;

Figure 6 is a schematic structural diagram of a device in Embodiment 4 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It can be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for convenience of description, only some but not all structures related to the present invention are shown in the drawings.

Embodiment 1

Figure 1 is a flow chart of a face recognition method provided by Embodiment 1 of the present invention. This embodiment can be applied to the situation of living face recognition in face recognition. This method can be executed by a face recognition device. The face recognition device can be implemented by software and/or hardware, and the face recognition device can be configured on a computing device, specifically including the following steps:

Step 11: Perform face recognition on the video to be detected.

Among them, the video to be detected can be understood as the video obtained by the camera continuously shooting in the area to be detected.

Among them, face recognition can be understood as identifying face-like contours in the video to be detected collected by the camera, so as to extract the face video containing the face to be recognized for live detection.

Specifically, the camera continuously shoots the area to be detected, extracts pictures from the acquired video at fixed time intervals, and performs contour recognition. When the contour of the face is recognized, the previous moment of the picture is extracted until the facial contour is continuously recognized. The video at the moment after the last extracted picture is used as the recognized face video.

Optionally, face detection can use SSD face detection technology, MTCNN face detection technology, S3FD face detection technology, etc.

The camera can be any device that can capture video. This is not limited in the embodiment of the present invention. The fixed time interval can be set manually or can be determined based on the properties of the camera device itself.

Step 12: Input the recognized face video into the liveness detection model to determine whether the face in the face video is a live face. The liveness detection model is trained based on the live face video and the non-living face video. Non-living face videos include videos of printed faces or videos of screen faces.

Among them, the living body detection model can be understood as a model trained based on multiple videos of non-living faces and live human faces to distinguish between living faces and non-living faces. Optionally, the training method may use a convolutional neural network training method or a big data statistics method, which is not limited in this embodiment of the present invention.

Specifically, the recognized face video is input into the trained living body detection model, the recognized face video is extracted according to a preset number of times and at a preset time interval, and the living face features and live body detection in the picture are extracted. The features of living faces in the model are compared, and based on the number of the same features, it is judged whether the face in the recognized face video is a living face.

The embodiment of the present invention performs face recognition on the video to be detected; inputs the recognized face video into a living body detection model to determine whether the face in the face video is a living human face. The living body detection model determines whether the face in the face video is a living human face. It is obtained by training on non-living face videos, including videos of printed faces or videos of screen faces. Performing face recognition on the video to be detected can prevent recognition errors caused by using photos to disguise faces for recognition. Inputting the face video into the liveness detection model for live face detection reduces the number of external devices, reduces the difficulty of user cooperation, improves the accuracy of liveness detection, and improves the simplicity of the model.

Embodiment 2

Figure 2 is a flow chart of a face recognition method provided in Embodiment 2 of the present invention. The technical solution of this embodiment is further refined based on the above technical solution, and specifically includes the following steps:

Step 21: Use the convolutional neural network to train the live face video and the non-live face video to obtain the live detection model.

Among them, convolutional neural network can be understood as a type of feed-forward neural network that contains convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

Optionally, the number of live face videos should be smaller than the number of non-living face videos. The number of live face videos and non-live face videos required varies depending on the training method, and this is not specifically limited in the embodiment of the present invention.

Specifically, Figure 3 provides a flow chart of the training process of a living body detection model, which specifically includes the following steps:

Step 211: Collect live face videos and non-live face videos.

Among them, non-living face videos may include videos of printed faces or videos of faces on the screen.

Optional, there should be no less than 5,000 videos of living faces, and no less than 10,000 videos of non-living faces.

Step 212: Scale the living face video and the non-living face video to a preset size.

Optional, the default size can be 32 pixels * 32 pixels.

Specifically, Figure 4 provides a flow chart for controlling the scaling of living face videos and non-living face videos, which specifically includes the following steps:

Step 2121: Extract a preset number of pictures from each video at a preset time interval, and scale the extracted pictures to a preset size.

Specifically, pictures are extracted at preset time intervals in each video until the number of pictures reaches the preset number, and the width and height of the extracted pictures are scaled to the preset size. The preset size can be set manually or based on training. The model needs to be set up.

Optionally, the preset time interval can be 20ms, and the preset number of pictures can be 8, which is not specifically limited in the embodiment of the present invention.

Step 2122: After linearly stretching the dense optical flow direction field of the extraction time period, the direction field picture is scaled to a preset size. The scaled extraction picture and the scaled direction field picture constitute the training picture of the convolutional neural network.

Among them, the extraction time period can be understood as the time period between the first extracted picture and the last extracted picture.

Among them, the dense optical flow direction field can be understood as a point-by-point matching image calibration for the image. This field is the offset of all points on the image within a preset time period, and its set is called the dense optical flow direction. field.

Optionally, the dense optical flow direction field can be linearly stretched from [0-360] to [0-255].

For example, the dense optical flow direction field collected during the extraction time period is linearly stretched from [0-360] to [0-255], and the length and width of the direction field picture are scaled to 32 pixels times 32 pixels, and the The scaled direction field picture and the 8 extracted pictures collected during the extraction time period together form a 96-pixel by 96-pixel training picture for the convolutional neural network.

Step 213: Use the convolutional neural network to train the scaled live face video and the non-live face video to obtain the live detection model.

For example, the living body detection model can be trained using the caffe framework, in which the model consists of a basic network and a custom network. The basic network uses the mobilenetV2 model, and the custom network design can be

Dense(240)->Activation(relu)->BatchNormalization()->Dropout(0.25)->Dense(2)

Optionally, the optimization method can use the Adam method.

Step 22: Extract a preset number of pictures from each video at a preset time interval according to a preset number of times, and scale the extracted pictures to a preset size.

Specifically, pictures are extracted from the recognized face video at preset time intervals until the number of extracted pictures reaches the preset number, the above behavior is repeated the preset number of times, and the preset number of times multiplied by the preset number of pictures is obtained. Picture, scale the dense optical flow direction field picture and the extracted picture within the above preset number of extraction times to the preset size and combine them to generate a large image.

Optionally, the preset time interval can be 20ms, the preset number of pictures can be 8, the preset size can be 32 pixels * 32 pixels, and the preset number of times can be 3 times.

For example, a picture of a recognized face video is extracted every 20ms, and every 8 pictures in the extracted pictures are resized so that the width and height are scaled to 32 pixels * 32 pixels, and every 8 pictures are When extracting pictures, calculate the dense optical flow direction field during that period, scale the direction field picture to 32 pixels * 32 pixels, and finally combine each 8 extracted pictures with their corresponding dense optical flow direction field pictures to generate a large image.

Step 23: If the scaled extracted picture is input to the liveness detection model and the number of times that the face in the face video is a live face reaches the threshold of times, then the face in the face video is a live face.

Optionally, the number of times threshold may be 2/3 of the preset number of times or other proportions or values preset by the user, which is not specifically limited in this embodiment of the present invention.

Specifically, when the scaled extracted picture and its corresponding dense optical flow direction field picture are combined and input into the living body detection model and the number of times that the number of times it is output as a living human face reaches a preset number or a preset ratio, the recognized face video is considered The face in the video is a living face; otherwise, the face in the recognized face video is considered to be a non-living face.

For example, when the preset number of times is 3, the combination of pictures obtained for 3 consecutive times is input into the living body detection model. If two or 3 of the 3 detections are judged to be live faces, the recognized person is considered to be a face. The faces in the face video are live human faces.

The technical solution of this embodiment is to conduct convolutional neural network training on mobile phone live face videos and non-live face videos, build a live body detection model, extract pictures from the face videos to be recognized at fixed time intervals, and compare them with the corresponding time The dense optical flow direction field pictures in the image are scaled and combined and then input into the living body detection model. Based on the times threshold, it is judged whether the face to be recognized in the video is a living face, which solves the recognition errors caused by using photos to disguise faces in face recognition. , interactively recognizes difficult problems for users to cooperate with, making living face recognition more convenient and accurate.

Embodiment 3

Figure 5 is a schematic structural diagram of a face recognition device provided in Embodiment 3 of the present invention. The face recognition device includes: a video face recognition module 31 and a living face determination module 32.

Among them, the video face recognition module 31 is used to perform face recognition on the video to be detected; the living face determination module 32 is used to input the recognized face video into the living body detection model to determine whether the face in the face video is For living faces, the liveness detection model is trained based on live face videos and non-living face videos. The non-living face videos include videos of printed faces or videos of screen faces.

The technical solution of this embodiment solves the problems of recognition errors caused by using photos to disguise faces in face recognition, high difficulty of user cooperation in interactive recognition, and many external devices, making living face recognition more convenient and accurate, and the model Easier and easier to promote.

Optionally, the device also includes:

A model training module is used to train live face videos and non-live face videos using convolutional neural networks to obtain the live detection model.

Optional, model training modules include:

The face video collection unit is used to collect live face videos and non-living face videos.

A video scaling unit, configured to scale the live face video and the non-living face video to a preset size.

A model training unit is used to train the scaled live face video and the non-live face video using a convolutional neural network to obtain the live detection model.

Optionally, the video scaling unit is specifically used to: extract a preset number of pictures from each video according to a preset time interval, and scale the extracted pictures to a preset size; extract the dense optical flow direction field of the time period After linear stretching, the orientation field image is scaled to a preset size. The scaled extraction image and the scaled orientation field image form the training image of the convolutional neural network.

Optional, living face determination module 32 includes:

The picture processing unit is used to extract a preset number of pictures from each video at a preset time interval according to a preset number of times, and scale the extracted pictures to a preset size.

The face determination unit is used to determine that the face in the face video is a live face if the number of times the scaled extracted picture is input to the live body detection model reaches a threshold of times, then the face in the face video is a live face.

The face recognition device provided by the embodiment of the present invention can execute the face recognition method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

Embodiment 4

Figure 6 is a schematic structural diagram of a device provided in Embodiment 4 of the present invention. As shown in Figure 6, the device includes a processor 41, a memory 42, an input device 43 and an output device 44; the number of processors 41 in the device can be One or more, one processor 41 is taken as an example in Figure 6; the processor 41, memory 42, input device 43 and output device 44 in the device can be connected through a bus or other means. In Figure C, the connection through a bus is taken as an example. .

As a computer-readable storage medium, the memory 42 can be used to store software programs, computer-executable programs and modules, such as program instructions/modules corresponding to the face recognition method in the embodiment of the present invention (for example, video face recognition module 31 and living face determination module 32). The processor 41 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 42, that is, implementing the above face recognition method.

The memory 42 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system and at least one application program required for a function; the stored data area may store data created according to the use of the terminal, etc. In addition, memory 42 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 42 may further include memory located remotely relative to processor 41, and these remote memories may be connected to the device through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

The input device 43 may be used to receive input numeric or character information and generate key signal input related to user settings and function control of the device, and may include a touch screen, a keyboard, a mouse, etc. The output device 44 may include a display device such as a display screen.

Embodiment 5

Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions. The computer-executable instructions, when executed by a computer processor, are used to perform a method of face recognition. The method includes:

Perform face recognition on the video to be detected;

The recognized face video is input into the liveness detection model to determine whether the face in the face video is a live face. The liveness detection model is trained based on the live face video and the non-living face video. The non-living face video Face videos include videos of printed faces or videos of screen faces.

Of course, the embodiments of the present invention provide a storage medium containing computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and can also execute the face recognition method provided by any embodiment of the present invention. related operations.

Through the above description of the implementation, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general hardware. Of course, it can also be implemented with hardware, but in many cases the former is a better implementation. . Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or that contributes to the existing technology. The computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk. , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions to make a computer device (which can be a personal computer, Server, or network device, etc.) performs the methods described in various embodiments of the present invention.

It is worth noting that in the above embodiments of the search device, the various units and modules included are only divided according to functional logic, but are not limited to the above divisions, as long as the corresponding functions can be realized; in addition, each function The specific names of the units are only for the convenience of distinguishing each other and are not used to limit the protection scope of the present invention.

Note that the above are only the preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, it can also include more other equivalent embodiments, and the present invention The scope is determined by the scope of the appended claims.

Claims (9)

1. A method of face recognition, characterized by including: Perform face recognition on the video to be detected; The recognized face video is input into the liveness detection model to determine whether the face in the face video is a live face. The liveness detection model is trained based on the live face video and the non-living face video. The non-living face video Face videos include videos of printed faces or videos of screen faces; Wherein, the step of inputting the recognized face video into a living body detection model to determine whether the face in the face video is a living face includes: According to the preset number of times, extract a preset number of pictures from each video at a preset time interval, and scale the extracted pictures to the preset size; If the scaled extracted image is input to the liveness detection model and the number of times it is determined that the face in the face video is a live face reaches the threshold of times, then the face in the face video is a live face; Wherein, scaling the extracted picture to a preset size includes: Scale the dense optical flow direction field picture within the preset number of extraction time periods with the extracted picture to a preset size, and combine the scaled dense optical flow direction field picture with the extracted picture As a scaled extracted image; Wherein, the extraction time period is the time period between the first extracted picture and the last extracted picture when extracting a preset number of pictures. 2. The method according to claim 1, characterized in that, before performing face recognition on the video to be detected, it further includes: Live human face videos and non-living human face videos are trained using convolutional neural networks to obtain the live body detection model. 3. The method according to claim 2, characterized in that the number of the live human face videos is smaller than the number of the non-living human face videos. 4. The method according to claim 2, characterized in that the living human face video and the non-living human face video are trained using a convolutional neural network to obtain the living body detection model, including: Collect living face videos and non-living face videos; Scale the living face video and the non-living face video to a preset size; The scaled live face video and the non-live face video are trained using a convolutional neural network to obtain the live detection model. 5. The method according to claim 4, wherein scaling the live face video and the non-living face video to a preset size includes: Extract a preset number of pictures from each video at a preset time interval, and scale the extracted pictures to the preset size; After linearly stretching the dense optical flow direction field in the extraction time period, the direction field picture is scaled to a preset size. The scaled extraction picture and the scaled direction field picture form the training picture of the convolutional neural network. 6. A face recognition device, characterized by including: Video face recognition module, used for face recognition of videos to be detected; A living face determination module, used to input the recognized face video into a living body detection model to determine whether the face in the human face video is a living human face. The living body detection model is based on the living human face video and the non-living human face video. It is obtained through training that the non-living face video includes a video of a printed face or a video of a screen face; Wherein, the living face determination module includes: The picture processing unit is used to extract a preset number of pictures from each video at a preset time interval according to a preset number of times, and scale the extracted pictures to a preset size; The face determination unit is used to determine that the face in the face video is a live face if the number of times the scaled extracted picture is input to the live body detection model reaches a threshold of times, then the face in the face video is a live face; Wherein, scaling the extracted picture to a preset size includes: Scale the dense optical flow direction field picture within the preset number of extraction time periods with the extracted picture to a preset size, and combine the scaled dense optical flow direction field picture with the extracted picture As a scaled extracted image; Wherein, the extraction time period is the time period between the first extracted picture and the last extracted picture when extracting a preset number of pictures. 7. The device of claim 6, further comprising: A model training module is used to train live face videos and non-live face videos using convolutional neural networks to obtain the live detection model. 8. A device, characterized in that the device includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the face recognition method as described in any one of claims 1-5. 9. A storage medium containing computer-executable instructions, characterized in that, when the computer-executable instructions are executed by a computer processor, they are used to perform the face recognition method according to any one of claims 1-5.