CN112395912B - A face segmentation method, electronic device and computer-readable storage medium - Google Patents

Abstract

The embodiment of the disclosure discloses a face segmentation method, an electronic device and a computer-readable storage medium, wherein the face segmentation method is applied to the electronic device, the electronic device is provided with a camera, and the face segmentation method comprises the following steps: acquiring pixel point information of the depth image through a camera; acquiring a foreground image in the depth image according to the pixel point information of the depth image; acquiring the face height of a shot object through a face detection algorithm; obtaining a face image in the foreground image according to a preset superposition depth model, pixel point information of the foreground image and the face height, wherein the preset superposition depth model is used for converting the foreground image into a foreground histogram; the face image is subjected to three-dimensional processing to obtain a three-dimensional face image of the shot object, and the three-dimensional face image is displayed, so that the problem of low efficiency caused by the fact that all pixel point information is required to be calculated in three-dimensional face segmentation can be solved, and the three-dimensional face segmentation efficiency is improved.

Description

A face segmentation method, electronic device, and computer-readable storage medium

technical field

Embodiments of the present disclosure relate to the field of image processing, and in particular to a face segmentation method, electronic equipment, and a computer-readable storage medium.

Background technique

With the rapid development of modern society, biometric authentication technologies represented by fingerprint recognition, iris recognition, and face recognition have received more and more attention as an important direction of security authentication. Among them, face segmentation, as the basis of face recognition, is widely used in the field of face recognition.

Since the 3D face contains a lot of information that the 2D face does not have, it can effectively solve the problems of 2D face recognition that are susceptible to changes in posture, expression, illumination, and self-occlusion. Therefore, in order to segment the face from the image When it comes out, three-dimensional face segmentation is usually used. However, the existing 3D face segmentation needs to calculate all the pixel information in the image, which has the problem of slow face segmentation speed, which leads to low face recognition efficiency.

Contents of the invention

Embodiments of the present disclosure expect to provide a face segmentation method, an electronic device, and a computer-readable storage medium, which can increase the speed of 3D face segmentation and further improve the efficiency of 3D face recognition.

The technical scheme of the embodiment of the present disclosure is realized in this way:

In a first aspect, an embodiment of the present disclosure provides a method for face segmentation, which is applied to an electronic device, where the electronic device is provided with a camera, and the method includes:

Obtaining pixel point information of a depth image through the camera, where the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point;

Acquiring a foreground image in the depth image according to the pixel point information of the depth image, the foreground image being used to represent the image where the subject is located;

Obtaining the face height of the subject through a face detection algorithm;

According to the preset superposition depth model, the pixel point information of the foreground image and the height of the human face, the face image in the foreground image is obtained, and the preset superposition depth model is used to convert the foreground image into a foreground histogram;

Performing three-dimensional processing on the face image to obtain a three-dimensional face image of the object to be photographed, and displaying the three-dimensional face image.

In a second aspect, an embodiment of the present disclosure provides an electronic device, the electronic device is provided with a camera, and the electronic device includes an acquisition unit, an acquisition unit, a segmentation unit, a three-dimensional processing unit, and a display unit, wherein,

The acquisition unit is configured to acquire pixel point information of a depth image through the camera, and the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point;

The acquisition unit is used to acquire the height of the face of the subject through a face detection algorithm;

The segmentation unit is configured to acquire a foreground image in the depth image according to the pixel point information of the depth image, and the foreground image is used to represent the image where the subject is located;

The segmentation unit is further configured to obtain a face image in the foreground image according to a preset superposition depth model, pixel information of the foreground image, and the face height, and the preset superposition depth model is used for converting the foreground image into a foreground histogram;

The three-dimensional processing unit is configured to perform three-dimensional processing on the face image to obtain a three-dimensional face image of the subject;

The display unit is configured to display the three-dimensional face image.

In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device is provided with a camera, and the electronic device includes at least a processor, a memory storing instructions executable by the processor, and a device for connecting the processing device, the memory, and the bus of the camera, and when the executable instructions are executed, the processor implements the steps in the above method for face segmentation.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which executable instructions are stored, and when the executable instructions are executed by a processor, the steps in the above-mentioned face segmentation method are implemented.

Embodiments of the present disclosure provide a face segmentation method, an electronic device, and a computer-readable storage medium. The face segmentation method is applied to an electronic device, and the electronic device is provided with a camera, including: acquiring pixels of a depth image through the camera Information, the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point; according to the pixel point information of the depth image, the foreground image in the depth image is obtained, and the foreground image is used to represent the shooting object The image where it is located; the face height of the subject is obtained through the face detection algorithm; according to the preset overlay depth model, the pixel information of the foreground image and the height of the face, the face image in the foreground image is obtained, and the preset overlay depth model is used To convert the foreground image into a foreground histogram; perform three-dimensional processing on the face image, obtain the three-dimensional face image of the subject, and display the three-dimensional face image, that is to say, on the one hand, the embodiment of the present disclosure first divides the depth image Obtain the foreground image; then the foreground image is segmented to obtain the face image, so that the face segmentation can be realized only by calculating the pixel information of the foreground image, which improves the speed of three-dimensional face segmentation; The stacking depth model is used to convert the foreground image into a foreground histogram, and the transformed foreground histogram is segmented by the height of the face, which can further improve the speed of face segmentation, thereby improving the efficiency of 3D face recognition.

Description of drawings

Fig. 1 provides a structural block diagram of a face segmentation method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of an implementation process of face segmentation provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of depth image segmentation provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of foreground image segmentation provided by an embodiment of the present disclosure;

FIG. 5 is a first schematic diagram of the composition and structure of an electronic device provided by an embodiment of the present disclosure;

FIG. 6 is a second schematic diagram of the composition and structure of an electronic device provided by an embodiment of the present disclosure.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the embodiments of the present disclosure. All other embodiments obtained by the skilled person without creative work belong to the scope of protection of the embodiments of the present disclosure.

In the process of face recognition, a three-dimensional face segmentation method is usually used to segment a face image from a complete image. At present, the main three-dimensional face segmentation methods can be divided into two types. One is the segmentation algorithm based on collective information. The segmentation process is to calculate the relevant geometric information, such as curvature and normal, according to each vertex involved in the image. , fitting surface, etc., and then segment the image into a rough face area according to the relevant geometric information, and then trim the image according to the curvature in the calculated geometric information to complete the segmentation; the other is based on the physiological characteristics of the face The segmentation algorithm, the segmentation process is to build a face model in advance, and then adjust the face model according to the key point information obtained from the detected image to complete the segmentation.

However, for a real 3D face model, the number of 3D pixels in the image and the faces composed of 3D pixels is large, and the calculation of the geometric information of all 3D pixels is huge, and there is a long time for face segmentation. low problem. In addition, at present, devices such as lasers are usually used to scan images to collect depth images, which also has the problem of high cost.

Therefore, considering that in 3D face segmentation, obtaining a face image from a whole depth image is only an intermediate step in obtaining a 3D face image, the face image needs to be filtered, back-projected, registered and displayed, etc. processing, the embodiment of the present disclosure proposes a face segmentation method. FIG. 1 is a structural block diagram of a face segmentation method provided by an embodiment of the present disclosure. As shown in FIG. 1 , the structural block diagram of the face segmentation method includes: an acquisition unit, a segmentation unit, a three-dimensional processing unit, and a display unit, wherein,

The acquisition unit is used to collect the depth image; the segmentation unit is used to segment the foreground image and the background image in the depth image, and the face image and the torso image in the segmentation foreground image; the three-dimensional processing unit is used to process the face image Three-dimensional processing such as filtering, back projection, registration, etc. to obtain a three-dimensional face image; a display unit for displaying the three-dimensional face image.

Through the structural block diagram of the face segmentation method provided by the embodiment of the present disclosure, it can be concluded that the embodiment of the present disclosure no longer calculates all pixel information in the depth image, but only needs to calculate the pixel information in the foreground image, which improves the accuracy of face segmentation. Speed, thereby improving the efficiency of face recognition.

FIG. 2 is a schematic diagram of a face segmentation implementation process provided by an embodiment of the present disclosure. As shown in FIG. 2, the face segmentation method is applied to an electronic device, the electronic device is provided with a camera, and the electronic device can realize the face segmentation method. Include the following steps:

S101. Obtain pixel point information of a depth image through a camera.

In the embodiment of the present disclosure, the electronic device is provided with a camera, through which the electronic device can obtain the pixel point information of the depth image, and the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinates corresponding to the pixel point information.

It should be noted that the above-mentioned depth image is composed of at least one pixel point, and each pixel point corresponds to the pixel point depth information and the corresponding plane coordinate information of the pixel point, and the pixel point depth information is used to represent the pixel point corresponding The distance between the plane coordinates of and the camera.

Exemplarily, in order to reduce the cost of acquiring a depth image, the electronic device may be a Kinect device, and the camera on the Kinect device captures an object to be displayed in three dimensions at a video rate to obtain a depth image. It should be noted that the The depth image is a sequence of depth images in video images collected by the Kinect device.

In the embodiment of the present disclosure, the electronic device is provided with a camera, and the number of the camera may be at least one. When the number of the camera of the electronic device is one, and the camera is a monocular camera, the electronic device obtains the depth through the camera. The process of the pixel point depth information in the pixel point information of the image is: collect the video image of the shooting object through the monocular camera; extract the depth image frame in the video image; analyze the relationship between different depth image frames in the video image Stereo matching, obtaining pixel depth information in the pixel information of the depth image.

When the number of cameras in the electronic device is set to two, the process for the electronic device to obtain pixel depth information in the pixel information of the depth image through the two cameras is: The first distance and the second distance of the point; according to the first distance, the second distance and the preset distance between the two cameras, the pixel depth information in the pixel information of the depth image is determined.

Exemplarily, when the electronic device is a Kinect device, the Kinect device is provided with two infrared cameras, and the Kinect device can obtain the pixel depth information in the pixel point information of the depth image through the two infrared cameras. No restrictions.

S102. Acquire a foreground image in the depth image according to pixel point information of the depth image.

In the embodiment of the present disclosure, after the electronic device acquires the depth image, the electronic device needs to acquire the foreground image in the depth image according to the pixel point information of the depth image.

It should be noted that the foreground image is used to represent the image of the subject in the depth image, and other images in the depth image except the foreground image can be background images, and the background image is used to represent the environment in the depth image Therefore, in order to obtain the face image of the subject, the electronic device needs to first segment the foreground image and the background image in the depth image to obtain the foreground image.

In the embodiment of the present disclosure, compared with the environment, the object in the foreground image has the shortest distance between the pixel corresponding to the object and the camera, and the depth value corresponding to the pixel depth information is used to represent the pixel object Therefore, the foreground image in the depth image can be obtained by the connected component analysis method, that is, the depth value corresponding to the depth information of two adjacent pixel points.

Specifically, the electronic device obtains the foreground image in the depth image according to the pixel point information of the depth image as follows: from the pixel point depth information of the depth image, obtains the depth information of two adjacent pixels; The pixel depth information is used to obtain the depth difference between two adjacent pixels; when the depth difference is less than the preset difference threshold, the image where the two adjacent pixels are located is used as the foreground image in the depth image.

It should be noted that when the depth difference is less than the preset difference threshold, the image where the two adjacent pixels are located is the foreground image, and when the depth difference is greater than or equal to the preset difference threshold, it means that the The image where two adjacent pixels are located is the background image, which needs to be removed. In this way, the foreground image and the background image in the depth image can be separated to obtain the required foreground image.

Exemplarily, as shown in FIG. 3, FIG. 3 is a schematic diagram of depth image segmentation provided by an embodiment of the present disclosure, wherein the image corresponding to the number 30 is a depth image, and the image corresponding to number 31 is a background image in the depth image. The background image is the image that needs to be removed; the image corresponding to the number 32 is the foreground image in the depth image, which is an image other than the background image in the depth image. In this way, the embodiment of the present disclosure combines the foreground image in the depth image and Segmenting the background image and removing the background image can facilitate the subsequent segmentation of the face image directly from the foreground image without calculating the pixel information in the background image, thereby improving the speed of face segmentation.

S103. Obtain the face height of the object to be photographed through a face detection algorithm.

In the embodiment of the present disclosure, in the process of face segmentation, the electronic device can obtain the face height of the object to be segmented through the face detection algorithm, so that the electronic device can perform face segmentation based on the face height .

It should be noted that the face height of the subject acquired by the electronic device is the real face height of the subject, which is the distance from the forehead to the chin of the subject.

Exemplarily, the above-mentioned face detection algorithm may be a deformable component DPM (Deformable Part Model) algorithm, a normalized pixel difference feature (Normalized Pixel Difference, NPD) algorithm and a Viola-Jones face detection algorithm, the embodiment of the present disclosure There is no limit here. It should be noted that the process of detecting the height of a human face by using the Viola-Jones face detection algorithm may be as follows: detecting the subject, and adjusting the rectangular frame to include the subject's face to mark the height of the human face.

S104. Obtain the face image in the foreground image according to the preset superposition depth model, the pixel point information of the foreground image, and the height of the face.

In the embodiment of the present disclosure, after the electronic device acquires the foreground image and the height of the face, the electronic device needs to obtain the face image in the foreground image according to the preset superimposed depth model, the pixel information of the foreground image, and the height of the face.

It should be noted that S103 can be performed in any step before S104, and the writing order of S103 does not imply a strict execution order, but constitutes any limitation on the implementation process of the embodiment of the present disclosure, and the specific execution order of the steps should be based on its Functionality and possible internal logic determined.

In the embodiment of the present disclosure, the face image is used to represent the image where the face of the subject is located in the foreground image, and the images other than the face image in the foreground image may be torso images. Therefore, in order to obtain the subject's For the face image, the electronic device needs to segment the face image and the torso image in the foreground image to obtain the face image.

Exemplarily, as shown in FIG. 4, FIG. 4 is a schematic diagram of foreground image segmentation provided by an embodiment of the present disclosure, wherein the image corresponding to the number 40 is the foreground image, and the image corresponding to the number 41 is the torso image in the foreground image, and the torso The image is the image that needs to be removed, and the image corresponding to the label 42 is the face image in the foreground image. In this way, the embodiment of the present disclosure only needs to calculate the pixel point information in the foreground image, and then the face image can be segmented, no need to calculate All the pixel information in the depth image improves the speed of face segmentation.

It should be noted that in the embodiment of the present disclosure, the preset stacking depth model is used to characterize the conversion of the foreground image into the foreground histogram.

Exemplarily, the preset stacking depth model can be formula (1):

Among them, h(v) is the superimposed depth feature value corresponding to the vertical coordinate v of the pixel point of the foreground image, and M _τ (u, v) is the depth feature value corresponding to the pixel plane coordinate (u, v) of the foreground image. In order to obtain the face image in the foreground image, the electronic device may first convert the foreground image into a foreground histogram through a preset stacking depth model, and then obtain the face image according to the face height of the subject and the foreground histogram.

Specifically, the electronic device obtains the face image in the foreground image according to the preset superposition depth model, the pixel point information of the foreground image, and the face height as follows: according to the preset superposition depth model and the pixel point information of the foreground image, obtain The foreground histogram corresponding to the foreground image; the foreground histogram is segmented by the face height to obtain the face image in the foreground image.

It can be understood that the electronic device acquires the foreground histogram corresponding to the foreground image by presetting the superimposed depth model and the pixel point information of the foreground image, and can convert the foreground image into the presentation form of the foreground histogram, so that the object can be captured through The horizontal line corresponding to the face height is used to segment the foreground histogram. In this way, the foreground image is directly segmented by the horizontal line corresponding to the face height, which improves the speed and accuracy of face segmentation.

In the embodiment of the present disclosure, the electronic device obtains the foreground histogram corresponding to the foreground image according to the preset superimposed depth model and the pixel point information of the foreground image as follows: through the depth value corresponding to the depth information of each foreground pixel point in the foreground image, obtain Each foreground depth feature value corresponding to each foreground pixel point; according to each foreground depth feature value and a preset stacking depth model, obtain at least one stacking depth feature value corresponding to the vertical coordinate of the foreground pixel point; by each foreground pixel point vertical coordinate and each The superimposed depth feature value corresponding to the ordinate of a foreground pixel constitutes a foreground histogram corresponding to the foreground image.

It should be noted that, in the embodiment of the present disclosure, converting the foreground image into a foreground histogram is used for subsequent electronic devices to directly segment the foreground histogram through the height of the face to obtain the face image, which is an important process to improve the segmentation speed.

Exemplarily, the electronic device obtains each foreground depth feature value corresponding to each foreground pixel point through the depth value corresponding to each foreground pixel point depth information in the foreground image, which may be that when the foreground pixel point depth information corresponds to a depth value, then the The foreground depth feature value corresponding to the foreground pixel is 1; when the depth information of the foreground pixel point does not correspond to a depth value, the foreground depth feature value corresponding to the foreground pixel point is 0, and at this time, the foreground depth feature value does not participate in the calculation.

时，通过对各前景像素点对应的各前景深度特征值进行叠加，便可以获取至少一个前景像素点纵坐标中前景像素点纵坐标v对应的叠加深度特征值。In addition, according to each foreground depth characteristic value and the preset superimposition depth model, the process of obtaining the superimposition depth characteristic value corresponding to at least one foreground pixel point ordinate can be as follows: when the preset superposition depth model is

, by superimposing each foreground depth feature value corresponding to each foreground pixel point, the superimposed depth feature value corresponding to the ordinate v of the foreground pixel point in the ordinate of at least one foreground pixel point can be obtained.

It can be understood that, in the embodiment of the present disclosure, the superimposed depth feature value corresponding to the ordinate v of the foreground pixel needs to first traverse all the abscissas of the foreground pixel corresponding to the ordinate v of the same foreground pixel to obtain the value corresponding to the ordinate v of the foreground pixel All the foreground pixels, and then superimpose the depth feature values corresponding to all the foreground pixels corresponding to the obtained foreground pixel ordinate v, to obtain the superimposed depth feature value corresponding to the foreground pixel ordinate v.

Exemplarily, if the foreground pixel coordinates are (1, 1), (1, 2), (2, 1), (2, 2), and the depth feature values corresponding to all the foreground pixels are 1, Then all the foreground pixels corresponding to the ordinate 2 of the foreground pixel are (1, 2) and (2, 2), by corresponding to the depth feature value 1 corresponding to the pixel point (1, 2) and the pixel point (2, 2) By superimposing the depth feature value 1 of the foreground pixel point, the superposition depth feature value corresponding to the vertical coordinate 2 of the foreground pixel point is 2.

In the embodiment of the present disclosure, the electronic device divides the foreground histogram by the height of the face, and the process of obtaining the face image in the foreground image is as follows: the ordinate of the foreground pixel in the current foreground histogram is less than or equal to the face height of the subject, And when the superimposed depth feature value corresponding to the vertical coordinate of the foreground pixel point is not equal to the preset superimposed depth feature value, the image where each foreground pixel point corresponding to the vertical coordinate of the foreground pixel point is located is used as the face image in the foreground image, so, the present disclosure In the embodiment, by comparing the ordinate corresponding to the foreground pixel information in the foreground histogram with the face height, the face image can be directly obtained, which improves the segmentation speed and accuracy of the face image.

Exemplarily, the preset characteristic value of the superimposed depth can be set according to the actual needs of the user, for example, the characteristic value of the superimposed depth can be set to 0, which is not limited in this embodiment of the present disclosure. When the superimposed depth feature value of the electronic device is 0, the process of segmenting the foreground histogram by the height of the face to obtain the face image in the foreground image can correspond to formula (2), where v is the ordinate of the foreground pixel point, y is the height of the face, h(v) is the superimposed depth feature value corresponding to the ordinate v of the foreground pixel point, and H is the area where the face image is located.

H＝{v丨v≤y and h(v)≠0} (2)

In the embodiment of the present disclosure, the electronic device divides the foreground histogram by the face height, and can also obtain the torso image in the foreground image at the same time. The process of obtaining the torso image in the foreground image is: the foreground pixel in the current foreground histogram When the vertical coordinate is greater than the height of the face of the subject, and the superimposed depth feature value corresponding to the vertical coordinate of the foreground pixel point is not equal to the preset superimposed depth feature value, the image where each foreground pixel point corresponding to the vertical coordinate of the foreground pixel point is located is taken as the foreground image torso image in .

Exemplarily, when the superimposed depth feature value of the electronic device is 0, the electronic device divides the foreground histogram by the height of the face, and the process of obtaining the torso image in the foreground image can correspond to formula (3), where v is the foreground pixel Point ordinate, y is the face height, h(v) is the superimposed depth feature value corresponding to the foreground pixel point ordinate v, J is the area where the torso image is located.

J＝{v丨v>y and h(v)≠0} (3)

In other embodiments, after the electronic device obtains each foreground depth feature value corresponding to each foreground pixel point through the depth value corresponding to the depth information of each foreground pixel point in the foreground image, and performs three-dimensional processing on the face image to obtain the photographed Before the 3D face image of the subject, the face image of the subject can also be obtained according to the foreground depth feature value corresponding to the foreground pixel point and the face height of the subject, specifically: the foreground depth feature value corresponding to the current foreground pixel point is When the depth feature value is preset, and the ordinate of the foreground pixel point is less than or equal to the height of the face of the subject, the image where the foreground pixel point is located is used as the face image in the foreground image. In this way, the embodiment of the present disclosure is also based on the foreground pixel point information The corresponding ordinate and the height of the face can directly obtain the face image in another form, which improves the segmentation speed and accuracy of the face image, and can flexibly obtain the face image at the same time.

It should be noted that the electronic device may also determine the face image of the subject by comparing the ordinate of the foreground pixel with the face height and determining whether the foreground pixel has a depth feature value corresponding to the depth information of the foreground pixel.

Exemplarily, the preset depth feature value may be set according to the actual needs of the user, for example, the preset depth feature value may be set to 1, which is not limited in this embodiment of the present disclosure.

When the preset depth feature value is 1, the electronic device obtains the face image of the subject according to the foreground depth feature value corresponding to the foreground pixel point and the face height of the subject, which can correspond to formula (4), where D _τ ( u, v) is the depth value corresponding to the pixel point depth information of the foreground image, D' _τ (u, v) is the depth value corresponding to the pixel point depth information of the face image in the foreground image, and M _τ (u, v) is The depth feature value corresponding to the plane coordinates (u, v) of the foreground pixel point, v is the vertical coordinate of the foreground pixel point, and y is the face height of the subject.

S105. Perform three-dimensional processing on the face image to obtain a three-dimensional face image of the subject, and display the three-dimensional face image.

In the embodiment of the present disclosure, after acquiring the face image, the electronic device may perform three-dimensional processing on the face image to obtain a three-dimensional face image of the subject.

It should be noted that the three-dimensional processing of the face image may be three-dimensional processing such as filtering, back-projection, and registration of the face image. Specifically, the electronic device performs three-dimensional processing on the face image to obtain the three-dimensional face image of the subject. The image process is: filter the face image to obtain the filtered face image; back-project the filtered face image to obtain the 3D point cloud of the filtered face image and the normal vector of the 3D point cloud; correspond to the filtered face image According to the 3D point cloud of the registered face image and the normal vector corresponding to the 3D point cloud, the 3D face image of the subject is obtained.

In the embodiment of the present disclosure, the face image obtained by the electronic device through the pixel information of the depth image may have jaggedness. In order to obtain an accurate three-dimensional face image, it is necessary to filter the face image. For example, the electronic device can pass The bilateral filter performs filtering processing on the face image to save the discontinuous depth value corresponding to the pixel depth information in the face image while removing the noise.

Among them, the above-mentioned filter processing of the face image, the specific process of obtaining the filtered face image is: from the face pixel depth information of the face image, obtain the depth variance value corresponding to the face pixel depth information; when the depth When the variance value is less than the preset depth variance threshold, the image where the face pixels are located is used as the filtered face image.

Exemplarily, the face image is filtered, and the specific process of obtaining the filtered face image may correspond to formula (5), wherein, D" _τ (u, v) is the depth value corresponding to the pixel point information of the filtered face image , D' _τ (u, v) is the depth value corresponding to the pixel depth information of the face image in the foreground image, δ(u, v) ² is the depth variance value corresponding to the face pixel depth information, and δ ² _max is Default depth variance threshold.

In the embodiment of the present disclosure, after obtaining the filtered face image, the electronic device also needs to back-project the filtered face image, so as to obtain a three-dimensional point cloud of the face image. The electronic device back-projects the filtered face image, and the process of obtaining the three-dimensional point cloud of the filtered face image and the normal vector corresponding to the three-dimensional point cloud is as follows: obtaining the three-dimensional coordinate model and the normal vector model; according to the pixel points of the filtered face image Obtain the 3D point cloud of the filtered face image based on the depth information, the pixel plane coordinate information of the filtered face image and the 3D calculation model; obtain the normal vector corresponding to the 3D point cloud according to the 3D point cloud and normal vector model of the filtered face image .

It should be noted that the 3D coordinate model is used to represent the 3D information corresponding to the pixels of the filtered face image, by inputting the pixel depth information of the filtered face image and the pixel plane coordinate information of the filtered face image into the 3D calculation model , the 3D point cloud of the filtered face image can be obtained.

Exemplarily, a three-dimensional pixel point coordinate [X, Y, Z] ^T in the filter face image is mapped to the pixel point plane coordinate [u, v] ^T of the filter face image in formula (6), wherein, D” _τ (u, v) is the depth value corresponding to the pixel point information of the filtered face image, f _x and f _y are the focal length of the camera, c _x and _cy are the projection centers of the camera, and the projection center is the three-dimensional point to Projective mapping of planar coordinates of 2D pixel points.

Among them, K is the inherent matrix of the camera, and K can be the expression (7).

By calculating the equation on the left side of the formula (6), the expression (8) of the plane coordinate u of the pixel point can be obtained.

u＝X*f _x /D” _τ (u,v)+c _x (8)

By calculating the equation on the left side of the formula (6), the expression (9) of the plane coordinate v of the pixel point can be obtained.

v=Y*f _y /D” _τ (u,v)+c _y (9)

According to formula (6), formula (8) and formula (9), the formula of the three-dimensional pixel point coordinate [X, Y, Z] ^T can be obtained as (10).

By transforming the formula (10), the formula of the three-dimensional point cloud V _τ (u, v) corresponding to the pixel plane coordinates (u, v) of the filtered face image can be obtained as (11).

V _τ (u,v)＝D” _τ (u,v)*K ^-1 *[u,v,1] ^T (11)

In the embodiment of the present disclosure, the normal vector model is used to represent the surface information of the 3D face image, and by inputting the adjacent 3D point clouds in the face image into the normal vector model, the plane formed by the adjacent 3D point clouds can be obtained The normal vector of , that is, the normal vector corresponding to the three-dimensional point cloud, the normal vector is the normal vector of the plane formed by the adjacent three-dimensional point cloud as the vertex.

Exemplarily, the normal vector model can be formula (12), where N _τ (u,v) is the normal vector corresponding to the three-dimensional point cloud, V _τ (u+1,v) and V _τ (u,v+ 1) are the three-dimensional point clouds of adjacent coordinates corresponding to the plane pixel points (u, v) of the filtered face image.

N _τ (u,v)＝[V _τ (u+1,v)-V _τ (u,v)]×[V _τ (u,v+1)-V _τ (u,v)] (12)

In the embodiment of the present disclosure, the depth image acquired by the electronic device is at least one depth image, and at least one filtered face can be obtained by first segmenting to obtain the foreground image, then segmenting to obtain the face image, and three-dimensional processing of filtering and back projection. For the 3D point cloud of the image, in order to obtain a 3D face image with high accuracy and clarity, the 3D point cloud corresponding to multiple filtered face images can be registered to obtain the 3D point cloud of the registered face image.

Exemplarily, the registration of the three-dimensional point clouds corresponding to the multiple filtered face images may be performed using a sparse iterative closest point algorithm, which is not limited in this embodiment of the present disclosure.

After obtaining the 3D point cloud of the registered face image, the electronic device obtains the 3D face image of the subject according to the 3D point cloud of the registered face image and the normal vector corresponding to the 3D point cloud. For the 3D point cloud of multiple registered face images, the 3D point cloud of multiple registered face images can be fused first, and then according to the fused 3D point cloud and the normal vector corresponding to the 3D point cloud, the shooting 3D face image of the subject.

Exemplarily, the fusion of the 3D point clouds of multiple registered face images may be fused using a stereo integration technology, which is not limited in this embodiment of the present disclosure.

In the embodiment of the present disclosure, after acquiring the three-dimensional face image of the subject, the electronic device draws the three-dimensional face image and displays the face image.

It should be noted that the drawn 3D face image is the current state of the subject, and the 3D face image can be updated by collecting depth images in real time. At the same time, the current state of the subject can also be used to redefine the 3D point cloud and corresponding normal vectors to update the acquired 3D face image to further improve the robustness.

Exemplarily, drawing a three-dimensional human face image can be selected Open Graphics Library (Open Graphics Library, OpenGL), this OpenGL is used to render the cross-language, cross-platform professional graphics program interface of three-dimensional vector image, can be by calling the bottom layer graphics database The image is used to draw a three-dimensional face image, which is not limited in this embodiment of the present disclosure.

Through the embodiments of the present disclosure, the electronic device first divides the depth image to obtain the foreground image; Three-dimensional face segmentation speed efficiency; on the other hand, the embodiment of the present disclosure converts the foreground image into a foreground histogram through the preset stacking depth model, and segments the converted foreground histogram through the height of the face, which can further improve the face segmentation. The speed of the three-dimensional face recognition is further improved, and the embodiment of the present disclosure performs three-dimensional processing such as filtering, back-projection and registration on the face image, which can remove noise and improve the accuracy of the three-dimensional face image.

In addition, the face segmentation method provided by the embodiments of the present disclosure can be applied in a 3D face recognition system and a 3D face matching system. By adopting a unified computing device architecture (Compute Unified Device Architecture, CUDA), the face segmentation method can be improved. The running speed makes the segmentation more efficient.

Based on the same inventive concept of the above-mentioned embodiments, FIG. 5 is a first schematic diagram of the composition and structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 1002, segmentation unit 1003, three-dimensional processing unit 1004 and display unit 1005, wherein,

The acquisition unit 1001 is configured to acquire pixel point information of the depth image through the camera, and the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point;

An acquisition unit 1002, configured to photograph the face height of the subject through a face detection algorithm;

The segmentation unit 1003 is configured to acquire a foreground image in the depth image according to the pixel point information of the depth image, and the foreground image is used to represent the image where the subject is located;

The segmentation unit 1003 is also used to obtain the face image in the foreground image according to the preset superimposed depth model, the pixel information of the foreground image and the face height, and the preset superimposed depth model is used to convert the foreground image into a foreground histogram;

A three-dimensional processing unit 1004, configured to perform three-dimensional processing on the face image to obtain a three-dimensional face image of the subject;

The display unit 1005 is configured to display a three-dimensional face image.

In other embodiments, the segmentation unit 1003 also includes:

The first segmentation unit 1006 is configured to obtain a foreground histogram corresponding to the foreground image according to the preset superimposed depth model and the pixel point information of the foreground image;

The second segmentation unit 1007 is configured to segment the foreground histogram by the face height to obtain the face image in the foreground image.

In other embodiments, the first segmentation unit 1006 is specifically configured to: obtain each foreground depth feature value corresponding to each foreground pixel point through the depth value corresponding to each foreground pixel point depth information in the foreground image; according to each foreground depth feature value and Preset the overlay depth model to obtain at least one foreground pixel ordinate corresponding to the overlay depth feature value; each foreground pixel ordinate and the overlay depth feature value corresponding to each foreground pixel ordinate form the foreground corresponding to the foreground image histogram.

In other embodiments, the second segmentation unit 1007 is specifically configured to: the ordinate of the foreground pixel in the current foreground histogram is less than or equal to the face height of the subject, and the superimposed depth feature value corresponding to the ordinate of the foreground pixel is not equal to the preset When superimposing the depth feature value, the image where each foreground pixel corresponding to the ordinate of the foreground pixel is used as the face image in the foreground image.

In other embodiments, the electronic device 1000 also includes:

The third segmentation unit 1008 is configured to obtain the foreground depth feature values corresponding to each foreground pixel point through the depth value corresponding to the depth information of each foreground pixel point in the foreground image, and perform three-dimensional processing on the face image to obtain the photographed Before the three-dimensional face image of the object, when the foreground depth feature value corresponding to the foreground pixel point is a preset depth feature value, and the ordinate of the foreground pixel point is less than or equal to the height of the face of the subject, the image where the foreground pixel point is located is taken as the foreground The face image in the image.

In other embodiments, the three-dimensional processing unit 1004 includes:

Filtering unit 1009, configured to filter the face image to obtain the filtered face image;

The back-projection unit 1010 is used to back-project the filtered face image, and obtain the normal vector of the three-dimensional point cloud and the three-dimensional point cloud of the filtered face image;

Registration unit 1011, configured to register the 3D point cloud corresponding to the filtered face image, and obtain the 3D point cloud of the registered face image;

The first acquiring unit 1012 is configured to acquire the 3D face image of the subject according to the 3D point cloud of the registered face image and the normal vector corresponding to the 3D point cloud.

In other embodiments, the filtering unit 1009 is specifically configured to: acquire the depth variance value corresponding to the face pixel depth information from the face pixel depth information of the face image; when the depth variance value is less than the preset depth variance When thresholding, the image where the face pixels are located is used as the filtered face image.

In other embodiments, the back-projection unit 1010 is specifically configured to: obtain a three-dimensional coordinate model and a normal vector model, the three-dimensional coordinate model is used to represent the three-dimensional information corresponding to the pixels of the filtered face image, and the normal vector model is used to represent the three-dimensional face The surface information of the image; according to the pixel depth information of the filtered face image, the pixel plane coordinate information of the filtered face image and the 3D calculation model, the 3D point cloud of the filtered face image is obtained; according to the 3D point cloud of the filtered face image and normal vector model to obtain the normal vector corresponding to the 3D point cloud.

In other embodiments, the segmentation unit 1003 is specifically configured to: obtain the depth information of two adjacent pixels from the depth information of pixels in the depth image; obtain the depth information of two adjacent pixels according to the depth information of two adjacent pixels The depth difference; when the depth difference is less than the preset difference threshold, the image where two adjacent pixels are located is taken as the foreground image in the depth image.

Through the embodiments of the present disclosure, on the one hand, the embodiments of the present disclosure firstly segment the depth image to obtain the foreground image; then segment the foreground image to obtain the face image, so that the face segmentation can be realized only by calculating the pixel point information of the foreground image, The speed of three-dimensional face segmentation is improved; on the other hand, the embodiment of the present disclosure converts the foreground image into a foreground histogram through a preset stacking depth model, and segments the converted foreground histogram through the height of the face, which can further improve the accuracy of the face. The speed of segmentation improves the efficiency of 3D face recognition. At the same time, the embodiment of the present disclosure performs 3D processing such as filtering, back-projection and registration on the face image, which can remove noise and improve the accuracy of the 3D face image.

Based on the same inventive concept of the foregoing embodiments, an embodiment of the present disclosure provides an electronic device. FIG. 6 is a schematic diagram of the composition and structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 6 , the electronic device is provided with a camera 05 , the electronic device further includes at least a processor 01, a memory 02, a communication interface 03 and a communication bus 04, wherein the communication bus 04 is used to realize the connection and communication of the processor 01, the memory 02, the communication interface 03 and the camera 05; the communication interface 03 uses The pixel point information of the depth image is obtained through the camera; the processor 01 is used to execute the executable instructions stored in the memory 02, so as to realize the steps in the face segmentation method provided by the above-mentioned embodiment.

Through the embodiments of the present disclosure, on the one hand, the embodiments of the present disclosure firstly segment the depth image to obtain the foreground image; then segment the foreground image to obtain the face image, so that the face segmentation can be realized only by calculating the pixel point information of the foreground image, The speed of three-dimensional face segmentation is improved; on the other hand, the embodiment of the present disclosure converts the foreground image into a foreground histogram through a preset stacking depth model, and segments the converted foreground histogram through the height of the face, which can further improve the accuracy of the face. The speed of segmentation improves the efficiency of 3D face recognition. At the same time, the embodiment of the present disclosure performs 3D processing such as filtering, back-projection and registration on the face image, which can remove noise and improve the accuracy of the 3D face image.

In addition, each component in this embodiment may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software function modules.

If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or It is said that the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, the computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: magnetic random access memory (FRAM, ferromagnetic random access memory), read-only memory (ROM, Read Only Memory), programmable read-only memory (PROM, Programmable Read-Only Memory), erasable Programmable Read-Only Memory (EPROM, Erasable Programmable Read-Only Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), Flash Memory (FlashMemory), Magnetic Surface Memory, CD-ROM, or CD-ROM (Compact Disc Read-Only Memory, Compact Disc Read-Only Memory) and other media capable of storing program codes, which are not limited by the embodiments of the present disclosure.

Based on the foregoing embodiments, embodiments of the present disclosure provide a computer-readable storage medium, on which executable instructions are stored, and when the executable instructions are executed by the above-mentioned processor, the steps in the face segmentation method in the above-mentioned embodiments are implemented.

Those skilled in the art should understand that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Accordingly, embodiments of the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present disclosure are described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the embodiments of the present disclosure, and are not intended to limit the protection scope of the embodiments of the present disclosure.

Claims (9)

1. A method for face segmentation, characterized in that it is applied in an electronic device, and the electronic device is provided with a camera, and the method comprises: Obtaining pixel point information of a depth image through the camera, where the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point; Acquiring a foreground image in the depth image according to the pixel point information of the depth image, the foreground image being used to represent the image where the subject is located; Obtaining the face height of the subject through a face detection algorithm; According to the preset superposition depth model, the pixel point information of the foreground image and the height of the human face, the face image in the foreground image is obtained, and the preset superposition depth model is used to convert the foreground image into a foreground histogram; Performing three-dimensional processing on the face image to obtain a three-dimensional face image of the subject, and displaying the three-dimensional face image; Wherein, said obtaining the face image in the foreground image according to the preset superposition depth model, the pixel point information of the foreground image and the face height includes: according to the preset superposition depth model and the The pixel point information of the foreground image, obtaining the foreground histogram corresponding to the foreground image; segmenting the foreground histogram by the face height to obtain the face image in the foreground image; Wherein, the pixel point information of the foreground image includes foreground pixel point plane coordinate information and foreground pixel point depth information, and the foreground pixel point plane coordinate information includes foreground pixel point ordinate, and according to the preset superposition depth model and The pixel point information of the foreground image, and obtaining the foreground histogram corresponding to the foreground image includes: obtaining the depth value corresponding to each foreground pixel point depth information in the foreground image Each foreground depth characteristic value; according to each foreground depth characteristic value and the preset superposition depth model, obtain at least one superimposition depth characteristic value corresponding to the ordinate of the foreground pixel point; from each foreground pixel point ordinate and the each The superimposed depth feature value corresponding to the ordinate of a foreground pixel point constitutes the foreground histogram corresponding to the foreground image; Wherein, said foreground histogram is segmented by said face height to obtain a face image in said foreground image, comprising: when the ordinate of said foreground pixel in said foreground histogram is less than or equal to said Face height, and when the superimposed depth feature value corresponding to the foreground pixel ordinate is not equal to the preset superimposed depth feature value, the image where each foreground pixel corresponding to the foreground pixel ordinate is used as the foreground image The face image in . 2. The method according to claim 1, characterized in that, after obtaining each foreground depth feature value corresponding to each foreground pixel point through the depth value corresponding to each foreground pixel point depth information in the foreground image , and before performing three-dimensional processing on the face image to obtain the three-dimensional face image of the subject, the method further includes: When the foreground depth feature value corresponding to the foreground pixel point is a preset depth feature value, and the ordinate of the foreground pixel point is less than or equal to the height of the face, the image where the foreground pixel point is located is used as the foreground image The face image in . 3. The method according to claim 1 or 2, wherein the three-dimensional processing of the human face image to obtain the three-dimensional human face image of the subject comprises: Filtering the face image to obtain a filtered face image; Back-projecting the filtered face image to obtain a three-dimensional point cloud of the filtered face image and a normal vector of the three-dimensional point cloud; Registering the three-dimensional point cloud corresponding to the filtered face image, and obtaining the three-dimensional point cloud of the registered face image; Acquiring the 3D face image of the subject according to the 3D point cloud of the registered face image and the normal vector corresponding to the 3D point cloud. 4. The method according to claim 3, wherein said filtering said face image and obtaining a filtered face image comprises: From the face pixel depth information of the face image, obtain the depth variance value corresponding to the face pixel depth information; When the depth variance value is smaller than the preset depth variance threshold, the image where the face pixels are located is used as the filtered face image. 5. method according to claim 3, is characterized in that, described filter face image is carried out back projection, obtains the three-dimensional point cloud of described filter face image and the normal vector corresponding to described three-dimensional point cloud, include: Obtain a three-dimensional coordinate model and a normal vector model, the three-dimensional coordinate model is used to represent the three-dimensional information corresponding to the pixels of the filtered face image, and the normal vector model is used to represent the surface information of the three-dimensional face image; Acquiring a three-dimensional point cloud of the filtered face image according to the pixel depth information of the filtered face image, the pixel plane coordinate information of the filtered face image, and the three-dimensional calculation model; According to the 3D point cloud of the filtered face image and the normal vector model, the normal vector corresponding to the 3D point cloud is acquired. 6. The method according to claim 1, wherein the pixel point information of the depth image comprises pixel point depth information of the depth image, and according to the pixel point information of the depth image, the The foreground image, including: Obtaining depth information of two adjacent pixels from the pixel depth information of the depth image; Acquiring the depth difference value of the two adjacent pixels according to the depth information of the two adjacent pixels; When the depth difference is smaller than a preset difference threshold, the image where the two adjacent pixels are located is used as the foreground image in the depth image. 7. An electronic device, characterized in that the electronic device is provided with a camera, and the electronic device includes an acquisition unit, an acquisition unit, a segmentation unit, a three-dimensional processing unit and a display unit, wherein, The acquisition unit is configured to acquire pixel point information of a depth image through the camera, and the pixel point information of the depth image is used to represent the pixel point depth information of the depth image and the plane coordinate information corresponding to the pixel point; The acquisition unit is used to acquire the height of the face of the subject through a face detection algorithm; The segmentation unit is configured to acquire a foreground image in the depth image according to the pixel point information of the depth image, and the foreground image is used to represent the image where the subject is located; The segmentation unit is further configured to obtain a face image in the foreground image according to a preset superposition depth model, pixel information of the foreground image, and the face height, and the preset superposition depth model is used for converting the foreground image into a foreground histogram; The three-dimensional processing unit is configured to perform three-dimensional processing on the face image to obtain a three-dimensional face image of the subject; The display unit is configured to display the three-dimensional face image; Wherein, the segmentation unit further includes: a first segmentation unit, configured to obtain the foreground histogram corresponding to the foreground image according to the preset superimposed depth model and pixel point information of the foreground image; Segment the foreground histogram to obtain the face image in the foreground image; Wherein, the pixel point information of the foreground image includes foreground pixel point plane coordinate information and foreground pixel point depth information, the foreground pixel point plane coordinate information includes foreground pixel point ordinate, and the first segmentation unit is specifically configured to: The depth value corresponding to the depth information of each foreground pixel point in the foreground image is obtained, and each foreground depth feature value corresponding to each foreground pixel point is obtained; according to each foreground depth feature value and the preset superposition depth model, at least one foreground pixel point is obtained. Superimposed depth eigenvalues; each foreground pixel ordinate and the superimposed depth eigenvalue corresponding to each foreground pixel ordinate form a foreground histogram corresponding to the foreground image; Wherein, the second segmentation unit is specifically used for: the ordinate of the foreground pixel point in the current foreground histogram is less than or equal to the face height of the subject, and the superimposed depth feature value corresponding to the ordinate of the foreground pixel point is not equal to the preset superimposed depth feature value, the image where each foreground pixel corresponding to the foreground pixel ordinate is taken as the face image in the foreground image. 8. An electronic device, characterized in that the electronic device is provided with a camera, and the electronic device at least includes a processor, a memory storing instructions executable by the processor, and a memory for connecting the processor, the The memory and the bus of the camera, when the executable instructions are executed, the processor implements the method according to any one of claims 1 to 6. 9. A computer-readable storage medium on which executable instructions are stored, wherein the executable instructions implement the method according to any one of claims 1 to 6 when executed by a processor.

Images