JPH0935070A - Face image processing device - Google Patents

Abstract

(57) Abstract: The inclination of a driver's face image is obtained from the inclination of a straight line passing through the center of gravity of the driver's binarized face image area and a representative point of the face image, and based on the inclination angle. The face image is rotated to obtain a face image without inclination. SOLUTION: A face image input means for inputting a face image of a driver, a feature candidate area extracting means for extracting a candidate area of a face feature point such as an eye, a nose, or a mouth from the face image, and two feature candidate areas are provided.
Binarizing means for binarizing, face area centroid calculating means for calculating the centroid of the area including the face in the binarized image, partial area representative point calculating means for calculating representative points of the face partial area, and the centroid. An inclination angle detecting means for detecting an angle formed by a straight line passing through two partial area representative points and a constant reference line as an inclination angle of the face.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face image processing apparatus having a function of detecting and determining the inclination of a face image by using image processing and correcting the face image to a position without inclination.

[0002]

2. Description of the Related Art A conventional driver face image processing apparatus is disclosed in, for example, Japanese Patent Application Laid-Open No. 6-227278. FIG. 29 is a block diagram showing the configuration of a conventional device. The image data taken into the apparatus by the image input means 55 is binarized by the binarizing means 50, and the eyeball existing area setting means 57 sets the eyeball existing area in the face image. The driver state determining means 59 is configured to determine the driver's state based on the pattern of the open / closed eyes in the eyeball presence region detected by the open / closed eye detecting means 58. The detection result of the open / closed eye detection means 58 is also sent to the erroneous detection determination means 61, and if it is determined that the detection is erroneous, the resetting means 60 resets the eyeball existing area. The setting of the eyeball existing area is performed in the order of the horizontal width and the vertical width.

FIG. 30 is an explanatory diagram of image scanning in the horizontal width setting, and FIG. 31 is an explanatory diagram of image scanning in the vertical width setting. The horizontal width is set by judging from the number of continuous white pixels in the left and right direction in the left and right regions with the scan start line (center of the X coordinate of the image) 62 of FIG. 32 as a boundary, and the black pixel 63 of FIG.
64 is searched and the vertical width is set. A broken line portion 65 in FIG. 34 is an eyeball existing region in which the horizontal width and the vertical width are set by the image scanning method of FIGS. 30 and 31. The horizontal width setting method counts the number of consecutive white pixels in the lateral direction of the face upwards from approximately the center of the face, as described in the flowchart of FIG.

Next, the X coordinate (horizontal coordinate of the face) at the maximum count is stored. As a result of counting the white pixels in the horizontal direction of the face in the vertical direction, if the sum of the number of consecutive white pixels on the left and right is, for example, 200 or more, the horizontal direction of the eyeball existing region is determined from the stored X coordinate at the maximum count. Set. However, when the total number of consecutive pixels is 200 or less, the number of consecutive white pixels in the horizontal direction is counted upward.

To set the vertical width of the eyeball existing area, as shown in the flow chart of FIG. 31, a black area in the vertical direction is searched from the number of black pixels. At this time, first, the maximum value of the number of pixels in the first black area is stored. After that, the maximum value of the pixels in the black area is stored vertically between the eyebrows and compared with the previous maximum value. Therefore, when the search for the black area in the vertical direction reaches between the eyebrows, the black area disappears, so that there is an eyeball between the coordinate value of the first maximum value and the coordinate value of the maximum value immediately before reaching the eyebrows. It is set as the vertical direction of the area. As a result,
As shown in FIG. 34, the left and right eyeball existing regions are set.

[0006]

As described above, the conventional face image processing apparatus always places the driver's face at a fixed position,
I am trying to capture the position of the eyeball from the image of the face,
Drivers do not always drive in the same posture, and often drive in a leaning posture. However, the conventional driver's face image processing device does not have a function of correcting the inclination of the face image, and therefore, if an attempt is made to process a face image that is greatly inclined and an eye presence region is detected, the presence of the detected eye presence is detected. The area is likely to be set at a position deviated from the actual position.

As a result, even if the eyeball image in the eyeball existing area is processed to make the eye open / closed determination, the eye open / closed determination cannot be accurately made, and the open / closed state may be erroneously detected.
There is a problem that even if the doze detection is performed, it is erroneously detected based on the processing result of the eyeball image.

Therefore, it is indispensable to recognize the movement of the head. When the human head is a sphere as shown in FIG. 28, the X axis is the horizontal axis of the face and the Y axis is the face. Is the vertical axis direction of the face, and the Z axis is the front-back axis direction of the face, the face is rotated about the X axis (for example, the amount of face down), or Y
It has been common to detect the direction of the face with respect to rotation about the axis (turning the face left and right) by image processing. (Refer to Shin-Gakkai Spring University D-541, 1990, etc.)

However, the method of detecting the inclination of the face with respect to the rotation around the z axis (tilting the face to the left and right) has not been found so often, and the erroneous detection of the open / closed state of the eyes cannot be eliminated.

The present invention has been made in order to solve the above-mentioned problems. Prior to setting the eyeball existing region, the inclination of the face image is detected and judged, and the inclination is corrected to determine the inclination. A face image processing device capable of obtaining a non-existing face image.

[0011]

According to a first aspect of the present invention, there is provided a face image processing apparatus for inputting a face image of a driver, face image input means, eyebrows, eyes, a nose in the input face image of the driver,
Feature candidate region extraction means for extracting candidate regions of face feature points such as mouth, binarization means for binarizing the image from which the feature candidate regions have been extracted, and the center of gravity (x for the region including the face in the binary image (x , Y), a face area center of gravity calculating means, a face part area setting means for setting an area including a part of the face based on the calculated center of gravity of the face area, and a partial area representative point for calculating a representative point for the face part area. Calculating means, the center of gravity (x,
y) and a straight line that passes through the two representative points of the partial area and the angle formed by one of the image axes is detected as the inclination detecting means.

A face image processing apparatus according to the invention of claim 2 is
Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, face area centroid calculating means for calculating the centroid (x, y) for the area including the face in the binary image, and face area centroid calculating means for calculating A face part area setting means for setting an upper area in the vertical direction of the face based on the center of gravity x of the face area center of gravity (x, y).
The set center of gravity of the upper region in the vertical direction of the face is calculated, and the partial region representative point calculating means using the calculated center of gravity as the representative point, and the straight line passing through the two points of the face region center of gravity (x, y) and the partial region representative point Inclination detection means for detecting the angle formed by the image axis of 1 as the inclination of the face is provided.

A face image processing apparatus according to the invention of claim 3 is
Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, face area centroid calculating means for calculating the centroid (x, y) for the area including the face in the binary image, and face area centroid calculating means for calculating Based on the horizontal center of gravity x of the face area center of gravity (x, y), the upper face vertical direction area is set, and the upper vertical face area is based on the vertical center of gravity y of the face area center of gravity (x, y). Face part area setting means for dividing the upper left and right areas into face areas, calculating the center of gravity of each of the set upper face left area and the upper face right area, and a partial area representative point calculating means whose midpoint is the representative point , The face area center of gravity (x, y) and the partial area representative point 2
A tilt detecting means for detecting an angle between a straight line passing through a point and one of the image axes as a tilt of the face is provided.

A face image processing apparatus according to a fourth aspect of the invention is
Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, feature candidate area grouping means for grouping feature candidate areas in the binary image, nostril area extracting means for extracting nostril areas in the feature candidate area group, Nostril region center of gravity calculating means for calculating the center of gravity for each nostril in the nostril region, inclination detecting means for detecting the angle between the straight line passing through the center of gravity for each nostril of the calculated nostril region and one image axis as the inclination of the face It is a thing.

A face image processing apparatus according to the invention of claim 5 is
Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, feature candidate area grouping means for grouping feature candidate areas in the binary image, nostril area extracting means for extracting nostril areas in the feature candidate area group, Feature area center of gravity calculating means for calculating the center of gravity for each feature candidate area group, between nostrils with the center of gravity for the area set by the nostril area extracting means as the center between the nostrils among the center of gravity calculated by the characteristic area center of gravity calculating means Center setting means, inclination detection means for detecting an average of angles formed by a straight line passing through two points of the center of gravity of each of the feature candidate areas excluding the nostril area and the center of the nostril and one image axis as the inclination of the face It is intended.

A face image processing apparatus according to the invention of claim 6 is
Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, face area barycenter calculating means for calculating the barycenter (x, y) for the area including the face in the binary image, and the feature candidate area in the binary image Feature candidate area grouping means for grouping, nostril area extracting means for extracting nostril areas in the feature candidate area group,
Inter-nostril center calculating means for calculating the center of the extracted nostrils between the nostrils, the angle between the straight line passing through the two points of the center of the nostrils and the center of gravity (x, y) of the face area and one image axis, and the inclination of the face. Inclination detecting means for detecting

A face image processing apparatus according to a seventh aspect of the invention is
The face image processing apparatus according to any one of claims 1 to 6, wherein a rotation center setting unit that sets a rotation center of a face image for correcting the face inclination detected by the inclination detection unit, and the rotation center setting unit. The rotation means is provided to correct the inclination of the face image by rotation based on the rotation center set by.

A face image processing apparatus according to the invention of claim 8 is
In the face image processing apparatus according to claim 7, the rotation center setting means sets the face area center of gravity (x, y) in the binary image as the rotation center.

A face image processing apparatus according to the invention of claim 9 is
In the face image processing apparatus according to the seventh aspect, the rotation center setting means sets the center between the nostrils as the rotation center.

A face image processing apparatus according to a tenth aspect of the present invention is the face image processing apparatus according to the seventh aspect, further comprising inclination determining means for determining whether or not the face image is inclined, and the face image is inclined. The image rotation process by the rotating means is continued until it is determined that there is no such image.

A facial image processing apparatus according to an eleventh aspect of the present invention is the facial image processing apparatus according to the tenth aspect, wherein the inclination determining means determines the lateral direction of the face in the binary image as the image vertical (y).
The vertical axis of the image is the horizontal (x) axis of the image, and an image vertical direction histogram showing the number of black pixels existing in the vertical axis of the image at each coordinate of the horizontal axis of the image is created. It is determined from the shape and the number of peaks whether or not the face image is tilted.

A face image processing apparatus according to a twelfth aspect of the present invention is the face image processing apparatus according to the tenth aspect, wherein the inclination determining means determines the horizontal direction of the face in the binary image as the image vertical (y).
The horizontal axis of the image and the vertical direction of the face are defined as the horizontal (x) axis of the image, and an image horizontal axis direction histogram indicating the number of black pixels existing in the image horizontal axis direction at each coordinate on the image vertical axis is created. It is determined from the symmetry whether or not the face image is tilted.

A face image processing apparatus according to a thirteenth aspect of the present invention is the face image processing apparatus according to the tenth aspect, wherein the inclination determining means obtains each of the feature candidate area centroid calculating means in the binary image. Whether or not the face image is tilted is determined from the arrangement of the center of gravity (xi) of the center of gravity (xi, yi) for the feature candidate region group.

A face image processing apparatus according to a fourteenth aspect of the present invention is the face image processing apparatus according to the tenth aspect, wherein an angle for rotating the face image is set according to the angle θ detected by the tilt detecting means. Rotation angle setting means, which comprises a rotation limit angle setting means for setting a limit angle of the rotation angle, when it is determined that the face image is not tilted by the tilt determination means while rotating the face image by the rotation means,
Alternatively, the image rotation processing by the rotating means is continued until the rotation limit angle is reached.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the invention are as follows. 1. The inclination of a straight line passing through the center of gravity of the binarized face image area of the driver and the representative point of the face image is obtained, and the face image is rotated based on this inclination to obtain a face image without inclination.

2. A representative point is found in the upper center area of the face image area of the face image area set with the center of gravity of the binarized face image area of the driver as a boundary, and the face image is obtained based on the inclination of a straight line passing through the center of gravity and the representative point. Rotation processing is performed to obtain a face image without inclination.

3. The midpoint between the center of gravity of the driver's binarized face image area and the center of gravity of the face upper left and right areas of the face image area set with this center of gravity as the representative point, and the center of gravity passes through the representative point. The face image is rotated based on the inclination of the straight line to obtain a face image without inclination.

4. 2 extracted from the driver's face image area
Calculate the center of gravity for each nostril in the quantized nostril region,
The face image is rotated based on the inclination of the straight line passing through each center of gravity.

5. After extracting candidate regions for each feature point of the face from the driver's face image region and binarizing and grouping the feature candidate regions, the calculated center of gravity is set as the center point between the nostrils in the feature candidate region including nostrils. The face image rotation processing is performed based on the average of the inclinations of the straight lines passing through the center point and the center of gravity calculated in each of the other feature candidate areas.

6. After calculating the center of gravity of the binarized face area, extracting candidate regions of each feature point of the face from the driver's face image region, binarizing the feature candidate regions and grouping them In the feature candidate area including the nostrils, the calculated center of gravity is set as the center point between the nostrils, and the face image is rotated based on the inclination of the straight line passing through the center point and the calculated center of gravity.

7. A predetermined coordinate position is set as a rotation center position of the face image, and the face image is rotated by a tilt angle of a straight line in a direction opposite to the tilt direction with reference to the rotation center position.

8. The center of rotation is set as the center of gravity of the face area, and the face image is rotated in the direction opposite to the tilt direction by the tilt angle of the straight line with the center of gravity as the reference.

9. The center of rotation is set as the center between the nostrils, and the face image is rotated in the direction opposite to the tilt direction by the tilt angle of the straight line with reference to this center.

10. The tilt determination means determines whether the face image is tilted, and the image rotation processing by the rotation means is continued until it is determined that the face image is not tilted.

11. A black color existing in the vertical axis direction of the image at each coordinate in the horizontal axis direction of the image is the horizontal axis of the image in the horizontal direction of the image and the vertical direction of the face is the horizontal axis of the image. An image vertical axis direction histogram showing the number of pixels is created, and it is determined whether or not the face image is tilted based on the shape and peak number of the image vertical axis direction histogram.

12. In the binary image, the horizontal direction of the face is the image vertical (y) axis, and the vertical direction of the face is the horizontal image (x) axis. Black that exists in the image horizontal axis direction at each coordinate in the image horizontal axis direction. An image horizontal axis direction histogram showing the number of pixels is created, and whether or not the face image is tilted is determined from the symmetry of the image horizontal axis direction histogram.

13. Whether or not the face image is tilted from the arrangement of the centroids xi in the horizontal axis of the centroids (xi, yi) for each feature candidate area group obtained by the feature candidate area centroid calculating means in the binary image To judge.

14. The rotation means rotates the face image, and the image rotation processing by the rotation means is continued when the inclination determination means determines that the face image is not inclined or until the rotation limit angle is reached.

Hereinafter, each embodiment will be described in detail. Embodiment 1. Hereinafter, the present embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the face image processing apparatus according to this embodiment. The face image processing apparatus according to the present embodiment includes a CCD camera 1 for capturing a face image of a driver, a C
The image memory 2 stores the image data of the face image output from the CD camera 1, and the CPU 3 that performs image processing based on the data in the image memory 2. FIG. 2 is a flowchart showing an outline of image processing of the CPU 3 in FIG.

First, an outline of image processing will be described with reference to the flowchart of FIG. The CPU 3 inputs the image data of the face image of the driver from the image memory 2 by the face image input means 4, and the feature candidate area extraction means 5 extracts the candidate regions of the feature points of the input face image. The candidate region of the face image extracted by the feature candidate region extracting means 5 is binarized by the binarizing means 6.

The center of gravity of the candidate area including the face in the binarized face image is calculated by the face area center of gravity calculating means 7, and then the face part area setting means 8 calculates the center of gravity within the face area based on the face area center of gravity. A face part area is newly set. Partial area representative point calculation means 9
Then, the angle formed by the straight line connecting the representative point in the set face area and the center of gravity of the face area and one of the image axes is obtained by the inclination detecting means 10, and the inclination of the face area is corrected by rotation. The rotation center is set by the rotation center setting means 11, and the rotation is performed by the rotation means 12.

Each means shown in the flow chart of FIG. 2 will be described in detail below with reference to each drawing. FIG. 3 is a face image of the driver, and FIG. 3A shows a face image 13 of the driver captured by the CCD camera 1. FIG. 3B shows the face image 14 after the processing by the feature candidate area extracting means 5 and the binarizing means 6, in which the feature candidate areas are binarized. 5A and 5B are explanatory diagrams of the feature candidate area extracting unit 5 and the binarizing unit 6 in the present embodiment.
4A, 4B, and 4C are explanatory diagrams of the max / min filter used in the feature candidate area extracting unit 5.

First, the max / min filter will be described. The image signal 15 after the max filter is applied to the image signal 15 of a part of the input image shown in FIG. 4A is 16, and the image signal 17 after the min filter is applied to this image signal 16 is shown. Reference numerals 18 to 20 in FIG. 4B correspond to the pixels in which the brightness values of the image signals 15 to 17 are respectively modeled. Here, one frame surrounding the brightness value represents one pixel, and the brightness value is set to a value of 0 to 20 for simplicity.

First, the max filter is applied to the input image. The max filter has a predetermined length (number of pixels a, hereinafter referred to as filter size), and the brightness value of the pixel at the center of the filter is set to the maximum brightness value in the filter. It is to convert.
For example, the number of pixels a is set to 5, and the squares in FIG.
When the ax filter is applied, since the maximum brightness is 10 here, the brightness value of the pixel 18b in the center of the filter is converted from 8 to 10.

When shifting by one pixel to the right and applying the max filter in the same manner, 18 in FIG. 4B is changed to 19 in FIG.
Is converted to. Next, a min filter is applied to the image after the max filter processing. min filter is also max
The filter has the same filter size (number of pixels a) as the filter, and the brightness value of the pixel at the center of the filter is converted into the minimum brightness value in the filter.

When the min filter is applied to the square portion 19 in FIG. 4 (b) after the max filter processing, the minimum luminance value is 10 here, so that the pixel 1 in the center of the filter is
The luminance value of 9c is converted from 12 to 10. When shifting to the right one pixel at a time and applying the min filter in the same manner,
The □ part of 19 in (b) is converted into 20.

However, in both the max filter and the min filter, the conversion starts from the center position of the filter applied first, and the conversion ends at the center position of the filter applied last. That is, the data after the max filter processing is shown in FIG.
The data after 19d to 19e and min filter processing of (b) are 20f to 20g of (b) of FIG. 4, and max / mi
The image after the n-filter processing is eventually smaller by a-1 pixel from the processing start position and the processing end position.

When the input image is subtracted from the image thus obtained after the max / min filter processing, a region having a predetermined brightness or less and a low luminance is extracted. 20 to 1 in FIG. 4 (b)
When 8 is subtracted, FIG. 4 (d) is obtained. When converted into an image signal, it becomes as shown in FIG.

The above method is used to extract the feature regions. In the present embodiment, processing is performed line by line along the horizontal axis (X axis) of the image (processing direction: arrow 23 in FIG. 3A).
Direction). Reference numeral 21 in FIG. 5A is an image signal after the input image has been subjected to a max / min filter having a length corresponding to the width of the characteristic region. Since the difference in brightness between the characteristic region of the face and the surroundings is large, the brightness value after the difference is large (21a).

On the other hand, shadows and clothes have a small difference in brightness from the surroundings (21b). Therefore, as shown by 22 in FIG. 5A, a threshold is set for the result after the difference, and based on this, 2
Quantify. FIG. 5B shows the signal after binarization. The image obtained by the above processing is shown in FIG.
It is.

FIGS. 6A, 6B and 6C are explanatory views of the face area center of gravity setting means 7, the face partial area setting means 8 and the partial area representative point calculating means 9. In the face area center of gravity setting means 7,
First, the center of gravity 2 for the entire area of the image 14 shown in FIG.
Calculate 4. The center of gravity is calculated in the black area (xj, y
j) is the total number of pixels, and the sum of the number of all yj pixels is Y.
Assuming that the sum of the pixel numbers of t and all xj is Xt, the coordinate point (X, Y) of the center of gravity 24 is obtained by the following equation (1).

[0052]

[Equation 1]

After calculating the center of gravity 24, the face part area setting means 8
Sets an area 25 including a face based on this coordinate point (X, Y) (hereinafter, an area including a face is referred to as a face area). Face area 2
5 is that the ratio of the face to the input image is large,
Considering that the center of gravity of the face is almost centered between the face and between the nose and the eyes, the face region 25 has the face feature amount (at least the eyes and the nose).
It is a rectangle that includes. The face area center of gravity 26 (coordinates (a, b)) for the face area 25 is calculated using the above equation (1) as in the case of the center of gravity 24.

Next, the face part area setting means 8 will be described. Based on a of the calculated coordinates (a, b) of the center of gravity 26 of the face area, the face area 25 is vertically upper part (including at least eyes) in the face direction.
It is divided into an area and a vertically lower area of the face. Here, the upper area is adopted as the face portion area 27. The processing by the face part area representative point calculating means 9 is performed on the face part area 27. In the present embodiment, the face part area representative point 28 (coordinates (c,
As shown in FIG. 6C, d)) is the center of gravity for the face portion area 27 obtained by the above equation (1).

FIG. 7 is an explanatory diagram of the inclination detecting means 10.
The inclination detecting means 10 executes the above-described processing to obtain the face area center of gravity 26 (a, b) and the face portion area representative point 28.
A straight line 29 passing through (c, d) is obtained, and an angle θ with one coordinate axis, which is the X axis in this embodiment, is obtained. The angle θ is calculated by the following equation (2).

[0056]

[Equation 2]

FIG. 8 shows the binary image after rotation. The center of gravity 26 of the face area is set as the rotation center by the rotation center setting means 11,
The face area 25 is rotated by θ in the counterclockwise direction by the rotating means 12. In this case, the rectangular area represented by the black background is the face area 25 before rotation, and the coordinates of each pixel of the face area 25 are generally represented by (e, f). The rectangular area represented by the intermediate color is the rotated face area 25, and the coordinates of each pixel in the face area 25 are generally represented by (E, F). Expression (3) is used for the rotation processing of the face area 25. However, a portion (30a in FIG. 8) where the rotated coordinate (E, F) appears from the image is cut, and the unrotated coordinate (e, f) does not exist (FIG. 8).
The luminance value of 30b) is 0.

[0058]

(Equation 3)

Embodiment 2 In the above-described first embodiment, the face part region representative point is obtained in the upper part of the face part region including the center of gravity of the face region, and the angle θ formed by the straight line connecting the face part region representative point and the center of gravity and the coordinate axis is set to the face. Although the corrected rotation angle of the area is used, the face area representative point may be displaced more accurately in the present embodiment because it may be displaced if noise or the like is added to the image data in which the facial area representative point is roughly determined. Therefore, the partial area setting means 8 and the partial area representative point calculating means 9 are changed as follows.

The changed face part area setting means 8 will be described. In the present embodiment, the face area center of gravity calculating means 7
The face part area 27 set based on a of the face area center of gravity 26 (a, b) calculated in step 3 is further divided into the left and right face areas 31 and 32 shown in FIG. Two areas, that is, the left and right areas 31 and 32 of the face are adopted as the face part area. The processing by the face part area representative point calculating means 9 is performed on the right and left areas 31 and 32 of the face. In the present embodiment, the face part area representative points are, as shown in FIG. 9, areas 31 and 3 obtained by the above equation (1).
Center point 34 of the center of gravity 33a, 33b for each of the two
(E, f).

When the face portion area representative point 34 is obtained in this way, a straight line connecting the face area center of gravity 26 and the face portion area representative point 34 (not shown), one coordinate axis, X in the present embodiment. The angle θ with the axis is calculated. The angle θ is obtained by the above equation (2). Then, the rotation center setting means 11
Thus, the center of gravity 26 of the face area is set as the center of rotation, and the face area 25 is rotated counterclockwise by the rotating means 12.

Embodiment 3 An outline of the processing of the third embodiment will be described based on the flowchart of FIG. In the present embodiment, the region of two nostrils that can be converted into coordinate values with the highest accuracy among the feature candidate regions is used as the feature candidate region. C
The PU 3 inputs the image data from the image memory 2 by the face image input means 4, and the feature candidate area extraction means 5 extracts the feature candidate area of the input face image. The image extracted in the feature candidate area is binarized by the binarizing means 6. The feature candidate area grouping means 35 sets the feature candidate areas in the binarized image.
13. As shown in FIG. 13, it is grouped into a plurality of groups.

The nostril region extracting means 36 extracts the nostrils from the grouped feature candidate region groups. Then, the nostril region center of gravity calculating means 37 calculates the center of gravity of each nostril region. The angle formed by the straight line that passes through the two points of the center of gravity of the nostril region and one of the image axes is calculated by the tilt detecting means 10, and the tilt is corrected by rotation. The center of rotation is set at the center of the center of gravity of each nostril region by the center of rotation setting means 11, and the rotating means 12 rotates the face area 25 in the direction opposite to the tilt direction of the face area with reference to this center of rotation.

Hereinafter, each means shown in the flow chart of FIG. 10 will be described in detail. The face image input means 4, the feature candidate area extraction means 5, and the binarization means 6 are the same as those in the first embodiment.
Since it has been described above, the feature candidate area grouping unit 35 will be omitted.
Let's start with. 11 (a), (b), and (c) are views for explaining the operation of the feature candidate area grouping unit 35, and FIG.
14A and 14B are diagrams for explaining the operation of the nostril region extracting means 36, and FIGS. 14A and 14B are the nostril region centroid calculating means 3
It is explanatory drawing of 7. Further, FIG. 13 is an image after the processing by the feature candidate area grouping means 36.

The feature candidate region grouping means 35 counts the number of black pixels in the vertical axis direction of the binarized face image for each X coordinate, and the vertical axis as shown in FIG. 11A is black. An image vertical direction histogram in which the number of pixels and the horizontal axis are X coordinates is created. Next, from the histogram, FIG.
As shown in, a peak with a maximum number of black pixels of 38a or more is searched for, and for each peak, a peak start position 38c with a number of black pixels of 38e or more and a distance from the maximum point of 38b or less.
i and the peak end position 38di are set.

This time, the number of black pixels in the direction of the horizontal axis of the image is counted for each Y coordinate for each of the band-shaped areas 39ai from the set peak start position 38ci to peak end position 38di, and as shown in FIG. 12 (a). An image horizontal direction histogram in which the vertical axis is the Y coordinate and the horizontal axis is the number of black pixels is created. In the histogram in the horizontal direction of the image, because of the characteristics of the feature candidate area extracting unit 5, only the area having a predetermined length or less is extracted, so that a part of the eye or a part of the eyebrow may be cut in some cases.

Therefore, the distance 3 as shown in FIG.
39c except 9c or more and 39b or less black pixels
i and 39di and the same strip-shaped region 39a
A feature candidate region group 40i belongs to i.

In the nostril region extracting means 36, two groups having almost the same width are arranged in the feature candidate region group 40i as shown in FIG. 14A, and the width between the two groups is narrower than the others. Let it be the nostril region 42. In the nostril region center of gravity calculating means 37, the nostril region 42 shown in FIG.
Center of gravity 43a for each black region in the two groups of
(A, b) and 43b (c, d) are obtained by the above equation (3).

Next, as shown in FIG. 15, the inclination detecting means 1
At 0, a straight line 44 passing through the nostril region centroids 43a (a, b) and 43b (c, d) obtained by executing the above-described processing is obtained, and one of the coordinate axes, that is, the Y axis in the present embodiment, is obtained. Find the angle θ. The angle θ is obtained by the above equation (2). By the rotation center setting means 11, the center of gravity 43a of the nostril region,
The midpoint 45 of 43b is set as the center of rotation, and the face area is rotated counterclockwise by the rotation means 12 by θ. The rotating means 12 is similar to that of the first embodiment, and uses the above formula (3).

Embodiment 4 In the third embodiment, the inclination is corrected by rotating the face area based on the inclination angle of a single straight line passing through the center of gravity of the nostril area, which is one of the feature candidate area groups. In the present embodiment, the inclination angle of each straight line connecting the midpoint obtained in the third embodiment and the center of gravity of each feature candidate area is obtained, and the average value of the inclination angles is calculated as the rotation angle θ. The outline of the processing of this embodiment will be described based on the flowchart of FIG.

The CPU 3 inputs the image data from the image memory 2 by the face image input means 4, and the feature candidate area extracting means 5 extracts the feature candidate area of the face image in the input image data. The face image extracted in the feature candidate area is binarized by the binarizing means 6. The feature candidate regions in the binarized face image are grouped by the feature candidate region grouping means 35. Nostrils are extracted by the nostril region extracting means 36 from the grouped feature candidate region groups.

The characteristic region center of gravity calculating means 46 calculates the center of gravity of each group set by the characteristic candidate area grouping means 35. The inclination detecting means 1 calculates the average of the angle between the straight line passing through the center of the nostril set by the internostril center setting means 47 and the center of gravity of each feature candidate area excluding the nostril area and one image axis.
Calculate with 0. The rotation center is set by the rotation center setting means 11. The face image is rotated by an average angle by the rotating means 12 with the center of rotation set as a reference to correct the inclination.

The details of each means shown in the flowchart of FIG. 16 will be described below. The face image inputting means 4, the feature candidate area extracting means 5, the binarizing means 6, the feature candidate area grouping means 35, and the nostril area extracting means 36 have been described in the first embodiment and the third embodiment, and are omitted. The characteristic region center of gravity calculating means 46 will be described. In the characteristic region center of gravity calculating means 46, as shown in FIG.
The center of gravity 48i (ci, di) for the black area in the group 40i existing in ai (i: 1-5) is calculated. The above equation (1) is used to calculate the center of gravity.

In the center-of-nostril setting means 47, the center of inter-nostril center 49 (a, b) is set as the feature candidate area center of gravity 48i for the nostril area 42, which is previously obtained by the nostril area extracting means 36.

Next, as shown in FIG. 18, the inclination detecting means 1
At 0, the center of gravity 48i (ci, di) of each feature candidate area and the center 49 between the nostrils obtained by executing the above-described processing.
An angle formed between a straight line passing through two points (a, b) and one coordinate axis, which is the X axis in the present embodiment, is calculated, and the average value is calculated as the angle θ.
And The angle θ is calculated by the following equation (4). The center 49 (a, b) between the nostrils is set as the center of rotation by the rotation center setting means 11, and is rotated counterclockwise by the rotation means 12. The rotating means 12 is similar to that of the first embodiment, and uses the above formula (3).

[0076]

(Equation 4)

Embodiment 5 In the above-described fourteenth embodiment, the center of gravity of each grouped feature candidate region is obtained and connected to the center of the nostril by a straight line, but the inclination of the straight line connecting the center of the nostril and the center of gravity of the face region is simplified to simplify the process. The tilt correction angle may be determined by the angle. The outline of the processing of this embodiment will be described based on the flowchart of FIG. CPU3
The face image input means 4 inputs the image data from the image memory 2, and the feature candidate area extracting means 5 extracts the feature candidate area of the face image in the input image data. The face image extracted in the feature candidate area is binarized by the binarizing means 6. After calculating the center of gravity for the area including the face in the binarized face image by the face area center of gravity calculating means 7, the feature candidate areas in the binarized face image are grouped by the feature candidate area grouping means 35. To do. Nostrils are extracted by the nostril region extracting means 36 from the grouped feature candidate region groups.

The internostril center calculation means 50 calculates the internostril center for the extracted nostril region. The angle between the straight line passing through the two points of the center of the nostril and the center of gravity of the face area and one of the image axes is calculated by the inclination detecting means 10. The face image is rotated by the rotating unit 12 on the basis of the center of rotation set by the center of rotation setting unit 11, and the inclination of the face image is corrected.

Details of each means shown in the flowchart of FIG. 19 will be described below. The face image input means 4, the feature candidate area extracting means 5, the binarizing means 6, the face area center of gravity setting means 7, the feature candidate area grouping means 35, and the nostril area extracting means 36 are the same as those in the first embodiment. Since it has been described in the third embodiment, it will be omitted, and the internostril center calculation means 50 will be described. However, the calculated face area center of gravity 2
6 is (a, b) in the first embodiment, but is (c, d) in the present embodiment.

The inter-nostril center calculating means 50 obtains the center of gravity 51 (a, b) for the black area in the nostril area group existing in the same strip-shaped area extracted by the nostril area extracting means 36 as shown in FIG. . The above equation (1) is used to calculate the center of gravity.

Next, in the inclination detecting means 10, the face area center of gravity 26 (c,
d) and a straight line passing through the nostril center 51 (a, b),
An angle θ with one coordinate axis, which is the X axis in this embodiment, is obtained. The angle θ is obtained by the above equation (2). The center 51 (a, b) between the nostrils is set as the center of rotation by the rotation center setting means 11, and is rotated counterclockwise by the rotation means 12. The rotating means 12 is the same as that of the first embodiment, and the above (2)
Use an expression.

Sixth Embodiment In the above-described first to fifth embodiments, the inclination determination of the face area is performed based on the result of calculating the face area center of gravity and the representative point. However, in order to simplify the processing, the number of black pixels in the binarized feature candidate area The inclination of the face area is directly determined from the histogram, and if the inclination is not determined, all processing is stopped there. If the inclination is determined, the same processing as in each of the above-described embodiments is performed, and the facial image rotation processing is performed.

FIG. 21 is a flow chart for explaining the outline of the processing of this embodiment. In the figure, the same step numbers as those in the flowchart of FIG. 2 indicate the same processing. In the figure,
In step 52, the inclination determining means 52 determines the presence / absence of inclination of the face area directly from the histogram of the number of black pixels in the binarized feature candidate area. Then, if it is determined that there is inclination, the process proceeds to the face area center of gravity calculating means 7, and if it is determined that there is no inclination, all processing is stopped.

In the inclination determining means 52, the horizontal face direction is the image vertical (y) axis, the vertical face direction is the image horizontal (x) axis in the binarized image, and the number of black pixels in the vertical image direction is Counting is performed for each X coordinate, and an image vertical axis direction histogram in which the vertical axis is the number of black pixels and the horizontal axis is the X coordinate is created. FIG. 22A is an image vertical direction histogram of a non-tilted face image, and FIG. 22B is an image vertical direction histogram of a tilted face image.

As can be seen from the figure, in a face image which is not tilted, the peak of the histogram is clear, and the number of black pixels is almost the same as the number of characteristic regions of the face (usually, eyebrows, eyes, nose, mouth are 4). peak). On the other hand, the histogram of the tilted image does not have clear peaks, and the number of peaks does not match the number of facial feature regions. Above, the shape of the histogram,
And it is judged whether it is inclined from the number of peaks.

Embodiment 7 In the sixth embodiment described above, the number of black pixels in the vertical axis direction of the image is counted for each X coordinate to create a histogram, and the non-tilt of the face image is determined from the clarity of the peak. Since it is bilaterally symmetric with respect to the line, the number of black pixels in the horizontal direction of the image is set to Y.
It is possible to determine the non-tilt of the face image from the appearance of symmetrical peaks having the same peak value by creating a histogram by counting the coordinates.

FIGS. 23 (a) and 23 (b) are explanatory views of the inclination determining means 52 in which the determination standard is changed. Tilt determination means 52
Then, with respect to the binarized image, the face horizontal direction is the image vertical (y)
The horizontal axis of the image is the horizontal axis of the image (x), the number of black pixels in the horizontal axis of the image is counted for each Y coordinate, and the histogram of the horizontal axis of the image in which the vertical axis is the number of black pixels and the horizontal axis is the Y coordinate is created. To do. FIG. 23A is an image horizontal axis direction histogram of an untilted image, and FIG. 23B is an image horizontal axis direction histogram of an inclined image.

As can be seen from the figure, the image which is not tilted has a clear histogram peak and is almost symmetrical (usually symmetrical at the center of the face). On the other hand,
The histogram of a tilted image has no clear peaks and is not symmetrical. As described above, it is determined from the shape and symmetry of the histogram whether the face image is tilted.

Eighth Embodiment FIG. 24 is a flowchart illustrating the face image processing according to this embodiment. Figure 25
(A) and (b) are the inclination determination means 52 according to the present embodiment.
FIG. As shown in FIG. 24, a tilt determining unit 52 is added after the feature region center of gravity calculating unit 46 of the fourth embodiment, and if the result of the determination is that there is no tilt, all processing is stopped, and the internostril center setting unit 47 After the processing up to the rotation means 12, the process returns to the feature candidate area extraction means 5 again, and the same processing is repeated thereafter.

The inclination determining means 52 determines whether or not the center of gravity calculated by the characteristic region center of gravity calculating means 46 is more inclined. FIG. 25 (a) shows an array of centroids of an untilted image, and FIG. 25 (b) shows an array of centroids of an inclined image. As can be seen from the figure, in the non-tilted image, the center of gravity is almost aligned with the vertical centerline of the face. On the other hand, the center of gravity of the tilted images is not constant. As described above, it is determined whether or not the center of gravity of the characteristic regions is inclined.

Embodiment 9 FIG. FIG. 27 shows the rotation angle setting means 53a and the rotation limit angle setting means 5 in this embodiment.
It is explanatory drawing of 3b. Embodiment 6 as shown in FIG.
The rotation angle setting means 53a and the rotation limit angle setting means 53b are added in front of the rotation means 12 of 8 to 8, and the rotation limit angle (θ + α) is the minimum rotation angle β and the maximum rotation angle depending on the detected angle θ. ) Is set.

However, as shown in FIG. 27, the rotation angle β does not exceed the rotation limit angle (θ + α), and the rotation limit angle (θ + α) is set based on the detection angle θ. After the processing up to the rotation means 12, the process returns to the feature candidate area extraction means 5 again, and the same processing is repeated thereafter. When repeating, the rotation limit angle (θ + α) is always constant. The rotation angle β does not have to be particularly constant, but it is considered that the closer the detection angle θ is, the smaller the rotation angle is and the more the number of rotations is, the more accurate the correction can be made. The above processing is continued until it is judged by the tilt judging means 52 that the image is not tilted or the rotation limit angle (θ + α) is reached.

The present invention has the following effects. 1. Face image inputting means for inputting a driver's face image, feature candidate area extracting means for extracting candidate areas of feature points of the face such as eyebrows, eyes, nose, mouth in the input driver's face image, feature candidate area Binarizing means for binarizing the extracted image, face area centroid calculating means for calculating the centroid (x, y) with respect to the area including the face in the binary image, based on the calculated centroid of the face area A face part region setting means for setting a region including a part of a face, a part region representative point calculation part for calculating a representative point for the face part region, and the face area center of gravity (x, y) and the part region representative point. Since the inclination detecting means for detecting the angle between the passing straight line and one of the image axes as the inclination of the face is provided, the inclination angle can be easily detected with high accuracy and at a high speed.
This has the effect of improving the accuracy of correcting the inclination of the face image.

2. Face image input means for inputting the face image of the driver, eyebrows, eyes, nose in the input face image of the driver,
Feature candidate region extraction means for extracting candidate regions of face feature points such as mouth, binarization means for binarizing the image from which the feature candidate regions have been extracted, and the center of gravity (x for the region including the face in the binary image (x , Y) for calculating a face area center of gravity, and a face portion for setting an upper area in the vertical direction of the face based on the center of gravity x in the horizontal axis of the face area center of gravity (x, y) calculated by the face area center of gravity calculating means. The area setting means, the center of gravity of the set face vertical direction upper area is calculated, and the partial area representative point calculation means using the calculated center of gravity as the representative point, the face area center of gravity (x, y) and the partial area representative point
Since the inclination detecting means for detecting the angle between the straight line passing through the point and one of the image axes as the inclination of the face is provided, the partial area representative point through which the straight line for detecting the inclination angle is passed together with the center of gravity of the face area accurately, and There is an effect that it can be easily obtained at high speed.

3. Face image input means for inputting the face image of the driver, eyebrows, eyes, nose in the input face image of the driver,
Feature candidate region extraction means for extracting candidate regions of face feature points such as mouth, binarization means for binarizing the image from which the feature candidate regions have been extracted, and the center of gravity (x for the region including the face in the binary image (x , Y) for calculating the face area center of gravity, and based on the center of gravity x in the horizontal axis of the face area center of gravity (x, y) calculated by the face area center of gravity calculating means, an upper face vertical direction area is set, and Face part area setting means for setting the upper left and right areas of the face by dividing the upper area in the vertical direction of the face based on the vertical center of gravity y of the center of gravity (x, y) of the face area, the upper left area of the set face, the upper right of the face A center of gravity of each area is calculated, and a partial area representative point calculating means having a middle point as a representative point, a straight line passing through the two points of the face area center of gravity (x, y) and the partial area representative point, and one image axis Since the inclination detecting means for detecting the angle formed as the inclination of the face is provided, the inclination angle can be detected. Linear more accurately the partial area representative point through with the face area center of gravity, yet there is an effect that easy can be obtained at high speed for.

4. Face image input means for inputting the face image of the driver, eyebrows, eyes, nose in the input face image of the driver,
Feature candidate region extracting means for extracting candidate regions of facial feature points such as mouth, binarizing means for binarizing the image from which the feature candidate regions are extracted, and feature for grouping feature candidate regions in a binary image Candidate area grouping means, nostril area extracting means for extracting a nostril area in the feature candidate area group, nostril area centroid calculating means for calculating the center of gravity for each nostril in the nostril area,
The calculated inclination of a straight line that passes through the center of gravity of each nostril region and an angle formed by one of the image axes is detected as the inclination of the face, and the inclination of the straight line that passes through the two nostrils whose position does not change in the face image. Since the face image is rotated based on the angle and the image is corrected, the correction accuracy is improved.

[0097] 5. Face image input means for inputting the face image of the driver, eyebrows, eyes, nose in the input face image of the driver,
Feature candidate region extracting means for extracting candidate regions of facial feature points such as mouth, binarizing means for binarizing the image from which the feature candidate regions are extracted, and feature for grouping feature candidate regions in a binary image Candidate area grouping means, nostril area extracting means for extracting nostril areas in the characteristic candidate area group, characteristic area centroid calculating means for calculating the centroid of each characteristic candidate area group, and centroids calculated by the characteristic area centroid calculating means Among these, a nostril center setting means having a center of gravity for the area set by the nostril area extraction means as a center between the nostrils, a straight line passing through two points of the center of gravity and the center of the nostril of each feature candidate area excluding the nostril area, and one Since the inclination detecting means for detecting the average of the angles formed by the image axes as the inclination of the face is provided, there is an effect that the inclination angle detection accuracy is improved.

6. Face image input means for inputting the face image of the driver, eyebrows, eyes, nose in the input face image of the driver,
Feature candidate region extraction means for extracting candidate regions of face feature points such as mouth, binarization means for binarizing the image from which the feature candidate regions have been extracted, and the center of gravity (x for the region including the face in the binary image (x , Y), a face area centroid calculating means, a feature candidate area grouping means for grouping feature candidate areas in a binary image, a nostril area extracting means for extracting a nostril area in the feature candidate area group, extracted And a nostril center calculating means for calculating the center between the nostrils of the nostril region, and the angle between the straight line passing through the two points of the center of the nostril and the face area center of gravity (x, y) and one image axis is detected as the face inclination. Since the tilt detecting means is provided, the tilt angle can be detected with high accuracy and easily.

7. The face image processing apparatus according to any one of claims 1 to 6, wherein a rotation center setting unit that sets a rotation center of the face image for correcting the inclination of the face detected by the inclination detection unit, and the rotation center setting unit. Since the rotation means for correcting the inclination of the face image by rotation based on the rotation center set by the above is provided, in addition to the effect of the face image processing apparatus according to any one of claims 1 to 6, There is an effect that the image processing quality is improved because the face image is corrected and processed.

8. In the face image processing apparatus according to claim 7, the rotation center setting means sets the face area center of gravity (x, y) in the binary image as the rotation center. Therefore, in addition to the effect of claim 7, rotation is performed. This has the effect of facilitating center setting.

9. In the face image processing apparatus according to claim 7, since the rotation center setting means sets the center of the nostril as the rotation center, in addition to the effect of the seventh aspect, the rotation center can be easily set. This has the effect of improving the center setting accuracy.

10. The face image processing apparatus according to claim 7, further comprising a tilt determination unit that determines whether or not the face image is tilted, and the image rotation processing by the rotation unit is continued until it is determined that the face image is not tilted. Therefore, in addition to the effect of the seventh aspect, there is an effect that fine inclination correction processing can be performed and the correction accuracy is improved.

11. 11. The face image processing apparatus according to claim 10, wherein the inclination determination means sets the horizontal face direction in the binary image as the image vertical (y) axis, the vertical face direction as the image horizontal (x) axis, and the horizontal image direction. An image vertical axis direction histogram indicating the number of black pixels existing in the vertical axis direction of the image at each coordinate is created, and it is determined whether or not the face image is tilted based on the shape and peak number of the image vertical axis direction histogram. Therefore, there is an effect that the inclination determination process is simplified.

12. 11. The face image processing device according to claim 10, wherein the inclination determination means sets the face horizontal direction in the binary image as the image vertical (y) axis, the face vertical direction as the image horizontal (x) axis, and on the image vertical axis. Since the image horizontal axis direction histogram showing the number of black pixels existing in the image horizontal axis direction at each coordinate is created, it is determined whether or not the face image is tilted based on the symmetry of the image horizontal axis direction histogram. Since the tilt determination can be visually performed, the tilt determination process is further simplified.

13. 11. The face image processing apparatus according to claim 10, wherein the inclination determining unit determines the center of gravity (xi, yi) of the center of gravity (xi, yi) for each feature candidate region group obtained by the feature candidate region center of gravity calculating unit in the binary image. Since it is determined whether or not the face image is tilted based on the arrangement of xi, there is an effect that the tilt determination process is simplified.

14. The face image processing apparatus according to claim 10, wherein a rotation angle setting unit that sets an angle for rotating the face image according to the angle θ detected by the tilt detection unit, and a rotation that sets a limit angle of the rotation angle. When the face image is rotated by the rotating unit and the tilt determining unit determines that the face image is not inclined, the image rotation process by the rotating unit is performed until the rotation limit angle is reached. Since I tried to continue, it is possible to limit the rotation range for tilt correction,
There is an effect that wasteful rotation can be suppressed and large erroneous rotation can be avoided.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing flow of the first and second embodiments.

FIG. 3 is a diagram of a face subjected to image processing.

FIG. 4 is an explanatory diagram of a max / min filter.

FIG. 5 is an explanatory diagram of binarization processing.

FIG. 6 is a diagram showing a part of the flow of processing according to the first embodiment.

FIG. 7 is an explanatory diagram of an inclination detection unit, a rotation center setting unit, and a rotation unit.

FIG. 8 is a face image after tilt correction.

FIG. 9 is a diagram showing a part of the flow of processing according to the second embodiment.

FIG. 10 is a flowchart showing a flow of processing according to the third embodiment.

FIG. 11 is an explanatory diagram of an image vertical axis direction histogram.

FIG. 12 is an explanatory diagram of an image horizontal axis direction histogram.

FIG. 13 is a face image after the feature candidate region grouping process.

FIG. 14 is an explanatory diagram of a nostril region extracting means.

FIG. 15 is an explanatory diagram of an inclination detection unit, a rotation center setting unit, and a rotation unit according to the third embodiment.

FIG. 16 is a flowchart showing a processing flow of the fourth embodiment.

FIG. 17 is an explanatory diagram of a characteristic region centroid calculation means.

FIG. 18 is an explanatory diagram of a tilt detecting unit, a rotation center setting unit, and a rotating unit according to the fourth embodiment.

FIG. 19 is a flowchart showing the flow of processing according to the fifth embodiment.

FIG. 20 is an explanatory diagram of a nostril center calculating means.

FIG. 21 is a flow chart showing the flow of processing in the sixth and seventh embodiments.

FIG. 22 is an image vertical axis histogram of an image.

FIG. 23 is an image horizontal axis direction histogram of an image.

FIG. 24 is a flowchart showing the flow of processing of the eighth embodiment.

FIG. 25 is a diagram showing an arrangement of centroids of an image.

FIG. 26 is a flowchart showing the flow of processing in the ninth embodiment.

FIG. 27 is an explanatory diagram of rotation angle setting output means.

FIG. 28 is a diagram in which a direction in which a face is tilted is expressed by coordinates.

FIG. 29 is a block diagram showing a configuration of a conventional example.

FIG. 30 is a flowchart showing a flow of lateral direction setting of an eyeball existing region in a conventional example.

FIG. 31 is a flowchart showing a flow of setting a vertical direction of an eyeball existing region in a conventional example.

FIG. 32 is a diagram showing a part of the processing flow of an eyeball existing region in a conventional example.

FIG. 33 is a diagram showing a part of the flow of processing of an eyeball existing region in a conventional example.

FIG. 34 is a diagram showing a part of the flow of processing of an eyeball existing region in a conventional example.

[Explanation of symbols]

1 camera, 2 image memory, 3 CPU, 4 face image input means, 5 feature candidate area extracting means, 6 binarizing means, 7 face area centroid calculating means, 8 face part area setting means, 9 partial area representative point calculating means 10 tilt detection means, 11 rotation center setting means, 12 rotation means.

Claims (14)

[Claims] 1. A face image input means for inputting a face image of a driver, and feature candidate region extraction for extracting candidate regions of face feature points such as eyebrows, eyes, nose and mouth in the input face image of the driver. Means, binarizing means for binarizing the image in which the feature candidate area is extracted, face area centroid calculating means for calculating the centroid (x, y) for the area including the face in the binary image, calculated face area Face area setting means for setting an area including a part of a face based on the center of gravity of the face, partial area representative point calculating means for calculating a representative point for the face area, the center of gravity (x, y) of the face area and the partial area representative A face image processing apparatus comprising a tilt detecting unit that detects an angle formed by a straight line passing through two points and one image axis as a tilt of a face. 2. Face image input means for inputting a driver's face image, and feature candidate region extraction for extracting candidate regions of facial feature points such as eyebrows, eyes, nose, mouth in the input driver's face image. Means, binarizing means for binarizing the image in which the feature candidate area is extracted, face area centroid calculating means for calculating the centroid (x, y) for the area including the face in the binary image, face area centroid calculation Based on the center of gravity x of the face area center of gravity (x, y) calculated by the means, the face part area setting means for setting the upper area of the face in the vertical direction, and the center of gravity of the set upper area of the face in the vertical direction are calculated. , A partial area representative point calculating means using that as a representative point, and an angle formed by a straight line passing through the two points of the face area center of gravity (x, y) and the partial area representative point and one image axis is detected as a face inclination. A face image processing apparatus comprising inclination detecting means. 3. Face image input means for inputting a driver's face image, and feature candidate region extraction for extracting candidate regions of facial feature points such as eyebrows, eyes, nose, mouth in the input driver's face image. Means, binarizing means for binarizing the image in which the feature candidate area is extracted, face area centroid calculating means for calculating the centroid (x, y) for the area including the face in the binary image, face area centroid calculation Based on the horizontal center of gravity x of the face area center of gravity (x, y) calculated by the means, an upper face vertical direction area is set, and the upper face vertical direction area is defined as the face area center of gravity (x, y).
y) The face part area setting means for setting the upper left and right areas of the face by dividing based on the vertical center of gravity y, the center of gravity of each of the set upper left area of the face and the upper right area of the face, and calculating the midpoint Partial area representative point calculating means as a representative point, inclination detecting means for detecting an angle formed by a straight line passing through two points of the face area center of gravity (x, y) and the partial area representative point and one image axis as the inclination of the face A face image processing apparatus comprising: 4. A face image input means for inputting a face image of a driver, and feature candidate region extraction for extracting candidate regions of face feature points such as eyebrows, eyes, nose and mouth in the input face image of the driver. Means, binarizing means for binarizing the image from which the feature candidate areas are extracted, feature candidate area grouping means for grouping the feature candidate areas in the binary image, extracting nostril areas within the feature candidate area group Nostril region extracting means, nostril region centroid calculating means for calculating the center of gravity for each nostril in the nostril region, and detecting the angle between the straight line passing through the center of gravity for each nostril in the calculated nostril region and one image axis as the face inclination A face image processing apparatus comprising inclination detecting means. 5. A face image input means for inputting a face image of a driver, and feature candidate region extraction for extracting candidate regions of face feature points such as eyebrows, eyes, nose and mouth in the input face image of the driver. Means, binarizing means for binarizing the image in which the feature candidate areas are extracted, feature candidate area grouping means for grouping the feature candidate areas in the binary image, and nostril areas in the feature candidate area group Nostril region extracting means, characteristic region center of gravity calculating means for calculating the center of gravity for each characteristic candidate region group, of the center of gravity calculated by the characteristic region center of gravity calculating means,
An inter-nostril center setting unit having a center of gravity for the region set by the nostril region extraction unit as a center between the nostrils, a straight line passing through two points of the center of gravity and the center of the nostrils of each feature candidate region excluding the nostril region, and one image A face image processing apparatus comprising a tilt detecting means for detecting an average of angles formed by axes as a tilt of a face. 6. A face image input means for inputting a face image of a driver, and a feature candidate region extraction for extracting candidate regions of facial feature points such as eyebrows, eyes, nose and mouth in the input face image of the driver. Means, binarizing means for binarizing the image in which the feature candidate area is extracted, face area centroid calculating means for computing the centroid (x, y) for the area including the face in the binary image, in the binary image Feature candidate region grouping means for grouping the feature candidate regions, Nostril region extracting means for extracting nostril regions in the feature candidate region group, Inter-nostril center calculating means for calculating the center between the nostrils of the extracted nostril region, A face image processing apparatus, comprising inclination detecting means for detecting an angle formed by a straight line passing through two points of the center of the nostril and the face area center of gravity (x, y) and one image axis as the inclination of the face. 7. A rotation center setting means for setting a rotation center of a face image for correcting the inclination of the face detected by the inclination detection means, and a face image by rotation based on the rotation center set by the rotation center setting means. 7. The face image processing apparatus according to claim 1, further comprising a rotating unit that corrects a tilt of the face image. 8. The face image processing apparatus according to claim 7, wherein the rotation center setting means sets the center of gravity (x, y) of the face area in the binary image as the rotation center. 9. The face image processing apparatus according to claim 7, wherein the rotation center setting means sets a center between the nostrils as a rotation center. 10. A tilt determining means for determining whether or not the face image is tilted, and the image rotating processing by the rotating means is continued until it is judged that the face image is not tilted. The face image processing device according to. 11. The tilt determining means uses the image horizontal (y) axis as the face horizontal direction and the image vertical (x) axis as the face vertical direction in the binary image, and the image vertical axis at each coordinate in the image horizontal axis direction. 11. The image vertical axis direction histogram showing the number of black pixels existing in the direction is created, and it is determined whether or not the face image is tilted from the shape and peak number of the image vertical axis direction histogram. The described face image processing apparatus. 12. The inclination determining means sets the horizontal face direction in the binary image as the image vertical (y) axis and the vertical face direction as the image horizontal (x) axis, and the image horizontal axis at each coordinate on the vertical axis of the image. 11. The image horizontal axis direction histogram showing the number of black pixels existing in the direction is created, and whether or not the face image is tilted is determined from the symmetry of the image horizontal axis direction histogram. Face image processing device. 13. The inclination determining means uses the center of gravity (xi, yi) of the center of gravity (xi, yi) of each feature candidate area group obtained by the feature center of gravity calculating means in the binary image in the horizontal axis direction xi.
The face image processing apparatus according to claim 10, wherein it is determined whether or not the face image is tilted based on the arrangement. 14. A rotation angle setting means for setting an angle for rotating the face image according to the angle θ detected by the inclination detection means, and a rotation limit angle setting means for setting a limit angle of the rotation angle, The image rotation processing is continued by the rotating means when the face image is rotated by the rotating means and when the inclination determining means determines that the face image is not inclined or until the rotation limit angle is reached. Item 10. The face image processing device according to item 10.