CN111291590B - Driver fatigue detection method, driver fatigue detection device, computer equipment and storage medium - Google Patents

Abstract

The invention relates to a driver fatigue detection method, device, computer equipment and storage medium. The method includes: acquiring a face video of a target driver, and performing human eye opening on each frame of face images in the face video respectively. degree detection to obtain the human eye opening value in each frame of the face image; determine a first opening degree threshold and a second opening degree threshold according to each of the human eye opening degree values, and the first opening degree threshold is greater than all the opening degree thresholds. the second opening degree threshold value; according to each of the human eye opening degree value, the first opening degree threshold value and the second opening degree threshold value, statistics that the human eye opening degree value is less than or equal to the first opening degree The first image frame value of the threshold, and the second image frame value of which the eye opening value is less than or equal to the second opening threshold; if the first image frame value and the second opening threshold are If the ratio is greater than the preset fatigue determination threshold, it is determined that the target driver is in a fatigue state. By adopting the scheme of the present invention, the accuracy of the fatigue detection result can be improved.

Description

Driver fatigue detection method, device, computer equipment and storage medium

technical field

The present invention relates to the technical field of image processing, and in particular, to a driver fatigue detection method, device, computer equipment and storage medium.

Background technique

Traffic accidents have always been one of the most serious threats to life and property safety faced by human beings, and most of the traffic accidents are caused by human factors of drivers. In the process of vehicle driving, driver fatigue is one of the important causes of serious traffic accidents, which seriously endangers traffic safety.

With the development of image recognition processing technology, it provides a new solution for preventing traffic accidents by identifying and processing the driver's facial image information during driving, judging the driver's fatigue state and giving an alarm.

The traditional driver fatigue detection method based on facial image information is to identify the open and closed state of human eyes, and finally perform fatigue detection through the detection state of continuous frames. In this way, the accuracy of the detection results is low.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a driver fatigue detection method, device, computer equipment and storage medium that can improve the detection accuracy in response to the above technical problems.

A driver fatigue detection method, the method comprises:

Obtaining the facial video of the target driver, performing eye opening detection on each frame of the facial image in the facial video, respectively, to obtain the eye opening value in the facial image of each frame;

Determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

According to each of the human eye opening degree value, the first opening degree threshold value and the second opening degree threshold value, count the first image frame values of which the human eye opening degree value is less than or equal to the first opening degree threshold value , and the second image frame value of which the eye opening value is less than or equal to the second opening threshold value;

If the ratio of the first image frame value to the second image frame value is greater than a preset fatigue determination threshold, it is determined that the target driver is in a fatigued state

A driver fatigue method device, the device comprises:

The detection module is used to obtain the face video of the target driver, and perform eye opening detection on each frame of the face image in the face video, and obtain the eye opening value in each frame of the face image. ;

a processing module, configured to determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

A statistics module, configured to count the number of eye openings less than or equal to the first opening threshold according to each of the human eye opening values, the first opening threshold and the second opening threshold. a first image frame value, and a second image frame value whose eye opening value is less than or equal to the second opening threshold;

A determination module, configured to determine that the target driver is in a fatigued state if the ratio of the first image frame value to the second image frame value is greater than a preset fatigue determination threshold.

A computer device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, the processor implements the following steps when executing the computer program:

Obtaining the facial video of the target driver, performing eye opening detection on each frame of the facial image in the facial video, respectively, to obtain the eye opening value in the facial image of each frame;

Determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

According to each of the human eye opening degree value, the first opening degree threshold value and the second opening degree threshold value, count the first image frame values of which the human eye opening degree value is less than or equal to the first opening degree threshold value , and the second image frame value of which the eye opening value is less than or equal to the second opening threshold value;

If the ratio of the first image frame value to the second image frame value is greater than a preset fatigue determination threshold, it is determined that the target driver is in a fatigued state.

A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtaining the facial video of the target driver, performing eye opening detection on each frame of the facial image in the facial video, respectively, to obtain the eye opening value in the facial image of each frame;

Determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

According to each of the human eye opening degree value, the first opening degree threshold value and the second opening degree threshold value, count the first image frame values of which the human eye opening degree value is less than or equal to the first opening degree threshold value , and the second image frame value of which the eye opening value is less than or equal to the second opening threshold value;

If the ratio of the first image frame value to the second image frame value is greater than a preset fatigue determination threshold, it is determined that the target driver is in a fatigued state.

The above-mentioned driver fatigue detection method, device, computer equipment and storage medium are to determine the first opening threshold and the second opening threshold according to the eye opening values in each frame of face image of the face video, and based on the first opening threshold and the second opening threshold. The first opening threshold and the second opening threshold are used to distinguish the state of the eyes, based on the first image frame value whose eye opening value is greater than the first opening threshold and the second image whose eye opening value is less than the second opening threshold. The ratio of the image frame values and the preset fatigue determination threshold determine that the target driver is in a fatigued state, which can improve the accuracy of the detection result.

Description of drawings

1 is an application environment diagram of a driver fatigue detection method in one embodiment;

2 is a schematic flowchart of a driver fatigue detection method in one embodiment;

3 is a schematic flowchart of an acquisition process of a human eye opening value in one embodiment;

4 is a schematic flowchart of an acquisition process of a first opening degree threshold and a second opening degree threshold in one embodiment;

FIG. 5 is a schematic diagram of the acquisition process of the facial feature image in one embodiment;

6 is a schematic diagram of a training process of an eye feature point positioning model in one embodiment;

7 is a schematic diagram of DPM feature extraction in one embodiment;

8 is a schematic diagram of a root filter (left) component filter (middle) Gaussian filtered 2x space model (right) in one embodiment;

9 is a schematic diagram of the effect comparison (a) and the formula comparison (b) of the DPM+Latent-SVM used in the traditional Hog+SVM and an embodiment;

FIG. 10 is a schematic diagram of cascade iteration effect in one embodiment;

11 is a schematic diagram of a hybrid tree model encoding topology changes due to viewpoints in one embodiment;

FIG. 12 is a schematic diagram of the positioning result of human eye feature points in one embodiment;

Fig. 13 is a structural block diagram of a driver fatigue method device in another embodiment;

FIG. 14 is an internal structure diagram of a computer apparatus in another embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not limit the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the description, claims and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence relationship. It is to be understood that the data so used may be interchanged under appropriate circumstances so that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein.

The driver fatigue detection method provided in this application can be applied to the application environment shown in FIG. 1 . Among them, the infrared camera collects the video information of the driver, and the video information collected by the infrared camera can be input into the terminal to carry out the driver fatigue method. Among them, the best installation position of the infrared camera is on the steering column below the steering wheel of the car. The infrared camera can communicate with the terminal in a wired or wireless manner. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, vehicle-mounted terminals and portable wearable devices.

In one embodiment, as shown in FIG. 2 , a method for detecting driver fatigue is provided, which is illustrated by taking the method applied to a terminal as an example, including the following steps:

Step S201: obtaining the face video of the target driver, and performing eye opening detection on each frame of the face image in the face video, to obtain the eye opening value in each frame of the face image;

Here, the face video is obtained by photographing the face of the target driver.

Specifically, the facial video of the target driver in one detection period can be obtained, and eye opening detection is performed on each frame of the facial image in the facial video to obtain the human eye in each frame of the facial image. opening value. The size of the detection period can be set according to actual needs.

Step S202: Determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

Generally, the first opening threshold is smaller than the maximum opening degree of the target driver's eyes, the second opening threshold is greater than 0, the first opening threshold is a threshold for determining whether the eyes are in a fully opened state, and the second opening threshold is The opening threshold is a threshold for determining whether the eyes are in a closed state.

Step S203: According to each of the human eye opening value, the first opening degree threshold and the second opening degree threshold, count the first opening degree value of the human eye opening degree value less than or equal to the first opening degree threshold value. an image frame value, and a second image frame value whose eye opening value is less than or equal to the second opening threshold;

Wherein, the human eye opening value is greater than the first opening degree threshold, indicating that the eyes are in a fully open state, and the human eye opening degree value is smaller than the second opening degree threshold, indicating that the eyes are in a closed state. The first image frame value is the number of image frames in which the human eye opening value in the face video is less than or equal to the first opening threshold value, and the second image frame value is the human eye opening degree in the face video. The number of image frames whose value is less than or equal to the second opening threshold.

Specifically, the first image frame value and the second image frame value in a detection period may be counted according to each of the human eye opening value, the first opening degree threshold and the second opening threshold.

Step S204: if the ratio of the first image frame value to the second opening threshold is greater than a preset fatigue determination threshold, determine that the target driver is in a fatigued state;

Among them, the size of the fatigue determination threshold can be set according to the actual situation.

In the above-mentioned driver fatigue detection method, the facial video of the target driver is obtained, and eye opening detection is performed on each frame of the facial image in the facial video, and the person in each frame of the facial image is obtained. Eye opening degree value, a first opening degree threshold and a second opening degree threshold are determined according to each of the human eye opening degree values, the first opening degree threshold value is greater than the second opening degree threshold value, and a first opening degree threshold value is determined according to each of the human eye opening degree thresholds. degree value, the first opening degree threshold value and the second opening degree threshold value, count the first image frame value whose eye opening degree value is less than or equal to the first opening degree threshold value, and the human eye opening degree value If the ratio of the first image frame value to the second image frame value is greater than the preset fatigue determination threshold, then determine the target The driver is fatigued. In the solution of this embodiment, the eye state is distinguished based on the first opening degree threshold and the second opening degree threshold, and the first image frame value based on the human eye opening degree value greater than the first opening degree threshold value and the human eye opening degree value less than The ratio of the second image frame value to the second opening threshold and the preset fatigue determination threshold to determine that the target driver is in a fatigued state can improve the accuracy of the detection result.

In one of the embodiments, in order to exclude the influence of the change of the distance between the human eye and the camera on the calculation of the human eye opening, a processing method for normalizing the human eye opening value is provided. As shown in FIG. 3 , specifically, the above-mentioned eye opening detection is performed on each frame of the face image in the face video, and the eye opening value in each frame of the face image is obtained, which may include: :

Step S301: Perform eye feature point positioning on each frame of the face image to obtain the eye feature point in each frame of the face image;

Step S302: According to the eye feature points in the facial images of each frame, respectively determine the interpupillary distance value of the human eye and the original value of the human eye opening in the facial image of each frame;

Specifically, the calibration point of the upper eyelid and the calibration point of the lower eyelid at the position facing the pupil in each frame of the facial image may be determined respectively according to the eye feature points in the facial image of each frame, The calibration point and the calibration point of the lower eyelid are used to calculate the original value of the human eye opening, wherein the original value of the human eye opening may be equal to the distance between the calibration point of the upper eyelid and the calibration point of the lower eyelid.

Step S303 : According to the interpupillary distance value of the human eye and the original value of the human eye opening in the facial image of each frame, determine the human eye opening value in each frame of the facial image respectively.

确定各帧所述脸部图像中的人眼开度值，其中，H_i表示第i帧所述脸部图像中的人眼开度值，h_i表示第i帧所述脸部图像中的人眼开度原始值，l_i表示第i帧所述脸部图像中的人眼瞳距值，C表示修正参数，该修正参数的大小可以根据情况设定或者调整。Specifically, according to

Determine the eye opening value in each frame of the face image, where H _i represents the human eye opening value in the ith frame of the face image, and hi represents the _ith frame of the face image. The original value of the human eye opening, li represents the interpupillary distance value of the human eye in the face image in the _ith frame, and C represents a correction parameter, and the size of the correction parameter can be set or adjusted according to the situation.

By adopting the solution in this embodiment, the influence of the change of the distance between the human eye and the camera on the calculation of the opening degree of the human eye can be effectively excluded.

In one embodiment, as shown in FIG. 4 , the above-mentioned determination of the first opening threshold and the second opening threshold according to each of the human eye opening values includes:

Step S401: Determine the maximum opening degree value of the human eye according to each of the human eye opening degree values;

Specifically, the magnitude of each of the human eye opening degree values may be compared to obtain the maximum value among the various human eye opening degree values, and the maximum value may be used as the maximum human eye opening degree value. It is also possible to sort each of the human eye opening values in descending order, obtain the top N eye opening values in the sorting, and take the average of these N human eye opening values, and the average value is used as the human eye. Maximum opening value.

Step S402: Multiply the maximum opening degree value by a preset first proportional coefficient and a second proportional coefficient, respectively, to obtain the first opening degree threshold and the second opening degree threshold;

The first proportional coefficient is greater than the second proportional coefficient, and generally, both the first proportional coefficient and the second proportional coefficient are values greater than 0 and less than 1, and the sizes of the first proportional coefficient and the second proportional coefficient can be based on actual needs Select, preferably, the first proportional coefficient is 0.8, and the second proportional coefficient is 0.2.

In this embodiment, the first opening threshold and the second opening threshold are obtained by multiplying the maximum opening degree value by the preset first proportional coefficient and the second proportional coefficient respectively, and the algorithm is easy to implement, and The first opening degree threshold and the second opening degree threshold are both determined based on the maximum opening degree value of the human eye, and the maximum opening degree value of the human eye is determined by the opening degree value of each human eye, which can further improve the detection accuracy.

In one embodiment, as shown in FIG. 5 , the above-mentioned positioning of the facial feature points of each frame of the facial image to obtain each facial feature image may include:

Step S501: extracting a first DPM feature map, where the first DPM feature map is a DPM feature map of a current facial image, and the current facial image is any frame of facial images;

Step S502: performing sampling processing on the first DPM feature map, and extracting a second DPM feature map, where the second DPM feature map is a DPM feature map of an image obtained by sampling the first DPM feature map;

Step S503: performing a convolution operation on the first DPM feature map with a pre-trained root filter to obtain a response map of the root filter;

Step S504: Perform a convolution operation on N times the second DPM feature map with a pre-trained component filter to obtain a response map of the component filter, and the resolution of the component filter is the root filter. N times the resolution of the device, N is a positive integer;

Step S505: obtain a target response diagram according to the response diagram of the root filter and the response diagram of the component filter;

Step S506: Acquire the current face feature image according to the target response map.

The facial image of each frame can be regarded as the current facial image in this embodiment, and the above steps S501 to S506 can be used to locate the facial feature points, and the facial feature image corresponding to each frame of the facial image can be obtained. .

In this embodiment, by adopting the DPM target detection algorithm for face detection, the detection accuracy of the algorithm is improved, and the false detection rate and the missed detection rate can be reduced at the same time.

In one embodiment, performing eye feature point positioning on each frame of the face image to obtain the eye feature point in each frame of the face image may include: Perform facial feature point location to obtain each facial feature image; input each of the facial feature images into a preset eye feature point location model respectively to obtain the eye feature points in each frame of the facial image.

In one embodiment, as shown in FIG. 6 , the training process of the above-mentioned eye feature point positioning model may include:

Step S601: Obtain the pixel value of each pixel point of the target image and the feature vector obtained by each of the pixel points;

In this embodiment, the model used may be based on a hybrid tree and a shared component V pool. Model each facial landmark as a part and use global blending to capture topological changes due to viewpoints.

Step S602: according to the pixel value and the feature vector, configure the local model of the tree structure, and determine the score function in the L part;

I表示目标图像，l_i＝(x_i,y_i)表示所述目标图像的第i个像素点的像素值，w表示部分模型，m表示树结构是混合型，部分模型是指将所述目标图像中的每个面部特征点分别作为一个部分进行建模得到，a、b、c和d表示弹性参数，α表示混合偏置标量；Among them, the score function is S(I,L,m)=App _m (I,L)+Shape _m (L)+α ^m ,

I represents the target image, li =(x _i , y _i ) represents the pixel value of the _ith pixel of the target image, w represents a partial model, m represents a hybrid tree structure, and a partial model refers to the Each facial feature point in the target image is modeled as a part, a, b, c, and d represent elastic parameters, and α represents a hybrid bias scalar;

求出了在l_i位置上i处(第i个像素点处)的模板

之和，φ(I,l_i)表示了目标图像l_i像素处的特征向量。In this embodiment,

Find the template at position _i (at the i-th pixel) at position li

The sum, _φ (I, _li ) represents the feature vector at the pixel of the target image li.

表示的是混合类型特定空间L排列的排列得分，其中dx＝x_i-x_j和dy＝y_i-y_j表示第i部分到第j部分的位移。公式中的每个参数(指a,b,c,d)可以被解释为不同部分之间的空间约束。In this embodiment,

is the permutation score of the hybrid type specific spatial L permutation, where dx = x _i -x _j and dy = y _i -y _j represent the displacement of the i-th part to the j-th part. Each parameter (referring to a, b, c, d) in the formula can be interpreted as a spatial constraint between different parts.

Step S603: Obtain the optimal configuration parameters of each part of each hybrid type by obtaining the values of L and m that enable the score function to obtain the maximum value;

Specifically, all hybrid types can be enumerated, and for each hybrid type, the best configuration parameters of each part can be found.

Step S604: establishing a training sample set, the training sample set includes positive samples and negative samples set with labels, the positive samples are images containing human faces, and the negative samples are images that do not contain human faces;

Specifically, a fully supervised scene is assumed, in which there are positive samples with faces and mixed labels and negative samples without faces. Shape parameters and appearance parameters can be learned differentially with a structure prediction framework.

Step S605: Construct a target vector according to the partial model, the elastic parameter and the mixed bias scalar, and modify the score function according to the target vector;

Specifically, part of the model w, elastic parameters (a, b, c, d) and mixed bias scalar α are all put into a vector β, and the above score function is modified into the following form: S(I,z)= β·Φ(I,z). The vector Φ(I,z) is sparse, with non-zero entries in a single interval corresponding to the mixture m.

Step S606: Learning to obtain the eye feature point positioning model according to the training sample set, the optimal configuration parameter, the modified score function and the predefined target prediction function;

Among them, the learned eye feature point localization model is:

Among them, β represents the target vector, z _n ={L _n ,m _n }, C represents the penalty term coefficient of the objective function, ξ _n represents the penalty term of the nth sample, pos and neg represent positive samples and negative samples, respectively , K represents the number of the target vector, and k represents the number of the corresponding target vector.

In this embodiment, the machine learning algorithm is used to locate the face feature points and the eye feature points respectively, which not only has a very high positioning accuracy, but also has a strong generalization ability for illumination and posture, which can improve the calculation of the degree of eye opening and closing. accuracy.

In order to facilitate the understanding of the solution of the present invention, the solution of the present invention is described in detail below with a preferred embodiment.

The driver fatigue detection method in this embodiment includes the following steps: the first step: video information input; the second step: face detection; the third step: facial feature point location; the fourth step: human eye feature point location ; Step 5: Blink detection - calculation of eye opening, fatigue detection and analysis.

Step 1: Collect video information. The monocular infrared camera for video information input (installed on the steering column under the steering wheel) inputs the driver's face status information (image) in real time while driving. The frequency of video input is 30Hz, and the size of each frame is 1280×1080 pixels. Among them, the infrared camera can adapt to different light conditions in the car, and accurately capture the driver's head posture information and facial information.

The second step: face detection. For each frame image of the input video, this embodiment uses a DPM (Deformable Part Model) target detection algorithm to perform face detection. Part of the principle of the HOG algorithm is applied to the DPM algorithm: first, the image is grayed; then, as in formula (1), the color space of the input image is standardized (normalized) by the Gamma correction method:

I(x,y)=I(x,y) ^gamma (1)

Among them, the value of gamma depends on the specific situation (for example, it can be taken as 1/2), which can effectively reduce the local shadow and illumination changes of the image; next, the gradient calculation is performed, and the gradient reflects the change between adjacent pixels. The change between adjacent pixels is relatively flat, the gradient is small, otherwise the gradient is large, and the gradient of any pixel (x, y) of the simulated image f(x, y) is a vector:

Among them, G _x is the gradient along the x direction, G _y is the gradient along the y direction, and the magnitude and direction angle of the gradient can be expressed by the following formula:

Pixels in a digital image are computed using difference:

Because the gradient operation using a simple one-dimensional discrete differential template [-1,0,1] has the best detection effect, the calculation formula used is as follows:

值，其梯度的幅值及方向计算公式如下：

The formula for calculating the magnitude and direction of the gradient is as follows:

Then, for the entire target image, it is divided into non-overlapping cells of the same size, and the gradient size and direction of each cell are calculated. DPM preserves the cell units of the HOG map, and then normalizes a cell unit on the image (the 8x8 cell unit in Figure 7) to its four cells in its diagonal neighborhood. Extracting signed HOG gradients, 0-360 degrees will generate 18 gradient vectors, extracting unsigned HOG gradients, 0-180 degrees will generate 9 gradient vectors. DPM only extracts unsigned features, which will generate 4*9=36-dimensional features, and add rows and columns to form 13 feature vectors (as shown in Figure 7, 9 columns are added, and 4 rows are added). In order to further improve the accuracy , the extracted 18-dimensional signed gradient features are also added (18 columns are added as shown in the figure), and finally 13+18=31-dimensional gradient features are obtained.

As shown in Figure 8, the DPM model uses an 8*8 resolution Root filter (left) and a 4*4 resolution Part filter (middle). Among them, the resolution of the middle image is 2 times that of the left image, and the size of the component filter is 2 times that of the root filter, so the gradient will be more refined. The picture on the right is the 2x space model after Gaussian filtering.

First, extract the DPM feature map (DPM feature map of the original image) from the input image and perform Gaussian pyramid upsampling (zooming the image), and then extract the DPM feature map of the Gaussian pyramid upsampled image. Convolve the DPM feature map of the original image and the trained root filter to obtain the response map of the root filter. At the same time, for the extracted DPM feature map (Gaussian pyramid upsampling) of the 2x image, the trained component filter is used for convolution operation to obtain the response map of the component filter. Fine Gausta downsampling is performed on the resulting component filter response map, so that the root filter response map and the component filter response map have the same resolution. Finally, the weighted average of the two is carried out to obtain the final response map. The greater the brightness, the better the response effect, and thus the face is detected. The response value score formula is:

为根模型的响应分数；

为部件模型的响应分；2(x₀,y₀)表示组件模型的像素为原始的2倍；b为用于与跟模型进行对齐的不同模型组件之间的偏移系数；2(x₀,y₀)表示组件模型的像素为原始的2倍，所以，像素点*2；v_i为像素点和理想检测点之间的偏移系数；其中其部件模型的详细响应得分公式如下：Among them, x ₀ , y ₀ , and l ₀ are the abscissa, ordinate, and scale of the feature point, respectively;

is the response score of the root model;

is the response score of the component model; 2(x ₀ , y ₀ ) means that the pixels of the component model are 2 times the original; b is the offset coefficient between different model components used to align with the model; 2(x ₀ ,y ₀ ) means that the pixel of the component model is twice the original, so, pixel*2; v _i is the offset coefficient between the pixel and the ideal detection point; the detailed response score formula of the component model is as follows:

Similar to formula (8), we hope that the larger the value of the objective function (D _i,l (x,y)), the better, the variables are dx, dy. In addition, in the above formula, (x,y) is the position of the ideal model for training; dx, dy is the offset of the ideal model position, and the range is the position from the ideal position to the edge of the picture; R _i,l (x+dx,y +dy) is the matching score of the component model; d _i *Φ _d (dx, dy) is the offset loss score of the component; d _i is the offset loss coefficient; Φ _d (dx, dy) is the pixel point sum of the component model The distance between the detection points of the component model. This formula shows that the higher the response of the component model, the smaller the distance between each component and its corresponding pixel point, the higher the response score, and the more likely it is the object to be detected.

When training the model, the DPM features obtained above should be trained. DPM uses the Latent-SVM classification method here. Compared with the Linear-SVM classification method, the Latent variable (latent variable) is added here. The Latent variable can be used to determine which sample in the positive sample is used as a positive sample. There are many Latent variables in LSVM, because given a picture of a positive sample, after marking the bounding box, it is necessary to propose a maximum sample at a certain position and a certain scale as the maximum sample of a certain part. Figure 9(a) is a comparison diagram of the effect of the general Hog+SVM and the applied DPM+Latent-SVM. The general Hog+SVM and the applied DPM+Latent-SVM formula are shown in Figure 9(b).

来预测更新形状向量，具体公式如下：The third step: facial feature point location. In this embodiment, the LBF algorithm is used, and a cascaded regressor is used to locate the facial feature points and the eye feature points in milliseconds. Each regression r _t (,) uses the current image I and shape vector

To predict the update shape vector, the specific formula is as follows:

表明当前的估计向量S，X_i表示图像I中的脸部特征点的(x,y)坐标。联级中最重要的步骤是回归器r_t(,)基于诸如像素灰度特征的预测，这些特征是基于图像I计算和当前形状向量

索引出来的。在这个过程中引入了几何形式的不变性，并且随着级联的进行可以人们可以更加确定脸部的精确语义位置正在被索引。in

Indicates the current estimated vector S, X _i represents the (x, y) coordinates of the facial feature points in the image I. The most important step in the cascade is the regressor r _t (,) based on predictions such as pixel grayscale features, which are computed based on the image I and the current shape vector

indexed. Invariance of geometric form is introduced in this process, and as the cascade progresses one can be more certain that the precise semantic position of the face is being indexed.

属于此空间，则确保由集合扩展的输出范围位于训练数据的线性子空间中。这样做不需要对预测实施额外的限制，这极大地简化了本方法。此外，简单地选择初始形状作为根据通用面部检测器的边界框输出居中和缩放的训练数据的平均形状。If the initial estimate

belongs to this space, it ensures that the output range extended by the set lies in the linear subspace of the training data. Doing so without imposing additional constraints on the prediction greatly simplifies the method. Furthermore, the initial shape is simply chosen as the average shape of the training data centered and scaled according to the bounding box output of the generic face detector.

如下：The next step is to learn each regressor in the cascade, using the training data set ((I ₁ ,S ₁ ),...,(I _n ,S _n )) to learn the regression functions r ₀ ,I _i represents a face image, and S _i represents a shape vector. Initialized shape estimation and target update)

as follows:

π _i ∈{1,...n} (13)

where i=(1,...,N). The total number of these triples is set here as N=nR, where R is the number of initializations each image I uses. Each initialized shape estimate for an image is uniformly sampled from (S ₁ ,...,S _n ) without replacement.

From this data, the regression function r ₀ can be learned by using the gradient tree boosting with the sum of the squared error loss. The specific algorithm is as follows:

学习率0<v<1，具体过程是：training data

The learning rate is 0<v<1, and the specific process is:

a.Initialization:

b. From 1 to K for k:

①i=1,…,N:

② Fit a regression tree to the regression function _rik and give a weak regression function

③Update:

c. Output:

联级中的下一个回归器r₁，被设置如下(t＝0)：The triplet training data will then update the training data as:

The next regressor in the cascade, r ₁ , is set as follows (t=0):

This process is iterated until the cascade r ₀ ,r ₁ ,...,r _T-1 combination of T regressions gives a sufficient level of accuracy.

At the heart of each regression function _rt is a tree-based regression function that fits the residual objective in the gradient boosting algorithm. At each separation node of the regression tree, we make a decision based on a threshold for the difference in intensity between two pixels. In the coordinate system defined based on the average shape, the pixel coordinates used in the test are (u, v). For an arbitrarily shaped face image, we want to index points that have the same position as their shape as u and v are for points of average shape. To achieve this, the image can be warped to an average shape based on the current shape estimate before extracting features. Since we are using a very sparse pixel representation of this image, it is more efficient to warp the positions of these points rather than warping the entire image.

Suppose k _u is the index of the closest facial landmark to u in the mean shape, and define its offset from u as:

Then, for a shape S _i defined in image I _i , position in I _i This is qualitatively similar to defining u in a shape image:

where s _i , R _i are the scale and rotation matrices, both of which are used to minimize the sum of the squared differences between the mean shape facial landmark points and twist points:

v' is similarly defined. Formally each split is a decision involving 3, the parameter θ = (τ, u, v) and is applied to each training and test sample.

Here u' and v' are defined by scale and rotation matrices. Computation of similarity transformations, the most computationally intensive part of the process at test time, is done only once at each level of the cascade.

For each regression tree, we approximate the underlying function using a piecewise constant function, where a constant vector is fitted to each leaf node. To train a regression tree, we randomly generate a set of candidate splits, θ, at each tree node. We then arbitrarily choose θ from these candidates, which minimizes the sum of squared errors. If Q is the indexed set of training examples on the node, this corresponds to minimizing:

where Q _θ,S is the index of the sample, ri is the vector of all residuals calculated for image _i in the gradient boosting algorithm, and μ _θ,s is defined by the following formula:

The optimal optimization point can be easily found because if we rearrange the formula or ignore the factors that depend on θ, we can see the following formula relationship:

When evaluating different θ, we only need to calculate μ _θ,l just as μ _θ,r can be calculated by μ and μ _θ,l , the process is as follows:

The decision at each node is based on thresholding the difference in intensity values at a pair of pixels. This is a fairly simple test, but it is more powerful than single threshold due to its relative insensitivity to changes in global illumination. Unfortunately, the disadvantage of using pixel disparity is that the number of possible segmentation (feature) candidates is quadratic in the number of pixels in the average image. This makes it hard to find a good θ without searching a lot of θ. However, this limitation can be alleviated to some extent by considering the structure of the image data. Let's first introduce an index

p(u,v)αe ^-λ||uv|| (29)

Pixel segmentation points within this distance range are easily selected, which can effectively reduce the number of data set prediction errors.

To deal with missing labels, we introduce a variable w _i,j ranging from 0 to 1 (representing the jth marker point of the ith image), then a new sum of squared difference formulas are derived:

where Wi is a vector (wi _{,i ,} ...wi _,p ) ^T deformed diagonal matrix. In addition, the formula for μ _θ,s is as follows:

The gradient boosting algorithm must also be modified to account for these weighting factors. This can be done simply by initializing the overall model with the weighted mean of the targets and fitting a regression tree to the weights. In addition, the weight value residual algorithm for fitting the regression tree is as follows:

Among them, the cascade iteration effect is shown in Figure 10.

The fourth step: human eye feature point positioning. In this example, the model used is based on a hybrid tree and a shared V pool of components. In this approach we model each facial landmark as a part and use global blending to capture topological changes due to viewpoints. As shown in FIG. 11, the hybrid tree model adopted in this embodiment encodes the topology changes due to viewpoints.

我们把一张图片标记为I，并且l_i＝(x_i,y_i)用来表示位置i处的像素。我们在L部分的得分配置为：Tree-structured local model: We write a linear tree-like structure for each parameter T _m = (V _m , E _m ), where m indicates that the structure is a hybrid, and

We label a picture as I, and li = (x _i , y _i ) is used to denote the pixel at position _i . Our scoring configuration for part L is:

S(I,L, _m )=Appm(I,L)+ _Shapem (L)+ ^αm (33)

之和，其中m表示这里是混合型。φ(I,l_i)表示了在图片I上l_i像素处的特征向量。Equation (34) finds the template at _i at position li

The sum, where m indicates that here is a mixed type. φ(I, _li ) represents the feature vector at _li pixels on image I.

Equation (35) represents the permutation score of the hybrid type specific space L permutation, where dx=x _i -x _j and dy=y _i -y _j represent the displacement from the i-th part to the j-th part. Each parameter (referring to a, b, c, d) in the formula can be interpreted as a spatial constraint between different parts. α ^m denotes a scalar with a bias.

Since the method of locating eye feature points is mainly used in the solution of this embodiment, the whole and part sharing factors are not considered.

Find the values of the parameters L and m that maximize the formula S(I, L, m):

Simply enumerate all mixes and find the best configuration for each part for each mix.

Since each hybrid T _m = (V _m , _Em ) is a tree-like structure, internal maximization can be efficiently accomplished by dynamic programming. Due to the lack of space, it is possible to omit the message passing equation. The total number of different part templates in the vocabulary in this embodiment is M'|V|, assuming that the dimension of each part is D and there are N candidate positions. The total cost of evaluating all parts at all locations is:

After the distance conversion is performed, the information transfer cost is converted into: O(NM|V|). This makes the overall model of this embodiment scheme linear in the number of parts and the size of the image.

In order to train the human eye feature point localization model. The solution of this embodiment assumes a fully supervised scene, in which there are positive samples and mixed labels and negative samples without face images. The solution of this embodiment uses the structure prediction framework to learn shape parameters and appearance parameters differently. It is first necessary to estimate the edge structure _Em for each mixed type. Although it is a natural process to use a tree structure to derive a human body shape model, the tree structure of human eye features is still not very clear.

The embodiment solution uses the Chow-Liu algorithm to find the maximum similarity tree structure, which can best explain the feature point positions of the Gaussian distribution. Given positive samples {I _n , L _n , m _n } and negative samples {I _n } of labels, a structural target prediction function will be defined in the embodiment solution, assuming z _n ={L _n ,m _n }. The formula S(I,L,m) is about the partial model w, the elastic parameters (a,b,c,d) and the mixing bias α. Putting all these parameters into a vector β, we can write the score function as follows:

S(I,z)=β·Φ(I,z) (38)

where the vector Φ(I,z) is sparse with non-zero entries in a single interval corresponding to the mixture m.

Next, you can learn a model of the following form:

In formula (39), C represents the penalty item coefficient of the objective function (hyperparameter, it is necessary to adjust the parameters to find the most suitable value), ξ _n represents the penalty item corresponding to different samples (the penalty item of the nth sample), n Corresponding to different samples, pos and neg represent positive and negative samples respectively, K represents the number of target vectors β, and k represents the corresponding target vector β number.

As shown in Figure 12, it is a schematic diagram of the positioning result of human eye feature points.

Step 5: Blink Detection - Calculate the opening of the eyes and perform fatigue analysis. Longer eye closure time when blinking is one of the important signs of driver fatigue. On the basis of the previous steps, the human eye region has been located and the human eye feature points have been found. In order to calculate the opening of the human eye, in the solution of this embodiment, the change of the distance between the human eye and the camera should be excluded first to prevent it from affecting the calculation of the opening of the human eye. unify. On this basis, the degree of fatigue of the driver is determined by the proportion of eye closure time per unit time. details as follows.

First, according to the opening degree value of the human eye, the opening and closing state of the human eye is judged. In this embodiment, the feature points of the human eye located in the previous step are selected, and the calibration points of the upper eyelid and the lower eyelid facing the pupil are found to calculate the opening degree of the human eye. However, it has been found in a large number of experiments and experience that the farther the distance between the human eye and the camera, the smaller the measured human eye opening, and the closer the distance between the human eye and the camera, the greater the measured human eye opening. This is not conducive to the detection of driver fatigue in the later stage. Therefore, it is necessary to normalize the abnormal changes in the opening degree of the human eye caused by the relative position change of the human eye and the camera. In this embodiment, the eye feature point positioning in the previous step can be used to measure the human pupillary distance l, and there is a linear relationship between the pupillary distance and the change of the eye opening. It is assumed that the actually measured human eye opening (equivalent to the above The original value of eye opening) is h, the normalized human eye opening is H, and the following formula is used to correct the human eye opening value:

where C represents a selected correction parameter.

Next is the human eye state division. In the solution of this embodiment, the maximum opening degree of the human eye is obtained according to the value of the opening degree of the human eye obtained above, which is recorded as MaxW. Suppose the measured eye opening is W, where state I means W>80%*MaxW

W>80%MaxW, the eyes are fully open at this time; state II means 20%*MaxW≤W≤80%*MaxW, the eyes are half-open; state III means ≤W≤20%*MaxW, the eyes are closed state.

In this embodiment, the number of frames in which the human eye opening is less than or equal to 80% of the maximum human eye opening in the period (equivalent to the above-mentioned detection period) is counted, and denoted as n, and the human eye opening is less than or equal to the statistics in this period. The frame number of 20% of the maximum human eye opening is recorded as m, and a ratio f can be obtained. The calculation formula is as follows:

The closer f is to 1, the closer the driver is to a fatigued state. After a large number of experiments, an experimental threshold T (equivalent to the above-mentioned fatigue determination threshold) can be obtained. If f>T, it indicates that the driver is in a fatigue state, and an early warning is given in the form of sound to remind the driver to pay attention to fatigue driving.

In the solution of this embodiment, the DPM algorithm is used for face detection, the detection accuracy of the algorithm is greatly improved, and the false detection rate and missed detection rate are reduced at the same time, and the robustness of illumination and face posture is improved; machine learning algorithm is used to locate people The facial feature points and the eye feature points are located separately. The positioning accuracy is very high, and it has a strong generalization ability for illumination and posture. The final algorithm can accurately estimate the degree of eye opening and closing; for fatigue detection, not only the open and closed eyes state As the main criteria, the eye closing time, the number of blinks per unit time, and the degree of eye opening and closing are also used as fatigue criteria.

In one embodiment, as shown in FIG. 13, a method and apparatus for driver fatigue is provided, including: a detection module 1301, a processing module 1302, a statistics module 1303 and a discrimination module 1304, wherein:

The detection module 1301 is used to obtain the facial video of the target driver, and perform eye opening detection on each frame of the facial image in the facial video to obtain the eye opening in each frame of the facial image. value;

a processing module 1302, configured to determine a first opening threshold and a second opening threshold according to each of the human eye opening values, where the first opening threshold is greater than the second opening threshold;

Statistics module 1303, configured to count the human eye opening value less than or equal to the first opening degree threshold according to each of the human eye opening degree value, the first opening degree threshold value and the second opening degree threshold value The first image frame value of , and the second image frame value of which the eye opening value is less than or equal to the second opening threshold;

The determination module 1304 is configured to determine that the target driver is in a fatigue state if the ratio of the first image frame value to the second image frame value is greater than a preset fatigue determination threshold.

In one embodiment, the detection module 1301 may perform eye feature point positioning on each frame of the face image, to obtain the eye feature points in each frame of the face image, according to the face image of each frame The eye feature points in each frame of the face image respectively determine the value of the interpupillary distance of the human eye and the original value of the human eye opening, according to the value of the interpupillary distance and the human eye opening in the facial image of each frame. The original value of the degree of human eye opening in each frame of the face image is determined respectively.

In one embodiment, the processing module 1302 may determine the maximum opening degree value of the human eye according to each of the human eye opening degree values, and multiply the maximum opening degree value by the preset first proportional coefficient and the second proportional coefficient respectively to obtain the first opening threshold and the second opening threshold.

In one embodiment, the detection module 1301 may perform facial feature point positioning on each frame of the facial image to obtain each facial feature image; input each of the facial feature images into preset eye feature points respectively Positioning the model to obtain the eye feature points in the facial image of each frame;

Wherein, the training process of the eye feature point positioning model includes: acquiring the pixel value of each pixel point of the target image and the feature vector of each pixel point; configuring the local tree structure according to the pixel value and the feature vector model, and determine the score function in the L part, the score function is S(I, L, m)=App _m (I, L)+Shape _m (L)+α ^m ; The way to obtain the values of L and m of the maximum value is to obtain the optimal configuration parameters of each part of each hybrid type; a training sample set is established, and the training sample set includes positive samples and negative samples set with labels. The sample is an image containing a human face, and the negative sample is an image that does not contain a human face; a target vector is constructed according to the partial model, the elastic parameter and the mixed bias scalar, and the score is modified according to the target vector function; according to the training sample set, the best configuration parameter, the modified score function and the predefined target prediction function, learn to obtain the eye feature point positioning model;

I表示目标图像，l_i＝(x_i,y_i)表示所述目标图像的第i个像素点的像素值，w表示部分模型，m表示树结构是混合型，部分模型是指将所述目标图像中的每个面部特征分别作为一个部分进行建模得到，a、b、c和d表示弹性参数，α表示混合偏置标量。in,

I represents the target image, li =(x _i , y _i ) represents the pixel value of the _ith pixel of the target image, w represents a partial model, m represents a hybrid tree structure, and a partial model refers to the Each facial feature in the target image is modeled separately as a part, where a, b, c, and d represent the elasticity parameters, and α represents the hybrid bias scalar.

In one embodiment, the detection module 1301 may extract a first DPM feature map, where the first DPM feature map is a DPM feature map of a current face image, and the current face image is any frame of face images, The first DPM feature map is sampled to extract a second DPM feature map, where the second DPM feature map is a DPM feature map of an image obtained by sampling the first DPM feature map, and the first DPM feature map is sampled. DPM feature map, perform convolution operation with the pre-trained root filter, obtain the response map of the root filter, and perform convolution operation on N times the second DPM feature map with the pre-trained component filter, Obtain the response diagram of the component filter, the resolution of the component filter is N times the resolution of the root filter, and N is a positive integer, according to the response diagram of the root filter and the component filter The response map of the device is obtained, and the target response map is obtained, and the current face feature image is obtained according to the target response map.

In one embodiment, a computer device is provided, and the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 14 . The computer equipment includes a processor, memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a method for analyzing facial features. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.

Those skilled in the art can understand that the structure shown in FIG. 14 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer equipment to which the solution of the present invention is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the driver in any of the above embodiments when the processor executes the computer program Fatigue detection method.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the driver fatigue detection method in any one of the above embodiments.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, all It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can also be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims (10)

1. A driver fatigue detection method, characterized in that the method comprises:

acquiring a face video of a target driver, and respectively detecting the opening degree of human eyes of each frame of face image in the face video to obtain the opening degree value of human eyes in each frame of face image;

determining a first opening threshold value and a second opening threshold value according to each human eye opening value, wherein the first opening threshold value is larger than the second opening threshold value;

according to each human eye opening value, the first opening threshold and the second opening threshold, counting a first image frame value of which the human eye opening value is smaller than or equal to the first opening threshold and a second image frame value of which the human eye opening value is smaller than or equal to the second opening threshold;

and if the ratio of the first image frame value to the second image frame value is greater than a preset fatigue judgment threshold value, judging that the target driver is in a fatigue state.

2. The method of claim 1, wherein the detecting the eye opening degree of each frame of face image in the face video to obtain the eye opening degree value of each frame of face image comprises:

respectively carrying out eye feature point positioning on each frame of the face image to obtain eye feature points in each frame of the face image;

respectively determining a human eye interpupillary distance value and a human eye opening original value in the face image of each frame according to the eye feature points in the face image of each frame;

and respectively determining the human eye opening value in the face image of each frame according to the human eye interpupillary distance value and the human eye opening original value in the face image of each frame.

3. The driver fatigue detection method according to claim 1 or 2, wherein the determining a first opening degree threshold value and a second opening degree threshold value from each of the human eye opening degree values includes:

determining the maximum eye opening degree value according to the eye opening degree values;

and multiplying the maximum opening degree value by a preset first proportional coefficient and a preset second proportional coefficient respectively to obtain the first opening degree threshold value and the second opening degree threshold value.

4. The method for detecting driver fatigue according to claim 2, wherein the performing eye feature point positioning on the face image of each frame to obtain eye feature points in the face image of each frame includes:

respectively carrying out face feature point positioning on the face image of each frame to obtain each face feature image;

respectively inputting each face feature image into a preset eye feature point positioning model to obtain eye feature points in each frame of the face image;

wherein, the training process of the eye feature point positioning model comprises the following steps:

acquiring a pixel value of each pixel point of a target image and a feature vector of each pixel point;

configuring a tree structure local model according to the pixel values and the feature vectors, and determining a score function in the L part, wherein the score function is S (I, L, m) App_m(I,L)+Shape_m(L)+α^m；

Wherein,

i denotes the target image,/_i＝(x_i,y_i) Representing the pixel value of the ith pixel point of the target image, w representing a partial model, m representing that the tree structure is a mixed type, the partial model is obtained by modeling each facial feature in the target image as a part, a, b, c and d representing elastic parameters, and alpha representing a mixed offset scalar; phi (I, l)_i) Representing l on the target image I_iA feature vector at the pixel;

obtaining optimal configuration parameters of each part of each hybrid type by calculating values of L and m which enable the score function to obtain a maximum value;

establishing a training sample set, wherein the training sample set comprises a positive sample and a negative sample which are set with labels, the positive sample is an image containing a human face, and the negative sample is an image not containing the human face;

constructing a target vector according to the partial model, the elasticity parameters and the mixed bias scalar, and modifying the score function according to the target vector;

and learning to obtain the eye characteristic point positioning model according to the training sample set, the optimal configuration parameters, the modified score function and a predefined target prediction function.

5. The method of claim 4, wherein the performing facial feature point location on the facial image of each frame to obtain each facial feature image comprises:

extracting a first DPM feature map, wherein the first DPM feature map is a DPM feature map of a current face image, and the current face image is any one frame of face image; DPM is a target detection algorithm Deformable Part Model;

sampling the first DPM feature map, and extracting a second DPM feature map, wherein the second DPM feature map is a DPM feature map of an image obtained by sampling the first DPM feature map;

performing convolution operation on the first DPM characteristic diagram by using a pre-trained root filter to obtain a response diagram of the root filter;

performing convolution operation on the N times of the second DPM characteristic diagram by using a pre-trained component filter to obtain a response diagram of the component filter, wherein the resolution of the component filter is N times of that of the root filter, and N is a positive integer;

obtaining a target response diagram according to the response diagram of the root filter and the response diagram of the component filter;

and acquiring a current face feature image according to the target response image.

6. The method of detecting driver fatigue as set forth in claim 4, wherein the eye feature point location model is:

wherein β represents the target vector, z_n＝{L_n,m_nC represents the penalty factor of the objective function, ξ_nAnd the penalty term of the nth sample is represented, pos and neg respectively represent a positive sample and a negative sample, K represents the number of the target vectors, and K represents the number of the corresponding target vectors.

7. A driver fatigue detecting device, characterized in that the device comprises:

the detection module is used for acquiring a face video of a target driver, and detecting the opening degree of human eyes of each frame of face image in the face video to obtain the opening degree value of human eyes of each frame of face image;

the processing module is used for determining a first opening threshold value and a second opening threshold value according to each human eye opening value, wherein the first opening threshold value is larger than the second opening threshold value;

a counting module, configured to count, according to each of the eye opening values, the first opening threshold and the second opening threshold, a first image frame value of which the eye opening value is smaller than or equal to the first opening threshold, and a second image frame value of which the eye opening value is smaller than or equal to the second opening threshold;

and the judging module is used for judging that the target driver is in a fatigue state if the ratio of the first image frame value to the second image frame value is greater than a preset fatigue judging threshold value.

8. The driver fatigue detection device according to claim 7, characterized in that:

the detection module positions eye feature points of each frame of the face image to obtain eye feature points of each frame of the face image, determines a human eye pupil distance value and a human eye opening original value of each frame of the face image according to the eye feature points of each frame of the face image, and determines a human eye opening value of each frame of the face image according to the human eye pupil distance value and the human eye opening original value of each frame of the face image.

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.

Images