CN101216885A - A Pedestrian Face Detection and Tracking Algorithm Based on Video - Google Patents

Abstract

The invention proposes an algorithm of moving object detection and moving object tracking for video pedestrian face detection and tracking, which belongs to the technical field of pattern recognition. The invention is composed of two parts: a pedestrian face detection algorithm based on moving object detection and a pedestrian face tracking algorithm based on moving object tracking. The invention proposes a pedestrian face detection algorithm based on moving object detection. This algorithm first uses motion analysis to detect pedestrians, then calculates the center of gravity of the human body and determines the face area according to the center of gravity of the human body, and finally uses the method of skin color model and template matching to detect the face in the face area. The invention proposes a pedestrian face tracking algorithm based on moving object tracking. This algorithm tracks pedestrians' faces by tracking pedestrians, thus effectively avoiding the impact of pedestrians' face swing, rotation, expression, occlusion and other factors on face tracking.

Description

A Pedestrian Face Detection and Tracking Algorithm Based on Video

technical field

The invention belongs to the technical field of pattern recognition, and in particular relates to a method for detecting and tracking pedestrian faces based on video by using a moving object detection and tracking method.

technical background

With the widespread deployment of video surveillance in large and medium-sized cities, the intelligent processing of massive information of surveillance video is also on the agenda. Video-based pedestrian face detection and tracking is a kind of surveillance video intelligent information processing, which has extensive and important applications in social security and public security criminal investigation.

At present, most face detection algorithms are based on images, not videos. These algorithms use a window to traverse the entire image and detect whether the window contains a face. In order to adapt to changes in the size of the face, these algorithms need to iterate through the image with windows of different sizes. Therefore, the image-based face detection algorithm is an algorithm with high time complexity. If the image-based face detection algorithm is directly applied to each frame of video without modification, it will not be able to meet the real-time requirements of video processing, not to mention that in practical applications, face detection is often followed by face recognition. The face recognition algorithm is also an algorithm with high time complexity. Therefore, if the time complexity of the face detection algorithm and the face recognition algorithm are taken into consideration, the real-time requirements for the face detection algorithm will be more stringent.

Video-based face tracking has important implications. First of all, face tracking can reduce the repeated operations of face detection in adjacent frames of the video, thereby greatly reducing the amount of calculation of the entire algorithm; secondly, face tracking can be applied to image enhancement to provide clear face images; finally, Face tracking can provide information from multiple perspectives of the face and enrich the basis for face recognition. However, face tracking based on video is very difficult, because in the video, the pose, shape and size of the face are constantly changing, coupled with the fact that the face occupies a small proportion of the entire image and factors such as occlusion and expression , it is difficult to find stable features of the face that can run through successive frames of the video.

The invention proposes an algorithm for monitoring and tracking people's faces with the help of a method of monitoring and tracking moving objects, which better solves the current difficulties encountered in the detection and tracking of pedestrians' faces.

Contents of the invention

The invention proposes an algorithm for pedestrian face detection and tracking by using the algorithm of moving object detection and tracking. This algorithm is based on the motion information of the human body in the video, and uses the information of the spatial correlation and temporal correlation of human motion. The specific content is as follows:

1. Apply the detection algorithm of moving objects to the detection of pedestrian faces (see Figure 1, Figure 2)

(1) Detection of moving regions: For a given video image sequence, the difference fixed background is generally used to subtract the background from the target image of the current frame, and after thresholding, the image is divided into moving regions and background regions.

(2) Discrimination of moving objects (pedestrians): Use a two-dimensional human body template constructed based on anthropometric data to discriminate the above-mentioned moving area to determine whether there are pedestrians in the area.

(3) Judgment of the human face area: After identifying the moving human body area, consider the three-quarters area above the center of gravity of the moving human body as the human face area, and mark this part of the area. Usually, image-based human face area determination is to detect human face area on the whole picture, and the human face area determination based on motion analysis in the present invention only determines the human face area on the motion area, which greatly reduces the search range and shortens the time. search time.

(4) Face detection in the face area: reposition the human body area on the original image, perform skin color segmentation and mathematical morphology processing, and exclude those that are too wide or too long or the aspect ratio is too large or too small skin tone area. Finally, the average face template matching method is used for detection to obtain the final face area.

According to the obtained rough detection results, the candidate face regions are extracted in turn, and after some grayscale processing, the face template is used to match the candidate face image window, and the image window that meets certain conditions and reaches the matching degree threshold is used as the face.

2. Apply the tracking algorithm of moving objects to the tracking of pedestrian faces

The invention applies the tracking algorithm of moving objects to the tracking of pedestrians' faces, and uses the existing mature human body tracking algorithm of pedestrians to achieve the purpose of real-time tracking of human faces. It mainly uses the characteristics of the Kalman filter recursive linear minimum variance estimation, which has the advantages of fast and accurate estimation, and is specifically divided into the following two steps:

(1) Track the face by tracking the human body: according to the detected position of the human body in the current frame, use the Kalman filter to estimate the position, velocity and acceleration of the human body in the current frame, and use this estimated value to estimate the position of the human body in the next frame. position in the frame to make predictions. When the tracking of the pedestrian's human body is completed, the pedestrian's face detection method described in 1 is used to detect the human face on the tracked human body again, and the pedestrian's face is tracked. The face tracking method based on the human body tracking of pedestrians in the present invention not only overcomes the unfavorable tracking factors caused by the characteristics of too small pedestrians and faces, but also reduces the search range when tracking and locating faces each time, greatly improving the tracking performance. speed.

(2) Analysis of other areas: Use the above-mentioned face detection method based on moving object detection to analyze the remaining areas to determine whether there are new faces. If so, continue to track it with the face tracking method based on moving object tracking. Obviously, at this time, the area where pedestrian detection needs to be performed is reduced, which greatly saves the amount of computation.

Refer to FIG. 3 for the whole algorithm flow chart of the present invention.

Features of the invention

The algorithm proposed by the invention constructs an algorithm framework for pedestrian face detection and tracking. Under this framework, most moving object detection and tracking algorithms can be combined into pedestrian face detection and tracking algorithms. The algorithm that the present invention proposes has two notable characteristics:

(1) The algorithm proposed by the present invention first finds pedestrians and then faces, and pedestrians are used as the basis for face area positioning. Since the target of pedestrians is larger than that of faces, and motion information is available, it is much easier to find pedestrians in videos than to find faces directly.

(2) The algorithm proposed by the present invention realizes the tracking of human faces by tracking pedestrians. Because the position of the pedestrian relative to the camera will change during the journey, and the head will also swing or turn, causing the shape, size, and direction of the face to change in successive frames of the video, coupled with factors such as expressions and occlusion, it is difficult to Extract stable features from the face as the basis for tracking. In contrast, the pedestrian target is larger, and there are more stable features on the pedestrian, which can be used as the basis for tracking in successive frames of the video.

Description of drawings

Figure 1. Flow chart of moving human detection

Figure 2. Flowchart of face detection

Fig. 3, the flowchart of whole algorithm in the present invention

specific implementation plan

Step 1: Use moving object detection for face detection

Face detection using the motion information of moving objects includes the following steps: motion area detection, motion body recognition, skin color segmentation, mathematical morphology processing, skin color-like area filtering, and average face template matching, which are described below.

1. Motion area detection

First of all, for a given image sequence in a video, this specific embodiment first detects the motion area by the method of Orthogonal Gauss Hermitian Moments (OGHMs), and separates the motion area from the background area.

Orthogonal Gaussian-Hermitian moment template window size is 3, which corresponds to a standard deviation of 0.2. When the window size is 3, the value range of the basis function is [-0.9972, 0.9972]. The template looks like this:

1.2598 0 -1.2598

first order template

4.0908 0 -4.0908

third-order template

7.9121 0 -7.9121

Fifth order template

After obtaining OGHMIs, moving objects cannot be directly extracted from them, and the obtained images must be segmented. The pixels in the same object in the same image have a strong spatial correlation. If only a simple threshold segmentation is set, this correlation will be destroyed and the robustness of the segmentation will be reduced. In terms of segmentation, the invariant moment algorithm does not take into account the spatial correlation of objects, so J.Shen et al. (Reference [1]: J.Shen.W.Shen, H.J.Sun.J.Y.Yang.Fuzzy Neural Nets with Non -Symmetric Membership Function and Application in Signal Processing and Image Analysis.Signal Process, 2000, 80: 965-983.) proposed the Non-symmetric membership function to get the membership function under OGHMIs (reference [2]: Youfu Wu, Jun Shen , Mo Dai.Traffic Object Detections and its Action Analysis.PattenRecognition Letters, 2005.), on this basis, apply the Fuzzy relaxation algorithm (FRM) for segmentation, thus ensuring a strong spatial correlation between the pixels of the object during segmentation .

The expression of the Membership function is shown in formula (1):

Through the Non-symmetric membership function, the membership function u of GHMIs can be deduced to achieve normalization, see formula (2):

u(M(x, y); T, M _min (x, y)) can be abbreviated as u(x, y).

After obtaining the membership function u(x, y), the moving object is extracted through the FRM algorithm. This algorithm is a region growing algorithm, which has several key steps: the determination of the starting point, the region growing method, the termination condition, and the interference filtering.

A. Determination of the starting point

The image is scanned sequentially from upper left to lower right, and the point where u(x, y)=1 is found as the starting point.

B. Growth method

Taking the starting point as the current point, search for u(x', y') in its four neighborhoods, if u(x', y')>0.7, set u(x', y')=1, and set (x ', y') as the seed point, continue to search the four-neighborhood; otherwise u(x', y')=0, not the point of the motion module.

C. Termination conditions

After all the seed points are searched, end the current fast search, continue to scan the image, find a new starting point, and repeat the above steps until the complete image is processed.

D. Interference filtering

During the region growing process, record the number of points belonging to the motion region, and when the size of the motion block is smaller than a certain threshold T, it will be filtered out as interference, and the corresponding u(x, y)=0.

Through the search process of the FRM algorithm, the image can be binarized to extract the moving object.

After the moving object is extracted, some optimization methods can be used to make the obtained target more accurate. In this paper, the size of the moving block and the static block in the image is calculated, and the corresponding threshold is set.

In this specific embodiment, the motion area is set to be white, and the static area is black. When the number of pixels in the black block in the white block is less than a certain threshold, it is considered that this part is also a motion area, and it is set to white, which can solve the problem of holes; When the number of white blocks in the block is less than a certain threshold, it is considered that this part is also a static area, and it is set to white, which can eliminate the background interference problem.

2. Sports human identification

In this specific embodiment, it is assumed that the human bodies in the scene are all in an upright state. Compared with other moving objects in nature, the human body in the upright state has a very special feature, that is, the height and width of the human body are relatively large. Animals that walk on all fours in nature are relatively small in height and width. There are also some moving objects, such as vehicles, in order to maintain a stable state during motion, and the center of gravity is generally all low, so that its height and width ratio are also small. For a specific scene, we can also determine certain rules based on some prior knowledge of the scene, and use them to help identify the existence of human beings. For example, for a known scene, the human body area has a general range, and by checking the area of the connected area, it can help remove some noise areas.

The eigenvalue of the area is extracted by calculating the number of pixels in the connected area in the binarized image. The threshold is set according to the human body area parameters obtained empirically in a specific scene, and then the threshold is segmented. The foreground connected area smaller than the area threshold The pixel value in is set to the background pixel value.

The eigenvalue of the height-to-width ratio of connected regions is obtained through the following steps:

A. Search for the first pixel in the connected area, record the value of its abscissa and ordinate, search for the subsequent pixels in the forward direction, record the value of its abscissa and ordinate, and find the maximum value X of the abscissa through the sorting algorithm _max and minimum value X _min , and the maximum value Y _max and minimum value Y _min of the ordinate.

B. Obtain the height of the connected region: H _n = (Y _max -Y _min ), width: W _n = (X _max -X _min ), (n is the number of the connected region during the forward search, the first searched connected The area number is n=1), store, and add 1 to n.

C. Search for the next connected region, go to step A, and exit if no connected region is found.

Through the test results of a large number of human body height and width ratios, it can be concluded that after the human body completely enters the scene, due to the influence of the swing range of the human body's arms and legs when walking, and the different angles of the camera, its height , The range of the width ratio is generally between 1 and 5. In order not to miss the human body in practical applications, we can appropriately relax the range scale.

3. Skin color segmentation

The distance between two points in the HSV space is consistent with the human vision, and the H component reflects the hue information of the object, which is easier for color image segmentation processing. The difference in skin color in the image is mainly caused by the light. Only considering the chroma information in the detection can reduce the influence of light. Therefore, only H (hue) can be used to extract the skin color alone.

In this specific implementation, the area determined after the motion detection is repositioned on the original image, and the conversion from RGB→HSV color space is completed for the pixels in this area according to the formula (3). According to the value of H, it is judged whether a certain pixel belongs to the skin color. In this embodiment, when 0.02<H<0.08, the pixel is considered to belong to the skin color.

V=max(R, G, B)

$\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; NaN</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; undefined</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}}{\mathrm{<span\; class="notranslate">\; 6</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Rand</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; b</span>}}{\mathrm{<span\; class="notranslate">\; 6</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Rand</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; G</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}}{\mathrm{<span\; class="notranslate">\; b</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Gand</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; R</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; r</span>}}{\mathrm{<span\; class="notranslate">\; 6</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Gand</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; r</span>}}{\mathrm{<span\; class="notranslate">\; 6</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; band</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; G</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; g</span>}}{\mathrm{<span\; class="notranslate">\; 6</span>}}& \mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; band</span>}\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; R</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

in

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; Delta</span>}}<span\; class="notranslate">\; ,</span>$

$\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}}{\mathrm{<span\; class="notranslate">\; Delta</span>}}<span\; class="notranslate">\; ,</span>$

$b=\frac{V-B}{\mathrm{Delta}},$ Delt=V-min(R, G, B).

4. Mathematical Morphological Processing

In this specific embodiment, the mathematical morphology processing method of erosion and expansion is adopted to process the area obtained in the previous step.

Erosion is the process of eliminating all boundary points of an object, with the result that the remaining object is one pixel smaller in area along its perimeter than the original object. If the width of any point of the object is less than three pixels, then it will become disconnected (become 2 objects) at that point. Objects that are no wider than 2 pixels wide in any direction will be removed. Erosion can be used to remove small and meaningless objects from a segmented image.

Dilation is the process of merging all background points that are in contact with an object into that object. The result of the process is to increase the area of the object by the corresponding number of points. If two objects are separated by less than 3 pixels at a certain point, they will be concatenated (merged into one object) at that point. Dilation can fill the void in the segmented object.

Five, similar skin color area filtering

After the filtering method based on mathematical morphology, most of the small block noises in the image are removed, but some small skin-like areas in the background still exist. In order to remove fake face regions, we have to analyze and compute these regions. First, mark the skin-like area, and then use the feature that the aspect ratio of the face meets a certain ratio to filter out, and exclude those areas that are too wide or too long, or the aspect ratio is too large or too small.

In order to determine the aspect ratio of a certain region, the length L and width W of the region must be calculated separately. However, due to the possible rotation and inclination of some faces, it is impossible to directly use the coordinate values of the left, right, top, and bottom vertices of the area (here, the coordinate system used is the bottom left corner of the image as the origin, and the horizontal direction The right is the positive direction of the x-axis, and the vertical upward is the positive direction of the y-axis) for judgment. The detailed process is: count the coordinate values of all points on the boundary of the area, find the coordinates (X _min , X _max ) with the smallest and largest X components on the x-axis, and the coordinates of the smallest and largest Y components on the Y-axis (Y _min , Y _max ), L=X _max -X _min , W=Y _max -Y _min The values are the length and width (width and length) parameter values of the face. Ratio of L to W $r=\frac{L}{W}$ That is, the ratio of length to width (or width to length) of the area sought. If the face is vertically frontal, the ratio should be close to 1.2. However, due to the rotation of the face and the similarity of skin color segmentation may cause the head and neck of the face to be segmented as the same area, the upper limit of the factory can be appropriately enlarged to prevent Treat correct segmentation regions as wrong judgments. In this specific embodiment, the value range of r is (0.5, 2), and the candidate regions that do not belong to this range are directly deleted.

6. Average face template matching

Manually cut out the area of the face in the selected sample image as a face sample, normalize its scale to 24×24, average the grayscale of all samples to obtain the average face image, and perform grayscale distribution on the average face image After normalization, it is used as a face template.

In order to adapt to different shapes of faces, the original templates were stretched according to the width-to-length ratios of 1:0.9, 1:1, 1:1.1, and 1:1.2, respectively.

Wherein, the standardization of the gray level distribution is to transform the gray level mean and variance of the image into μ ₀ =128 and σ ₀ =80. Let the gray value matrix of the image be D[W][H] (where W and H are the width and height of the image respectively), calculate its mean and variance, and perform the following transformation:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; WH</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

${\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; WH</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

According to the obtained rough detection results, the candidate face regions are extracted in turn, converted into grayscale images, and the grayscale distribution of the images is standardized. Then use the face template to match the face image window to be selected, and use the image window that meets certain conditions and reaches the matching degree threshold as the face.

The following matching criteria are adopted in the matching process. Suppose the grayscale matrix of the face template is T[M][N], the grayscale mean and variance are _μT and _σT respectively, the grayscale matrix of the face image area to be selected is R[M][N], and the grayscale The degree mean and variance are μ _R and σ _R respectively, then the correlation coefficient r(T, R) between them is:

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; MN</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

When using the face template for matching, if the correlation coefficient r(T, R) exceeds the threshold value t (t=0.6), it is considered to have passed the average face matching screening and is considered as a human face.

Step 2: Use the moving object tracking algorithm for face tracking

There are many algorithms for moving object tracking. This specific embodiment uses a Kalman filter, but is not limited to a Kalman filter.

The Kalman filter is a recursive linear minimum variance estimation, which has the advantages of fast and accurate estimation, and has a mature application in video pedestrian tracking. This specific implementation plan uses the Kalman filter to estimate the motion state of pedestrians, and tracks pedestrians according to the estimated motion state, and detects faces on the tracked pedestrians to achieve the purpose of tracking pedestrians' faces. The process is as follows:

1. Pedestrian tracking

A. Discrete Kalman filter

The discrete system state equation of a dynamic system driven by system noise with deterministic control can be written as:

X[k]=AX[k-1]+BU[k]+W[k]

The measurement equation of the observation system is:

Z[k]=HX[k]+V[k]

Where X[k] is the state vector of the system at time k, U[k] is the input vector of the system at time k, Z[k] is the output vector of the system at time k, W[k] and V[k] are The noise vectors at time k follow the Gaussian distribution and are independent of each other. A.

B and H are coefficient matrices. The corresponding Kalman filter basic equation is:

State one-step prediction equation: X[k/k-1]=AX[k-1/k-1]+BU[k]

State estimation: X[k/k]=AX[k/k-1]+K _g [k](Z[k]-HX[k/k-1])

Filter gain matrix: $K_g\left[k\right]=\frac{P[k/k-1]h^T}{\mathrm{HP}[k/k-1]h^T+R}$

One-step prediction mean square error matrix: P[k/k-1]=AP[k-1/k-1] ^AT +Q

Estimated mean square error matrix: P[k/k]=(IK _g [k]H)P[k/k-1]

According to the specific situation in the application, the initial value of the filter is selected, and then the Kalman filter estimation of the system state can be performed through the newly obtained measurement vector and the above recursive formula.

It can be seen from the discrete Kalman filtering process in FIG. 3 that the Kalman filtering has two calculation loops: a gain calculation loop and a filter calculation loop. The gain calculation loop is independent, while the filter calculation loop depends on the gain calculation loop. There are two update processes, the time update process and the measurement update process, in one cycle. If the estimated value of the predicted state at time k-1 and the measured value at time k are known, as well as the one-step forecast mean square error matrix at time k-1, the optimal estimated value of the state vector at time k can be obtained. And it can predict the state estimation value and measurement value of the system at k+1 time.

B. Pedestrian Human Model Creation

The center of gravity of the human body found in step 1 is selected as the feature point. This is because the shape of the human body is symmetrical, and the center of gravity of the human body will translate steadily along the direction of motion of the human body without being constrained by the self-motion of the human body, avoiding the periodicity of the human body shape impact of changes. In order to reduce the computational complexity, two Kalman filters are set for each human body to estimate the motion state of the human center of gravity in the X direction and the Y direction respectively.

The motion state of the human body can be expressed by the vector X=(s _x , v _x , a _x , s _y , v _y , a _y ) ^T , where s _x , v _x , and a _x respectively represent the displacement of the center of gravity of the human body in the X direction , velocity, and acceleration, s _y , v _y , and a _y represent the displacement, velocity, and acceleration of the center of gravity of the human body in the Y direction, respectively. Since each human body uses two Kalman filters to estimate the motion state of its center of gravity in the X direction and Y direction, the vector X can be decomposed into two vectors X _x = (s _x , v _x , a _x ) ^T and X _y = (s _y , v _y , a _y ) ^T respectively represent the movement state of his center of gravity in the X direction and the Y direction. The movement of the human body in these two directions is independent of each other, and the processing method is the same , so here we only elaborate on the processing in the X direction.

The system equation of the motion state of the center of gravity of the human body is:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&Delta;T</span>}<span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&times;</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;T</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&Delta;T</span>}\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span>\end{array}\right.$

In the formula, s _x [k] represents the displacement of the human body's center of gravity in the X direction at the kth frame, v _x [k] represents the velocity of the human body's center of gravity in the X direction at the kth frame, and a _x [k] represents the X direction of the human body's center of gravity at the kth frame The acceleration of , ΔT represents the time interval, the above formula is written in matrix form as

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; \&Delta;T</span>}& \frac{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;T</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; \&Delta;T</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span>\end{array}\right)<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; A</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>$

In the formula $A=\left(\begin{array}{ccc}1& \mathrm{\&Delta;T}& \frac{{\left(\mathrm{\&Delta;T}\right)}^2}{2}\\ 0& 1& \mathrm{\&Delta;T}\\ 0& 0& 1\end{array}\right)$ , W[k] represents noise.

In practical applications, only the displacement of the center of gravity of the human body in the image can be observed, and the velocity and acceleration cannot be directly observed, so the measurement equation is:

${\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 100</span>}<span\; class="notranslate">\; )</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>\end{array}\right)<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; HX</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>$

Where H=(100), the measurement noise V _x [k] can be regarded as white noise.

It can be seen from the above that the form of the state equation and measurement equation of the system is the same as that of the standard Kalman filter, so the basic equation of the discrete Kalman filter can be used to estimate the state of the system. to track video pedestrians.

C. Tracking the human body of pedestrians

Initialize two Kalman filters for the pedestrian body obtained by the moving object detection method in step 1, which are used to estimate the motion state of the pedestrian's center of gravity in the X direction and Y direction respectively. Since the Kalman filter uses a recursive estimation method, as long as the initial state of the filter equation and the initial estimated mean square error matrix are given, the current measurement value can be used to obtain the estimated value of the system state. This specific implementation plan Initialize the system state vector of the pedestrian body using the measured values obtained for the first time:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right)<span\; class="notranslate">\; ,</span>$ ${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ]</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right)$

where z _x [0] and z _y [0] are measurements in the X and Y directions of the center of gravity of the human body. The initial values of the estimated mean square error for both Kalman filters are set to:

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 100</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 100</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 100</span>}\end{array}\right)$

In this way, the displacement, velocity, and acceleration of the human body's center of gravity in the X and Y directions can be estimated through the Kalman filter based on the current measurement value, and the displacement, velocity, and acceleration of the human body's center of gravity in the next frame can be predicted according to the filter equation . If the predicted center of gravity of the human body falls within the tracking window of a pedestrian human body detected in the next frame, then the human body in the previous frame is considered to match this human body, and then the system state is calculated according to the newly obtained measurement value. renew.

2. Locate the face on the tracked pedestrian body

Use the face detection method in step 1 to detect faces on the tracked moving body and mark them with a red rectangle. Because the face detection based on motion analysis of this specific embodiment only locates the face in the area above the three-quarters of the center of gravity of the moving human body, the search range is greatly reduced, the running time of the algorithm is shortened, and the face detection is closely related to the human body. Tracking is performed synchronously in almost real time, so as to achieve the purpose of accurately tracking pedestrian faces.

3. Analyze other areas

Use the face detection method based on moving object detection in step 1 to analyze the remaining areas to determine whether there are new faces. If so, continue to track it with the face tracking method based on moving object tracking. Apparently, at this time, the area that needs to be detected for pedestrians and faces is significantly reduced, which saves the amount of computation.

Claims (3)

1. the present invention proposes a kind of algorithm based on the pedestrian human face detection of video and tracking, it is characterized in that utilize the algorithm of moving object detection and the algorithm of moving object tracking to carry out the detection and tracking of pedestrian human face. 2. As described in right 1, the detection algorithm of moving objects is applied to the detection of pedestrians and faces: for a given image sequence in a video, the motion area is first obtained by the method of motion detection, and the motion area and background The area is divided, and the moving body is recognized in the moving area obtained by the motion detection. The face area must be located above the center of gravity of the moving body. Mark this part of the area and reposition it on the original image, and then in this face detection in the area. 3. as described in claim 1, utilize the tracking algorithm of moving object to carry out the tracking of pedestrian's face: the target of moving human body is larger than human face, easier to track, utilize the algorithm of moving object tracking to the described in power 2 The detected moving human body is tracked, and then the face detection method in Power 1 is used to perform face detection on the tracked moving human body, which overcomes the adverse effects of facial posture changes on tracking, so as to achieve the purpose of tracking human faces .

Images