JP2008112462A - Face area-tracing method by using face image, and device therefor - Google Patents

Abstract

A high-speed and precise face area tracking device is provided.
An image input unit, a front face detection unit, a front face dictionary storage unit, a front face evaluation unit, a face template map storage unit, a template setting unit, and tracking template information storage. 8 and a template tracking unit 9, and by performing a search that is centered on the previous parameter value and having a dense center, rough but fast tracking is performed for fast movement of the face. Precise tracking is possible for slow movements.
[Selection] Figure 1

Description

  The present invention relates to a face region tracking method using a face image and a device for tracking a face region of a person in an image with high robustness, high accuracy, and high speed.

  An active interface using facial movement is useful when the hand is handicapped or when the hand is full of luggage. As a means of human communication in normal life, in addition to words and gesture gestures, there is a case of expressing affirmation by shaking the face vertically or pointing the face in the direction in the sense of pointing something . This indicates that such an input support method using the gesture of moving the face is also useful for the input interface of a computer.

  If the movement of the face can be captured, it can be used not only as an active operation method but also for grasping the object of interest / attention, preference (market research), and mental state of the person.

  Here, in order to recognize the movement of the face, it is necessary to first detect the face from the image and track the detected face. Until now, there were problems such as restrictions on the position of the face, the number of cameras, slow speed, low detection accuracy, etc., and it was difficult to accurately track fast face movements.

  On the other hand, there is a method for detecting head movement without tracking a face. This is a method for detecting the three-dimensional movement of the head based on the contrast between the face and the hair (see Non-Patent Document 1). However, since this method uses the density of the image, it is easily affected by changes in illumination conditions, and a background image that does not change is required. Furthermore, it is influenced by whether the hair is hanging near the ear.

  In addition, there is a method of using a stereo camera to manually register facial feature points and tracking the feature points by normalized correlation matching to obtain the face direction and line-of-sight direction (see Non-Patent Document 2). However, a plurality of cameras are necessary, and the influence of the deformation of the feature point pattern due to the change in the face direction is not taken into consideration.

  There is a method of preparing a face template for each face orientation angle and detecting a face corresponding to various face orientations (see Non-Patent Document 3). However, a face image taken at a preset angle must be used as the face orientation angle, and a template cannot be created by classifying each face orientation angle from the face image itself.

The method of tracking an object to be tracked using a template is mainly based on a raster scan type pattern search in which full scanning is sequentially performed on the image from the upper left to the lower right (see, for example, Non-Patent Document 4). Furthermore, no method has been established for tracking or searching for a pattern while changing the enlargement magnification (reduction magnification) when the template is associated with the target image. This is one factor that requires a very long calculation time when tracking the face area.
"Head reader: Real-time detection of head movement by image" IEICE Transactions, Vol.J74-D-II, No.3, pp.398-406, March 1991 `` An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement ''. In Proc. Of the Int'l Conf. On Automatic Face and Gesture Recognition, 2000 “View-based and modular eigenspaces for face recognition”. In Proc. IEEE Conf. On Computer Vision and Pattern Recognition, p.84, 1994) Akio Okazaki, “First Image Processing Technology”, Industrial Research Committee, 2000

  Accordingly, the present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a fast and precise face area tracking method and apparatus.

  The present invention relates to a face region tracking method based on a face image that compares a template having a predetermined number of pixels with an input image and detects the degree of coincidence between the image and the template. The position is (x, y), the magnification of the template for making the template correspond to a region of an arbitrary size on the image is r, and the template is applied to the target region rotated on the image. The rotation angle of the template for correspondence is φ, and the center of the search points is densely distributed around the value of (x, y, r, φ) in the previous tracking result (x, y), This is a face area tracking method based on a face image in which a target area is tracked by associating a template in which r or φ is changed.

  According to the present invention, high-speed and precise face area tracking can be performed by performing tracking according to a dense distribution centered on the previous position.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  In the present embodiment, a face area tracking method using a face image will be described. FIG. 1 is a configuration diagram showing an example of a face area tracking method using a face image according to the present embodiment and a system for realizing the method. FIG. 2 is a conceptual diagram illustrating a face template map according to the embodiment. FIG. 3 is a flowchart for explaining the flow of the face area tracking process according to the embodiment.

  A major feature of the face area tracking system 1 shown in FIG. 1 is that it has a face template map storage 6 inside the system and has a function of learning and updating the face template map sequentially. The face area tracking system 1 includes an image input unit 2, a front face detection unit 3, a front face dictionary storage unit 4, a front face evaluation unit 5, a face template map storage unit 6, a template setting unit 7, a tracking It comprises a template information storage unit 8 and a template tracking unit 9. By including such components, the face area tracking system 1 outputs information such as the position of the face area, face orientation information, and results of face area tracking calculation when an image is input. Build a face template map inside the system by updating.

  The image input unit 2 receives an image of a subject photographed by a video camera or the like from the outside of the system, converts it into digital data so that it can be easily processed, and converts the electronic data to the front face detection unit 3 and template tracking. Send to part 9.

  The front face detection unit 3 receives a template for front face detection from the front face dictionary storage unit 4 in which a template for detecting a front image of a person is stored. The image input from the image input unit 2 and its template Are compared to detect a human face image from the input image. For example, it is possible to use a method in which images of human eyes and nostril regions are detected by pattern matching and a circularity separation filter, and face detection is performed by using this in combination with whole face pattern matching. In this case, the template is a dictionary pattern of the subspace method corresponding to each of the face, eyes, and nose.

  The front face evaluation unit 5 receives the face image of the person and the position information of the detected face region from the front face detection unit 3, receives the front face template 13 from the face template map storage unit 6, and detects the detected face region data. Is more suitable as the front face template 13, the front face template 13 in the face template map storage 6 is updated. The detected face area position information is sent to the template setting unit 7.

  The determination as to whether or not it is more suitable as a front face template can be made based on, for example, the similarity value of template matching, the arrangement of coordinates of eyes and nostrils, and the like.

  As shown in FIG. 2, the face template map storage unit 6 stores large and small face templates that differ depending on the face orientation angle in association with the upward and lateral angles, and constitutes a face template map as a whole. Yes. Here, the large face template is an image template of the entire face including facial features such as both eyes and mouth, and the small face template is a facial feature portion (for example, nose) that can be detected three-dimensionally. ).

  In the face template map stored in the face template map storage unit 6, for example, in the initial state, each template for each face orientation angle is blank (blank). First, the front face evaluation unit 5 converts the template into a front face image. A large and small face template is created using an image determined to be suitable, and a front face template 13 is generated. Next, by the function of the template setting unit 7, face templates other than the front face are learned and updated as face images with various face orientations are input.

  The template setting unit 7 receives front face information detected from the front face evaluation unit 5, determines a tracking template 22 to be used for the next face area tracking, and stores the tracking template 22 in the tracking template information storage unit 8.

  Here, the template setting unit 7 determines whether or not to newly register the detected face area as a face template based on the relative position information of the large and small face templates. This is because, for example, when the face direction changes, such as when the subject faces sideways, the matching position of the small face template shifts in the face direction direction (for example, the horizontal direction), and the degree of shift increases as the face direction angle increases. This utilizes the property that (relative position from the front position) increases. If the shift amount becomes larger than a certain threshold value, the detected face area is registered in the face template map storage unit 6 as a face template such as left and right or up and down as necessary.

  A face template map can be constructed by using a plurality of templates having three or more sizes, and face area tracking can be performed. As shown in FIGS. 9, 10, and 11, when three types of templates (large, medium, and small) are used, for example, the position of the face template (medium) with respect to the face template (large) and the face template (medium) The position of the face template (small) is related to the size of the face orientation angle. Based on the comprehensive judgment based on these values, the detected face area is used as a face template such as left and right, up and down, and the face template map. Register in the storage unit 6. As a method for comprehensive determination, an addition sum of relative magnitudes of shifts can be used. Thus, by increasing the number of templates, it is possible to perform more stable and robust registration processing and face area tracking.

  The tracking template information storage unit 8 receives the tracking face template from the template setting unit, stores the tracking template 22 used for the tracking processing of the face direction that is sequentially updated, and the template setting unit 7 and the template tracking as necessary. The tracking template 22 is sent to the department.

  The template tracking unit 9 receives the input image from the image input unit 2, receives the tracking template 22 from the tracking template information storage unit 8, and compares the input image with the tracking template to perform a face area tracking process. The tracking processing results (success / failure of face area tracking, face area size, position, face orientation (angle), etc.) are output, and face area information is sent to the template setting unit 7.

  In the tracking process, as shown in FIG. 4, for example, the tracking template 22 consisting of 9 pixels × 9 pixels = 81 pixels is made to correspond to a certain position on the input image area 21, and the similarity is calculated by the template matching process. This is achieved by changing a plurality of search positions and magnifications. When the maximum similarity exceeds a predetermined threshold value, it is determined that a face area has been detected at the position and magnification.

  The magnification indicates the enlargement / reduction ratio of the image. If the number of pixels of the image area corresponding to the template is 9 × 9 pixels, the magnification is exactly 1 ×, but if it is n × n pixels, the magnification is n / 9 ×. When the magnification is not 1, it can be made to correspond to each pixel of the tracking template by enlarging or reducing the image. Although the number of pixels of the tracking template is 9 × 9 here, any number may be used, and it may be set appropriately according to the problem.

  The search for the image position and the magnification in the tracking process is limited in calculation time. In practice, the search for the image position and the magnification must be performed sparsely with a certain interval, but in this embodiment, the position is shown in FIGS. If the value shown in FIG. 8 is used and the value shown in FIG. 6 is used for the magnification, the search process can be performed more efficiently.

  Specifically, when the previous position is (x, y), the vicinity of the position is searched in detail using the previous position as a reference, and the search process is roughly performed as the search position moves away. For example, (x, y) of the input image is searched using the tracking template (0th search), then (x ± 1, y), (x, y ± 1), (x + 1, y ± 1) , (X−1, y ± 1) (first search), then (x ± 3, y), (x, y ± 3), (x + 3, y ± 3), (x−3, Search y ± 3) (second search) until the n-th search.

  Regarding the magnification search, specifically, when the previous magnification is r, an image near the magnification is generated and searched in detail with the previous search magnification as a reference. For example, using a small constant a (0 <a <1), an input image is first searched using a tracking template of r times (0th search), and then r * (1 ± a) The search is performed using the tracking template with the magnification (first search), and then r * (1 ± 3a) is searched (second search) until the m-th search.

  As for the search for rotation, a search is made in detail using a template that has been rotated in the vicinity of the previous rotation angle with reference to the previous rotation angle, and a coarse search is performed as the rotational displacement from the rotation angle increases. For example, if the rotation angle of the previous template is α, the input image is searched using the tracking template that has been subjected to the rotation processing of α (0th order search), and then the rotation of α (1 ± a) is performed. The search is performed using the tracking template (primary search), and the search (secondary search) is continued using the tracking template that is further rotated by α (1 ± 3a).

  As described above, by performing a search that centers on the previous parameter value and the center is dense, it performs rough but fast tracking for fast movement of the face, and for slow movement of the face Precise tracking can be performed.

  FIG. 3 is a flowchart showing the tracking procedure of the face area tracking method in this embodiment.

  First, it is determined whether or not to perform face area tracking based on input from the subject or the system (ST1), and if not, the process ends. When face area tracking is performed, an input image from the image input unit is processed to perform front face detection and template tracking (ST2). Next, it is determined whether or not the previous face area tracking process was successful (ST3). If the previous success, the process proceeds to step ST8, and if the previous failure, the process proceeds to step ST4.

  In step ST4, the front face detection unit 3 performs a front face detection process to determine whether a front face has been found (ST5). If found, the process proceeds to step ST6, and if not, the process returns to step ST1. .

  In step ST6, the front face evaluation unit 5 performs front template update processing, copies the front template to the tracking template 22 (ST7), and the template tracking unit performs large / small template tracking processing (ST8). Next, it is determined whether or not the template tracking is successful (ST9). When the template tracking is unsuccessful, the process returns to step ST1, but when the template tracking is successful, the face position information is output (ST10). Accordingly, the tracking template 22 is updated when it is large, and in either case, the process returns to step ST1.

It is a block diagram which shows an example which concerns on one Embodiment of this invention. It is a conceptual diagram explaining the face template map which concerns on the same embodiment. It is a flowchart which shows an example of a process of the face area tracking method which concerns on the embodiment. It is a conceptual diagram explaining face area tracking using the tracking template concerning the embodiment. It is a conceptual diagram explaining the face area position search according to the dense distribution according to the embodiment. It is a conceptual diagram explaining the face template magnification search according to the dense distribution according to the embodiment. It is a conceptual diagram explaining the face area position search according to the dense distribution according to the embodiment. It is a conceptual diagram explaining the face area position search according to the dense distribution according to the embodiment. It is a conceptual diagram explaining the face template map which concerns on the same embodiment. It is a conceptual diagram explaining the face template map which concerns on the same embodiment. It is a conceptual diagram explaining the face template map which concerns on the same embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face area tracking system 2 Image input part 3 Front face detection part 4 Front face dictionary storage part 5 Front face evaluation part 6 Face template map storage part 7 Template setting part 8 Tracking template information storage part 9 Template tracking part 11 Face template (large )
12 Face template (small)
13 Front face template 14 Face template (middle)
21 Input image area 22 Tracking template

Claims (2)

Comparing a template with a predetermined number of pixels and an input image, a face region tracking method using a face image for detecting the degree of matching between the image and the template,
The position on the input image that corresponds to the template is (x, y), and the magnification of the template for making the template correspond to an area of an arbitrary size on the image is r, and the image is rotated on the image. The rotation angle of the template for associating the template with the target region is φ, and the centers of the search points are densely distributed around the value of (x, y, r, φ) in the previous tracking result. Tracking the region of interest by matching templates with varying (x, y), r, or φ,
A method for tracking a face area using a face image. A face area tracking device using a face image that compares a template having a predetermined number of pixels with an input image and detects the degree of coincidence between the image and the template,
The position on the input image that corresponds to the template is (x, y), and the magnification of the template for making the template correspond to an area of an arbitrary size on the image is r, and the image is rotated on the image. The rotation angle of the template for associating the template with the target region is φ, and the centers of the search points are densely distributed around the value of (x, y, r, φ) in the previous tracking result. Tracking the region of interest by matching templates with varying (x, y), r, or φ,
Facial region tracking device based on facial images.

Images