JP2010072723A - Tracking device and tracking method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique executing tracking processing of a person in a moving picture with high reliability from a stage as early as possible. <P>SOLUTION: The tracking device for tracking and recording movement of a person in a moving picture includes: a detection means which detects, from an image of each of successively input frames, a person and a candidate that is an object which cannot be determined to be the person; a person tracking means which executes tracking processing with the person as tracking object; a candidate tracking means which executes tracking processing with the candidate as tracking object; a storage means which stores tracking information recording position information of the person of the tracking object and tracking information recording position information of the candidate of the tracking object; and an update means which updates the tracking information in the storage means. The update means changes, when a person who is evaluated as the same object as the candidate taken as the tracking object by the candidate tracking means is present among persons detected by the detection means, the tracking information of the candidate stored in the storage means to the tracking information of the person. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for tracking and recording the movement of a person in a moving image.

  A technique for automatically tracking and recording the movements of an unspecified number of people appearing in a moving image using a moving image obtained by a surveillance camera, a camera in a store, or the like has attracted attention. In this type of technology, the human face is generally used as a target for detection and tracking, but when the face is hidden behind an obstacle or another person, or when the face turns sideways or back Has a problem that face detection and tracking becomes difficult.

  As a technique for solving such a problem, the present applicant determines the position of the face from the information around the face (such as the color and shape of clothes) when the face to be tracked becomes invisible during the tracking process. A method of estimating and continuing tracking using the estimated position is proposed (see Patent Document 1).

  However, since the conventional method focuses on how to keep track of the person being tracked, it cannot deal with the following cases.

Even if a person begins to appear in a moving image, tracking of the person is not always started immediately. This is because if the face is hidden, not facing the front, or small, the face cannot be detected, and the apparatus cannot recognize that a new tracking target person has appeared in the moving image. In such a case, since the tracking process starts only after the person's face is clearly recognized, the motion information from the appearance of the person to the successful face detection cannot be captured. In particular, in an image in which a large number of persons are photographed, there is a high possibility that face hiding will occur, and such lack of motion information becomes significant. In addition, if the detection of face detection is made sweet, it is possible to detect a face at a certain early stage, but in that case, false detection and mistracking increase, and the reliability of the device itself decreases. Bad effects occur.
JP 2007-42072 A

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of performing tracking processing of a person in a moving image with high reliability from the earliest possible stage. It is in.

  In order to achieve the above object, the present invention employs the following means or processing.

  The tracking device according to the present invention is a tracking device that tracks and records the movement of a person in a moving image, and from each frame image that is sequentially input, a person and a candidate that cannot be determined whether or not the person is Detection means for detecting a person, tracking means for performing tracking processing for tracking a person, candidate tracking means for executing tracking processing for tracking a candidate, and position information of a person to be tracked are recorded Storage means for storing the tracking information recorded and the tracking information in which the position information of the candidate to be tracked is recorded, and update means for updating the tracking information in the storage means, wherein the update means includes the detection When there is a person who is evaluated as the same target as the candidate tracked by the candidate tracking means among the persons detected by the means, the tracking information of the candidate stored in the storage means The person To change between the tracking information.

  According to this configuration, not only an object that is clearly identified as a person ("person"), but also a gray zone object ("candidate") that is difficult to determine whether it is a person due to a hidden face or the like. The tracking process is started and tracking information is recorded. Then, when the “candidate” is tracked as “candidate”, when the “candidate” is found to be “person”, the information recorded as the “candidate” tracking information becomes the tracking information of “person”. And changed. Thereafter, the tracking process is continued as “person”. Therefore, it is possible to track and record the movement of the person from the time when it is difficult to determine whether or not the person is a person. Something other than a person is detected as a “candidate” and tracking processing may be performed, but in that case, since “candidate” does not change to “person”, the tracking information of “person” Reliability is not reduced. That is, according to the configuration of the present invention, it is possible to simultaneously satisfy the conflicting purpose of starting the tracking process from the earliest possible stage and suppressing the decrease in reliability due to erroneous detection or mistracking. .

  The update means, when there is a person evaluated as the same target as the person tracked by the person tracking means among the persons detected by the detection means from the image of the current frame, It is preferable to add the position information of the person in the current frame to the tracking information of the storage means.

  Conventionally, when the tracking in the current frame is successful, the tracking information of the person is generally updated. In contrast, in the present invention, both tracking processing and detection processing are performed on the image of the current frame, and when both tracking and detection are successful for the tracking target person, the tracking information of the person is updated. The structure to be adopted is adopted. Therefore, it is possible to provide a highly reliable tracking result with less mistracking than in the past.

  In addition, the updating unit may include a person or a candidate evaluated as the same target as the candidate to be tracked by the candidate tracking unit among the persons and candidates detected by the detection unit from the image of the current frame. In the case where there is, it is preferable to add the position information of the candidate in the current frame to the tracking information of the storage means.

  Thus, also in the case of tracking a candidate, if both tracking processing and detection processing are performed on the image of the current frame, and both tracking and detection are successful for the candidate, tracking information of the candidate is displayed. By adopting a configuration for updating, it is possible to provide a highly reliable tracking result with less mistracking.

  Preferably, the update means obtains position information to be added to the tracking information of the storage means using the position detected by the detection means.

  There may be a slight difference between the position (tracking position) of the person (or candidate) specified by the tracking process and the position (detection position) of the person (or candidate) specified by the detection process. In general, the detection position can be expected to be more accurate than the tracking position. Therefore, the reliability of the tracking result can be improved by determining the position information to be recorded in the tracking information using the detected position as in the present invention.

  The detection means includes a plurality of types of detectors that output the detected object together with the reliability of the detection result, and the detected object is detected when the reliability of the detection result of at least one of the detectors is equal to or greater than a threshold value. It is preferable to determine that the detected object is a candidate when the reliability of detection results of all detectors is smaller than a threshold.

  By using a plurality of types of detectors, it can be expected that an object that may be a person can be detected without omission.

  It is preferable that the detection means sequentially execute the plurality of types of detectors one by one and stop the detection process when a detection result having a reliability equal to or higher than a threshold is obtained.

  As a result, the number of executions of the detector can be reduced as much as possible, and the time required for the detection process can be shortened.

  The plurality of types of detectors include a detector that detects a head and a detector that detects a face, and the detector that detects the head is executed before the detector that detects the face. It is preferable.

  By performing head detection with a shorter processing time than face detection first, it is possible to omit face detection processing for those that can be determined as a person only from the result of head detection, thereby reducing the time required for detection processing. It becomes possible.

  Preferably, the candidate tracking unit stops the tracking process of the candidate after a predetermined period from a frame in which the candidate is first detected.

  In other words, if the change from “candidate” to “person” is not performed during a predetermined period, the “candidate” is regarded as something other than a person and is excluded from the tracking target. Thereby, useless tracking processing can be reduced and processing time can be shortened.

  The present invention may be regarded as a tracking device or a tracking system including at least a part of the above means, a tracking method including at least a part of the above processing, or a program for causing a computer to execute the tracking method, or a program thereof It can also be understood as a computer-readable recording medium on which the program is recorded. Any combination of the above means and the above processing can be taken as much as possible.

  For example, the tracking method according to the present invention is a tracking method for tracking and recording the movement of a person in a moving image, and the computer cannot determine whether a person is a person or not from the first frame image. A candidate is detected; a person detected in the image of the first frame is set as a tracking target; a tracking process is performed on an image of a second frame after the first frame; and the first The candidate detected in the image of the frame is set as the tracking target, the tracking process is performed on the image of the second frame, the tracking information including the position information of the person who is the tracking target, and the tracking target Recording tracking information including candidate position information in a storage device, detecting a person from the second frame image, and detecting a person from the second frame image, A step of changing the tracking information of the candidate stored in the storage device to the tracking information of the person when there is a person evaluated as the same target as the tracking target in the second frame; , Is executed.

  According to the present invention, tracking processing of a person in a moving image can be executed with high reliability from the earliest possible stage.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings.

(overall structure)
FIG. 1 is a block diagram showing a functional configuration of a tracking device according to an embodiment of the present invention. First, the functional configuration of the tracking device will be described with reference to FIG.

  This tracking device is a device for capturing a moving image (input image) from a camera or a recording medium (both not shown) and automatically tracking and recording the movement of an unspecified person appearing in the moving image. When a plurality of persons are captured in the moving image, an ID is assigned to each person, and movement information of each person is individually tracked and recorded. Such a tracking device can be applied to various applications such as a device that analyzes the movement and number of intruders and visitors from the video of the surveillance camera, and a device that analyzes the flow of the customer from the video of the in-store camera. is there.

  As shown in FIG. 1, the tracking device includes a detection processing unit 10, a tracking processing unit 20, an integrated processing unit 30, and a tracking information database 40 as its functions. Although not shown, it is also preferable to provide a camera for capturing a moving image, a display unit for outputting the moving image and tracking results, and the like. These functional elements are realized by an arithmetic processing unit (computer) executing software (program) and controlling hardware resources such as a camera, a memory, and a display as necessary. However, at least some of these functional elements may be configured by a dedicated chip.

(Detection processing unit)
The detection processing unit 10 is a function (detection means) that detects “person” and “candidate” from images of respective frames that are sequentially input. Here, “person” refers to an object detected by the detection processing unit 10 that is determined to be a person (human), and “candidate” refers to an object that cannot be determined whether or not it is a person. .

  The detection processing unit 10 includes three types of detectors: a head detector 11, a face detector 12, and an upper body detector 13. These detectors 11 to 13 have a function of performing person detection processing using different algorithms (or feature amounts). Each detector detects all objects (regions) that may be a person from the image, and outputs the detected objects together with the reliability of the detection result. The reliability is a score representing the probability (likeness of person) that the detected object is a person, and indicates that the higher the reliability, the higher the possibility of being a person. In this embodiment, when a plurality of types of detectors 11 to 13 are used and a detection result with high reliability is obtained with at least one detector, it is determined as “person” and the reliability of all detectors is low. Are determined as “candidates”. As a result, it is possible to detect an object that may be a person without omission.

  Here, the three detectors 11 to 13 are not executed simultaneously (in parallel), but one in the order of the head detector 11 → the face detector 12 → the upper body detector 13 as shown in FIG. It is executed one by one. If any one of the detectors is determined to be “person”, the detection processing is stopped at that time, and the processing of the remaining detectors is omitted. Thereby, the number of executions of the detector is reduced as much as possible, and the time required for the detection process is shortened.

  The processing order of the detectors is determined in consideration of the characteristics of each detector. In a situation where the human body is hidden due to congestion or a change in the orientation of the face, the most easily detected part of the person is the head. For example, it is difficult to detect a face when wearing a hat or sunglasses, when the face is facing sideways, or when the facial organ is not clearly shown due to lighting conditions. In addition, when the human bodies overlap, the upper body may not be detected. However, even such an image can be detected if the outline of the head is clearly visible. The head detection algorithm is relatively simple and has the advantage of a short processing time. Therefore, in the present embodiment, first, head detection processing by the head detector 11 is executed. When the reliability obtained by the head detection is sufficiently high, it is determined that the person is a “person” only from the result of the head detection. However, since head detection tends to erroneously detect an elliptical object or the like as a head, it is preferable to set a high threshold for determining whether or not it is a “person”.

The face detector 12 is a function for detecting whether or not the face is a feature using the features of the facial organs. When the face is clearly shown in the image, the detection error is the smallest among the three detectors. There are advantages. However, secondary detection that is performed only when the reliability of head detection is low due to the fact that processing time is longer than head detection and that it is easily affected by conditions such as lighting and face orientation. Use as a container.

  The upper body detector 13 has a function of detecting whether or not the human body is focused on the outline of the upper body. Similar to head detection, the algorithm is relatively simple and has the advantage of a short processing time. Even when the size of a person in the image is small, or when it is difficult to discriminate a face due to a hat, face orientation, lighting conditions, or the like, the upper body can often be detected. Therefore, in the present embodiment, the upper body detector 13 is used as a tertiary detector that is executed when the reliability of face detection is low.

  Note that the type (algorithm), number, and order of detectors to be used are not limited to those shown in this embodiment, and may be changed as appropriate. For example, only two types of head detectors and face detectors may be used, or four or more types of detectors may be combined. If the detection accuracy is prioritized over the processing time, all detectors are executed regardless of the reliability, and the result of the detection is combined to determine whether the person is a candidate. Also good.

  For a specific algorithm of the detector and a method for creating the detector, a conventionally known technique can be used. For example, using a large number of sample images (in the case of a head detector, prepare an image that shows the head and an image that does not show the head) and use the Boosting method such as Real AdaBoost to calculate the feature amount. A detector is created by learning the trend. As the feature amount, for example, any type of feature amount such as a HoG (Histogram of Gradient) feature amount, a Sparse feature amount, or a Haar feature amount can be used. In addition to the boosting method, the learning means can also use an SVM (support vector machine), a neural network, or the like.

  The detection result of the detection processing unit 10 is recorded in the detection information DB 41 in the tracking information DB 40. The detection result includes the position and size in the image, reliability, and a person flag (ON for a person, OFF for a candidate) for each detected object.

(Tracking processing part)
The tracking processing unit 20 includes a person tracking unit 21 (person tracking unit) that performs tracking processing for “person” as a tracking target, and a candidate tracking unit 22 (candidate tracking) that performs tracking processing for “candidate” as a tracking target. Means).

  Information of the person to be tracked by the person tracker 21 (tracked person information) is recorded in the tracked person information DB 42 in the tracking information DB 40. The tracked person information includes the ID of the tracked object, the position and size in the image (a movement history corresponding to the number of frames that are being tracked is accumulated), a feature amount, and a person flag.

  The person tracker 21 performs tracking processing for each tracking target while referring to the tracked person information. The tracking process is a process of searching for an object (region) whose position, size, feature amount, and the like are similar to the tracking target in the previous frame from the image of the current frame. While the tracking process searches for a tracking target using the similarity between frames, the detection process described above is a process of searching the head, face, upper body, etc. from an image of one frame. Is different.

On the other hand, information on candidates to be tracked by the candidate tracker 22 (tracking candidate information) is recorded in the tracking candidate information DB 43 in the tracking information DB 40. This tracking candidate information includes a tracking target ID, a position and size in the image, a feature amount, and a person flag. The tracking target ID is
A unique number is assigned regardless of whether it is a person or a candidate.

  The candidate tracker 22 performs tracking processing for each tracking target while referring to the tracking candidate information. The specific tracking algorithm is the same as that of the person tracker 21 except that the tracking target is different. However, the candidate tracker 22 continues the candidate tracking process only for a predetermined period (for example, 10 frames from the frame in which the candidate is first detected). The reason why such a restriction is provided is to reduce useless tracking processing for things other than a person and shorten processing time. As will be described later, when it is determined that the candidate is a person during the 10-frame period, the process is taken over by the person tracker 21, and the tracking process is continued.

  For a specific algorithm of the tracker and a method for creating the tracker, a conventionally known technique can be used. For example, as a tracking method, particle filter, mean shift, online boosting, or the like can be used. For comparison between frames, a template obtained in the previous frame, color information, a feature amount obtained from a detector, and the like can be used.

(Integrated processing department)
The integration processing unit 30 integrates the detection result of the detection processing unit 10 and the tracking result of the tracking processing unit 20, and updates the contents of the tracking person information DB 42 and the tracking candidate information DB 43 in the tracking information DB 40 (update means). It is. As illustrated in FIG. 1, the integration processing unit 30 includes a detection-person tracking result integration unit 31, a detection-candidate tracking result integration unit 32, and a person-candidate result integration unit 33.

  The main function of the detection-person tracking result integration unit 31 is recorded in the tracking person information DB 42, the function of identifying the “person” detected by the detection processing unit 10 and the “person” being tracked by the person tracker 21. And a function of registering “person” newly detected by the detection processing unit 10 in the tracking person information DB 42. Further, the main function of the detection-candidate tracking result integration unit 32 is the “candidate” being tracked by the object (including both “person” and “candidate”) detected by the detection processing unit 10 and the candidate tracker 22. , A function for updating the tracking candidate information recorded in the tracking candidate information DB 43, and a function for registering “candidate” newly detected by the detection processing unit 10 in the tracking candidate information DB 43. Further, the main function of the person-candidate result integration unit 33 is a function for eliminating duplicate registration in the tracking person information DB 42 and the tracking candidate information DB 43, and the “candidate” being tracked was “person”. In this case, the tracking candidate information is changed to tracking person information. Details of these functions will be described later.

(Example of tracking process)
Now, the flow of the tracking process in the tracking device of the present embodiment will be described with reference to a flowchart and an image example. The following (1) detection process, (2) tracking process, and (3) integration process are executed for each frame.

(1) Detection Processing FIG. 2 is a flowchart showing the flow of detection processing. When an image of a certain frame is input, the head detector 11 first detects the head from the input image (S30). And the process of S32-S39 is performed about each of the detected N heads (S31). In S32, for head nI, head detection reliability Conf1 is compared with threshold value TH1. When the head detection reliability Conf1 is high (S32; NO), the person flag of the head nI is set to ON (S33), and the detection result (position / size in the image, reliability, person flag) is detected. It records in information DB41 (S34).

If the head detection reliability Conf1 is lower than the threshold value TH1 in S32, the face detector 12 detects a face from a predetermined range centered on the head nI (S35). Here, by limiting the face detection range to the vicinity of the head nI, the time required for the face detection process can be shortened. In step S36, the face detection reliability Conf2 is compared with a threshold value TH2. When the reliability Conf2 is high (S36; NO), the person flag is set to ON (S33), and the detection result is recorded in the detection information DB 41 (S34).

  If the face detection reliability Conf2 is lower than the threshold value TH2 in S36, the upper body detector 13 detects the upper body from a predetermined range based on the head nI (S37). Again, the processing time is reduced by limiting the upper body detection range. In S38, the reliability Conf3 of upper body detection is compared with the threshold value TH3. When the reliability Conf3 is high (S38; NO), the person flag is set to ON (S33), and the detection result is recorded in the detection information DB 41 (S34).

  In S38, when the upper body detection reliability Conf3 is lower than the threshold value TH3, the person flag is set to OFF (S39), and the detection result is recorded in the detection information DB 41 (S34).

  Through the above flow, detection results of all “persons” and “candidates” included in the input image are recorded in the detection information DB 41. FIG. 3 shows an example of the input image and the detection result. Two people are shown in this image. The front person A is recognized as “person” (person flag = ON), but the rear person B is “candidate” (person flag = OFF) because a part of the body is hidden. Further, a non-human object C is also detected as a “candidate”.

(2) Tracking Process FIG. 4 is a flowchart showing the flow of the person tracking process. When the image is input, the person tracking device 21 acquires the tracking person information from the tracking person information DB 42 (S50). And the process of S52-S56 is repeated by the number of persons made into the tracking object (S51). In the example of FIG. 3, the number M of tracking targets is 1 (tracking target ID: 001).

  In S52, the person tracker 21 performs the tracking process of the tracking target nJ. Specifically, based on the position, size, and feature amount of the tracking target nJ read from the tracking person information DB 42 (these are information obtained from the image of the previous frame), the image of the current frame The tracking target nJ is searched from the list. If the tracking is successful (S53; YES), the tracking success flag is set to ON (S54), and information such as the position in the current frame and the tracking success flag is output as the tracking result (S56). On the other hand, when tracking fails (S53; NO), the tracking success flag is set to OFF (S55), and the tracking result is output (S56). The tracking result output here is used for the integration process of the detection-person tracking result integration unit 31 described later.

  FIG. 5 is a flowchart showing the flow of candidate tracking processing. When the image is input, the candidate tracker 22 acquires the tracking candidate information from the tracking candidate information DB 43 (S60). And the process of S62-S67 is repeated by the number of candidates made into the tracking object (S61). In the example of FIG. 3, the number P of tracking targets is 2 (tracking target ID: 002, 003).

  In S62, the candidate tracker 22 checks the number of tracking of the tracking target nK. The number of times of tracking can be easily checked from the number of history of positions recorded in the tracking candidate information, for example. If the number of times of tracking has reached 10 (S62; NO), the candidate tracker 22 does not execute the tracking process and sets the tracking success flag of the tracking target nK to OFF (S66). This limits candidate tracking to a period of at most 10 frames.

In S63, the candidate tracker 22 performs the tracking process of the tracking target nK. Specifically, based on the position, size, and feature amount of the tracking target nK read from the tracking candidate information DB 43 (these are information obtained from the image of the previous frame), the image of the current frame Search the tracking target nK. If the tracking is successful (S64; YES), the tracking success flag is set to ON (S65), and the tracking result (position in the current frame, tracking success flag, etc.) is output (S67). On the other hand, if tracking fails (S64; NO), the tracking success flag is set to OFF (S66), and the tracking result is output (S67). The tracking result output here is used for the integration process of the detection-candidate tracking result integration unit 32 described later.

  With the above flow, the tracking processing is executed for all “persons” (tracking target ID: 001) and “candidates” (tracking target ID: 002, 003) that are tracked. Note that the processing in FIG. 4 and the processing in FIG. 5 may be executed in parallel or in order.

(3) Integration Processing FIG. 6 is a flowchart showing the flow of detection-person tracking result integration processing.

  Upon receiving the person tracking result (see S56 in FIG. 4) output from the person tracker 21, the detection-person tracking result integration unit 31 checks the tracking success flag of each tracking target, and the tracking target whose tracking success flag is OFF. Is deleted from the tracking person information DB 42 (S71).

  Next, the detection-person tracking result integration unit 31 acquires a detection result from the detection information DB 41 (S72). And the process of S75-S78 is repeated with respect to the detection result (namely, each of the "person" detected by the detection process part 10) of a person flag (S73, S74).

  In S75, the detected person nI and the tracking target (person) whose tracking success flag is ON are identified. For example, if the distance between the position of the person nI specified by the detection process and the position of the tracking target specified by the person tracking process is smaller than a predetermined threshold, it is sufficient to determine that both are the same target. . Further, not only the position but also size comparison and feature amount comparison may be performed.

  When the same target exists (S76; YES), the detection-person tracking result integration unit 31 adds the position information of the tracking target in the current frame to the tracking person information DB 42 (S77). At this time, instead of directly recording the position specified by person tracking (tracking position), it is preferable to obtain position information to be recorded in the DB using the position specified by the detection process (detection position). Specifically, the detection position may be recorded, or the average (intermediate point) of the detection position and the tracking position may be recorded. In general, there is a slight difference between the tracking position and the detection position, but the detection position can be expected to be more accurate than the tracking position. By determining (or correcting) the position of the tracking target using the detection position as in this embodiment, the success rate of tracking and the reliability of the tracking result can be improved.

  On the other hand, when the same target does not exist (S76; NO), the detection-person tracking result integration unit 31 assigns a new tracking target ID to the person nI and newly registers it in the tracking person information DB 42 (S78). ).

  FIG. 7 is a flowchart showing a flow of detection-candidate tracking result integration processing.

  Upon receiving the candidate tracking result (see S67 in FIG. 5) output from the candidate tracker 22, the detection-candidate tracking result integration unit 32 checks the tracking success flag of each tracking target, and the tracking target whose tracking success flag is OFF. Is deleted from the tracking candidate information DB 43 (S81).

  Next, the detection-candidate tracking result integration unit 32 acquires a detection result from the detection information DB 41 (S82). And the process of S84-S88 is repeated with respect to all the detection results (namely, each of the "person" and "candidate" detected by the detection process part 10) (S83).

  In S84, the detected target (person or candidate) nI and the tracking target (candidate) whose tracking success flag is ON are identified. Also in this case, as in S75 of FIG. 6, it is only necessary to determine whether or not they are the same target by comparing the position, size, feature amount, and the like.

  When the same target exists (S85; YES), the detection-candidate tracking result integration unit 32 adds the position information of the tracking target in the current frame to the tracking candidate information DB 43 (S86). Also in this case, the tracking position is not recorded as it is, but the detection position or the average of the detection position and the tracking position may be recorded. Furthermore, the detection-candidate tracking result integration unit 32 also updates the person flag in the tracking candidate information DB 43 in accordance with the person flag of the target nI. Here, the fact that the person nI person flag is ON means that the object that has been tracked as a “candidate” until now has been found to be “person”. Such a case may occur, for example, when the positional relationship of the person changes from the state shown in FIG. 3 to the state shown in FIG. 8 so that the face or upper body of the person behind can be clearly recognized.

  On the other hand, when the same target does not exist (S85; NO), the detection-candidate tracking result integration unit 32 checks the person flag of the target nI (S87), and when the person flag is OFF, a new tracking target ID Is newly registered in the tracking candidate information DB 43 (S88).

  FIG. 9 is a flowchart showing the flow of the person-candidate result integration process.

  The person-candidate result integration unit 33 acquires information about each person that is a tracking target from the tracking person information DB 42 (S100), and acquires information about each candidate that is a tracking target from the tracking candidate information DB 43 (S100). S101). And the process of S103-S106 is repeated about each candidate nK (S102).

  In S103, identification of the candidate nK and the person to be tracked acquired in S100 is performed. In this case as well, as in S75 of FIG. 6, it is only necessary to determine whether or not they are the same target by comparing the position, size, feature amount, and the like.

  When the same person as the candidate nK exists in the tracking person information DB 42 (S104; YES), the information on the candidate nK is deleted from the tracking candidate information DB 43 (S105). Such cases include, for example, when the same person is registered in both databases due to false detection or tracking, or when two people approach each other and one person is completely behind the other. This can happen if you're hiding in

  On the other hand, in S106, the person-candidate result integration unit 33 checks the person flag of the candidate nK. When the candidate nK person flag is ON, that is, in the case of a “candidate” that is found to be “person” in S84 to S86 in FIG. It moves to DB42 (S107). Thereby, the information tracked and recorded as a candidate is changed into the tracking information of a person (refer FIG. 8). For the candidate nK, the tracking processing is continued as “person” from the processing of the next frame.

  Note that the process of FIG. 6 and the process of FIG. 7 may be executed in parallel or in order. The processing in FIG. 9 is executed after the processing in FIG. 6 and the processing in FIG. 7 are completed.

(Advantages of this device)
According to the above configuration, not only an object that is clearly identified as a person ("person") but also a gray zone object ("candidate") that is difficult to determine whether it is a person due to a hidden face or the like. The tracking process is started and tracking information is recorded. Then, when the “candidate” is tracked as “candidate”, when the “candidate” is found to be “person”, the information recorded as the “candidate” tracking information becomes the tracking information of “person”. And changed. Thereafter, the tracking process is continued as “person”. Therefore, it is possible to track and record the movement of the person from the time when it is difficult to determine whether or not the person is a person. Something other than a person is detected as a “candidate” and tracking processing may be performed, but in that case, since “candidate” does not change to “person”, the tracking information of “person” Reliability is not reduced. That is, according to the configuration of the present apparatus, it is possible to simultaneously satisfy the conflicting purpose of starting the tracking process as early as possible and suppressing the decrease in reliability due to erroneous detection and tracking. .

  In addition, the detection process and the tracking process are executed for each frame, and only when both the detection and the tracking are successful, the tracking information of the tracking target is updated. Results can be provided.

  The above configuration is only a specific example of the present invention. The scope of the present invention is not limited to the above configuration, and various modifications are possible within the scope of the technical idea.

FIG. 1 is a block diagram showing a functional configuration of a tracking device according to an embodiment of the present invention. FIG. 2 is a flowchart showing the flow of detection processing. FIG. 3 is a diagram illustrating examples of input images, detection results, tracking person information, and tracking candidate information. FIG. 4 is a flowchart showing the flow of the person tracking process. FIG. 5 is a flowchart showing the flow of candidate tracking processing. FIG. 6 is a flowchart showing the flow of detection-person tracking result integration processing. FIG. 7 is a flowchart showing the flow of detection-candidate tracking result integration processing. FIG. 8 is a diagram illustrating examples of input images, detection results, tracking person information, and tracking candidate information. FIG. 9 is a flowchart showing the flow of person-candidate result integration processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Detection process part 11 Head detector 12 Face detector 13 Upper body detector 20 Tracking process part 21 Person tracker 22 Candidate tracker 30 Integrated process part 31 Detection-person tracking result integration part 32 Detection-candidate tracking result integration part 33 Person-candidate result integration unit 33 candidate result integration 40 tracking information DB
41 Detection information DB
42 Tracking person information DB
43 Tracking candidate information DB

Claims (10)

A tracking device that tracks and records the movement of a person in a moving image,
Detecting means for detecting a person and a candidate that cannot be determined whether or not a person from images of each frame sequentially input;
A person tracking means for performing a tracking process for tracking a person,
Candidate tracking means for executing a tracking process for tracking candidates,
Storage means for storing tracking information that records position information of a person who is a tracking target, and tracking information that records position information of a candidate that is a tracking target;
Updating means for updating the tracking information of the storage means,
The update means is stored in the storage means when a person evaluated by the candidate tracking means as the same target as the candidate tracked by the candidate tracking means exists among the persons detected by the detection means. A tracking device, wherein the tracking information of the candidate is changed to tracking information of a person.   The update means, when there is a person evaluated as the same target as the person tracked by the person tracking means among the persons detected by the detection means from the image of the current frame, The tracking apparatus according to claim 1, wherein position information of the person in the current frame is added to tracking information in the storage unit.   The update means includes a person or a candidate that is evaluated as the same target as the candidate to be tracked by the candidate tracking means among the persons and candidates detected by the detection means from the image of the current frame. 3. The tracking device according to claim 1, wherein, in a case where the candidate is located, position information of the candidate in the current frame is added to the tracking information of the storage unit.   The tracking device according to claim 2, wherein the update unit obtains position information to be added to the tracking information of the storage unit, using the position detected by the detection unit.   The detection means includes a plurality of types of detectors that output the detected object together with the reliability of the detection result, and the detected object is detected when the reliability of the detection result of at least one of the detectors is equal to or greater than a threshold value. 5 is determined to be a person, and when the reliability of detection results of all detectors is smaller than a threshold value, it is determined that the detected target is a candidate. The tracking device according to item.   The detection unit according to claim 5, wherein the detection unit sequentially executes the plurality of types of detectors one by one, and stops the detection process when a detection result having a reliability equal to or higher than a threshold is obtained. Tracking device.   The plurality of types of detectors include a detector that detects a head and a detector that detects a face, and the detector that detects the head is executed before the detector that detects the face. The tracking device according to claim 6.   The tracking apparatus according to claim 1, wherein the candidate tracking unit stops tracking the candidate after a predetermined period from a frame in which the candidate is first detected. A tracking method for tracking and recording the movement of a person in a moving image,
Computer
Detecting from a first frame image a person and a candidate that is a subject that cannot be determined whether or not it is a person;
Performing tracking on a person detected in the image of the first frame as a tracking target and performing an image on a second frame after the first frame;
Subjecting the candidate detected in the first frame image to a tracking target and executing a tracking process on the second frame image;
Recording tracking information including position information of a person to be tracked and tracking information including position information of a candidate to be tracked in a storage device;
Detecting a person from the image of the second frame;
If there is a person who is evaluated as the same target as the tracking target in the second frame among the persons detected from the image of the second frame, the person stored in the storage device Changing candidate tracking information to person tracking information;
The tracking method characterized by performing.   The program for making a computer perform each step of the tracking method of Claim 9.

Images