JP2013090063A - Monitoring device, monitoring method, and monitoring program - Google Patents

Abstract

A monitoring device, a monitoring method, and a monitoring program are provided that can more reliably image a person who needs to be the target of the most enlarged image.
A behavior evaluation value N of a person included in a first image captured by a first imaging unit 1 is calculated, and a second imaging unit 2 that captures a second image based on the calculated behavior evaluation value N. An arithmetic processing unit 20 that determines the imaging target of the image processing unit, and a control unit 23 that adjusts the second imaging unit 2 so that the imaging target determined by the arithmetic processing unit 20 is included in the second image. When the first image is newly picked up by the first image pickup unit 1 in parallel with the image pickup of the second image by the second image pickup unit 2, an action evaluation value N in the newly picked up first image is obtained. The calculation unit is updated again to update the imaging target, and when the imaging target is updated by the arithmetic processing unit 20, the control unit 23 adjusts the second imaging unit 2 so that the updated second image of the imaging target is captured. To do.
[Selection] Figure 3

Description

  The present invention relates to a monitoring device, a monitoring method, and a monitoring program, and more particularly to a monitoring device, a monitoring method, and a monitoring program for monitoring a suspicious person by linking two cameras having different angles of view.

  Conventionally, a wide-angle camera that captures a wide-angle image with a large angle of view, a telephoto camera that can move the shooting direction vertically and horizontally, and that has a high-power zoom lens and captures an enlarged image with a small angle of view; There has been proposed a monitoring apparatus that monitors suspicious persons who have entered the monitoring area in conjunction with each other.

  As such a monitoring device, for example, a first imaging unit that captures a wide-angle image, a second imaging unit that captures an image with a smaller angle of view than the first imaging unit, and a wide-angle imaged by the first imaging unit. A detection unit that detects a plurality of objects from an image, a creation unit that creates history information including position information in a wide-angle image for each detected object, and a plurality of objects based on the created history information There is an object tracking device including a selection unit that selects a tracking target and a drive unit that drives a second imaging unit so as to image the tracking target (Patent Document 1).

JP 2006-033224 A

  However, according to the object tracking device described in Patent Literature 1, since the most suspicious object is selected as a tracking target from a plurality of objects based on history information created for each object, the object tracking device is a tracking target. If another object starts suspicious movement after selecting, the object tracking device cannot select the object as a tracking target. For this reason, according to the object tracking device described in Patent Document 1, when the object is the most suspicious person, there is a problem in that imaging of an object that needs to be captured most of the enlarged image is missed. was there.

  The present invention has been made in order to solve the above-described problems, and an object of the present invention is to provide a monitoring device, a monitoring method, and a monitoring method that can more reliably image a person who needs to be the target of the most magnified image. Is to provide a program.

(1) In order to solve the above-described problem, a monitoring apparatus according to an aspect of the present invention calculates a behavior evaluation value of a person included in a first image captured by a first imaging unit, and calculates the calculated behavior evaluation Based on the value, the arithmetic processing unit for determining the imaging target of the second image captured by the second imaging unit that can be adjusted to a smaller angle of view than the first imaging unit, and the arithmetic processing unit A controller for adjusting the second imaging unit so that the imaging target is included in the second image, and the arithmetic processing unit is in parallel with the imaging of the second image by the second imaging unit. Then, when the first image is newly captured by the first imaging unit, the behavior evaluation value in the first image that has been newly captured is calculated again, and the imaging target is determined based on the calculated behavior evaluation value. The control unit is updated by the arithmetic processing unit. When the imaging target is updated, the second image is updated in the imaging target adjusting the second image pickup unit so as to be imaged.

  With such a configuration, even after the arithmetic processing unit determines the imaging target, the imaging target is updated in parallel with the imaging by the second imaging unit based on the new imaging by the first imaging unit. For example, even when a person different from the person imaged by the second imaging unit newly performs a suspicious action, the person who newly performed the suspicious action can be set as an imaging target. Therefore, it is possible to more reliably capture the person most required to be the imaging target of the second image.

(2) More preferably, the control unit adjusts the second imaging unit so that a second image obtained by enlarging a part of the imaging target is captured by the second imaging unit.

  With such a configuration, for example, when a crime occurs, it is possible to grasp the face of a suspicious person or an object held by a hand, so that a large amount of evidence can be collected.

(3) Preferably, the arithmetic processing unit performs person determination as to whether or not the subject included in the first image is a person based on the first image picked up by the first image pickup unit. The behavior evaluation value of the subject determined to be is calculated, and when the control unit determines that the person determination cannot be performed by the arithmetic processing unit, the second imaging unit determines the subject. The second imaging unit is adjusted so that the second image of the subject is captured, and the arithmetic processing unit performs the person determination of the subject based on the second image of the subject captured by the second imaging unit. Do it again.

  With such a configuration, it is possible to prevent erroneous detection in which an action evaluation value is calculated for an object that is not a person, or non-detection in which an action evaluation value is not calculated despite being a person.

(4) Preferably, the said control part adjusts the said illuminating device so that near infrared rays may be irradiated to a monitoring object area | region by an illuminating device.

  With such a configuration, it is possible to prevent light damage caused by unnecessary light. In addition, stable monitoring can be performed constantly even at night. Furthermore, stable monitoring can be performed constantly without being greatly affected by changes in sunlight or backlight.

(5) More preferably, when the second image is picked up by the second image pickup unit, the control unit controls the lighting device when the periphery of the image pickup target is dark enough to require illumination. By doing so, the near infrared rays irradiated to the imaging object from the illumination device are switched to visible rays.

  With such a configuration, a clear second image can be captured even when the periphery of the imaging target is dark. Further, a warning by light can be given to the imaging target, and crimes can be prevented in advance.

(6) Preferably, the monitoring apparatus further includes a recording unit for recording the second image captured by the second imaging unit, and the arithmetic processing unit corresponds to the determined imaging target. When the behavior evaluation value satisfies a predetermined condition, the second image to be imaged is recorded by controlling the recording unit.

  In this way, the second image can be recorded only when there is a high possibility that the imaging target is a suspicious person, for example, when the behavior evaluation value corresponding to the imaging target is equal to or greater than a predetermined threshold. A suspicious person can be easily identified by checking the displayed image. In addition, since the capacity of the memory that is the storage destination of the second image can be suppressed, the cost can be reduced.

(7) In order to solve the above-described problem, a monitoring method according to an aspect of the present invention calculates a behavior evaluation value of a person included in a first image captured by a first imaging unit, and calculates the calculated behavior evaluation. Based on the value, the step of determining the imaging target of the second image captured by the second imaging unit that can be adjusted to a smaller angle of view than the first imaging unit, and the determined imaging target is included in the second image Adjusting the second imaging unit so that the second imaging unit determines the imaging target in parallel with the imaging of the second image by the second imaging unit by the first imaging unit. When one image is newly captured, the behavior evaluation value in the newly captured first image is calculated again, the imaging target is updated based on the calculated behavior evaluation value, and the second imaging unit is updated. In the adjustment step If at the step of determining the imaging target the imaging object is updated, the second image is updated in the imaging target adjusting the second image pickup unit so as to be imaged.

  With such a configuration, even after the imaging target is determined, the imaging target is updated in parallel with the imaging by the second imaging unit based on the new imaging by the first imaging unit. Even when a person different from the person imaged by the imaging unit newly performs a suspicious action, the person who newly performed the suspicious action can be set as the imaging target, and the second image A person who needs to be imaged can be more reliably imaged.

(8) In order to solve the above-described problem, a monitoring program according to an aspect of the present invention calculates a behavior evaluation value of a person included in a first image captured by a first imaging unit, and calculates the computer. Based on the behavior evaluation value, a step of determining an imaging target of a second image captured by a second imaging unit that can be adjusted to a smaller angle of view than the first imaging unit, and the determined imaging target is a second Adjusting the second imaging unit to be included in the image, and executing the step of determining the imaging target in parallel with the imaging of the second image by the second imaging unit. Then, when the first image is newly captured by the first imaging unit, the behavior evaluation value in the first image that has been newly captured is calculated again, and the imaging target is determined based on the calculated behavior evaluation value. In the step of newly adjusting the second imaging unit, when the imaging target is updated in the step of determining the imaging target, the updated second image of the imaging target is captured. 2 Adjust the imaging unit.

  With such a configuration, even after the imaging target is determined, the imaging target is updated in parallel with the imaging by the second imaging unit based on the new imaging by the first imaging unit. Even when a person different from the person imaged by the imaging unit newly performs a suspicious action, the person who newly performed the suspicious action can be set as the imaging target, and the second image A person who needs to be imaged can be more reliably imaged.

  According to the present invention, it is possible to more reliably image a person who is most required to be a target for capturing an enlarged image.

It is a figure which shows the external appearance of the monitoring apparatus which concerns on embodiment of this invention. (A) is a figure which shows the wide angle image imaged with the wide angle camera with which the monitoring apparatus which concerns on embodiment of this invention was equipped, (b) is equipped with the monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows the enlarged image imaged with the obtained PTZ camera. It is a block diagram which shows the structure of the monitoring apparatus which concerns on embodiment of this invention. It is a figure explaining the change of the largest action evaluation value, and the operation | movement of the determination part, control part, recording part, and communication part accompanying this change. It is a flowchart which shows the procedure of the monitoring operation | movement by the monitoring apparatus which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the human detection tracking process by the human detection tracking part of the monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows the wide-angle image by a wide-angle camera in the time t and the time t + 1. (A) is a table | surface which shows the action evaluation value computed by the determination part of the monitoring apparatus which concerns on this Embodiment in the time t, (b) is the monitoring apparatus which concerns on this Embodiment in the time t + 1. It is a table | surface which shows the action evaluation value computed by the determination part. It is a flowchart which shows the procedure of the expansion tracking process by the control part of the monitoring apparatus which concerns on embodiment of this invention.

[Configuration and basic operation]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

  FIG. 1 is a diagram illustrating an appearance of a monitoring device 100 according to an embodiment of the present invention, and FIG. 2A is an image captured by a wide-angle camera 1 provided in the monitoring device 100 according to an embodiment of the present invention. FIG. 2B is a diagram illustrating an enlarged image captured by the PTZ camera 2 provided in the monitoring apparatus 100 according to the embodiment of the present invention. FIG. 3 is a block diagram showing the configuration of the monitoring apparatus 100 according to the embodiment of the present invention. FIG. 4 shows a change in the maximum action evaluation value Nmax, and a determination unit 22 and a control unit associated with this change. 23, operations of the recording unit 24 and the communication unit 25.

(Monitoring device structure)
Referring to FIG. 1, the monitoring device 100 includes a monitoring processing unit 10, a wide-angle camera 1 (first imaging unit), and a PTZ (pan / tilt / zoom) camera 2 (second imaging unit). Yes.

  With reference to FIG. 2A, the wide-angle camera 1 always captures the entire monitoring region to be monitored with a wide-angle image (first image). The wide-angle camera 1 is sensitive to light from near infrared to visible light.

  The PTZ camera 2 can move the shooting direction vertically and horizontally, and can adjust the angle of view. Referring to FIG. 2B, the PTZ camera 2 is adjusted to a smaller angle of view than the wide-angle camera 1 by the monitoring processing unit 10, and a specific area of the monitoring area is captured by an enlarged image (second image). To do. Specifically, the PTZ camera 2 normally images a preset place such as a place where a person frequently enters and exits in the monitoring area, but the monitoring processing unit 10 sets a specific person as an imaging target, for example. When adjustment is performed so as to capture an enlarged image of the imaging target, the imaging direction is moved so as to capture an enlarged image of the specific person, and imaging is performed.

  Further, the PTZ camera 2 has at least sensitivity to visible light, and preferably has sensitivity to light from near infrared rays to visible light as in the wide-angle camera 1.

  The wide-angle camera 1 and the PTZ camera 2 capture images in parallel. The wide-angle camera 1 and the PTZ camera 2 may be separated from the monitoring device 100 and externally attached to the monitoring device 100. Further, instead of the wide-angle camera 1, a PTZ camera having a larger maximum angle of view than the PTZ camera 2 can be used.

  Referring to FIG. 3, monitoring device 100 further includes an external monitor 3 and a lighting device 4. The external monitor 3 displays a wide-angle image captured by the wide-angle camera 1 and an enlarged image captured by the PTZ camera 2. For example, the external monitor 3 displays a wide-angle image captured by the wide-angle camera 1 and an enlarged image captured by the PTZ camera 2 on one screen.

  The illuminating device 4 is controlled by the monitoring processing unit 10 and normally irradiates the entire monitoring target region with near infrared rays. Further, the illumination device 4 is a case where the PTZ camera 2 is controlled by the monitoring processing unit 10 to capture an enlarged image of a specific imaging target. Is dark, it is controlled by the monitoring processing unit 10 to irradiate the imaging target with visible light.

(Detailed structure of the monitoring processor)
The monitoring processing unit 10 includes a video input unit 11, an A / D conversion unit 12, a RAM 13, a video input unit 14, an A / D conversion unit 15, an information processing unit 16, a memory 17, and a D / A. A conversion unit 18 and an output unit 19 are provided.

  The video input unit 11 receives an analog signal of a wide-angle image captured by the wide-angle camera 1 and outputs this analog signal to the A / D conversion unit 12. The A / D converter 12 converts the analog signal of the wide-angle image received from the video input unit 11 into a digital signal and outputs it to the RAM 13. In this manner, the wide-angle image digital signals captured by the wide-angle camera 1 are sequentially recorded in the RAM 13 and overwritten.

  The video input unit 14 receives an analog signal of an enlarged image captured by the PTZ camera 2 and outputs the analog signal to the A / D conversion unit 15. The A / D conversion unit 15 converts the analog signal of the enlarged image received from the video input unit 14 into a digital signal and outputs the digital signal to the RAM 13. As described above, the RAM 13 sequentially records and overwrites the digital signals of the enlarged images captured by the PTZ camera 2.

  The D / A converter 18 acquires the digital signals of the wide-angle image and the enlarged image from the RAM 13, converts the digital signals into analog signals, and outputs them to the output unit 19. The output unit 19 outputs analog signals of the wide-angle image and the enlarged image output from the D / A conversion unit 18 to the external monitor 3. As a result, the wide-angle image and the enlarged image are displayed on the external monitor 3 by one screen. Note that the output unit 19 may output an analog signal of either the wide-angle image or the enlarged image to the external monitor 3 so that either the wide-angle image or the enlarged image is displayed on the external monitor 3.

  The information processing unit 16 is, for example, a CPU (Central Processing Unit), and includes an arithmetic processing unit 20, a control unit 23, a recording unit 24, and a communication unit 25. The arithmetic processing unit 20 includes a human detection tracking unit 21 and a determination unit 22.

(A) Human detection and tracking unit The human detection and tracking unit 21 acquires a digital signal of a wide-angle image recorded in the RAM 13, and based on the wide-angle image, for example, a background subtraction method is applied to one or more subjects included in the wide-angle image. By using it, it is determined whether or not it is a person.

  Specifically, first, the human detection tracking unit 21 acquires the luminance value of each pixel in the wide-angle image based on the digital signal of the wide-angle image, and the acquired luminance value of each pixel and a person created in advance are obtained. A background difference value D1 (i, j), which is an absolute value of the difference between the luminance value of each pixel in the image of the monitoring area that is not shown (hereinafter referred to as “background image”), is calculated. The human detection tracking unit 21 then binarizes “1” or “0” for the wide-angle image based on the background difference value D1 (i, j). Then, the human detection tracking unit 21 reads a plurality of binarized wide-angle images using a frame approximately the size of a person as a detection frame, and a ratio P1 where “1” is given in the read detection frame is a predetermined value. If the value is equal to or greater than Pa, it is determined that a person is shown in this detection frame.

  If the person detection tracking unit 21 cannot clearly distinguish whether the subject in the detection frame is a person and determines that the person cannot be determined, an enlarged image of the detection frame is displayed. A control signal is output to the control unit 23 so that an image is taken. For example, the human detection tracking unit 21 determines whether or not the ratio P1 to which “1” is given in the read detection frame is less than the predetermined value Pa and greater than or equal to the predetermined value Pb. However, Pa> Pb. When the human detection tracking unit 21 is less than the predetermined value Pa and greater than or equal to the predetermined value Pb, the human detection tracking unit 21 determines that the subject in the detection frame cannot be clearly distinguished as a person, and performs control. A control signal is output to the unit 23.

  Upon receiving the control signal, the control unit 23 adjusts the PTZ camera 2, and the PTZ camera 2 captures an enlarged image of the detection frame. The analog signal of the enlarged image is output to the A / D conversion unit 15 via the video input unit 14 and converted into a digital signal by the A / D conversion unit 15. The digital signal of the enlarged image is recorded in the RAM 13. Then, the human detection tracking unit 21 acquires a digital signal of the enlarged image recorded in the RAM 13, and performs person determination for the subject in the detection frame based on the enlarged image.

  The human detection tracking unit 21 tracks a detection frame in which it is determined that a person is captured using, for example, a template matching method.

  Specifically, the human detection tracking unit 21 uses, as a template, a detection frame that is determined to include a person in the enlarged image at time t. Then, the human detection tracking unit 21 reads out a plurality of detection frames having the same size as the template from the enlarged image at time t + 1. Then, the human detection tracking unit 21 calculates the sum of the absolute values of the differences between the luminance values of the pixels in the template and the luminance values of the pixels in the read detection frame, and moves the person in the detection frame having the minimum sum. Specify as a destination. In this way, the person detection tracking unit 21 specifies the movement destination of the person in the wide-angle image, tracks the detection frame in which the person is captured, and outputs the tracking result to the determination unit 22.

  In addition, when the wide-angle camera 1 captures a new wide-angle image, the human detection tracking unit 21 detects a person included in the newly-captured wide-angle image and performs detection of a person included in the newly captured wide-angle image. Do tracking.

(B) Determination Unit The determination unit 22 determines a person included in the wide-angle image based on the wide-angle image captured by the wide-angle camera 1, that is, a subject tracked by the person detection and tracking unit 21 that is determined to be a person. An action evaluation value N is calculated.

  Here, the action evaluation value N is, for example, the movement path of a person, the position of a person with respect to a place where a protection target such as an important cultural property exists, a trap at a specific place, a stay for a predetermined time or more, It is a value based on an action such as a moving object and a person's moving speed, and is set in advance.

  For example, when a person stays at the same position for a predetermined time or more and the staying position is close to the place where the important cultural property exists, the action evaluation value N for each action is added up. The

  Note that, as described above, if the person detection tracking unit 21 does not clearly distinguish whether the subject in the detection frame is a person, an enlarged image of the subject is captured, and the person detection tracking unit 21 Distinguishes whether the subject is a person based on the enlarged image. In such a case, the determination unit 22 can also calculate the action evaluation value N of a person based on an enlarged image used for distinguishing whether or not the subject is a person, not a wide-angle image.

  When the person detection tracking unit 21 tracks a plurality of persons, the determination unit 22 calculates the behavior evaluation value N for all the persons being tracked. Then, the determination unit 22 determines a person corresponding to the maximum action evaluation value Nmax among the calculated action evaluation values N as an object to be imaged by the PTZ camera 2. Then, the determination unit 22 outputs the coordinates of the detection frame in which the imaging target is captured and the luminance value of each pixel in the detection frame to the control unit 23.

  In parallel with the imaging of the enlarged image by the PTZ camera 2, a wide-angle image is newly imaged by the wide-angle camera 1, and the person detection and tracking unit 21 detects and tracks a person included in the newly-imaged wide-angle image. When the determination is made, the determination unit 22 calculates again the behavior evaluation value N of the person in the newly captured wide-angle image, and updates the imaging target based on the calculated behavior evaluation value N. That is, since an enlarged image of a specific person has already been captured by the PTZ camera 2, the imaging target may be switched by updating the imaging target of the enlarged image.

  In addition, when the maximum action evaluation value Nmax satisfies a predetermined condition, the determination unit 22 controls the recording unit 24 so that an enlarged image to be captured is recorded. That is, with reference to FIG. 4, the determination unit 22 determines whether or not the maximum action evaluation value Nmax is equal to or greater than a predetermined threshold Th1, and when the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th1. The recording start signal is output to the recording unit 24 on the assumption that the person corresponding to the maximum action evaluation value Nmax is likely to be a suspicious person.

  Further, when the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th1, the determination unit 22 further determines whether or not the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th2. However, Th1 <Th2. When the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th2, the determination unit 22 determines that the person corresponding to the maximum action evaluation value Nmax is more likely to be a suspicious person, and determines the communication start signal. Is output to the communication unit 25.

(C) Control Unit Referring to FIG. 3 again, the control unit 23 captures the imaging target output by the determination unit 22 so that an enlarged image of the person who is the imaging target determined by the determination unit 22 is captured. The PTZ camera 2 is adjusted based on the coordinates of the detected detection frame. In addition, when the determination unit 22 updates the imaging target, the control unit 23 adjusts the PTZ camera 2 so that an enlarged image of the person who is the imaging target after the update is captured.

  At this time, the control unit 23 assumes that the upper periphery of the detection frame in which the imaging target is captured is the position of the face of the person, and further, that the portion where the movement is intense or the waist height is the vicinity of the hand. The PTZ camera 2 is adjusted so that an enlarged image obtained by enlarging the vicinity of a face or a person who is a part of the imaging target is captured by the camera 2. In addition, when the object that the person who is the imaging target is holding in the monitoring area and is away from the object, the control unit 23 causes the PTZ so that an enlarged image of the object away from the imaging target is captured. The camera 2 may be adjusted.

  Further, the control unit 23 does not clearly distinguish whether or not a person is shown in the detection frame in the wide-angle image by the human detection tracking unit 21, and when receiving a control signal from the human detection tracking processing unit 21, The PTZ camera 2 is adjusted so that the camera 2 captures an enlarged image of the subject included in the detection frame.

  Moreover, the control part 23 controls the illuminating device 4 so that near infrared rays may be normally irradiated to the whole monitoring area | region. Further, when the PTZ camera 2 is adjusted by the monitoring processing unit 10 and the PTZ camera 2 captures an enlarged image of a specific imaging target, and the surroundings of the imaging target are dark, The illumination device 4 is controlled so that visible light is irradiated around the imaging target.

  Specifically, the control unit 23 determines whether or not the person who is the imaging target is dark based on the luminance value of each pixel in the detection frame output from the determination unit 22. And the control part 23 controls the illuminating device 4 so that a visible ray may be irradiated to the said person from the illuminating device 4, when the circumference | surroundings of the person who is an imaging target are dark.

  Note that, for example, the monitoring device 100 includes a light detection sensor, and the control unit 23 can determine whether or not the periphery of the imaging target is dark based on the detection result of the light detection sensor.

(D) Recording unit When the recording unit 24 receives a recording start signal from the determining unit 22 at time t1, for example, the recording unit 24 is slightly ahead of the timing at which the recording start signal is received (for example, time t1-1). Going back, an enlarged image recorded in the RAM 13 is acquired (see FIG. 3). Then, the recording unit 24 outputs the enlarged image acquired from the RAM 13 to the memory 17, and the enlarged image is recorded in the memory 17. For example, recording by the recording unit 24 is performed until time t4 when the maximum action evaluation value Nmax becomes less than the predetermined threshold Th1. The memory 17 is a removable hard disk such as an SD memory card, for example.

(E) Communication unit When receiving a communication start signal from the determination unit 22 at the time t2 when the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th2, for example, the communication unit 25 reads an enlarged image recorded in the memory 17 Then, the read enlarged image is output to the communication device 30 such as a smartphone registered in advance (see FIG. 4). For example, the communication unit 25 communicates with the communication device 30 until the time t3 when the maximum action evaluation value Nmax becomes less than the predetermined threshold Th2. Note that the communication unit 25 may output not only the recorded enlarged image but also a text sentence for reporting the suspicious situation to the communication device 30.

[Operation]
Next, the monitoring operation by the monitoring apparatus 100 according to the embodiment of the present invention will be described. FIG. 5 is a flowchart showing the procedure of the monitoring operation by the monitoring apparatus 100 according to the embodiment of the present invention, and FIG. 6 shows the human detection by the human detection tracking unit 21 of the monitoring apparatus 100 according to the embodiment of the present invention. It is a flowchart which shows the procedure of a tracking process.

  The monitoring apparatus 100 reads out a program including each step of the flowchart from a memory (not shown) and executes the program. This program can be installed externally. The installed program is distributed in a state stored in a recording medium, for example.

  Referring to FIG. 5, first, a digital signal that is data of a wide-angle image captured by wide-angle camera 1 is output to RAM 13 (step S1).

  Next, based on the wide-angle image data recorded in the RAM 13, the person detection / tracking unit 21 performs a detection / tracking process for a person included in the wide-angle image (step S2). That is, the person detection tracking unit 21 determines whether or not a person is captured in a plurality of detection frames included in the wide-angle image, and performs tracking for a detection frame that is determined to include a person.

  Specifically, referring to FIG. 6, first, the human detection tracking unit 21 acquires the luminance value of each pixel in the wide-angle image based on the data of the wide-angle image, and the luminance value I (i of the acquired wide-angle image. , J) and the luminance value B (i, j) of the background image, D1 (i, j) = | I (i, j) −B (i, j) |, which is the difference (absolute value), is calculated. (Step S11).

  Next, the human detection tracking unit 21 performs a binarization process of “1” or “0” on the background difference value D1 (i, j) using a predetermined threshold (step S12).

  Next, the human detection tracking unit 21 reads out a predetermined detection frame about the size of a person in the enlarged image, and obtains a ratio P1 where “1” is given in the read detection frame (step S13). ).

  Next, the human detection tracking unit 21 determines whether or not the ratio P1 is greater than or equal to the reference value Pa (step S14). When the ratio P1 is less than the reference value Pa (“No” in step S14), the human detection tracking unit 21 further determines whether the ratio P1 is greater than or equal to the reference value Pb (Pb> Pa). Judgment is made (step S15).

  When the ratio P1 is greater than or equal to the reference value Pb, that is, when the ratio P1 is less than the reference value Pa and greater than or equal to the reference value Pb (“Yes” in step S15), the human detection tracking unit 21 reads If it is unclear whether or not a person is captured in the predetermined detection frame, a control signal is output to the PTZ camera 2 so that an enlarged image of the subject included in the detection frame is captured. Thereby, the PTZ camera 2 captures an enlarged image (step S16). Data of the enlarged image captured by the PTZ camera 2 is recorded in the RAM 13 via the video input unit 14 and the A / D conversion unit 15.

  Next, the human detection tracking unit 21 acquires enlarged image data from the RAM 13 and calculates a background difference value D2 (i, j) for the enlarged image recorded in the RAM 13 in the same manner as in step S11 (step S17). ). Then, the human detection tracking unit 21 performs binarization processing on the background difference value D2 (i, j) as in step S12 (step S18).

  Next, the human detection tracking unit 21 acquires a ratio P2 where “1” is given in the enlarged image (step S19). Next, the person detection tracking unit 21 determines whether or not the ratio P2 is equal to or greater than the reference value Pc (step S20). Then, when the ratio P2 is less than the reference value Pc (“No” in step S20), the human detection tracking unit 21 determines that no person is captured in the predetermined detection frame (step S21).

  On the other hand, when the ratio P1 is greater than or equal to the reference value Pa (“Yes” in step S14) or when the ratio P2 is greater than or equal to the reference value Pc (“Yes” in step S20), the human detection tracking unit 21 Then, it is determined that a person is captured in the predetermined detection frame (step S22).

  Then, the human detection tracking unit 21 determines whether or not the above determination has been made for all detection frames in the enlarged image (step S23), and if there is a detection frame for which no determination has been made (in step S23). “No”), step S13 to step S22 are repeated for the detection frame.

  On the other hand, when the human detection tracking unit 21 determines all the detection frames (“Yes” in step S23), for example, the detection frame in which the person is shown is tracked using the template matching method, and the tracking result is obtained. Is output to the determination unit 22 (step S24).

  Next, referring to FIG. 5 again, the determination unit 22 calculates the action evaluation value N for all the persons tracked by the human detection tracking unit 21 (step S3). And the determination part 22 determines an imaging target based on the calculated action evaluation value N (step S4).

  Here, FIG. 7 is a diagram showing wide-angle images by the wide-angle camera 1 at time t and time t + 1, and FIG. 8A is a determination unit 22 of the monitoring apparatus 100 according to the present embodiment at time t. 8B is a table showing the behavior evaluation value N calculated by the determination unit 22 of the monitoring device 100 according to the present embodiment at time t + 1. is there.

  Referring to FIG. 7, the wide-angle image (frame 1) captured by wide-angle camera 1 at time t includes persons A, B, and C. Further, the wide-angle image (frame 2) captured by the wide-angle camera 1 at time t + 1 includes persons A, B, C, and D.

  Specifically, when the wide-angle camera 1 captures the frame 1 at time t, the determination unit 22 calculates an action evaluation value N for each of the persons A, B, and C, as shown in FIG. Create a table. At this time, since the maximum action evaluation value Nmax is “120” and the corresponding person is “A”, the determination unit 22 determines “A” as an imaging target.

  When the wide-angle camera 1 captures the frame 2 at time t + 1, the determination unit 22 calculates the action evaluation value N for each of the persons A, B, C, and D, and displays a table as shown in FIG. Create At this time, the maximum action evaluation value Nmax is “130”, and the corresponding person is “B”. For this reason, the determination unit 22 updates the imaging target from “A” to “B” and determines “B” as the imaging target.

  Next, when the determination unit 22 determines the imaging target, the control unit 23 starts an enlargement tracking process (step S5 shown in FIG. 7). Here, FIG. 9 is a flowchart showing the procedure of the expansion tracking process by the control unit 23 of the monitoring apparatus 100 according to the embodiment of the present invention.

  Referring to FIG. 9, first, the determination unit 22 outputs the coordinates of the detection frame in which the person determined as the imaging target is captured and the luminance value of each pixel in the detection frame to the control unit 23 (step S31). ).

  Next, the control unit 23 determines whether or not the luminance value around the imaging target is smaller than a predetermined value (step S32). When the control unit 23 determines that the luminance value of the detection frame that is the imaging target is smaller than the predetermined value (“Yes” in step S32), the light emitted by the illumination device 4 is visible from near infrared to visible light. The illumination device 4 is controlled so that visible light is irradiated around the imaging target (step S33).

  Next, the control unit 23 adjusts the PTZ camera 2 based on the coordinates of the detection frame received by the determination unit 22 so that a magnified image to be captured is captured, and the PTZ camera 2 captures the magnified image. (Step S34).

  In this way, the illumination device 4 and the PTZ camera 2 are controlled by the control unit 23, and an enlarged image to be imaged is captured. Here, as described above, when a wide-angle image is newly captured, the determination unit 22 updates the imaging target. For this reason, for example, in the example shown in FIGS. 7 and 8, even when the enlarged image “A” is captured by the PTZ camera 2 at time t, the enlarged image “A” at time t + 1. Imaging is stopped, the imaging target is switched to “B”, and an enlarged image of “B” is captured by the PTZ camera 2.

  The enlarged image data captured by the PTZ camera 2 in this manner is sequentially recorded in the RAM 13 via the video settlement unit 14 and the A / D conversion unit 15.

  Next, referring to FIG. 5 again, the determination unit 22 determines whether or not the maximum action evaluation value Nmax is equal to or greater than a predetermined threshold Th1 (step S6). If the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th1 (“Yes” in step S6), the determination unit 22 outputs a recording start signal to the recording unit 24.

  Next, when the recording unit 24 receives a recording start signal from the determining unit 22, the recording unit 24 acquires an enlarged image to be captured from the RAM 13 and outputs it to the memory 17. Thereby, the enlarged image to be imaged is recorded in the memory 17 (step S7).

  Next, the determination unit 22 determines whether or not the maximum action evaluation value Nmax is equal to or greater than a predetermined threshold Th2 (Th1 <Th2) (step S8). When the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th2 (“Yes” in step S40), the determination unit 22 outputs a communication start signal to the communication unit 25.

  When the communication unit 25 receives a communication start signal from the determination unit 22, the communication unit 25 reads an enlarged image recorded in the memory 17, and reads the read video to the communication device 30 such as a smartphone registered in advance. And the text sentence for telling a suspicious situation is output (step S9).

  On the other hand, when the maximum action evaluation value Nmax is less than the predetermined threshold Th1 (“No” in step S6), it is determined that the person corresponding to the maximum action evaluation value Nmax is unlikely to be a suspicious person. The recording start signal is not output to the recording unit 24 by the unit 22.

  When the maximum action evaluation value Nmax is equal to or greater than the predetermined threshold Th1 and less than the threshold Th2 (Th1 ≦ Nmax <Th2) (“No” in step S8), the person corresponding to the maximum action evaluation value Nmax is determined. Although there is a possibility of being a suspicious person, the determination unit 22 does not output a communication start signal on the assumption that there is a low risk of communication with the communication device 30 at the present time.

  Specifically, in the example illustrated in FIGS. 7 and 8, if the predetermined threshold Th1 is “125” and the predetermined threshold Th2 is “135”, the maximum action evaluation value Nmax is “ 120 ”, the recording start signal is not output to the recording unit 24 by the determination unit 22 and the communication start signal is not output to the communication unit 25. On the other hand, since the maximum action evaluation value Nmax is “130” at time t + 1, the recording start signal is output to the recording unit 24 by the determination unit 22 and the communication start signal is not output to the communication unit 25. .

  By the way, in the technique described in Patent Document 1, it is necessary to select the most magnified image as an imaging target, such as when there is another object that has started suspicious movement after the object tracking device has selected the tracking target. There was a problem of missing the imaging of the object.

  On the other hand, the monitoring apparatus 100 according to the embodiment of the present invention calculates the action evaluation value N of the person included in the wide-angle image captured by the wide-angle camera 1, and based on the calculated action evaluation value N, A determination unit 22 for determining an imaging target of the PTZ camera 2 that captures an enlarged image is provided. In addition, the monitoring apparatus 100 includes a control unit 23 for adjusting the PTZ camera 2 so that the imaging target determined by the determination unit 22 is included in the enlarged image. Further, when the wide-angle image is newly captured by the wide-angle camera 1 in parallel with the capturing of the enlarged image by the PTZ camera 2, the determination unit 22 calculates again the behavior evaluation value N in the newly captured wide-angle image. The imaging target is updated based on the calculated behavior evaluation value N. Further, when the imaging target is updated by the determination unit 22, the control unit 23 adjusts the PTZ camera 2 so that an enlarged image of the updated imaging target is captured.

  With such a configuration, even after the arithmetic processing unit determines the imaging target, the imaging target is updated in parallel with the imaging by the second imaging unit based on the new imaging by the first imaging unit. For example, even when a person different from the person imaged by the second imaging unit newly performs a suspicious action, the person who newly performed the suspicious action can be set as an imaging target. Therefore, it is possible to more reliably capture the person who is most required to be the target of the enlarged image.

  In the monitoring device 100 according to the embodiment of the present invention, the control unit 23 adjusts the PTZ camera 2 so that an enlarged image obtained by enlarging a part of the imaging target is captured by the PTZ camera 2.

  With such a configuration, for example, when a crime occurs, it is possible to grasp the face of a suspicious person or an object held by a hand, so that a large amount of evidence can be collected.

  In the monitoring device 100 according to the embodiment of the present invention, the human detection tracking unit 21 determines whether the subject included in the wide-angle image is a person based on the wide-angle image captured by the wide-angle camera 1. Make a decision. In addition, the determination unit 22 calculates an action evaluation value N of the subject that is determined to be a person by the human detection tracking unit 21. Further, the control unit 23 adjusts the PTZ camera 2 so that the PTZ camera 2 captures an enlarged image of the subject when it is determined that the person detection / tracking unit 21 cannot perform person determination. Then, the human detection tracking unit 21 performs another person determination of the subject based on the enlarged image of the subject captured by the PTZ camera 2.

  With such a configuration, it is possible to prevent erroneous detection in which an action evaluation value is calculated for an object that is not a person, or non-detection in which an action evaluation value is not calculated despite being a person.

  Moreover, in the monitoring apparatus 100 which concerns on embodiment of this invention, the control part 23 controls the illuminating device 4 so that near infrared rays are irradiated to the monitoring object area | region by the illuminating device 4. FIG.

  With such a configuration, it is possible to prevent light damage caused by unnecessary light. In addition, stable monitoring can be performed constantly even at night. Furthermore, stable monitoring can be performed constantly without being greatly affected by changes in sunlight or backlight.

  In the monitoring apparatus 100 according to the embodiment of the present invention, when the enlarged image is captured by the PTZ camera 2, the control unit 23 performs illumination when the surrounding of the imaging target is dark enough to require illumination. By controlling the device 4, the near infrared ray irradiated from the illumination device 4 to the imaging target is switched to visible light.

  With such a configuration, it is possible to capture a clear enlarged image even when the periphery of the imaging target is dark. Further, a warning by light can be given to the imaging target, and crimes can be prevented in advance.

  The monitoring apparatus 100 according to the embodiment of the present invention further includes a recording unit 24 for recording an enlarged image captured by the PTZ camera 2, and the determination unit 22 performs an action corresponding to the determined imaging target. When the evaluation value Nmax satisfies a predetermined condition, the enlarged image to be captured is recorded by controlling the recording unit 24.

  In this way, an enlarged image can be recorded only when there is a high possibility that the imaging target is a suspicious person, for example, when the behavior evaluation value corresponding to the imaging target is greater than or equal to a predetermined threshold, and thus the recorded image is recorded. The suspicious person can be easily identified by confirming the image that is displayed. In addition, since the capacity of the memory that is the storage destination of the enlarged image can be suppressed, the cost can be reduced.

  The above embodiment should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 Wide-angle camera (first imaging unit)
2 PTZ camera (second imaging unit)
3 External monitor 4 Illuminating device 10 Monitoring processing unit 11, 14 Video input unit 12, 15 A / D conversion unit 13 RAM
DESCRIPTION OF SYMBOLS 16 Information processing part 17 Memory 18 D / A conversion part 19 Output part 20 Arithmetic processing part 21 Human detection tracking part 22 Determination part 23 Control part 24 Recording part 25 Communication part 30 Communication apparatus 100 Monitoring apparatus

Claims (8)

The second imaging that can be adjusted to a smaller angle of view than the first imaging unit based on the calculated behavior evaluation value based on the calculated behavior evaluation value of a person included in the first image captured by the first imaging unit. An arithmetic processing unit for determining an imaging target of the second image captured by the unit;
A control unit for adjusting the second imaging unit so that the imaging target determined by the arithmetic processing unit is included in the second image,
When the first image is newly captured by the first image capturing unit in parallel with the second image capturing by the second image capturing unit, the arithmetic processing unit is configured to perform the action on the newly captured first image. Recalculate the evaluation value, update the imaging target based on the calculated behavior evaluation value,
When the imaging target is updated by the arithmetic processing unit, the control unit adjusts the second imaging unit so that the updated second image of the imaging target is captured.   The monitoring apparatus according to claim 1, wherein the control unit adjusts the second imaging unit so that a second image obtained by enlarging a part of the imaging target is captured by the second imaging unit. The arithmetic processing unit determines whether or not the subject included in the first image is a person based on the first image captured by the first imaging unit, and determines that the subject is a person. Calculate the behavior evaluation value of the subject,
When the arithmetic processing unit determines that the person determination cannot be performed, the control unit causes the second imaging unit to capture the second image of the subject by the second imaging unit. Adjust
The monitoring apparatus according to claim 1, wherein the arithmetic processing unit performs the person determination of the subject again based on a second image of the subject imaged by the second imaging unit.   The monitoring device according to any one of claims 1 to 3, wherein the control unit controls the lighting device such that near-infrared rays are irradiated to the monitoring target region by the lighting device.   When the second image is captured by the second imaging unit, the control unit controls the illumination device when the surrounding of the imaging target is dark enough to require illumination, thereby controlling the illumination device. The monitoring apparatus according to claim 4, wherein the near infrared ray irradiated to the imaging target is switched from visible light to visible light. The monitoring device further includes a recording unit for recording the second image captured by the second imaging unit,
The arithmetic processing unit records the second image of the imaging target by controlling the recording unit when the behavior evaluation value corresponding to the determined imaging target satisfies a predetermined condition. The monitoring device according to any one of claims 1 to 5. The second imaging that can be adjusted to a smaller angle of view than the first imaging unit based on the calculated behavior evaluation value based on the calculated behavior evaluation value of a person included in the first image captured by the first imaging unit. Determining an imaging target of the second image captured by the unit;
Adjusting the second imaging unit so that the determined imaging target is included in the second image,
In the step of determining the imaging target, when the first image is newly imaged by the first imaging unit in parallel with the imaging of the second image by the second imaging unit, the newly imaged first Recalculating the behavior evaluation value in the image, updating the imaging target based on the calculated behavior evaluation value,
In the step of adjusting the second imaging unit, the second imaging unit is configured such that when the imaging target is updated in the step of determining the imaging target, the updated second image of the imaging target is captured. Adjust the monitoring method. On the computer,
The second imaging that can be adjusted to a smaller angle of view than the first imaging unit based on the calculated behavior evaluation value based on the calculated behavior evaluation value of a person included in the first image captured by the first imaging unit. Determining an imaging target of the second image captured by the unit;
Adjusting the second imaging unit so that the determined imaging target is included in the second image, and a program for executing the program,
In the step of determining the imaging target, when the first image is newly imaged by the first imaging unit in parallel with the imaging of the second image by the second imaging unit, the newly imaged first Recalculating the behavior evaluation value in the image, updating the imaging target based on the calculated behavior evaluation value,
In the step of adjusting the second imaging unit, the second imaging unit is configured such that when the imaging target is updated in the step of determining the imaging target, the updated second image of the imaging target is captured. Adjust the monitoring program.

Images