JPH0954894A - Moving object monitoring and measuring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately monitor and measure the movement of a mobile object by a comparatively simple constitution and to effectively utilize the monitoring and measurement for the improvement of security and a marketing strategy. SOLUTION: A mirror 011 is arranged in the upper part of a passage on which a pedestrian 015 passes, the image from above the pedestrian is reflected by the mirror 011, the mirror is turned in the extending direction of the passage, and the first image of the upper part of the mobile object reflected by the mirror 011 by the television camera 012 provided above the passage and the second image in the extending direction of the passage of the pedestrian 015 existing in the passage below the mirror 011 are imaged without overlapping. Image processings are interactively performed for the first and second images picked up by the television camera 012 in an image processor 013. The measured data and the image of the pedestrian 015 are outputted from an image output device 014.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention monitors and measures moving objects such as pedestrians and vehicles passing through passages, and counts the number of them, detects attributes, detects the direction of movement, and measures residence time. The present invention relates to a moving object monitoring / measuring device that performs an operation and outputs an image of the moving object.

[0002]

2. Description of the Related Art In recent years, there are many needs to install a television camera in a place where people pass, such as in an underground mall or a building, in order to improve security, reflecting the social situation. However, the conventional surveillance video system does not have a function of measuring a moving object and storing and transmitting a video in synchronization with the measurement. For this reason,
It is an operation that prepares a tape for a long time, records an image for the time being, and searches for a recording tape when an intended event occurs, which points out the efficiency of equipment and the difficulty of use.

On the other hand, in shopping districts and department stores, there is an increasing tendency to carry out marketing strategies based on pedestrian flow measurement data, and the number of customers who enter the store and passersby is increased by sensors. However, since the conventional count sensor uses infrared rays or the like, it is possible to count, but it is difficult to acquire the attribute information of a person, and the function of investigating the customer base is impossible in principle.

[0004]

As described above, the conventional method merely records an image or counts the number of passers-by, and is effective in, for example, marketing research. However, there is a problem in that it is not possible to obtain data such as people flow information and person attribute information, and the device configuration is relatively large, so it cannot be said that it meets the needs of the user. However, there is a problem that the cost is high, and as a result, there is a problem that it is insufficient to use for the sales strategy based on the statistical data.

[0005] The present invention has been made in view of the above,
It is an object of the present invention to provide a moving object monitoring / measuring device which is capable of appropriately monitoring and measuring the movement of a moving object with a relatively simple structure and improving security and effective for marketing strategies.

[0006]

To achieve the above object, the present invention according to claim 1 is arranged in an upper portion of a passage through which a moving object passes, and a reflection from the upper portion of the moving object passing through the passage. A ray direction changing means for changing a ray containing light to direct the light in a direction of extension of the passage, a first image above the moving object converted by the ray direction changing means, and a passage below the ray direction changing means. Imaging means arranged in the extension direction of the passage so as to capture the second image of the existing moving object from the extension direction without overlapping, and the first and first images captured by the imaging means. The gist of the present invention is to have an image processing means for interactively image-processing the two images and an output means for outputting the measurement data and the image of the moving object obtained by interactively image-processing with the image processing means.

According to the first aspect of the present invention, by including the light beam direction converting means, the moving object can be accurately imaged even when the image pickup means is provided only at a low position, and the image pickup means is provided. Can be reduced. Furthermore, by performing image processing on the first and second images interactively, it is possible to simultaneously monitor, measure, and extract features of a moving object.

[0008] The present invention according to claim 2 is based on claim 1.
In the invention described above, the gist is that the light beam direction converting means has a curved mirror.

According to the second aspect of the present invention, since the curved mirror is used, a wide range image can be obtained along the extending direction of the passage, and the height lowering from the ceiling is higher than that of the flat mirror. It can be installed in an environment with a lower ceiling.

[0010] Further, the present invention according to claim 3 provides the invention according to claim 1.
In the invention described above, the ray direction changing means is provided near the intersection of at least two or more element passages of the intersection passage, and the imaging means is provided at the intersection corresponding to each of the ray direction changing means. A moving object that moves between the element passages or stays between the element passages by extracting a moving object from the image captured by each of the image capturing means and associating the moving object with each other. The gist is to have means for detecting and outputting an image at the time of detection.

According to the third aspect of the present invention, a plurality of element passages in the intersecting passage are provided with light ray direction changing means, and image pickup means is provided corresponding to the light ray direction changing means, whereby the element passages are separated from each other. It is possible to detect a moving object that moves or stays, and measure its moving direction and staying time.

According to a fourth aspect of the present invention, there is provided a first image pickup means which is arranged at an upper portion of a passage through which a moving object passes, and which picks up an image of the moving object passing through the passage from above, and an extension of the passage. Second image pickup means arranged to pick up an image of a moving object from a direction, and synthesizing means for synthesizing the first and second images respectively picked up by the first and second image pickup means into one screen, Image processing means for interactively performing image processing on the first and second images composed of the synthesizing means;
The gist of the present invention is to have output means for outputting measurement data and an image of a moving object obtained by interactively performing image processing by the image processing means.

According to the fourth aspect of the present invention, the first image pickup means images the moving object downward from above, and the second image pickup means horizontally images the moving object from the extending direction of the passage. However, it is possible to adjust the orientation of the image pickup means in an interlocking manner, it is possible to capture a large moving object or a wide range according to the situation, and it is easy to acquire only a specific event.

Further, the present invention according to claim 5 is the same as claim 1.
Alternatively, in the invention described in the fourth aspect, the gist is that the image processing means has a measuring means for measuring a moving object by a moving object extracting process for extracting the moving object from the first image.

According to the present invention of claim 5, it is possible to output an image when a moving object appears by the moving object extraction processing,
Only necessary scenes can be detected efficiently.

The present invention according to claim 6 provides the invention according to claim 5.
In the invention described above, the image processing means uses the moving object passing position information or moving object passing time information from the moving object extracting process to extract a moving object from the second image, and the extracting means. Attribute detection means for detecting an attribute of the moving object including at least one of the length, color, or shape in the vertical direction of the moving object extracted from the image of the moving object extracted by The gist of the present invention is to have output means for adding and outputting the attribute information detected by the attribute detection means.

According to the present invention of claim 6, attributes such as the length, color and shape of the moving object in the vertical direction can be detected, and additional information effective for marketing and security can be provided. it can.

Further, the present invention according to claim 7 provides the invention according to claim 4.
In the invention described above, the first image pickup means is provided in an upper portion of at least two or more element passages near the intersection portion of the intersection passage, and the second image pickup means corresponds to each of the first image pickup means. Provided at the intersection, the image processing means extracts a moving object from the images captured by each of the first and second image capturing means, and associates the moving object with each other so as to correlate the element passages. It is a gist to have means for detecting a moving object that moves or stays between the element passages and outputs an image at the time of the detection.

According to another aspect of the present invention, the first image pickup means images the moving object downward from above, and the second image pickup means horizontally images the moving object from the extending direction of the passage. However, the direction of the image pickup means can be adjusted in conjunction with each other, a moving object can be captured large or a wide range depending on the situation, and it is easy to acquire only a specific event.

The present invention according to claim 8 is the same as claim 5
In the invention described above, the image processing means stores the first and second images in the image storage means based on the passing position information, the passing direction information, or the passing time information of the measurement result. The gist is to have an image selection and output means for selecting and outputting a desired accumulated image.

According to the eighth aspect of the present invention, a desired image stored in the image storage means is selected and output based on the passing position information, passing direction information, or passing time information of the measurement result. As a result, a desired image can be output even when a desired image cannot be obtained when passage is detected due to the angle of view and direction of the image pickup means, the influence of obstacles, the order of occurrence of events, and the like.

Furthermore, the present invention according to claim 9 provides the invention according to claim 5.
In the invention described above, the gist is that the measuring means performs a moving object extraction process based on image sequence data when passing through a slit provided in an image.

According to the ninth aspect of the present invention, since the measurement is performed based on the image sequence data when passing through the slits provided in the image, the processing amount can be reduced and the speed can be increased. be able to.

According to a tenth aspect of the present invention, in the invention according to the ninth aspect, the measuring means is provided on the first measurement slit provided on the image, and on both sides of the first measurement slit. A second and a third pair of slits, and a determination unit for processing a spatiotemporal image formed by stacking the images on the slits in the time direction to measure a moving object and determine the passing direction are summarized. And

According to the tenth aspect of the present invention, the method for measuring the moving object and determining the passing direction by processing the spatio-temporal image formed by stacking the images on each slit in the time direction is described. Even when installed in a simple environment, it is possible to accurately detect a moving object by direction with a small amount of processing.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

First, a moving object monitoring and measuring apparatus according to a first embodiment of the present invention will be described with reference to FIGS. 1, 2 and 3. 1 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to the first embodiment, FIG. 2 is a diagram showing an image captured by the moving object monitoring and measuring apparatus of FIG. 1, and FIG.
3 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

In FIG. 1, 011 is a mirror installed on the ceiling which is a light beam direction changing device for changing the traveling direction of the installed light beam, and 012 is a television camera also installed on the ceiling. Is adjusted such that the mirror is imaged on top of the television camera image and the directly imaged passage is imaged on the bottom of the television camera image without overlapping. Reference numeral 013 is an image processing apparatus, 014 is a monitoring image output apparatus, and 015 is a pedestrian imaged by the television camera 012, which stands directly under the mirror. Reference numeral 016 is the position of the virtual TV camera, 017 is the optical axis of the actual TV camera, and 018 is the optical axis of the virtual TV camera.

In FIG. 2, reference numeral 021 denotes one image in the image sequence taken by the television camera 012 of FIG.
Reference numeral 2 is an area in which the mirror 011 of FIG. 1 is imaged, and corresponds to the conversion area of the light beam direction changing device including the mirror 011. 023 is shown in FIG.
Image of pedestrian 015 reflected in the mirror, 024 in FIG.
An image directly captured from the pedestrian 015 of the above, 025 is a virtual measurement line used in the method using the measurement line which is one of the measurement methods.

In FIG. 3, 011 is the mirror that constitutes the light beam direction conversion device, 012 is a television camera that constitutes the image input device, 013 is an image processing device, 031 is an interface unit inside the image processing device, and 032 is an image memory. , 0
Reference numeral 33 is a feature extraction unit that extracts features from an image, 034 is a measurement unit that counts the number of pedestrians, and 014 is an image output device that displays a monitoring image.

Next, the operation of the moving object monitoring and measuring apparatus configured as described above will be described.

The TV camera 012 installed on the ceiling is
It is adjusted so that a pedestrian 015 walking in the aisle can be imaged through the mirror 011 installed on the ceiling, and at the same time, the pedestrian 015 can be directly imaged. The image sequence captured in this manner is the image 021 of FIG. 2, and the image 023 of the pedestrian is captured from above in the mirror 022 shown in the image. An image 024 taken from the front is shown.

To count the number of persons, it is effective to process an image of the person captured from above. Generally, the higher the TV camera is mounted, the less the image distortion is, and the passing people can be imaged without overlapping, and therefore, the extraction and counting can be performed with high accuracy. However, if the ceiling is not high enough, it is impossible to attach the TV camera as shown by 016 in FIG. In addition, it is desirable to image the pedestrian from the front as much as possible in order to use it for monitoring.

When the purpose is to measure and monitor as described above, the position of the television camera usually differs depending on the purpose.
In the present embodiment, measurement and monitoring can be realized with one TV camera by using a viewpoint conversion mirror.

By photographing the pedestrian 015 with the television camera 012 through the mirror 011, the image reflected in the mirror is virtually the same as when the television camera is installed at the position 016. It has the effect of increasing the apparent installation height. Here, the length of the optical axis from the television camera 012 to the mirror 011 shown by 017 is equal to the length from the virtual television camera 016 to the mirror 011 shown by 018.

The image 021 thus picked up is passed from the television camera 012 to the image processing apparatus 013. In the image processing apparatus 013, the interface unit 031 first receives the image and writes it in the memory 032.

The feature extraction unit 033 reads out the images on the measurement line 025 shown in FIG. 2 from the images written in the memory 032 and creates a spatiotemporal image which is created by stacking them in the time direction, and creates a slit image for each slit image. On the other hand, for example, as shown in Japanese Patent Application No. 7-29220, the difference between the slit-shaped background image obtained by statistically processing the time change of each pixel value in the processing target area and the acquired slit image. Two
The person area 023 in the image area 022 is processed by the binarization process.
To extract.

The feature extraction unit 033 further includes a memory 032.
The human area 024 is extracted from the image of the portion of the image 021 written in the image 021 directly captured without the mirror 011 and the 023 and 024 are associated with each other. When it is found that there is a person, the measuring unit 034 measures the pedestrian from the area and shape of 023, and the pedestrian 023 and 0
An interactive process is performed by referring to the information of 24 and outputting only the image when 024 is reflected to the image output device 014 as the image for monitoring.

In the present embodiment, the TV camera and the mirror are explained as one, but both may be plural. As the method of extracting the person area, an example of a method using difference processing accompanied by background updating is shown, but the area may be extracted by difference processing with a fixed background image, edge extraction by differential processing, or the like. Further, although the display of the image is taken as an example of the output of the monitor image, the output method may be filing, transmission, or the like.

Further, as the interactive processing, not only the images are associated with each other, but also the attribute information in the area is obtained, the temporal and spatial positioning is performed when the area is cut out, and the background for area extraction is used. A process of generating a trigger for updating an image and extracting a region with high accuracy is also included.

Next, a moving object monitoring and measuring apparatus according to a second embodiment of the present invention will be described with reference to FIGS. 1, 2 and 9. FIG. 9 is a block diagram showing the circuit configuration of the moving object monitoring and measuring apparatus according to the second embodiment.

In FIG. 9, 011 is a mirror which constitutes a light beam direction conversion device, 012 is a television camera which constitutes an image input device, 099 is an image processing device, 031 is an interface unit inside the image processing device, 032 is an image memory, 033
Is a feature extraction unit that extracts features from the image, 034 is a measurement unit that counts the number of pedestrians, 095 is a color identification unit that detects the clothing color of the pedestrian, and 096 is attribute information added to the output monitoring image. This is an image storage device that stores the information.

As shown in FIG. 1, the television camera 012 installed on the ceiling is adjusted so that the pedestrian 015 walking in the aisle can be imaged through the mirror 011 installed on the ceiling, and at the same time, the pedestrian 015 can be directly imaged. Has been done. The image sequence captured in this manner is the image 021 in FIG. 2, and the mirror 022 shown in the image has the image 023 obtained by capturing the pedestrian from above, which is different from the area of the mirror. An image 024 captured from the front is reflected in the image.

This image 021 is passed to the image processing apparatus 099, and in the image processing apparatus 099, the interface unit 031 first receives the image and writes it in the memory 032.
The feature extraction unit 033 reads out the image on the measurement line 025 from the images written in the memory 032, creates a spatiotemporal image that is created by stacking in the time direction, and extracts the person region 023 in the image region 022. Of the image 021 written in the memory 032, the person region 024 is extracted from the image of the part directly imaged without a mirror, and the 023 and 024 are associated with each other.

The method of extracting the person area is, for example, Japanese Patent Application No. 7-
As shown in No. 29220, the difference processing between the background image obtained by statistically processing the time change of each pixel value in the processing target area and the input image may be performed. When it is found that there is a person, the measuring unit 034 counts pedestrians from the area and shape of 023, measures the passing position and passing time in real time from the position in the spatiotemporal image, and the color identifying unit 095 Performs an interactive process in which the clothing color is extracted as attribute information from the directly photographed person area 024, and the image storage device 096 records and stores the image and the attribute information at that time together with the time information and the measurement result. The clothing color extraction method is
For example, the hue of each pixel of the person region directly imaged may be classified into the colors having the closest hue among the predetermined representative colors of several colors, and the representative color occupying the maximum area may be the clothing color of the person.

In the present embodiment, as an example of the interactive processing, the clothing color of the human area is extracted from the color information of the extracted object. However, clothing or hairstyle is extracted from the shape feature of the extracted object, The height of the person may be extracted from the vertical length of the region in the image directly captured without passing through the mirror when the moving body stands in the position specified by the image reflected in the mirror. By integrating such information, information on each person area such as gender and age group, and configuration information of a pedestrian group by detecting a person group (whether it is one person,
It is also possible to perform extraction of a pair, parent and child, family or the like, or to detect a suspicious person by searching for a wearer of a mask or sunglasses from a cutout image of a person area.

Further, although the present embodiment has been described as an apparatus for accumulating measurement results including images and attribute information, an apparatus for displaying and transmitting images and allocating filings depending on the passing direction and attributes of pedestrians. Alternatively, the device may be a device that efficiently searches and browses by adding attribute information to the image, and may also be a device that outputs the result of statistically processing the attribute information.

Next, a moving object monitoring and measuring apparatus according to a third embodiment of the present invention will be described with reference to FIGS. 4, 5 and 6. 4 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to the third embodiment, FIG. 5 is a diagram showing an image taken by the moving object monitoring and measuring apparatus of FIG. 4, and FIG.
3 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

In FIG. 4, reference numeral 041 denotes a television camera installed on the ceiling and capturing an image directly below, 042 a television camera installed on the ceiling and oriented in a substantially horizontal direction, and 043 a television camera also installed on the ceiling and oriented in a substantially horizontal direction. Yes,
The TV camera 042 images a person passing directly under the TV camera of the TV camera 041 from the traveling direction of the passage, and the TV camera 043 is at a position symmetrical to the TV camera 042 with the TV camera 041 as a center and from the opposite direction. It is a TV camera that captures images. Reference numeral 044 denotes an image synthesizing device, which synthesizes the images captured by the television camera 041, the television camera 042, and the television camera 043, and
Output as one image. Reference numeral 045 denotes an image processing device, 01
Reference numeral 4 is an image output device, and 046 is a pedestrian, which stands directly under the television camera 041.

In FIG. 5, reference numeral 051 denotes one image in the image sequence synthesized by the image synthesis device 044 shown in FIG.
2 is a part of the composite image 051 which corresponds to the image captured by the television camera 041, and 053 is a part of the composite image 051.
The part corresponding to the image captured by the TV camera 042, 0
54 is a part of the composite image 051 which corresponds to the image captured by the TV camera 043, and 055 is the TV camera 041.
, 056 is an image of the pedestrian 046 imaged by the television camera 042, 057 is an image of the pedestrian 046 imaged by the television camera 043, 058,
Reference numeral 059 is a virtual measurement line used in a method using a measurement line which is one of measurement methods, and 058 is a counting line, 0
Reference numeral 59 is a direction determination line installed on both sides of 058.

In FIG. 6, 041, 042, and 043 are the television cameras constituting the image input device, 044 is an image synthesizing device, 045 is an image processing device, 031 is an interface unit inside the image processing device, and 032 is an image memory, 0
Reference numeral 63 is a feature extraction unit that extracts features from the image, 064 is a measurement unit that counts the number of pedestrians in each direction, 065 is an image selection unit that selects an image to be output from the stored images, 066 is a television camera control unit, and 014 Is an image output device for displaying a monitoring image.

The three TV cameras 041, 042 and 043 used in this embodiment are all installed on the ceiling of the passage, but the TV camera 041 faces downward from the installation position. Cage, TV camera 0
42 and the TV camera 043 are installed at positions apart from the TV camera 041 in the front-rear direction of the passage.
Both of them face the TV camera 041. Furthermore, the television cameras 042 and 043 have functions such as pan, tilt, and zoom.

Television cameras 041 and 0 which are image input units
The images input from 42 and 043 are the video synthesizing device 04.
4 is combined and passed as one video to the image processing apparatus 045. In the image processing apparatus 045, the interface unit 031 receives an image and writes it in the memory 032, as in the first embodiment. In the feature extraction unit 063, the memory 0
Of the images written in 32, measurement lines 058,0
The images on 59 are read out to create spatiotemporal images that are stacked in the temporal direction, and the person region 055 in the image region 052 captured by the television camera 041 is extracted from each spatiotemporal image.

The method of extracting the person area is, for example, Japanese Patent Application No. 7-
As shown in No. 29220, the difference processing between the background image obtained by statistically processing the time change of each pixel value in the processing target area and the input image may be performed. When it is found that there is a person, the measuring unit 064 counts the number of pedestrians in each direction and determines the passing position and size of the pedestrian. A method of counting pedestrians by direction is, for example, Japanese Patent Application No. 6-20632.
As described in No. 0, if a method is used to determine the presence or absence of a person area image on the direction determination line at the moment when a pedestrian passes the counting line, based on the spatiotemporal image created for each measurement line. Good.

The television camera control unit 06 is arranged so that the pedestrian who has passed through can be seen as easily as possible on the basis of the pedestrian's passing position, passing time, and size information obtained by the above method.
6 sequentially controls panning, tilting, and zooming of the television cameras 042 and 043.

In parallel with the above processing, the feature extraction unit 063 converts the person area 056 captured by the television camera 042 from the image corresponding to the image area 053 in the image written in the memory 032 into the image area. The person area 057 captured by the television camera 043 is extracted from the image corresponding to 054. Further, the person regions 055, 056, 057 captured by the respective TV cameras are also associated with each other.

In the image selection unit 065, when the measurement unit 064 counts the pedestrians by direction, the pedestrians are present in the image in the image regions 053, 054 depending on the direction, and the pedestrians face the front. Select the image of the
Perform an interactive process of sending to 4. The video output device 014 outputs only the selected video as a monitoring video. As a result, it is possible to always monitor the image facing the television camera, and the image is adjusted so that the pedestrian to be monitored can easily see it.

In the present embodiment, as an example of the method for extracting the human region, the method using the difference processing accompanied by the background update is shown. However, it is possible to use the difference processing with the fixed background image or the edge extraction by the differentiation processing. Regions may be extracted.
Further, although the display of the image is taken as an example of the output of the monitor image, the output method may be filing, transmission, or the like. Further, in the present embodiment, it has been described that one TV camera facing directly below and two TV cameras facing in the horizontal directions opposite to each other are used to output only the person image facing the TV camera. , It is not limited to the image of the person facing the TV camera, and multiple TV cameras facing the same direction are installed in parallel, or one of the TV cameras facing in the horizontal direction is installed. May be

Next, a moving object monitoring and measuring apparatus according to a fourth embodiment of the present invention will be described with reference to FIGS. 7 is a diagram showing the configuration of the moving object monitoring and measuring apparatus according to the fourth embodiment, and FIG. 8 is a diagram showing an image captured by the moving object monitoring and measuring apparatus of FIG.

In FIG. 7, reference numeral 071 is a light beam direction changing device for changing the traveling direction of the installed light beam, and a mirror with a cylindrical cross section is installed on the ceiling. 012 is a television camera also installed on the ceiling. The arrangement relationship between the camera and the mirror is adjusted so that the mirror is reflected in the upper part of the television camera image and the directly photographed passage is reflected in the lower part of the television camera image without overlapping. 073 is an image processing device,
Reference numeral 014 is an image output device, and 015 is a television camera 0.
A pedestrian imaged at 12, standing directly under the mirror.

In FIG. 8, reference numeral 081 denotes one image in the image sequence captured by the television camera 12 in FIG.
7 is an area showing the mirror 071 in FIG. 7, 083 is an image formed by reflecting the pedestrian 015 in FIG. 7, 084 is an image directly captured by the pedestrian 015 in FIG. 7, and 085 and 086 are measurement values. A virtual measurement line used in a method using a measurement line which is one method, 085 is a counting line, and 086 is a direction determination line arranged on both sides of 085.

The television camera 012 installed on the ceiling is
The pedestrian 015 who walks in the passage is adjusted so that the pedestrian 015 can be directly imaged at the same time as the image is taken through the mirror 071 having a cylindrical cross section installed on the ceiling. The row of the image captured in this manner is 081 in FIG. 8, and an image 083 obtained by capturing the pedestrian from above is present in the mirror 082 shown in the image, which is in a place different from the area of the mirror. An image 084 taken from the front is shown. This image 081 is passed to the image processing apparatus 073, and in the image processing apparatus 073, the interface section receives the image and writes it in the memory. The feature extraction unit reads out the images on the measurement lines 085 and 086 from the images written in the memory, creates spatiotemporal images that are stacked in the time direction, and creates an image region 082 from each spatiotemporal image. Inside person area 0
Image 0 written in memory while extracting 83
A person region 084 is extracted from an image of a portion directly photographed out of 81, and 083 and 084 are associated with each other.

The method of extracting the person area is, for example, Japanese Patent Application No. 7-
As shown in No. 29220, the difference processing between the background image obtained by statistically processing the time change of each pixel value in the processing target area and the input image may be performed. When it is known that a person exists, the measuring unit counts pedestrians by direction. A method of counting pedestrians according to directions is, for example, Japanese Patent Application No. 6-206320.
As described in No. 6, a method of determining the presence or absence of a person area image on the direction determination line at the moment when a pedestrian passes the counting line based on the spatiotemporal image created for each measurement line may be used. When the number of pedestrians is counted, an interactive process is performed in which only the image when 084 is reflected is output to the image output device 014 as an image for monitoring.

In the present embodiment, the mirror has a cylindrical cross-sectional shape, but the shape is not limited to a cylindrical shape, and if the characteristics are known in advance and a curved surface shape that can be measured by a pedestrian by the measurement method adopted, It may be any curved surface.

A convex curved surface such as a cylindrical surface has an integral function, and a wide real space can be imprinted on a narrow mirror surface. As a result, the measurement line can be measured at any position before and after the passage by slightly moving the measurement line up and down in the image. Further, in particular, the cylindrical shape is widely used as an interior, and there is an advantage that existing equipment can be used and it can be easily accepted as an interior even if newly installed.

Next, a moving object monitoring and measuring apparatus according to a fifth embodiment of the present invention will be described with reference to FIGS. 10 is a diagram showing the configuration of a moving object monitoring and measuring apparatus according to the fifth embodiment, and FIG. 11 is a block diagram showing the circuit configuration of the moving object monitoring and measuring apparatus of FIG.

In FIG. 10, 101, 102, 10
3, 104 changes the traveling direction of light rays installed on the ceiling of each element passage (each passage extending from the intersection, if there is a cross road, there are four, and if there are three difference roads, there are three element passages) Mirrors 105, 106, 1 which are light beam direction changing devices
Reference numerals 07 and 108 denote TV cameras installed on the ceiling. The mirrors 101, 102, 103, and 104 are displayed above the respective TV camera images, and the image of the direct passage is imaged below the TV camera images. Is adjusted. Reference numerals 109, 110, 111, and 112 denote image processing apparatuses that process the images captured by the television cameras 105, 106, 107, and 108, respectively.
Reference numeral 114 denotes an image output device, which is a device for collecting measurement information and feature information obtained by the image processing devices 0, 111, and 112.

In FIG. 11, 101, 102, 10
Reference numerals 3 and 104 denote the mirrors which constitute the light beam direction changing device, and 10
5, 106, 107, and 108 are the television cameras constituting the image input device, 109, 110, 111, and 112 are image processing devices, 031 is an interface unit inside the image processing device, 032 is an image memory, and 115 is a feature from an image. Of the number of pedestrians, reference numeral 113 represents a measurement information / feature information totaling device, reference numeral 116 represents a measurement target association unit, reference numeral 117 represents a residence time measurement unit, 11
An image output device 4 displays a monitoring image.

The present embodiment is an example in which the device is installed at the crossroads. Each passage extending from the intersection will be called each element passage. If it is a crossroads, there are four, and if there are three differences, there are three elemental passages. Mirrors 101, 102, 103, 104 are installed on the ceiling of each element passage of the crossroads, and are adjusted so that the area directly below the installation location can be seen when viewed from the intersection side. On the ceiling of the intersection, the television cameras 105, 106, 107, 1 face the mirrors 101, 102, 103, 104 installed on the ceilings of the element passages.
08 are installed, and the mirrors 101, 102, 1 are respectively installed.
03 and 104 are at the top of each TV camera image,
It is adjusted so that the image of the direct passage is captured in the lower part of the TV camera image. The images captured in this way are respectively processed by the image processing devices 109, 110, 11
Passed to 1,112.

The processing in the image processing apparatus will be described by taking the image processing apparatus 109 as an example. In the image processing apparatus 109, the interface unit 031 first receives an image, and the memory 032 receives the image.
Write in. The feature extraction unit 115 extracts a person region from the image written in the memory 032, from both the image reflected in the mirror and the image of the portion directly captured without passing through the mirror, and extracted from each partial image. Corresponds to the person area. For example, Japanese Patent Application No.
As indicated by No. 29220, difference processing between the background image and the input image obtained by statistically processing the temporal change of each pixel value in the processing target area may be performed.

The measuring unit 034 measures the pedestrian by direction from the person image reflected in the mirror. The pedestrian direction-by-direction counting method is, for example, as in Japanese Patent Application No. 6-206320, draw three measurement lines (one counting line and two direction determination lines) in the image reflected on the mirror, A method of determining the presence / absence of a human region image on the direction determination line at the moment when the pedestrian passes the counting line based on the spatiotemporal image created for each measurement line may be used.

When the pedestrian is measured, the feature extraction unit 115
Converts (mosaic) an image of a person directly imaged by each TV camera into an image of a predetermined resolution (for example, m × n pixels), and creates a vector (m
Xn-dimensional vector) is extracted as one of the feature amounts of the person area, and the pedestrian measurement information (passing time, passing position, passing direction, passing speed) obtained by each image processing device and the pedestrian's It is sent to the aggregation device 113 together with the image. In the aggregation device 113, the measurement target association unit 116 causes the consistency of the measurement information (passing time, passing position, passing direction, passing speed) of each pedestrian at each point and the magnitude of the vector distance of the feature vector of each pedestrian. It is made to correspond by a method to which the method shown in Japanese Patent Application No. 6-38866, such as comparing each point with each other, is applied.
The time from when the pedestrian associated by the measurement object association unit 116 appears in the corresponding area to when the pedestrian disappears is measured as the residence time.

On the image output device 114, the moving direction of the pedestrian, the passing time, the staying time, etc. are displayed together with the image of the pedestrian whose staying time is associated with each other as described above.

In the present embodiment, the case where the present apparatus is installed at the crossroads has been described, but the present invention can be carried out not only at the crossroads but also at any intersection, and it can also be installed in a room or a building having a plurality of entrances and exits in an underground mall, It is also possible to measure the residence time etc. inside a room, building, underground mall, etc.

Correspondence of pedestrians for measuring the staying time is as follows: each pedestrian's passing time, passing position, passing direction,
Although a method of associating from the luminance value of the mosaiced image with the passing speed is shown, a feature amount other than the above may be used, and the pedestrians are not associated with each other to measure the residence time, It is also possible to measure the change in the passage density of pedestrians at each point as a feature amount and estimate the residence time from the change in passage density at each point with time.

Next, a moving object monitoring and measuring apparatus according to a sixth embodiment of the present invention will be described with reference to FIGS. 12, 13 and 14. FIG. 12 is a diagram showing the configuration of a moving object monitoring and measuring apparatus according to the sixth embodiment, and FIG.
FIG. 14 is a diagram showing a configuration of the moving object monitoring and measuring apparatus viewed from above, and FIG. 14 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

12 and 13, 121, 1
Reference numerals 32, 123, and 131 are mirrors, which are light beam direction changing devices installed on the ceiling of each element passage to change the traveling direction of the light beam.
Reference numeral 24 is a television camera that is an image input device installed on the ceiling, and 122 is a mirror that is also a light beam direction conversion device that changes the traveling direction of the light beam installed on the ceiling. The mirror 131 and the mirror 132 are viewed from the TV camera 124. Is adjusted so that is reflected in. Reference numeral 125 is an image processing apparatus, and 114 is an image output apparatus.

In FIG. 14, reference numeral 141 designates 121, 122, 1 of FIGS.
Mirrors 23, 131, and 132, 124 is a television camera that constitutes an image input device, 125 is an image processing device, and 031
Is an interface unit inside the image processing apparatus, 032 is an image memory, 142 is an image feature extraction unit, 143 is a measurement unit that counts the number of pedestrians, 144 is a measurement target association unit, and 14 is a measurement target association unit.
Reference numeral 5 is a residence time measuring unit, and 114 is an image output device for displaying a monitoring image.

The present embodiment is an example in which the device is installed at the crossroads. A mirror 12 is placed on the ceiling of each passage of the crossroads.
1, 132, 123, 131 are installed, and a mirror 122 having a shape bent at the center is additionally provided on the ceiling.
Is installed. A television camera 124 is installed on the ceiling far from the intersection with respect to the mirror of the element passage on which the mirror 123 is installed.
The angle of view and orientation of the television camera and the installation location and orientation of the mirror are adjusted so that the image of 23 does not overlap and the image of a pedestrian passing through the intersection is directly captured. Further mirror 122
Are adjusted so that the entire mirrors 131 and 132 can be seen from the television camera 124. Further, each mirror is adjusted so that the pedestrian can be imaged when the pedestrian appears directly below the mirrors 121, 132, 123, 131. The image captured by the television camera 124 is the image processing device 12.
Passed to 5. In the image processing device 125, first, the interface unit 031 receives an image and writes it in the memory 032. The feature extraction unit 142 extracts a human area from both the image written in the memory 032 and the image captured in the mirrors 121, 122, and 123 and the image of the portion directly captured without passing through the mirrors. The person regions extracted from the image are associated with each other.

The method of extracting the person area is, for example, Japanese Patent Application No. 7-
As indicated by No. 29220, the difference processing between the background image and the input image obtained by statistically processing the temporal change of each pixel value in the processing target area may be performed. In addition, the measuring unit 143 measures the pedestrian for each direction for each element passage from the human image reflected in the mirror. The pedestrian direction-by-direction counting method is, for example, as in Japanese Patent Application No. 6-206320, draw three measurement lines (one counting line and two direction determination lines) in the image reflected on the mirror, A method of determining the presence / absence of a human region image on the direction determination line at the moment when the pedestrian passes the counting line based on the spatiotemporal image created for each measurement line may be used.

When the pedestrian is measured, the feature extraction unit 142
However, a directly-captured person image has a predetermined resolution (for example, m
The image is converted (mosaiced) into an image of (× n pixels), and a vector (m × n dimensional vector) having the pixel value of each pixel as an element is extracted as one of the feature amounts of the person area. Based on the pedestrian's measurement information (passing time, passing position, passing direction, passing speed) and characteristic information obtained by the image processing device 125 as described above, the measurement target association unit 144 passed under the mirror. Correspond to pedestrians. As a method of associating, for example, the method shown in Japanese Patent Application No. 6-38866 is applied, and the consistency of the passing time, passing position, passing direction, and passing speed of each pedestrian and the vector distance of the feature vector of each pedestrian This can be done by comparing the sizes. The dwell time measuring unit 145 measures the dwell time of the pedestrian associated by the measurement target associating unit 144. The image output device 114, together with the image of the pedestrian whose residence time was measured in association with the above,
The pedestrian's moving direction, passing time, staying time, etc. are displayed.

In the present embodiment, the device for measuring the stay time of a pedestrian has been described. However, a device for simply measuring a pedestrian and outputting a pedestrian image without associating a pedestrian that has passed therewith. It may be. Further, in the present embodiment, the case where the present device is installed at the crossroads has been described, but it can be carried out not only at the crossroads but also at any intersection, and also in a room or a building having a plurality of doorways, installed at each doorway of an underground mall, It is also possible to measure the residence time, etc. inside buildings, underground malls, etc.

Next, a moving object monitoring and measuring apparatus according to a seventh embodiment of the present invention will be described with reference to FIG. In the present embodiment, a case will be described in which an image when a pedestrian appears a certain distance behind the mirror as viewed from the television camera in FIG. 1 is displayed.

The TV camera 012 installed on the ceiling is
A pedestrian 015 walking in the aisle is imaged through a mirror 011 installed on the ceiling, and at the same time, a video of the aisle is also imaged. The image input from the television camera 012 is passed to the image processing device 013, and the image processing device 013 first
The interface unit receives the image and writes it in the memory. Of the images written in the memory in the feature extraction unit,
At the same time as extracting the person image from the image reflected in the mirror, the person image is also extracted from the image reflected without the mirror. The extraction method of the person area is, for example, as shown in Japanese Patent Application No. Hei 7-29220, a difference processing between a background image and an input image obtained by statistically processing the time change of each pixel value in the processing target area. Good.

When it is found that a person exists, the measuring unit counts pedestrians by direction from the image reflected in the mirror. A method of counting pedestrians by direction is, for example, Japanese Patent Application No. 6-20632.
As described in No. 0, if a method is used to determine the presence or absence of a person area image on the direction determination line at the moment when a pedestrian passes the counting line, based on the spatiotemporal image created for each measurement line. Good. When pedestrians are counted by direction,
From the passing direction and speed of the pedestrian, the time when the pedestrian is at or comes to a predetermined position (here, a certain distance behind the mirror seen from the TV camera) where the pedestrian image is desired to be calculated,
If the pedestrian is moving in the direction of the TV camera, select the image when the pedestrian was in a predetermined position from the images stored in the memory, and move the pedestrian in the direction opposite to the direction of the TV camera. If so, wait until the pedestrian reaches a predetermined position to acquire an image, and
Perform interactive processing such as displaying in 4.

[0086]

As described above, according to the present invention as set forth in claim 1, the image pickup means is provided at the high position even when the image pickup means can be provided only at the low position by having the light beam direction changing means. Similar to the case, it is possible to accurately image a moving object, reduce the number of imaging means, there are no severe restrictions on installation conditions, and the device configuration is relatively small. Can be accurately measured and monitored. Further, it is possible to obtain traffic data and the like, which are indispensable for marketing, as well as monitoring pedestrians, which are moving objects. In addition, interactive image processing of the first and second images enables simultaneous monitoring, measurement, and feature extraction of a moving object.

According to the second aspect of the present invention, by using the curved mirror as the light beam direction changing means, a wide range of images can be obtained along the extending direction of the passage, so that various positions can be obtained. In addition to being able to perform measurement, the height from the ceiling can be made smaller than that of a plane mirror, and it can be installed even in an environment with a lower ceiling.

Further, according to the present invention as set forth in claim 3, a plurality of element passages of the intersection passage are provided with a light ray direction changing means,
By providing an image pickup means corresponding to the light ray direction changing means, it is possible to detect a moving object moving or staying between the element passages, and measure the moving direction and staying time thereof, for example, determining the layout in a store, Measuring the degree of interest in a product,
It can also be used to control traffic flow.

According to the present invention as set forth in claim 4, the first image pickup means performs the measurement and the second image pickup means performs the monitoring separately, and the images of the respective image pickup means are interactive. In this way, it is possible to freely operate pan, tilt, zoom, etc. according to the purpose by linking the monitoring imaging means with the measurement result, and to provide an advanced security system for recording media and transmission. The road can be saved and the amount of work can be reduced. Further, the first image pickup means images the moving object downward from above, and the second image pickup means horizontally images the moving object from the extending direction of the passage,
It is possible to adjust the orientation of the image pickup unit in conjunction with each other, to capture a moving object in a large range or a wide range according to the situation, and it is easy to acquire only a specific event.

Further, according to the present invention of claim 5, it is possible to output an image when a pedestrian, which is a moving object, appears by the moving object extraction process, and to efficiently detect and display only a necessary scene. , Records can be stored and can be transmitted to a remote place, effective measurement and monitoring can be performed, and effective information for marketing and security can be provided.

According to the present invention of claim 6, not only the traffic data but also the vertical length of the moving object,
Color, shape, etc., that is, the height of the passerby, clothing color, clothing,
It is possible to detect attributes such as hairstyle, provide additional information effective for marketing and security, and use these attribute information to easily search and browse images stored and transmitted. You can

Further, according to the present invention as set forth in claim 7, the first image pickup means performs the measurement and the second image pickup means performs the monitoring separately, and the images of the respective image pickup means are interactively arranged. In this way, it is possible to freely operate pan, tilt, zoom, etc. according to the purpose by linking the monitoring imaging means with the measurement result, and to provide an advanced security system for recording media and transmission. The road can be saved and the amount of work can be reduced. Further, the first image pickup means images the moving object downward from above, and the second image pickup means horizontally images the moving object from the extending direction of the passage,
It is possible to adjust the orientation of the image pickup unit in conjunction with each other, to capture a moving object in a large range or a wide range according to the situation, and it is easy to acquire only a specific event.

According to the present invention, the desired image stored in the image storage means is selected and output based on the passing position information, passing direction information or passing time information of the measurement result. By doing so, it is possible to output the desired image even when the desired image cannot be obtained when the passage is detected due to the angle of view and direction of the imaging means, the influence of obstacles, the order of occurrence of events, etc. It is expected that the monitoring accuracy will be improved and higher safety will be secured.

Further, according to the present invention as set forth in claim 9, since the measurement is performed based on the image sequence data when passing through the slit provided in the image, it is possible to reduce the processing amount and speed up. The measurement process can be performed in real time.

According to the present invention of claim 10,
Since the spatiotemporal image formed by stacking the images on each slit in the time direction is processed to measure the moving object and determine the passing direction, the moving object can be optimized with a small amount of processing regardless of the environment. It is possible to accurately detect each direction.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an image captured by the moving object monitoring and measuring apparatus of FIG.

3 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

FIG. 4 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to a third embodiment of the present invention.

5 is a diagram showing an image captured by the moving object monitoring and measuring apparatus of FIG.

6 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

FIG. 7 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to a fourth embodiment of the present invention.

8 is a diagram showing an image captured by the moving object monitoring and measuring apparatus of FIG.

FIG. 9 is a block diagram showing a circuit configuration of a moving object monitoring and measuring apparatus according to a second embodiment of the present invention.

FIG. 10 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to a fifth embodiment of the present invention.

11 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

FIG. 12 is a diagram showing a configuration of a moving object monitoring and measuring apparatus according to a sixth embodiment of the present invention.

13 is a diagram showing a configuration of the moving object monitoring and measuring apparatus of FIG. 12 viewed from above.

14 is a block diagram showing a circuit configuration of the moving object monitoring and measuring apparatus of FIG.

[Explanation of symbols]

 011 mirror 012 TV camera 013 image processing device 014 image output device 015 pedestrian 016 virtual TV camera position 031 interface unit 032 memory 033 feature extraction unit 034 measurement unit

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Akira Banno 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation

Claims (10)

[Claims] 1. A light ray direction changing means arranged at an upper portion of a passage through which a moving object passes, for changing a light ray including reflected light from an upper portion of the moving object passing through the passage so as to be directed in an extending direction of the passage. Capturing a first image of the upper part of the moving object converted by the light ray direction converting means and a second image of the moving object existing in the passage below the light ray direction converting means from the extending direction without overlapping. So as to be arranged in the extending direction of the passage, image processing means for interactively image-processing the first and second images captured by the imaging means, and interactively by the image processing means. A moving object monitoring and measuring apparatus, comprising: an output unit that outputs measurement data and an image of a moving object obtained by performing image processing on the moving object. 2. The moving object monitoring and measuring apparatus according to claim 1, wherein the light beam direction converting means has a curved mirror. 3. The light ray direction changing means is provided near an intersection of at least two element passages of the intersection passage, and the image pickup means is provided at the intersection corresponding to each of the light ray direction changing means. The image processing means is provided and extracts a moving object from the image captured by each of the image capturing means, and associates the moving object to move between the element passages or stay between the element passages. The moving object monitoring and measuring apparatus according to claim 1, further comprising means for detecting a moving object and outputting an image at the time of the detection. 4. A first device which is arranged at an upper portion of a passage through which a moving object passes and which images the moving object passing through the passage from above.
Image pickup means, a second image pickup means arranged to pick up an image of a moving object from the extending direction of the passage, and first and second images picked up by the first and second image pickup means, respectively. A synthesizing unit for synthesizing on one screen, an image processing unit for interactively image-processing the first and second images formed by the synthesizing unit, and a movement obtained by interactively image-processing by the image processing unit. A moving object monitoring and measuring apparatus comprising: an output unit that outputs measurement data and an image of the object. 5. The movement according to claim 1, wherein the image processing means has a measuring means for measuring a moving object by a moving object extracting process for extracting the moving object from the first image. Object monitoring and measurement device. 6. The extracting means for extracting a moving object from the second image by using the moving body position information or moving body time information from the moving body extracting process, and the extracting means. Attribute detection means for detecting an attribute of the moving object including at least one of the length, color, or shape of the moving object in the vertical direction from the image of the moving object extracted by the measuring means; The moving object monitoring and measuring apparatus according to claim 5, further comprising an output unit that adds and outputs the attribute information detected by the attribute detection unit. 7. The first image pickup means is provided at an upper portion of at least two or more element passages of the intersection passage near the intersection portion, and the second image pickup means is provided in each of the first image pickup means. Correspondingly provided at the intersection, the image processing means extracts the moving object from the images picked up by the first and second image pickup means, and associates the moving object with the moving object. The moving object monitoring and measuring apparatus according to claim 4, further comprising means for detecting a moving object moving between the element passages or staying between the element passages and outputting an image at the time of the detection. 8. The image processing means stores the first and second images, and the image storage means based on passing position information, passing direction information, or passing time information of the measurement result. The moving object monitoring and measuring apparatus according to claim 5, further comprising: an image selecting and outputting unit that selects and outputs a desired image stored in the. 9. The moving object monitoring and measuring apparatus according to claim 5, wherein the measuring means performs a moving object extraction process based on image sequence data when passing through a slit provided in the image. 10. The measuring means includes a first measuring slit provided on the image, a pair of second and third slits provided on both sides of the first measuring slit, and on each of the slits. 10. The moving object monitoring and measuring apparatus according to claim 9, further comprising a determination unit that processes a spatiotemporal image formed by stacking images in the time direction to measure a moving object and determine a passing direction.