JP2007278869A - Ranging device, vehicle periphery monitoring device, ranging method, and ranging program - Google Patents

Abstract

An object of the present invention is to accurately measure a distance of an object based on parallax while variably setting a region for extracting an object from captured images of two imaging devices.
An object extraction area for extracting an object is set in each of the captured images of two image pickup apparatuses 2R and 2L. The position data of the reference image area in each captured image and the reference parameter that defines the relationship between the parallax and distance of the object in the reference image area are stored and held in advance in the storage unit. A parameter that defines the relationship between the parallax and distance of the object in the set object extraction region is determined based on the positional relationship between the object extraction region and the reference image region in each captured image and the reference parameter, Based on the determined parameters, the distance of the object is calculated from the parallax of the object.
[Selection] Figure 1

Description

  The present invention relates to a distance measuring device, a distance measuring method, and a distance measuring program for measuring the distance (relative distance from a vehicle) of an object based on the parallax of the object in two captured images mounted on the vehicle. . Furthermore, it is related with the periphery monitoring apparatus of the vehicle carrying this distance measuring device.

  2. Description of the Related Art Conventionally, as a vehicle periphery monitoring device for monitoring an object such as a person existing around a vehicle based on a captured image of an image pickup device mounted on the vehicle, for example, the one shown in Patent Document 1 is disclosed by the present applicant. Proposed.

  In the technique shown in Patent Document 1, a pair of left and right imaging devices (infrared cameras) are mounted on the front portion of a vehicle, and an object such as a person is extracted from images captured by these imaging devices, and left and right imaging is performed. Parallax of the same object included in each image, that is, the position of the object in the right captured image (lateral position in the captured image) and the position of the object in the left captured image The relative distance from the vehicle to the object is detected based on the difference from (the lateral position in the captured image). In this case, the horizontal position of the object in each captured image is the distance from the vertical straight line passing through the center point of the captured image (hereinafter referred to as the horizontal center line) to the center of gravity of the object ( (Interval with the right or left side of the horizontal center line being positive). Based on the detected relative distance, the real space position of the object (relative position with respect to the vehicle) is recognized, and the object should avoid contact with the vehicle based on the recognized real space position. It is determined whether or not the vehicle is a target, and when it is determined that the vehicle is to be avoided, a warning is given to the driver.

An apparatus found in Patent Document 2 is also known. In this device, an imaging device capable of mechanical zoom operation is mounted on a vehicle, and the captured image is displayed on a front window by a head-up display device. Then, the angle of view (zoom amount) of the imaging device is changed according to the vehicle speed of the vehicle, the steering angle of the vehicle, the inter-vehicle distance detected by the radar device, and the like.
JP 2001-6096 A Japanese Patent Laying-Open No. 2005-81860

  By the way, as can be seen in Patent Document 1, in a vehicle periphery monitoring device that calculates the relative distance of an object using the parallax of the object obtained from two captured images, the traveling environment of the vehicle It is desirable to be able to expand and contract the imaging range as appropriate according to (the driving situation of the vehicle and the situation around the vehicle). For example, in order to reduce the processing load of image processing, the imaging range is usually narrowed. Then, when the distance from the vehicle to the object is relatively short and the entire image of the object cannot be included in two captured images in a narrow imaging range, the imaging range can be expanded. It is considered desirable.

  And when expanding and reducing the imaging range in this way, as seen in Patent Document 2, it is conceivable that each imaging device has a mechanical zoom function so that the imaging range of each captured image can be changed.

  However, if each imaging device is provided with a zoom function, the optical axis and focal length of each imaging device are likely to change over time due to the mechanical operation of the zoom function. For this reason, as can be seen in Patent Document 1, when the relative distance of the target object is detected using the parallax of the same target object in the two captured images, the relative distance is accurately detected. It tends to be difficult. This is because the relationship between parallax and relative distance changes due to changes in the optical axis and focal length of each imaging device, and it is difficult to detect changes in the optical axis and focal length.

  In view of this, it is possible to variably set an image area for extracting an object in each captured image so that the substantial imaging range can be expanded or reduced without providing each image capturing device with a mechanical zoom function. Conceivable.

  In this case, when the optical axes of the left and right imaging devices are parallel to each other and the heights of the optical axes are the same, as shown in Patent Document 1, they are included in the right and left captured images. What is necessary is just to obtain | require the difference of the space | interval from the horizontal direction center line of each captured image as the parallax of the said target object, and to calculate the relative distance of a target object based on the calculated | required parallax.

  However, when the orientation and height of the optical axes of the two imaging devices are accurately aligned as described above and mounted on the vehicle, it is actually caused by errors in the position or orientation of the imaging device mounted on the vehicle. It is extremely difficult. In addition, it is very time-consuming to adjust the optical axes of both imaging devices after the imaging devices are assembled to the vehicle.

  If the optical axes of the two imaging devices are not aligned, the accuracy of the relative speed of the target obtained from the calculated parallax decreases depending on how the image area is set for extracting the target. There was a fear.

  The present invention has been made in view of such a background, and accurately measures the distance of an object based on parallax while variably setting an area for extracting an object from captured images of two imaging devices. An object of the present invention is to provide a distance measuring device, a distance measuring method, and a distance measuring program that can be performed. Furthermore, it aims at providing the periphery monitoring apparatus of a vehicle provided with such a ranging device.

  According to a first aspect of the first aspect of the distance measuring device of the present invention, in order to achieve the above object, a first captured image and a second captured image respectively obtained from two image capturing devices mounted on a vehicle. The same object included in each captured image is extracted from the image, and the difference between the position of the object on the first captured image and the position of the object on the second captured image In the distance measuring device that calculates the distance of the object to the vehicle based on the parallax, the reference image area is predetermined in the entire area of each of the first captured image and the second captured image. Reference parameters defining the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted, and position data representing the position of the reference image area in each captured image are stored and held in advance. Storage means; An object extraction area setting means for variably setting an object extraction area for extracting an object for each of the first captured image and the second captured image, the first captured image, and the second captured image. Based on the positional relationship between the position of the set object extraction region set in each captured image and the position of the reference image region represented by the position data, and the reference parameter, the first captured image and Parameter determining means for determining a parameter that defines the relationship between the parallax of the object and the distance of the object when the same object is extracted in the object extraction region set in each of the second captured images When the same object is extracted in the object extraction area set for each of the first captured image and the second captured image, the parameter determination is performed corresponding to the object extraction area. Based on the relationship defined by the determined parameters by means, characterized in that a distance calculating means for calculating the distance of the object from the disparity of the object in the object extraction area.

  According to the first invention, the position between the position of the set object extraction region and the position of the reference image region represented by the position data in each of the first captured image and the second captured image. Based on the relationship and the reference parameter, the parallax of the object when the same object is extracted in the object extraction region set in each of the first captured image and the second captured image, and the parallax Since the parameter that defines the relationship with the distance of the object is determined, for each type of object extraction area variably set by the object extraction area setting means, the object extraction area of each captured image It is possible to set an appropriate parameter that defines the relationship between the parallax and the distance of the same object extracted in step (1). The reference image area and the reference parameters and position data corresponding to the reference image area may be determined experimentally, for example, at the time of manufacturing the vehicle or during maintenance / inspection. In this case, the position of the reference image region in each of the first and second captured images is generally due to a shift in the direction of the optical axis of each captured image and the like, and generally the first captured image and the second captured image. It differs from the image.

  Therefore, for each type of object extraction area variably set by the object extraction area setting means, the object extraction is performed based on the relationship between the parallax and the distance defined by the parameters determined as described above. By obtaining the distance of the object from the parallax of the object in the region, it is possible to obtain the distance accurately and appropriately.

  Therefore, according to the first aspect of the present invention, the distance of the object based on the parallax (relative distance from the vehicle) is measured while the region for extracting the object from the captured images of the two imaging devices is variably set. It becomes possible to carry out with high accuracy.

  In the first invention, the position of the object extraction region set in each of the first and second captured images (relative position with respect to each captured image) is the same between the first captured image and the second captured image. It is suitable in some cases. Further, when obtaining the parallax of the object in the object extraction area, the relative position of the object with respect to the object extraction area of the first captured image (relative position with respect to the reference point fixed with respect to the object extraction area) and The difference between the relative position of the object with respect to the object extraction area of the second captured image (relative position with respect to the reference point fixed with respect to the object extraction area) may be obtained as the parallax of the object. Similarly, for the parallax related to the reference parameter, the difference between the relative position of the object with respect to the reference image area of the first captured image and the relative position of the object with respect to the reference image area of the second captured image is determined as the target. What is necessary is just to obtain | require as a parallax of an object.

  In order to achieve the above object, a second invention, which is a second aspect of the distance measuring device of the present invention, includes a first captured image and a second captured image respectively obtained from two imaging devices mounted on a vehicle. The same object included in each captured image is extracted from each captured image, and the position of the object on the first captured image and the position of the target on the second captured image are In a distance measuring device that calculates the distance of the object to the vehicle based on a parallax that is a difference between the reference image, a reference image that is predetermined within the entire area of each of the first captured image and the second captured image Reference parameters that define the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted in the area and position data that represents the position of the reference image area in each captured image are stored in advance. Retained hand Object extraction area setting means for variably setting an object extraction area for extracting an object for each of the first captured image and the second captured image, and the first captured image and the first captured image. When the same object is extracted in the object extraction area set in each of the two captured images, the object is determined from the parallax of the object in the object extraction area based on the relationship defined by the reference parameter. Distance calculating means for calculating the distance of the object, and the object extraction area setting means extracts the same object in the object extraction area set in each of the first captured image and the second captured image. The position of the reference image area represented by the position data so that the relationship between the parallax of the object and the distance of the object is equal to the relationship defined by the reference parameter. And sets the object extraction area Te.

  In the second invention, the object extraction area setting means is configured to extract the object when the same object is extracted in the object extraction area set in each of the first captured image and the second captured image. Since the object extraction region is set according to the position of the reference image region represented by the position data so that the relationship between the parallax of the object and the distance of the object is equal to the relationship defined by the reference parameter For each type of object extraction area variably set by the object extraction area setting means, the reference parameter is the parallax of the same object extracted in the object extraction area of each captured image It is an appropriate parameter that defines the relationship with distance. The method for setting the reference image area may be the same as in the first invention.

  Therefore, for each type of object extraction area variably set by the object extraction area setting means, the object in the object extraction area is based on the relationship between the parallax and the distance defined by the reference parameter. By obtaining the distance of the object from the parallax of the object, the distance can be obtained appropriately with high accuracy.

  Therefore, according to the second invention, the distance (relative distance from the vehicle) of the object based on the parallax is measured while the region for extracting the object from the captured images of the two imaging devices is variably set. It becomes possible to carry out with high accuracy.

  In the second invention, more specifically, the object extraction area set in each of the first and second captured images is, for example, between the object extraction area and the reference image area in the first captured image. What is necessary is just to set to each captured image so that a positional relationship and the positional relationship between the target object extraction area | region and reference | standard image area | region in a 2nd captured image may correspond. Further, when obtaining the parallax of the object in the object extraction area, the relative position of the object with respect to the object extraction area of the first captured image (relative position with respect to the reference point fixed with respect to the object extraction area) and The difference between the relative position of the object with respect to the object extraction area of the second captured image (relative position with respect to the reference point fixed with respect to the object extraction area) may be obtained as the parallax of the object. Similarly, for the parallax related to the reference parameter, the difference between the relative position of the object with respect to the reference image area of the first captured image and the relative position of the object with respect to the reference image area of the second captured image is determined as the target. What is necessary is just to obtain | require as a parallax of an object.

  Next, in order to achieve the above object, a third invention, which is a third aspect of the distance measuring device of the present invention, includes a first captured image and a second captured image respectively obtained from two imaging devices mounted on a vehicle. The same target object included in each captured image is extracted from the two captured images, and the position of the target object on the first captured image and the position of the target object on the second captured image are extracted. In the distance measuring device that calculates the distance of the object to the vehicle based on the parallax that is the difference between the first captured image and the second captured image, a plurality of predetermined numbers are determined in the entire area of each of the first captured image and the second captured image. For each type of image area, a parameter that defines the relationship between the parallax of the target and the distance of the target when the same target is extracted in each type of image area is set for each type of image area. Corresponding notes that are stored in advance Means, and object extraction area setting means for selectively setting an object extraction area for extracting an object from the plurality of types of image areas in each of the first captured image and the second captured image. When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, based on the relationship defined by the parameter corresponding to the object extraction area And a distance calculating means for calculating the distance of the object from the parallax of the object in the object extraction area.

  In the third aspect of the invention, the same object is extracted in each type of image area for each of a plurality of types of image areas determined in advance in the entire areas of the first captured image and the second captured image. In this case, a parameter that defines the relationship between the parallax of the target object and the distance of the target object is determined and stored in advance, and the target object extraction area setting means selectively selects from the plurality of types of image areas. The object extraction area is set in. For this reason, for each type of object extraction region selected, the corresponding parameter can be appropriate as representing the relationship between the parallax and distance of the object. In other words, the parameters need only be determined experimentally so that the relationship between the parallax and the distance is appropriate for each type of object extraction region, so that the parameters can be set appropriately.

  Therefore, for each type of object extraction area selectively set by the object extraction area setting means, the object extraction is performed based on the relationship between the parallax and the distance defined by the corresponding parameter. By obtaining the distance of the object from the parallax of the object in the region, it is possible to obtain the distance accurately and appropriately.

  Therefore, according to the third aspect of the invention, measurement of the distance of the object (relative distance from the vehicle) based on the parallax is performed while variably setting the area for extracting the object from the captured images of the two imaging devices. It becomes possible to carry out with high accuracy.

  In the third invention, as in the case of the first invention, the position of the object extraction region set in each of the first and second captured images (relative position with respect to each captured image) is the same as that of the first captured image. This is suitable when the second captured image is the same. Moreover, the method of obtaining the parallax in each type of object extraction region may be the same as in the case of the first invention.

  In addition, according to a fourth aspect of the fourth aspect of the distance measuring device of the present invention, each captured image is obtained from the first captured image and the second captured image respectively obtained from the two image capturing devices mounted on the vehicle. Based on the parallax that is the difference between the position of the target object on the first captured image and the position of the target object on the second captured image. In a distance measuring device that calculates a distance of the object to the vehicle, reference point position data indicating a position of a reference point as a reference of the position of the object in each of the first captured image and the second captured image; In addition, a reference parameter that defines the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted in each of the first captured image and the second captured image is stored and held in advance. Storage means and the first Object extraction area setting means for variably setting an object extraction area for extracting an object for each of the image image and the second captured image, and each of the first captured image and the second captured image When the same object is extracted in the object extraction area set to, the relative position of the object with respect to the reference point indicated by the reference point position data on the first captured image, and the second imaging A difference between the relative position of the object with respect to the reference point indicated by the reference point position data on the image is obtained as a parallax of the object, and the parallax is obtained from the obtained parallax based on the relationship defined by the reference parameter. A distance calculating means for calculating the distance of the object is provided.

  According to the fourth aspect of the invention, for each type of object extraction area set by the object extraction area setting means, the object relative to the reference point indicated by the reference point position data on the first captured image. Between the relative position of the object and the relative position of the object with respect to the reference point indicated by the reference point position data on the second captured image is obtained as the parallax of the object, and from the obtained parallax, the reference Since the distance of the object is calculated based on the relationship defined by the parameters, the same reference parameter is extracted in the object extraction area of each captured image for each type of object extraction area. An appropriate parameter that defines the relationship between the parallax and the distance of the object can be used. That is, since the reference point is a point determined for each of the first and second captured images, when the parallax is obtained as described above, the parallax value does not depend on the object extraction region. . Since the reference point position data and the reference parameters may be determined experimentally, they can be set appropriately.

  Accordingly, the parallax of the object is obtained as described above for each type of object extraction area variably set by the object extraction area setting means, and the parallax defined by the reference parameter is determined from the obtained parallax. By obtaining the distance of the object based on the relationship between the distance and the distance, the distance can be obtained appropriately and accurately.

  Therefore, according to the fourth invention, the distance of the object based on the parallax (relative distance from the vehicle) is measured while the region for extracting the object from the captured images of the two imaging devices is variably set. It becomes possible to carry out with high accuracy.

  In the fourth invention, the object extraction area can be set arbitrarily.

  Further, the vehicle periphery monitoring device (the fifth invention) of the present invention is a vehicle periphery monitoring device provided with the distance measuring device of the present invention described above, and the object extraction area setting means is a travel environment of the vehicle. The object extraction area is variably set according to the above.

  According to the fifth aspect of the invention, since the object extraction area is set according to the traveling environment of the vehicle, it is possible to extract and monitor the object in a spatial area in a range suitable for the traveling environment of the vehicle. .

  Note that the travel environment includes, for example, the vehicle speed and the state of the vehicle in the traveling direction (for example, whether the vehicle is traveling straight, turning right, turning left) ), The degree of approach between the vehicle and the object.

  Next, according to a sixth aspect of the first aspect of the distance measuring method of the present invention, each of the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle is captured. The same object included in the image is extracted, and based on the parallax that is the difference between the position of the object on the first captured image and the position of the object on the second captured image In the distance measuring method for calculating the distance of the object to the vehicle, the same object is extracted in a reference image area determined in advance in the entire area of each of the first captured image and the second captured image. In this case, a reference parameter that defines the relationship between the parallax of the target object and the distance of the target object and position data representing the position of the reference image area in each captured image are determined in advance, and the first Captured image and second shot A step of variably setting an object extraction region in which an object is to be extracted in each of the images; and the set target object extraction region in each of the first captured image and the second captured image The object extraction set for each of the first captured image and the second captured image based on the positional relationship between the position of the reference image region represented by the position data and the position of the reference image region and the reference parameter Determining a parameter that defines the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted in the region; and for each of the first captured image and the second captured image When the same object is extracted in the set object extraction area, the function defined by the parameter determined by the parameter determination means corresponding to the object extraction area Based on, characterized by comprising the step of calculating the distance of the object from the disparity of the object in the object extraction area.

  According to the sixth invention, as in the first invention, for each type of object extraction region that is variably set, based on the relationship between the parallax and the distance defined by the parameter, By obtaining the distance of the object from the parallax of the object in the object extraction region, the distance can be obtained appropriately with high accuracy. As a result, it is possible to accurately measure the distance of the object (relative distance from the vehicle) based on the parallax while variably setting the area for extracting the object from the captured images of the two imaging devices. Become.

  In addition, according to a seventh aspect of the second aspect of the distance measuring method of the present invention, each captured image is obtained from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Based on the parallax that is the difference between the position of the target object on the first captured image and the position of the target object on the second captured image. In the distance measuring device that calculates the distance of the object to the vehicle, the same object is extracted in a predetermined reference image area in each of the entire areas of the first captured image and the second captured image. In this case, a reference parameter that defines the relationship between the parallax of the target object and the distance of the target object and position data representing the position of the reference image area in each captured image are determined in advance, and the first imaging Image and second shot A step of variably setting an object extraction area for extracting an object for each of the images, and the same object in the object extraction area set for each of the first captured image and the second captured image And calculating the distance of the object from the parallax of the object in the object extraction area based on the relationship defined by the reference parameter, and setting the object extraction area The step is a relationship between the parallax of the target object and the distance of the target object when the same target object is extracted in the target object extraction region set in each of the first captured image and the second captured image. The object extraction region is set in accordance with the position of the reference image region represented by the position data so that is equal to the relationship defined by the reference parameter. .

  According to the seventh invention, similarly to the second invention, as described above, the object extraction area is variably set, and each kind of object extraction area is defined by the reference parameter. By calculating the distance of the target object from the parallax of the target object in the target object extraction area based on the relationship between the parallax and the distance, it is possible to determine the distance accurately and appropriately. As a result, it is possible to accurately measure the distance of the object (relative distance from the vehicle) based on the parallax while variably setting the area for extracting the object from the captured images of the two imaging devices. Become.

  In addition, according to an eighth aspect of the third aspect of the distance measuring method of the present invention, each captured image is obtained from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Based on the parallax that is the difference between the position of the target object on the first captured image and the position of the target object on the second captured image. In the distance measuring method for calculating the distance of the object to the vehicle, for each of a plurality of types of image areas determined in advance in the entire areas of the first captured image and the second captured image, Parameters that define the relationship between the parallax of the object and the distance of the object when the same object is extracted in the image area of the type are determined in advance in association with each type of image area, First captured image and second In each of the captured images, a step of selectively setting an object extraction region from which the object is to be extracted from the plurality of types of image regions, and the first captured image and the second captured image are set. When the same object is extracted in the object extraction area, the distance of the object from the parallax of the object in the object extraction area based on the relationship defined by the parameter corresponding to the object extraction area And a step of calculating.

  According to the eighth invention, similarly to the third invention, the parameters are determined corresponding to each of a plurality of types of object extraction areas, and each type of object extraction area set selectively. On the other hand, by obtaining the distance of the target object from the parallax of the target object in the target object extraction region based on the relationship between the parallax and the distance defined by the corresponding parameter, the distance can be accurately determined. It becomes possible to ask appropriately. As a result, it is possible to accurately measure the distance of the object (relative distance from the vehicle) based on the parallax while variably setting the area for extracting the object from the captured images of the two imaging devices. Become.

  In addition, according to a ninth aspect of the fourth aspect of the distance measuring method of the present invention, each captured image is obtained from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Based on the parallax that is the difference between the position of the target object on the first captured image and the position of the target object on the second captured image. In a distance measuring method for calculating a distance of the object to the vehicle, reference point position data indicating a position of a reference point as a reference of the position of the object in each of the first captured image and the second captured image; And a reference parameter that prescribes the relationship between the parallax of the object and the distance of the object when the same object is extracted in each of the first captured image and the second captured image. The first captured image is And a step of variably setting an object extraction region for extracting an object for each of the second captured images, and an object extraction set for each of the first captured image and the second captured image When the same object is extracted in a region, the relative position of the object with respect to the reference point indicated by the reference point position data on the first captured image, and the reference point on the second captured image The difference between the relative position of the object with respect to the reference point indicated by the position data is obtained as the parallax of the object, and the distance of the object is calculated from the obtained parallax based on the relationship defined by the reference parameter. And a step.

  According to the ninth aspect, as in the fourth aspect, the parallax of the object is obtained as described above for each type of object extraction region that is variably set, and the parallax is obtained from the obtained parallax. By obtaining the distance of the object based on the relationship between the parallax and the distance defined by the reference parameter, it is possible to obtain the distance accurately and appropriately. As a result, it is possible to accurately measure the distance of the object (relative distance from the vehicle) based on the parallax while variably setting the area for extracting the object from the captured images of the two imaging devices. Become.

  The tenth aspect of the first aspect of the distance measuring program of the present invention is the same extracted from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Processing for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position of the object on the first captured image and the position of the object on the second captured image When the same object is extracted in a predetermined reference image area in each of the entire areas of the first captured image and the second captured image. A reference parameter that defines the relationship between the parallax of the object and the distance of the object, position data representing the position of the reference image area in each captured image, a process of reading from the storage means, the first captured image, A process of variably setting an object extraction region for extracting an object for each of the second captured images, and the set object in each of the first captured image and the second captured image Based on the positional relationship between the position of the object extraction area and the position of the reference image area represented by the position data, and the reference parameter, each of the first captured image and the second captured image is set. A process for determining a parameter that defines a relationship between a parallax of the target object and a distance of the target object when the same target object is extracted in the target object extraction region, the first captured image, and the second captured image When the same object is extracted in the object extraction area set in each of the images, it is defined by the parameter determined by the parameter determination means corresponding to the object extraction area Based on engagement, characterized in that the parallax of the object in the object extraction area comprises a function to execute a process of calculating the distance of the object in the computer.

  According to the tenth aspect of the present invention, it is possible to cause a computer to execute processing that can achieve the effects described with respect to the first aspect of the present invention.

  The eleventh aspect of the second aspect of the distance measuring program of the present invention is the same extracted from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Processing for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position of the object on the first captured image and the position of the object on the second captured image When the same object is extracted in a predetermined reference image area in each of the entire areas of the first captured image and the second captured image. Processing for reading from the storage means a reference parameter that defines the relationship between the parallax of the object and the distance of the object, and position data representing the position of the reference image area in each captured image; and the first captured image And Processing for variably setting an object extraction region for extracting an object in each of the second captured images, and an object extraction region set for each of the first captured image and the second captured image A function of causing a computer to execute a process of calculating the distance of the object from the parallax of the object in the object extraction region based on the relationship defined by the reference parameter when the same object is extracted in And the function of causing the computer to execute the process of setting the object extraction area is such that the same object is extracted in the object extraction area set in each of the first captured image and the second captured image. The reference image region represented by the position data so that the relationship between the parallax of the target object and the distance of the target object is equal to the relationship defined by the reference parameter. Characterized in that depending on the position is a function of executing a process of setting the object extraction area to the computer.

  According to the eleventh aspect of the invention, it is possible to cause a computer to execute a process that can achieve the effects described with respect to the second aspect of the invention.

  Further, the twelfth aspect of the third aspect of the distance measuring program of the present invention is the same extracted from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Processing for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position of the object on the first captured image and the position of the object on the second captured image Is a distance measurement program for causing a computer to execute each type of image for each of a plurality of types of image areas determined in advance in the entire area of each of the first captured image and the second captured image. Processing for reading parameters defining the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted from the area, the first captured image, and the second captured image It's In addition, a process for selectively setting an object extraction area to be extracted from the plurality of types of image areas, and the object set for each of the first captured image and the second captured image. When the same object is extracted in the extraction area, the distance of the object is calculated from the parallax of the object in the object extraction area based on the relationship defined by the parameter corresponding to the object extraction area A function of causing a computer to execute processing is provided.

  According to the twelfth aspect of the present invention, it is possible to cause a computer to execute processing that can achieve the effects described with respect to the third aspect of the present invention.

  Further, the thirteenth aspect of the fourth aspect of the distance measuring program of the present invention is the same extracted from the first captured image and the second captured image respectively obtained from the two imaging devices mounted on the vehicle. Processing for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position of the object on the first captured image and the position of the object on the second captured image In the distance measuring program for causing the computer to execute, the reference point position data indicating the position of the reference point as the reference of the position of the object in each of the first captured image and the second captured image, and the first A process of reading from the storage means a reference parameter that defines the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted in each of the captured image and the second captured image; First A process of variably setting an object extraction region for extracting an object for each of the captured image and the second captured image, and an object set for each of the first captured image and the second captured image When the same object is extracted in the object extraction area, the relative position of the object with respect to the reference point indicated by the reference point position data on the first captured image and the second captured image on the second captured image The difference between the relative position of the object with respect to the reference point indicated by the reference point position data is obtained as the parallax of the object, and the distance of the object is calculated from the obtained parallax based on the relationship defined by the reference parameter. It has the function to make a computer perform the process to calculate.

  According to the thirteenth aspect, it is possible to cause a computer to execute processing that can achieve the effects described with respect to the fourth aspect.

[First Embodiment]
A first embodiment of the present invention will be described below with reference to the drawings. This first embodiment is an embodiment of the first invention, the fifth invention, the sixth invention, and the tenth invention of the present application. First, with reference to FIG. 1 and FIG. 2, the system configuration | structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. FIG. 1 is a block diagram showing the overall configuration of the periphery monitoring device, and FIG. 2 is a perspective view showing the appearance of a vehicle (vehicle) equipped with the periphery monitoring device. In FIG. 2, illustration of some components of the periphery monitoring device is omitted.

  With reference to FIGS. 1 and 2, the periphery monitoring device of the present embodiment includes an image processing unit 1. The image processing unit 1 is connected to infrared cameras 2R and 2L as two imaging devices that capture an image in front of the vehicle 1, and the yaw rate of the vehicle 10 is used as a sensor that detects the traveling state of the vehicle 10. A yaw rate sensor 3 for detecting, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle 10, and a brake sensor 5 for detecting a brake operation (specifically, whether or not a brake pedal is operated) of the vehicle 10 are connected. Has been. Further, the image processing unit 1 includes a speaker 6 for outputting auditory alarm information such as sound, and a display device for displaying images taken by the infrared cameras 2R and 2L and visual alarm information. 7 is connected.

  Although not shown in detail, the image processing unit 1 is composed of an electronic circuit including an A / D conversion circuit, a microcomputer (CPU, RAM, ROM), an image memory, and the like. The infrared cameras 2R and 2L, the yaw rate sensor 3. Outputs (analog signals) of the vehicle speed sensor 4 and the brake sensor 5 are digitized and inputted via an A / D conversion circuit. Further, although not shown in the figure, the image processing unit 1 includes an ON / OFF signal indicating whether or not the operation signal of the right / left turn direction indicator (so-called blinker) of the vehicle 10 is operated. Signal) is also input. Then, the image processing unit 1 performs processing for detecting an object such as a person (pedestrian) based on the input data, and the real space position of the detected object with respect to the vehicle 10 (the distance from the vehicle 10). And the like, a process for issuing a warning regarding the object to the driver via the speaker 6 and the display device 7 and the like are executed by the microcomputer. These processes are realized by the microcomputer executing a program pre-installed in the ROM of the microcomputer, and the program includes the distance measuring program of the present invention.

  Supplementally, the image processing unit 1 includes a function as a distance measuring device of the present invention and a function as a vehicle periphery monitoring device.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion of the vehicle 10 (the front grill portion in the figure) in order to image the front of the vehicle 10. In this case, the infrared cameras 2R and 2L are respectively arranged at a position on the right side and a position on the left side of the center of the vehicle 10 in the vehicle width direction. Their positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L basically have their optical axes extending in the front-rear direction of the vehicle 10 in parallel with each other, and the heights of the respective optical axes from the road surface are equal to each other. It is fixed to the front part of the vehicle 10. In practice, however, the orientation and height of the optical axes of the infrared cameras 2R and 2L vary to some extent due to structural variations of the infrared cameras 2R and 2L and variations in the assembly position with respect to the vehicle 10. Produce. Each infrared camera 2R, 2L is an imaging device having sensitivity in the far-infrared region, and the higher the temperature of the object imaged by the infrared camera 2R, 2L, the higher the level of the output signal of the object image (the object's image). (The brightness of the image is increased).

  In the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as HUD 7a) that displays information such as an image on the front window of the vehicle 10, for example. The display device 7 includes a display provided integrally with a meter that displays a traveling state such as the vehicle speed of the vehicle 10 in place of the HUD 7a or together with the HUD 7a, or a display provided in an in-vehicle navigation device. But you can.

  Next, the overall operation (processing of the image processing unit 1) of the periphery monitoring device of the present embodiment will be described with reference to the flowcharts of FIGS. Of the processes in the flowcharts of FIGS. 3 and 4, the same processes as those described in Patent Document 1 are not described in detail in this specification. Supplementally, the processes in the flowcharts of FIGS. 3 and 4 are processes realized by a program executed by the microcomputer of the image processing unit 1.

  The image processing unit 1 repeatedly executes the processing of STEP1 to STEP20 at a predetermined arithmetic processing cycle. In the following, the image processing unit 1 first acquires infrared images, which are output signals of the infrared cameras 2R and 2L (STEP 1), performs A / D conversion (STEP 2), and stores each image in the image memory. (STEP 3). As a result, images captured by the infrared cameras 2R and 2L are taken into the image processing unit 1. Hereinafter, an image obtained from the infrared camera 2R is referred to as a right image, and an image obtained from the infrared camera 2L is referred to as a left image. These right image and left image are both grayscale images. Note that the entire right image and left image are both rectangular two-dimensional images, and their sizes (vertical and horizontal dimensions) and the number of pixels are the same. The right image and the left image correspond to the first captured image and the second captured image in the present invention.

  Supplementally, the processes of STEP 1 to 3 are the same as the processes of S 11 to S 13 of FIG.

  Next, the image processing unit 1 sets an object extraction region in which an object is to be extracted in the right image and the left image according to the traveling environment of the vehicle 10 (STEP 4). In this case, as the traveling environment considered when setting the object extraction area, in this embodiment, the vehicle speed of the vehicle 10 (the vehicle speed detected by the vehicle speed sensor 4), the presence / absence of the blinker operation, the image processing unit 1 And the relative distance of the object extracted in the past calculation processing cycle (previous calculation processing cycle) with respect to the vehicle 10 (the relative distance of the object in the front-rear direction of the vehicle 10).

  In the present embodiment, there are two types of object extraction areas set in STEP 4; an enlarged area Sb that is a relatively large area area and a standard area Ss that is a relatively small area area. Is set in the area. Here, the enlarged region Sb and the standard region Ss will be described with reference to FIG. FIG. 5 shows the right image and the left image arranged side by side in the vertical direction.

  The enlargement area Sb set for each of the right image and the left image is an area (square area) having the same shape as each image, and the sizes (the horizontal dimension FHb and the vertical dimension FVb) are mutually equal. It is the same area. In the present embodiment, each enlarged region Sb of the right image and the left image is a rectangular region having the same size (the same area) as the entire region of the right image and the left image. Further, the standard area Ss set for each of the right image and the left image is a square area having an area smaller than the entire area of the right image and the left image, and its size (the horizontal dimension FHs and the vertical direction). , Dimensions FVs) are the same region.

  In this case, the position of the enlarged region Sb in each of the right image and the left image (enlarged region in a coordinate system (for example, the xy coordinate system Cb in FIG. 5) fixed in the same positional relationship with respect to the left and right images). The position of Sb is the same for the right image and the left image. In the present embodiment, since the size of the enlarged area Sb is the same as the entire area of each of the left and right images, the enlarged area Sb matches the entire area of the left and right images. Supplementally, “region position” means the position of a representative point (such as the center point or vertex of the region) fixed to the region.

  On the other hand, the position of the standard area Ss in each of the left and right images (the position of the standard area Ss in the coordinate system (for example, the xy coordinate system Cb in FIG. 5) fixed to the left and right images in the same positional relationship) Are generally different as in the example shown in FIG. That is, the representative point of the standard area Ss is, for example, the upper left vertex of the standard area Ss, and the position (coordinates) of the upper left vertex of the standard area Ss in the coordinate system Cb of the right image is (xrb) as shown in FIG. , Yrb), and the position (coordinates) of the upper left vertex of the standard area Ss in the coordinate system Cb of the left image is (xlb, ylb), generally xrb ≠ xlb or yrb ≠ ylb. The standard area Ss is an area in which an arrangement position in each image is set in advance by an operator when the vehicle 10 is manufactured, maintained, or inspected. How to set the position of the standard area Ss in the left and right images will be described later.

  In FIG. 5, the subscript “b” in the sign of the coordinate component value of the position regarding the point in the image indicates the coordinate component value in the coordinate system Cb of FIG. 5, and the subscript “s”. Means to indicate a position in the coordinate system Cs of FIG. The coordinate system Cs is a coordinate system fixed with respect to the standard area Ss. In any of the coordinate systems Cb and Cs, the x axis is the horizontal axis of the image, and the y axis is the vertical axis of the image. Supplementally, in this embodiment, since the entire area of each of the left and right images and the enlarged area Sb coincide with each other, the coordinate system Cb has a meaning as a coordinate system fixed to each image and at the same time enlarges It has a meaning as a coordinate system fixed to the region Sb.

  In STEP4, for example, the vehicle speed of the vehicle 10 is equal to or lower than a predetermined speed (relatively low), and the requirement that the turn indicator for turning right and left of the vehicle 10 is operating (vehicle 10). And the relative distance to the vehicle 10 (the distance in the front-rear direction of the vehicle 10) among the objects extracted in the past (for example, the previous calculation processing cycle) in advance. When any one of the requirements that there is an object (an object having a relatively short distance from the vehicle 10) that is equal to or less than the predetermined value is satisfied, the enlarged area Sb is used as the object extraction area. Is set. When none of the above requirements is satisfied, the standard area Ss is set as the object extraction area.

  Note that the processing of STEP 4 constitutes the object extraction area setting means in the present invention.

  Supplementally, since an object existing in front of the vehicle 10 is detected in the object extraction area of the right image and the left image as described later, changing the object extraction area means that the infrared camera 2R is changed. , The same effect as changing the viewing angle (view angle) of 2L. In this case, the horizontal dimension of the object extraction region corresponds to the horizontal viewing angle (angle of view) of the infrared cameras 2R and 2L, and the vertical dimension of the object extraction region is vertical to the infrared cameras 2R and 2L. This corresponds to the viewing angle (view angle) of the direction. Accordingly, the horizontal dimension and the vertical dimension of each of the object extraction regions Sb and Ss may be set according to the angle of view required corresponding to each.

  In the present embodiment, the enlarged regions Sb of the right image and the left image are matched with the entire region of each image, but may be a region having an area smaller than the entire region. For example, an area indicated by an imaginary line area Sb ′ in FIG. 5 may be an enlarged area. However, in the method of calculating the distance of the object in the present embodiment, the horizontal position of the enlarged region in each of the left and right images is set to be the same position in the right image and the left image. In this case, the position of the upper left vertex of the enlarged area in the coordinate system Cb of FIG. 5 is (xb, yb), the horizontal sampling number of the entire area of the left and right images is FHm, and the vertical sampling number is FVm. Then, the enlarged regions may be set in the left and right images so that the values of xb and yb (values in units of pixels) are values given by the following equations (1) and (2), respectively. Here, the “sampling number in the horizontal direction” and the “sampling number in the vertical direction” are respectively the number of divisions [pixel] of the entire area of the image in the horizontal direction and the number of divisions [pixel] of the entire area of the image in the vertical direction. means.

xb = (1/2) * FHm * (1-RHfov / MHfov) (1)
yb = (1/2) * FVm * (1-RVfov / MVfov) (2)

In the equations (1) and (2), RHfov is a required field angle in the horizontal direction (lateral direction) of the infrared cameras 2R and 2L, and RVfov is a required field angle in the vertical direction (vertical direction) of the infrared cameras 2R and 2L. , MHfov is the maximum horizontal field angle of the infrared cameras 2R and 2L, and RVfov is the maximum vertical field angle of the infrared cameras 2R and 2L.

  Returning to the description of the flowcharts of FIGS. 3 and 4, the image processing unit 1 then uses one of the right image and the left image as a reference image, and binarizes the reference image (STEP 5). The reference image is a right image in the present embodiment. In this binarization process, the luminance value of each pixel of the reference image is compared with a predetermined luminance threshold value, and an area (relatively bright area) having a luminance value higher than the predetermined luminance threshold value in the reference image. This is a process of setting “1” (white) and setting an area having a luminance value lower than the luminance threshold (a relatively dark area) to “0” (black). Hereinafter, an image (monochrome image) obtained by the binarization process is referred to as a binarized image. In the binarized image, an area “1” is referred to as a high luminance area. The binarized image is stored in the image memory separately from the grayscale image (right image and left image). The binarization process is performed in the object extraction area set in STEP 4 in the reference image.

  Next, the image processing unit 1 performs the processing of STEP 6 to 9 on the binarized image, and extracts an object (more precisely, an image portion corresponding to the object) from the binarized image. That is, the pixel group constituting the high luminance area of the binarized image is classified into lines having a width of one pixel in the vertical direction (y direction) of the reference image and extending in the horizontal direction (x direction). Each line is converted into run-length data composed of the coordinates and length (number of pixels) of the position (two-dimensional position on the reference image) (STEP 6). A label (identifier) is assigned to each line group that overlaps in the vertical direction of the reference image among the lines represented by the run-length data (STEP 7), and each line group is extracted as an object. (STEP 8). Thereby, the object is extracted within the object extraction area of the reference image.

  Note that the objects extracted by the processing in STEPs 6 to 8 generally include not only people (pedestrians) but also artificial structures such as other vehicles. In addition, a plurality of local parts of the same object may be extracted as the target object.

  FIG. 5 shows a schematic example of a right image and a left image obtained from the infrared cameras 2R and 2L at night. In this example, another vehicle exists in front of the vehicle 10, and the entire other vehicle is captured in the right image and the left image as schematically indicated by a broken line. In this case, the portions denoted by reference symbols T1 to T5 in FIG. 5 are high luminance regions, and those existing in the object extraction region among these portions T1 to T5 are extracted as the objects. T1 and T2 are portions corresponding to the tail lamp of the other vehicle, T4 and T5 are portions corresponding to the rear wheels of the other vehicle, and T3 is a portion corresponding to the exhaust pipe of the other vehicle. In FIG. 5, T1 to T5 are schematically shown by solid lines. In general, the shape of an object extracted from an actual image does not have a uniform shape as shown in FIG.

  Supplementally, the binarization processing in STEP 5 may be performed in the entire area of the reference image, not in the object extraction area set in STEP 4. In this case, the processing of STEPs 6 to 8 may be performed within the object extraction area set in STEP 4. Moreover, the processing of STEP5-8 is the same as the processing of S14-S17 of the said patent document 1 of FIG. 3 except the target object extraction area | region which performs these processes.

  Next, the image processing unit 1 obtains the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle (STEP 9). The position of the center of gravity of each object is obtained by multiplying the coordinates of the position of each line (the center position of each line) of the run length data included in the object by the length of the line. It is obtained by adding all the lines of included run length data and dividing the addition result by the area of the object. Further, instead of the center of gravity of each object, the position of the center (center) of the circumscribed rectangle of the object may be obtained.

  Next, the image processing unit 1 traces the object extracted in STEP 8 during the time, that is, recognizes the same object for each calculation processing cycle of the image processing unit 1 (STEP 10). In this process, when the object A is extracted by the processing of STEP 8 at a time (discrete system time) k in a certain arithmetic processing cycle, and the object B is extracted by the processing of STEP 8 at the time k + 1 of the next arithmetic processing cycle. The identity of these objects A and B is determined. This identity determination may be performed based on, for example, the shape and size of the objects A and B on the binarized image, the correlation of the luminance distribution on the reference image (grayscale image), and the like. . When it is determined that the objects A and B are the same, the label of the object B extracted at time k + 1 (the label attached in STEP 7) is changed to the same label as the label of the object A. The

  Next, the image processing unit 1 reads the outputs (the detected vehicle speed value and the detected yaw rate value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (STEP 11). In STEP 11, the turning angle (azimuth angle) of the vehicle 10 is also calculated by integrating the read detected yaw rate value. Supplementally, the processing of STEPs 9 to 11 is the same as the processing of S18 to S20 of Patent Document 1.

  On the other hand, the image processing unit 1 executes the processes of STEPs 12 to 14 in parallel with the processes of STEPs 10 and 11. The processing of STEPs 12 to 14 is processing for obtaining the distance (relative distance) of each object extracted in STEP 8 from the vehicle 10. The process will be schematically described. First, an area including each object is extracted as a search image R1 from the right image (reference image) (STEP 12). The search image R1 is, for example, a circumscribed square area of the object.

  Next, a search that is an area for searching for the same object as the object included in the search image R1 of the right image within the object extraction area of the left image set in STEP 4 (an area wider than the search image R1). A region R2 is set, and a region having the highest correlation with the search image R1 (region having the same shape and size as the search image R1) within the search region R2 is an image corresponding to the search image R1 (with the search image R1). (Corresponding image) is extracted as the corresponding image R3 (STEP 13).

  Specifically, regions having the same shape and size as the search image R1 are set as correlation determination regions at a plurality of positions in the search region R2 of the left image, and the luminance distribution and search of the correlation determination region at each position are searched. A feature amount representing the degree of coincidence with the luminance distribution of the image R1 is calculated. As the feature amount, for example, the absolute value of the difference between the luminance value of each pixel in the correlation determination area and the luminance value of the pixel of the search image R1 corresponding to the pixel is obtained for all the pixels in the correlation determination area. An absolute difference sum obtained by integration is used. In this case, the smaller the feature value (absolute difference sum) is, the higher the degree of coincidence between the luminance distribution of the correlation determination region and the luminance distribution of the search image R1 is. Then, the correlation determination area where the value of the feature amount (absolute difference sum) is the smallest is the area having the highest correlation with the search image R1, and that area is extracted as the corresponding image R3. Note that the processing in STEP 13 is performed using a grayscale image, not a binarized image.

  Supplementally, the search area R2 may be set in an area where there is a high possibility that the corresponding image R3 exists in the search area R2, and corresponds to an object from which the search image R1 is extracted in time series from the past. In some cases, the set position and dimensions of the search region R2 may be changed as appropriate based on the past history of the distance of the object (relative distance from the vehicle 10). In order to search the corresponding image R3 in as short a time as possible, it is desirable to set the search region R2 narrower.

  Next, the position of the center of gravity of the search image R1 in the object extraction area of the right image (or the position of the center of gravity of the object (high luminance area) included in the search image R1) and the correspondence in the object extraction area of the left image The number of pixels in the lateral difference from the position of the center of gravity of the image R3 (or the position of the center of gravity of the target object (high luminance region) included in the corresponding image R3) is calculated as the parallax Δd, and the target A distance z of the object from the vehicle 10 (a distance in the front-rear direction of the vehicle 10) is calculated (STEP 14). Details of the parallax calculation and the distance calculation processing will be described later.

  The above is the outline of the processing of STEPs 12 to 14. In addition, the processing of STEP12-14 is performed with respect to each target object extracted by said STEP8. Supplementally, the processing of STEP 14 includes functions as parameter determination means and distance calculation means in the present invention.

  After the processing of STEP 11 and STEP 14, the image processing unit 1 next calculates a real space position that is a position (relative position with respect to the vehicle 10) of each object in the real space (STEP 15). Here, as shown in FIG. 2, the real space position is a position (X, Y, Z) in the real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R, 2L as the origin. ). The X direction and Y direction of the real space coordinate system are the vehicle width direction and the vertical direction of the vehicle 10, respectively. These X direction and Y direction are the x direction (lateral direction) and y direction of the right image and the left image, respectively. (Vertical direction) and the same direction. The Z direction in the real space coordinate system is the front-rear direction of the vehicle 10. Then, the real space position (X, Y, Z) of the object is calculated by the following equations (3), (4), (5).

X = x × z × p / f (3)
Y = y × z × p / f (4)
Z = z (5)

Note that x and y are the x-coordinate and y-coordinate of the object on the reference image. However, the coordinate system in this case is an xy coordinate system having an origin in the vicinity of the center point of the reference image, although illustration is omitted. The origin is a point determined in advance so that the x coordinate and the y coordinate on the reference image of the target object are both zero when the target object exists on the Z axis of the real space coordinate system. .

  Next, the image processing unit 1 compensates for the influence of the change in the turning angle of the vehicle 10 and increases the accuracy of the real space position of the object, out of the real space position (X, Y, Z) of the object. The position X in the X direction is corrected according to the time-series data of the turning angle obtained in STEP 10 from the value obtained by the above equation (3) (STEP 16). Thereby, the real space position of the object is finally obtained. In the following description, “the real space position of the object” means the real space position of the object subjected to this correction.

  Next, the image processing unit 1 obtains a movement vector of the object with respect to the vehicle 10 (STEP 17). Specifically, a straight line approximating time series data in a predetermined period (a period from the current time to a predetermined time before, hereinafter referred to as a monitoring period) of the real space position for the same object is obtained, and the time before the predetermined time is obtained. A vector from the position (point) of the object on the straight line to the position (point) of the object on the straight line at the current time is obtained as a movement vector of the object. This movement vector is proportional to the relative velocity vector of the object with respect to the vehicle 10. Note that the processing of STEPs 15 to 17 is the same as the processing of S21 to S23 of FIG.

  Next, the image processing unit 1 executes an avoidance target determination process for determining whether or not each object extracted in STEP 8 is an avoidance object that should avoid contact with the vehicle 10 (STEP 18). This avoidance target determination process will be described below with reference to FIGS. FIG. 6 is a flowchart showing the avoidance target determination process in STEP 18, and FIG. 7 is a diagram for explaining an area used in this avoidance target determination process.

  Referring to FIG. 6, in the avoidance target determination process, first, a first object position determination process is executed as a first determination process related to the real space position of the target object (STEP 31). The first object position determination process is a process for determining whether or not contact between the vehicle 10 and the object can be avoided with sufficient margin by steering or braking operation of the vehicle 10. Specifically, in the first object position determination process, the vehicle within the area within the viewing angle of the infrared cameras 2R and 2L corresponding to the object extraction area set in STEP 4 (hereinafter referred to as an imaging monitoring area) The current real space position of the object (current value of the real space position) is in an area where the distance in the Z direction from 10 (the distance in the front-rear direction of the vehicle 10) is a predetermined value or less (hereinafter referred to as the first area). It is determined whether or not it exists.

  In this case, the predetermined value related to the distance from the vehicle 10 is set for each object. Specifically, by dividing the Z direction component of the movement vector by the time of the monitoring period for obtaining the movement vector in STEP 17, the average speed Vz of the object in the monitoring period (before and after the vehicle 10) Average value Vz) of the relative velocity of the object in the direction is obtained, and a value Vz · T obtained by multiplying this average velocity Vz by a predetermined constant T (constant of time dimension) is the Z direction of the first region. Is set as the predetermined value that defines the boundary.

  Here, the first region will be described with reference to FIG. In FIG. 7, the imaging monitoring area when the object extraction area is set to the enlarged area Sb is represented by an area between the straight lines L1b and L2b in the drawing in plan view, and the object extraction area is The imaging monitoring area in the case where the standard area Ss is set is represented by an area between the straight lines L1s and L2s in the drawing in plan view. Accordingly, the first region is a region surrounded by the triangle abc or ade in FIG. 7 in plan view. Note that the first region abc or ade is a region having a predetermined height (for example, about twice the height of the vehicle 10) in the vertical direction.

  The first object position determination process in STEP 31 is a process for determining whether or not the object exists in the first region abc or ade, and the Z-direction position of the current real space position of the object is Vz. When it is T or less and the Y-direction position is a position not more than a predetermined height, it is determined that the object is present in the first region. When the relative speed Vz of the object in the front-rear direction of the vehicle 10 is a relative speed away from the vehicle 10, it is determined that the object does not exist in the first region.

  In STEP 31, when it is determined that the object does not exist in the first region abc or ade (when the determination result in STEP 31 is NO), the object and the vehicle 10 are moved by steering or braking operation of the vehicle 10. This is a situation where contact can be avoided with a margin. In this case, the image processing unit 1 determines in STEP 36 that the object is not an avoidance target, and ends the avoidance target determination process for the object.

  On the other hand, when it is determined in STEP 31 that the object is present in the first region abc or ade (when the determination result in STEP 31 is YES), the image processing unit 1 further relates to the real space position of the object. A second object position determination process as a second determination process is executed (STEP 32). This second object position determination process is a process for determining whether or not the possibility of contact between the vehicle 10 and the object is high when the real space position of the object is maintained at the current position. is there. Specifically, in the second object position determination process, the object extends in the front-rear direction on both sides of the vehicle 10 as shown in FIG. 7 (extending in parallel with the vehicle width center line L0 of the vehicle 10). It is determined whether or not it exists in an area AR2 (hereinafter referred to as a second area AR2) between a pair of boundary lines L3 and L4 set as follows.

  In this case, the left and right boundary lines L3 and L4 of the second area AR2 have the same distance W / 2 left and right from the vehicle width center line L0 of the vehicle 10, as shown in FIG. It is set to the position to have. The interval W between the boundary lines L3 and L4 is set to be slightly wider than the vehicle width α of the vehicle 10. Whether or not the object exists in the second area AR2 depends on whether the value of the X direction component of the current real space position of the object is the X direction position of the boundary line L3 and the X direction position of the boundary line L4. It is determined by whether the value is between.

  Supplementally, the width W of the second area AR2 is changed in accordance with the traveling environment of the vehicle 10 (the vehicle speed of the vehicle 10, the operation of the blinker, etc.) considered for setting the object extraction area in STEP4. May be. In this case, in the traveling environment in which the enlarged region Sb is set as the object extraction region, it is preferable that the width of the second area AR2 is made wider than in the traveling environment in which the standard region Ss is set.

  In STEP 32, when it is determined that the real space position of the object exists in the second area AR2 (when the determination result in STEP 32 is YES), the object remains at the current real space position. There is a high possibility that the object comes into contact with the vehicle 10. In this case, the image processing unit 1 next performs a process of determining whether the object is a pedestrian (person) (more accurately, whether or not there is a high possibility of being a person) (STEP 33). A determination as to whether or not the object is a pedestrian may be made using a known method (for example, a method described in Japanese Patent Application Laid-Open No. 2003-284057), and the shape of the object on the grayscale image This can be done based on features such as size and luminance distribution.

  In this pedestrian determination process of STEP 33, if it is determined that the object is likely to be a pedestrian, the process proceeds to STEP 34 in order to increase the reliability of the determination that the object is a pedestrian. A determination process (artificial structure determination process) is performed as to whether or not the object is an artificial structure such as another vehicle or a utility pole. Whether or not the object is an artificial structure may be determined by using a known method (for example, a method described in JP2003-16429A or JP2003-230134A). The image (grayscale image or binarized image) can be performed based on the presence or absence of a straight line portion or a right angle portion, the presence or absence of a window shield portion of another vehicle, or the like. Alternatively, it may be determined whether or not the object is an artificial structure by a pattern matching method.

  When it is determined in STEP 34 that the object is not an artificial structure (when the determination result in STEP 34 is NO), the image processing unit 1 determines in STEP 35 that the object is an avoidance target. Then, the avoidance target determination process ends. Therefore, when it is determined that the object is present in the second area AR2 in the first area abc or ade and the object is a pedestrian and is not an artificial structure, It is determined that the object is an avoidance target.

  If it is determined in STEP 33 that the object is not a pedestrian, or if it is determined in STEP 34 that the object is an artificial structure, the process proceeds to STEP 36 and it is determined that the object is not an avoidance target. The avoidance target determination process ends.

  On the other hand, when it is determined in STEP 32 that the object does not exist in the second area AR2 (when the determination result in STEP 32 is NO), the image processing unit 1 next relates to the moving direction of the object. An approach contact determination process is executed (STEP 37). This approach contact determination process is a process for determining whether or not an object enters the second area AR2 and is highly likely to come into contact with the vehicle 10. Specifically, assuming that the movement vector of the object is maintained as it is, the position in the X direction of the intersection point between the straight line including this movement vector and the XY plane of the real space coordinate system at the front end position of the vehicle 10 is obtained. It is done. Then, the obtained X-direction position must be within a predetermined range (a range slightly wider than the vehicle width of the vehicle 10) centered on the X-direction position of the vehicle width center line L0 of the vehicle 10 (hereinafter referred to as an approach). It is determined whether or not this approach contact requirement is satisfied.

  In STEP 37, when the object satisfies the entry contact requirement (when the determination result in STEP 37 is YES), the object is highly likely to come into contact with the vehicle 10 in the future. Therefore, in this case, the image processing unit 1 determines in STEP 35 that the target object is an avoidance target, and ends the avoidance target determination process.

  In STEP 37, when the object does not satisfy the entry contact requirement (when the determination result in STEP 37 is NO), the object is less likely to come into contact with the vehicle 10, so the image processing unit 1 In STEP 36, it is determined that the object is not an avoidance target, and the avoidance target determination process is terminated.

  The above is the details of the avoidance target determination process of STEP18.

  Returning to the description of the flowcharts of FIGS. 3 and 4, when it is determined in STEP 18 that the target is not an avoidance target (when there is no target determined to be an avoidance target). The determination result of STEP18 is NO. In this case, the processing of the current processing cycle of the image processing unit 1 ends, and the processing from STEP 1 is restarted at the next processing cycle.

  Further, in the avoidance target determination process of STEP 18, when the target is determined to be the avoidance target (when there is an target determined to be the avoidance target), the determination result of STEP 18 is YES. In this case, proceeding to STEP 19, the image processing unit 1 executes an alarm output determination process for determining whether or not an actual alarm regarding an object determined to be an avoidance target should be performed. In this alarm output determination process, it is confirmed from the output of the brake sensor 5 that the driver has operated the brake of the vehicle 10, and the deceleration acceleration of the vehicle 10 (acceleration in the vehicle speed decreasing direction is positive). ) Is larger than a predetermined threshold (> 0), it is determined that no alarm is given. Further, when the driver does not perform a brake operation or when the deceleration acceleration of the vehicle 10 is equal to or less than a predetermined threshold even if the brake operation is performed, it is determined that an alarm should be issued.

  If the image processing unit 1 determines that an alarm should be issued (when the determination result in STEP 19 is YES), the image processing unit 1 issues an alarm by the speaker 6 and the display device 7 to the driver of the vehicle 10. An alarm generation process is executed (STEP 20). Then, after this alarm generation process, the process in the current calculation process cycle is completed, and the process from STEP 1 is resumed in the next calculation process cycle. In the alarm generation process, for example, the reference image is displayed on the display device 7 and the image of the target object to be avoided in the reference image is highlighted. Furthermore, a voice guidance is provided from the speaker 6 to the driver that such an object exists. This alerts the driver to the object. Note that the warning for the driver may be performed by only one of the speaker 6 and the display device 7. When the vehicle 10 is a vehicle that can perform automatic steering, the steering control of the vehicle 10 may be executed so as to avoid contact with an object that is determined to be an avoidance object in STEP 18. .

  Further, when it is determined in STEP 19 that no warning is given (when it is determined that no warning is given for all objects), the determination result in STEP 21 is NO, and in this case, the current calculation processing cycle is continued as it is. The processing is completed, and the processing from STEP1 is resumed in the next calculation processing cycle.

  The above is the overall operation of the periphery monitoring device of this embodiment.

  Next, the processing of STEP 14 (processing for calculating the parallax and distance of the target object) and the setting of the standard area Ss, which will be described later, will be described in detail.

  In the present embodiment, the standard area Ss corresponds to the reference image area in the present invention. The standard area Ss is an area that is set in advance by an operator at the time of manufacturing the vehicle 10 or during maintenance / inspection. The setting is performed as follows.

  That is, the aiming object is arranged in front of the vehicle 10 in a state where the vehicle 10 is stopped at the time of manufacturing or maintenance / inspection of the vehicle 10. In this case, the relative position of the aiming object with respect to the vehicle 10 is a predetermined position. In this state, the relative positional relationship between the object for aiming in the captured images (right image and left image) of the infrared cameras 2R and 2L and the standard area Ss of each image (fixed with respect to the standard area Ss). The position of the standard region Ss in each image is set so that the determined reference point (for example, the positional relationship with the upper left vertex of the standard region Ss) has a predetermined positional relationship. In other words, the coordinate value of the position of the object for aiming (the center of gravity or the position of the center of the object) in the coordinate system (two-dimensional coordinate system) fixed with respect to the standard region Ss of each image is a predetermined value. The position of the standard area Ss in each image is set so that the coordinate value becomes. In this case, the predetermined position in the vertical direction of the object for aiming with respect to the standard area Ss is the same in the left and right images, but the predetermined position in the horizontal direction of the object for aiming with respect to the standard area Ss is It is different in the image.

  Here, the optical axes of the infrared cameras 2R and 2L are accurately extended in parallel with the longitudinal direction of the vehicle 10, and the distance and the height of the optical axes are as designed (hereinafter referred to as the design values). In the ideal assembled state of the infrared cameras 2R and 2L), when the standard area Ss is set as described above, each of the standard areas Ss is located at a position where the center point coincides with the center point of the left image and the right image. Set to image.

  However, in general, the assembled state of the infrared cameras 2R and 2L is deviated from the ideal assembled state (shift of the optical axis and height). Therefore, in general, the standard region Ss is not set at the center of the left and right images, but is set at a position shifted from the center of each image, for example, as shown in FIG.

  Further, after the standard area Ss is set as described above, the lateral difference between the relative position of the aiming object with respect to the standard area Ss of the left image and the relative position of the aiming object with respect to the standard area Ss of the right image The number of pixels Δds is obtained as the parallax of the aiming object, and the parallax Δd0 and the relative distance z0 of the aiming object in the front-rear direction (Z direction) of the vehicle 10 (this is a default value) The reference parallax offset amount Ldisp (amount in units of the number of pixels) is determined so as to satisfy the relationship (5).

z0 = (f × D) / ((Δd0 + Ldisp) × p) (5)

In Expression (5), f is the focal length of the infrared cameras 2R and 2L, D is the base length that is the distance between the optical axes of the infrared cameras 2R and 2L, and p is the pixel interval (interval between adjacent pixels). . f and D are design values in the ideal assembled state. Further, the relative position of the aiming object with respect to the standard area Ss is generally for aiming with respect to a reference point (reference point having a predetermined positional relationship with respect to the standard area Ss) fixed with respect to the standard area Ss. The relative position of the object. In other words, it is the position of the aiming object in the coordinate system (two-dimensional coordinate system) fixed with respect to the standard region Ss. In the present embodiment, the relative position of the object with respect to the standard area Ss in the left and right images is, for example, as shown in FIG. 5, the position of the object in the coordinate system Cs with the upper left vertex of the standard area Ss as the origin. (This means the relative position of the object with respect to the reference point when the upper left vertex of the standard area Ss is used as the reference point).

  Here, since the position of the object in each of the left and right images, and thus the parallax, can be calculated only in units of pixels, the parallax that can be calculated is a discrete value in units of length ([m], [cm], etc.). It becomes. For this reason, assuming that Ldisp in Expression (5) is 0, the relative distance obtained by the calculation of the right side of Expression (5) causes an error with respect to the actual relative distance. A correction parameter for compensating for this error is the reference parallax offset amount Ldisp.

In the present embodiment, the reference parallax offset amount Ldisp determined as described above at the time of manufacturing or maintenance / inspection of the vehicle 10 defines the relationship between the parallax of the object extracted on the standard area Ss and the relative distance. One of the reference parameters is stored and held in a non-illustrated non-volatile memory (writeable / erasable non-volatile memory) such as an EEPROM provided in the image processing unit 1.
This nonvolatile memory corresponds to the storage means in the present invention.

  Note that f, D, and p on the right side of Expression (5) are also reference parameters that define the relationship between the parallax of the object extracted on the standard region Ss and the relative distance. It is a fixed value and is stored and held in advance in the ROM or EEPROM of the image processing unit 1. Data indicating the position of the standard area Ss in the left and right images is also stored and held in the nonvolatile memory together with the reference parallax offset amount Ldisp. Data indicating the position of the standard area Ss in the left and right images is the position of the representative point of the standard area Ss in the coordinate system fixed to each image. In the present embodiment, for example, the upper left vertex of the standard region Ss in the coordinate system Cb with the upper left vertex of the entire area of each of the left and right images (in the present embodiment, this corresponds to the enlarged region Sb) as the origin ( The position (coordinates) of the representative point of the standard area Ss is stored and held in the nonvolatile memory as position data representing the position of the standard area Ss in each image.

Next, the parallax and distance calculation processing in STEP 14 will be described in detail below.
This processing is realized by executing a subroutine program pre-installed in the ROM of the microcomputer by the microphone computer.

  In the present embodiment, the parallax Δd of each object extracted in STEP 8 is the object extraction in the right image regardless of whether the object extraction area is set to the standard area Ss or the enlarged area Sb. It is calculated as the number of pixels of the difference between the relative position of the object relative to the area (lateral relative position) and the relative position of the object relative to the object extraction area (lateral relative position) in the left image. In this case, the positional relationship between the enlarged region Sb and the standard region Sa in the right image and the positional relationship between the enlarged region Sb and the standard region Sb in the left image are generally different from each other. The value of generally depends on the object extraction area.

  Specifically, referring to FIG. 5, when the object extraction area is set to the standard area Ss, the relative position of the object to the standard area Ss in the right image is the standard area Ss of the right image. As the position (coordinates) of the object in the coordinate system Cs with the origin at the upper left vertex. The same applies to the left image. Therefore, the parallax of the target object when the target object extraction area is set to the standard area Ss is the lateral position of the target object in the coordinate system Cs in the right image (x coordinate in the coordinate system Cs) and the left image. In the coordinate system Cs in the horizontal direction position (x coordinate in the coordinate system Cs). For example, the position (coordinates) of the object T3 in the right image in the coordinate system Cs is (x3rs, y3rs), and the position (coordinates) of the object T3 in the left image in the coordinate system Cs is (x3ls, y3ls). Then, the parallax Δd3s of the object T3 when the object extraction area is set to the standard area Ss is calculated as the number of pixels of | x3rs−x3ls |.

  The left and right coordinate systems Cs are set on each image based on the position data of the standard area Ss stored and held in the nonvolatile memory as described above.

  When the object extraction area is set to the enlarged area Sb, the relative position of the object in the right image with respect to the standard area Ss is the upper left vertex of the enlarged area Sb of the right image (in this embodiment, This is obtained as the position (coordinates) of the object in the coordinate system Cb with the origin at the top left vertex of the entire area of the right image. The same applies to the left image. Therefore, the parallax of the target object when the target object extraction area is set to the enlarged area Sb is the horizontal position of the target object in the coordinate system Cb in the right image (x coordinate in the coordinate system Cb) and the left image. In the coordinate system Cb in the horizontal direction position (x coordinate in the coordinate system Cb). For example, if the position (coordinates) of the object T3 of the right image in the coordinate system Cb is (x3rb, y3rb) and the position (coordinates) of the object T3 of the left image in the coordinate system Cb is (x3lb, y3lb). The parallax Δd3b of the object T3 when the object extraction area is set to the enlarged area Sb is calculated as the number of pixels of | x3rb−x3lb |.

  Hereinafter, when the object extraction area is set to the standard area Ss, the parallax of the same object (for example, T3 in the figure) extracted in the respective standard areas Ss of the left and right images is generally represented by Δds. Similarly, when the object extraction area is set to the enlarged area Sb, the parallax of the same object (for example, T1 to T5 in the figure) extracted in the enlarged areas Sb of the left and right images is generally Δdb Represented by

  Then, the relative distance of the object to the vehicle 10 is calculated from the parallax Δds and Δdb obtained as described above.

  That is, when the object extraction area is set to the standard area Ss, the standard area Ss matches the reference image area, so that the image processing unit 1 is stored and held in the nonvolatile memory as described above. The focal length f, the baseline length D, the pixel interval p, and the reference parallax offset Ldisp are determined as parameters that define the relationship between the parallax of the object and the relative distance.

  Then, the relative distance z of the object is calculated from the parallax obtained as described above by the following equation (6) equivalent to the equation (5).

z = (f × D) / ((Δds + Ldisp) × p) (6)

On the other hand, when the object extraction area is set as the enlarged area Sb, the positional relationship between the enlarged area Sb in each of the right image and the left image and the positional relation in the standard area Ss (reference image area) are generally set. Since there is a difference, the relationship between the parallax and the distance of the target object when the target object extraction area is set to the enlarged area Sb is the relationship between the target object when the target object extraction area is set to the enlarged area Sb. In general, the relationship between parallax and distance is different. Therefore, even if the parallax Δdb determined as described above is applied to the equation (6) as it is, the relative distance of the object cannot be calculated correctly.

  Therefore, in this case, the image processing unit 1 corrects the reference parallax offset amount Ldisp among the reference parameters as follows, and determines a parameter that defines the relationship between the parallax and the relative distance.

  That is, based on the position data of the standard area Ss (reference image area) stored and held in the nonvolatile memory, the horizontal position of the standard area Ss in the coordinate system Cb of the right image and the coordinate system Cb of the left image A deviation Δx (= xrb−xlb) from the horizontal position of the standard area Sb is calculated. Then, by adding this deviation Δx to the reference parallax offset amount Ldisp stored in the nonvolatile memory, the parallax offset amount Sdisp corresponding to the enlarged region Sb is calculated.

  That is, the parallax offset amount Sdisp is calculated by the following equation (7).

Sdisp = (xrb−xlb) + Ldisp
= Δx + Ldisp (7)

Then, the set of the parallax offset amount Sdisp and the focal length f, the base length D, and the pixel interval p obtained in this way is determined as a parameter representing the relationship between the parallax and the distance.

  Here, xrb indicates the horizontal position of the standard area Ss (reference image area) in the coordinate system Cb of the right image, and represents the positional relationship between the enlarged area Sb and the standard area Ss in the right image. It becomes. Further, xlb indicates the horizontal position of the standard area Ss (reference image area) in the coordinate system Cb of the left image, and thus represents the positional relationship between the enlarged area Sb and the standard area Ss in the left image. Become.

  Therefore, the parallax offset amount Sdisp is determined based on the positional relationship between the enlarged region Sb and the standard region Ss in the left and right images and the reference parallax offset amount Ldisp.

  Then, after determining the parameters (including the parallax offset amount Sdisp) corresponding to the enlarged region Sb as described above, the distance z of the object is calculated from the parallax Δdb of the object by the following equation (8).

z = (f × D) / ((Δdb + Sdisp) × p) (8)

The above is the details of the processing of STEP14.

  Supplementally, the process of determining a parameter (particularly the amount of parallax offset) that defines the relationship between the parallax and the distance corresponding to each object extraction region corresponds to the process of the parameter determination unit in the present invention. And the process which calculates the distance z of a target object by Formula (6) or (8) is equivalent to the process of the distance calculation means in this invention.

  By calculating the distance of the object by obtaining the parallax as described above, the distance of the object is accurately calculated regardless of whether the object extraction area is set to the enlarged area Sb or the standard area Ss. be able to.

  Moreover, in this embodiment, the target object extraction area | region suitable for a driving environment can be set by setting a target object extraction area | region according to the above-mentioned driving environment of the vehicle 10. FIG. For example, as shown in FIG. 5, when the other vehicle in front of the vehicle 10 is close to the vehicle 10, by setting the object extraction region to the enlarged region Sb, the entire other vehicle that cannot be captured in the standard region Ss. Since it can be captured in the enlarged region Sb, for example, in the artificial structure determination process of STEP 34, it is easy to recognize that the objects T1 to T5 are components of another vehicle. In addition, when the winker of the vehicle 10 is operating, that is, when the vehicle 10 is about to turn left or right, the object extraction area is set to the enlarged area Sb, so a wide area in front of the vehicle 10 It is possible to extract the object in (a wide area in the vehicle width direction). For this reason, it is possible to alert the driver over a wide range in front of the vehicle 10.

  Supplementally, in the present embodiment, as described above, the standard region Ss as the reference image region has the same vertical position (height) of the aiming object with respect to the standard region Ss in the left and right images. The standard area Ss is set. Therefore, although the vertical position of the standard area Ss in each image is generally different between the left and right images, if the same object exists in the standard area Ss of both images, the standard area Ss of the object is on the standard area Ss. Are substantially the same height. Therefore, when searching for the corresponding image on the standard area Ss of the left image in STEP 13, the corresponding image is searched at the same height position as the height of the search image on the standard area Ss of the right image. The search can be performed efficiently.

On the other hand, since the vertical positional relationship between the enlarged regions Sb of the left and right images is generally different from the vertical positional relationship of the standard regions Ss of the left and right images, it is the same as the enlarged regions Sb of both images. In general, the height of the object on the enlarged region Sb is different. In this case, when searching for a corresponding image on the standard area Ss of the left image in STEP 13, the vertical position of the standard area Ss in the right image (for example, the vertical position “yrb” in the coordinate system Cb of FIG. 5). And a height different from the height of the search image on the enlarged region Ss of the right image by the difference between the vertical position of the standard area Ss in the left image (for example, the vertical position “ylb” in the coordinate system Cb of FIG. 5). By searching for the corresponding image at the vertical position (the height position of the left image on the enlarged region Ss), the search can be performed efficiently. When the enlarged area Sb is set to be smaller than the entire area of each image, the difference in the vertical position of the enlarged area Sb in each image is the same as the difference in the vertical position of the standard area Ss. As described above, if the enlarged region Sb in each image is set, the height of the same object on the enlarged region Sb in each of the left and right images becomes substantially the same in the left and right images.
[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG. This second embodiment is an embodiment of the second invention, fifth invention, seventh invention, and eleventh invention of the present application. FIG. 8 is a diagram showing the object extraction area set in the present embodiment. The present embodiment is different from the first embodiment only in the method of setting the object extraction region and the method of calculating the distance of the object. Therefore, the same components as in the first embodiment or The description of the same processing portion will be omitted, and the description will focus on the difference portion.

  In the present embodiment, there are two types of object extraction areas set in STEP4, the standard area Ss and the enlarged area S2b, as in the first embodiment, and the traveling of the vehicle 10 as in the first embodiment. Set according to the environment. In this case, the standard area Ss corresponds to the reference image area, and the shape and size of the standard area Ss and how to set the positions in the left and right images are the same as those in the first embodiment. Further, the reference parallax offset amount Ldisp related to the standard region Ss is the same as that in the first embodiment, and is obtained as described in the first embodiment at the time of manufacturing or maintenance / inspection of the vehicle 10. The reference parallax offset amount Ldisp is stored and held in a non-volatile memory (storage means) such as an EEPROM as a reference parameter that defines the relationship between the parallax and the distance, together with the focal length f, the base line length D, and the pixel interval p. The The nonvolatile memory also stores and holds position data representing the position of the standard area Ss in the left and right images, as in the first embodiment.

  On the other hand, in the present embodiment, in STEP 4, as shown in FIG. 8, the enlarged region S2b is not the entire region of the right image and the left image, but the positional relationship between the enlarged region S2b and the standard region Ss in the right image. The position of the enlarged region S2b in each of the left and right images is set so that the positional relationship between the enlarged region S2b and the standard region Ss in the left image matches.

  Specifically, in the present embodiment, as shown in FIG. 8, the enlarged region S2b is a rectangular region, and its size (vertical dimension FV2b and horizontal dimension FH2b) is that of the left and right images. It is determined to be a slightly smaller area than the size of the entire area. Then, the enlarged area S2b in the right image is set at a position where the center point Pr coincides with the center point of the standard area Ss of the right image. Similarly, the enlarged area S2b in the left image is set at a position where the center point Pl coincides with the center point of the standard area Ss of the left image. Note that the center point of the standard area Ss of each of the left and right images is determined based on the position data of the standard area Ss stored and held in the nonvolatile memory.

  Supplementally, the vertical positions of the center points Pr and Pl of the enlarged region S2b in the left and right images may not coincide with the vertical position of the center point of the standard region Ss. However, when the same object is captured in the enlarged areas S2b of the left and right images, the vertical areas of the objects on the enlarged areas S2b are the same. The amount of vertical displacement between the center point of S2b and the center point of the standard area Ss is preferably the same in the left and right images.

  When the enlarged region S2b is set as described above, the positional relationship between the enlarged region S2b and the standard region Ss in the right image matches the positional relationship between the enlarged region S2b and the standard region Ss in the left image. . For this reason, when the object extraction area is set to the enlargement area S2b and the same object is captured in each enlargement area S2b, the relative position of the object with respect to the enlargement area S2b of the right image (for example, the The coordinates of the object in the coordinate system Cb with the upper left vertex of the enlarged area S2b as the origin) and the relative position of the object with respect to the enlarged area S2b of the left image (for example, the upper left vertex of the enlarged area S2b is the origin) When the number of pixels in the lateral direction with respect to the coordinates of the object in the coordinate system Cb is calculated as the parallax of the object, the relationship between the parallax and the distance of the object is expressed by the above equation (6). Will be. That is, the reference parameter stored and held in the nonvolatile memory not only defines the relationship between the parallax and the distance in the standard area Ss but also defines the relationship between the parallax and the distance in the enlarged area Sb. This is because the parallax of the object in the coordinate system fixed with respect to the standard area Ss and the parallax of the object in the coordinate system fixed with respect to the enlarged area S2b have the same value.

  In the present embodiment, when the object extraction area is set to the standard area Ss in the processing of STEP 14, the parallax and the distance of the object are obtained by the same method as in the first embodiment. Calculated.

  On the other hand, when the object extraction area is set to the enlargement area S2b, as described above, the position (coordinates) of the object in the coordinate system Cb having the origin at the upper left vertex of the enlargement area S2b of the right image Then, the number of pixels in the lateral difference from the position (coordinates) of the object in the coordinate system Cb with the upper left vertex of the enlarged area S2b of the left image as the origin is calculated as the parallax of the object. The method of calculating the parallax is the same as in the first embodiment. However, in the present embodiment, when calculating the distance of the object from the parallax, the distance z of the object is calculated by the above equation (6).

  Therefore, regardless of whether the object extraction region is set to the enlarged region S2b or the standard region Ss, the reference parallax offset amount Ldisp, the focal length f, the baseline stored in the nonvolatile memory as described above is used. Based on the reference parameter composed of the length D and the pixel interval p, the distance z of the object is calculated from the parallax of the object according to the equation (6).

The configuration and processing other than those described above are the same as those in the first embodiment. In this embodiment, the same effect as that of the first embodiment can be obtained.
[Third Embodiment]
Next, a third embodiment of the present invention will be described. This third embodiment is an embodiment of the third invention, sixth invention, eighth invention, and twelfth invention of the present application. Note that this embodiment is different from the first embodiment only in part of the processing of STEP 14, so the description will focus on the differences, and the same components and processing parts as those of the first embodiment will be described. Description is omitted.

  In the present embodiment, the method of setting the object extraction area in STEP 4 is the same as in the first embodiment, and is set to either the enlarged area Sb or the standard area Ss as shown in FIG.

  Here, in the first embodiment, when the enlarged region Sb is set, the reference parameters (reference parallax offset amount Ldisp, focal length f, and so on) that define the relationship between the parallax and the distance in the standard region Ss (reference image region) are set. Based on the base line length D and the pixel interval p), in STEP 14, the parameters (the parallax offset amount Sdisp, the focal length f, the base line length D, and the pixel interval p) that define the relationship between the parallax and the distance in the enlarged region Sb are set. Among them, the parallax offset amount Sdisp is calculated and the parameter is determined.

  On the other hand, in the present embodiment, not only the reference parallax offset amount Ldisp is obtained at the time of manufacturing or maintenance / inspection of the vehicle 10, but also the parallax offset amount Sdisp corresponding to the enlarged region Sb is calculated according to the equation (7). It is stored and held in a non-volatile memory (storage means) together with the reference parallax offset amount Ldsip. As a result, parameters that define the relationship between the parallax and the distance of the object in advance corresponding to each of the standard area Ss and the enlarged area Sb, which are each type of area set as the object extraction area, are nonvolatile. It will be stored in memory. Note that position data representing the position of the standard area Ss and the position of the enlarged area Sb in the left and right images is also stored in the nonvolatile memory. In STEP 4, the object extraction area is set at a position represented by the position data stored and held in the nonvolatile memory correspondingly.

  In the present embodiment, when the object extraction area is set to the standard area Ss in the processing of STEP 14, the parameter corresponds to the standard area Ss when calculating the distance from the parallax of the object. Using the reference parallax offset amount Ldisp, the distance of the object is calculated by the equation (6). In addition, when the object extraction area is set to the enlarged area Sb, when calculating the distance from the parallax of the object, the above expression is used with the parallax offset amount Sdisp that is a parameter corresponding to the enlarged area Sb. The distance of the object is calculated from (8). In any case, the parallax calculation method is the same as in the first embodiment.

  Configurations and processes other than those described above are the same as those in the first embodiment.

  In this embodiment, the same effect as that of the first embodiment can be obtained. Also in this embodiment, the enlarged region Sb does not have to coincide with the entire region of each image, and for example, the virtual line region Sb ′ in FIG. 5 may be used as the enlarged region.

Further, when the enlarged area Sb is made to coincide with the entire area of each of the left and right images, a parameter (such as the parallax offset amount Sdisp) that defines the relationship between the parallax and the distance related to the enlarged area Sb is used for manufacturing the vehicle 10. By storing in advance in storage means such as a non-volatile memory at the time of maintenance or inspection, only the enlarged area Sb is used as the object extraction area, and the object extraction process or the vehicle 10 of the object is It is also possible to perform the distance calculation process.
[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIG. The fourth embodiment is an embodiment of the fourth, fifth, ninth, and thirteenth inventions of the present application. FIG. 9 is a diagram showing the object extraction area set in the present embodiment. Note that this embodiment is different from the first embodiment only in the method of calculating the parallax and the method of calculating the distance in the object extraction region, and thus the same components or the same as the first embodiment. The description of the processing part will be omitted, and the difference will be mainly described.

  In the first embodiment, regardless of whether the object extraction area is set to the standard area Ss or the enlarged area Sb, when the parallax of the object is obtained, the object extraction area set to the right image is determined. The number of pixels in the lateral difference between the relative position of the object and the relative position of the object with respect to the object extraction area set in the left image (the object extraction area of the same type as the right image) Calculated as the parallax of the object.

  On the other hand, in the present embodiment, regardless of whether the object extraction area is set to the standard area Ss or the enlarged area Sb, in STEP 14, the object for the standard area Ss (reference image area) of the right image is set. Is calculated as the parallax of the object in the horizontal direction between the relative position of the object and the relative position of the object with respect to the standard area Ss (reference image area) of the left image.

  Specifically, the parallax calculation method when the object extraction region is set to the standard region Ss is the same as that in the first embodiment. That is, the position (coordinate) of the object in the coordinate system Cs fixed with respect to the standard area Ss of the right image and the position of the object in the coordinate system Cs fixed with respect to the standard area Ss of the left image ( The number of pixels in the horizontal difference from the coordinate) is calculated as the parallax of the object.

  On the other hand, the parallax calculation method when the target object extraction area is set to the enlarged area Sb is different from that of the first embodiment. That is, as in the case where the object extraction area is set to the standard area Ss, the position (coordinates) of the object in the coordinate system Cs fixed with respect to the standard area Ss of the right image and the standard of the left image The number of pixels in the lateral difference from the position (coordinates) of the object in the coordinate system Cs fixed with respect to the region Ss is calculated as the parallax of the object. For example, the parallax in the enlarged region Sb of the object T1 (the portion of the left tail lamp of the other vehicle in front of the vehicle 10) in FIG. 9 is that of the object in the coordinate system Cs fixed with respect to the standard region Ss of the right image. It is obtained as the number of pixels of the difference between x1rs which is the x coordinate and x1ls which is the x coordinate of the object in the coordinate system Cs fixed with respect to the standard area Ss of the left image.

  When the parallax of the object is calculated in each type of object extraction area in this way, the relationship between the parallax and the distance of the object is not related to the type of the object extraction area, ). Accordingly, when the vehicle 10 is manufactured, the reference parameter stored and held in the nonvolatile memory at the time of maintenance / inspection defines the relationship between the parallax and the distance with respect to both the standard area Ss and the enlarged area Sb.

  Therefore, in the present embodiment, the parallax of the object is obtained as described above in STEP14, and the distance of the object is obtained from the obtained parallax according to the equation (6). The configuration and processing other than those described above are the same as those in the first embodiment.

  In the present embodiment, the position data stored in the non-volatile memory at the time of manufacturing the vehicle 10 or at the time of maintenance / inspection is used as the position of the standard area Ss in the left and right images. This corresponds to the reference point position data in the ninth and thirteenth inventions. As described in the first embodiment, the position data is the standard region Ss in the coordinate system (coordinate system fixed to each image) having the origin at the upper left vertex of the entire region of the left and right images. Is the position (coordinates) of the top left vertex (representative point of the standard area Ss). The top left vertex of the standard area Ss corresponds to the reference point in the fourth, ninth, and thirteenth inventions.

  In this embodiment, the same effect as that of the first embodiment can be obtained.

  Supplementally, in this embodiment, the enlarged region Sb is set to be the same as that of the first embodiment, but may be set to be the same as that of the second embodiment, for example. More generally, the size and position of the enlarged region in each image may be arbitrary, and the size and position may be different between the right image and the left image.

  In the embodiment described above, two types of object extraction areas are used. However, more object extraction areas may be variably set according to the driving environment.

1 is a block diagram showing the overall configuration of a vehicle periphery monitoring device in a first embodiment of the present invention. The perspective view which shows the external appearance of the vehicle (vehicle) carrying the periphery monitoring apparatus of FIG. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. The figure which shows the picked-up image and target object extraction area | region of the infrared camera with which the periphery monitoring apparatus of FIG. 1 was equipped. 5 is a flowchart showing the processing of STEP18 in the flowchart of FIG. The figure for demonstrating the process of the flowchart of FIG. The figure which shows the captured image and target object extraction area | region in 2nd Embodiment of this invention. The figure which shows the captured image and target object extraction area | region in 4th Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit (ranging device), STEP4 ... Object extraction area setting means, STEP14 ... Parameter determination means, distance calculation means

Claims (13)

The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring device that calculates the distance of the object with respect to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object.
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image Storage means for preliminarily storing and holding reference parameters that define the relationship and position data representing the position of the reference image area in each captured image;
Object extraction area setting means for variably setting an object extraction area for extracting an object for each of the first captured image and the second captured image;
The positional relationship between the position of the set object extraction area and the position of the reference image area represented by the position data in each of the first captured image and the second captured image, and the reference parameter Based on the relationship between the parallax of the object and the distance of the object when the same object is extracted in the object extraction region set in each of the first captured image and the second captured image based on A parameter determining means for determining a parameter to be defined;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the parameter determined by the parameter determination unit corresponding to the object extraction area A distance measuring device comprising: distance calculating means for calculating the distance of the object from the parallax of the object in the object extraction region based on the relationship defined by The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring device that calculates the distance of the object with respect to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object.
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image Storage means for preliminarily storing and holding reference parameters that define the relationship and position data representing the position of the reference image area in each captured image;
Object extraction area setting means for variably setting an object extraction area for extracting an object for each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the object in the object extraction area is based on the relationship defined by the reference parameter. Distance calculating means for calculating the distance of the object from the parallax of the object,
The object extraction area setting means is configured to extract the parallax of the object and the object when the same object is extracted in the object extraction area set in each of the first captured image and the second captured image. Ranging characterized in that the object extraction region is set according to the position of the reference image region represented by the position data so that the relationship with the distance of the object is equal to the relationship defined by the reference parameter apparatus. The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring device that calculates the distance of the object with respect to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object.
In the case where the same object is extracted in each type of image area for each of a plurality of types of image areas determined in advance in the entire area of each of the first captured image and the second captured image. Storage means for storing and holding in advance parameters that define the relationship between the parallax of the object and the distance of the object in association with each type of image area;
Object extraction area setting means for selectively setting, from each of the plurality of types of image areas, an object extraction area for extracting an object for each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, based on the relationship defined by the parameter corresponding to the object extraction area, A distance measuring device comprising distance calculating means for calculating the distance of the object from the parallax of the object in the object extraction region. The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring device that calculates the distance of the object with respect to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object.
Reference point position data indicating the position of a reference point as a reference of the position of the object in each of the first captured image and the second captured image, and each of the first captured image and the second captured image Storage means for preliminarily storing and holding a reference parameter that defines the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted;
Object extraction area setting means for variably setting an object extraction area for extracting an object for each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set for each of the first captured image and the second captured image, the reference point indicated by the reference point position data on the first captured image The difference between the relative position of the object and the relative position of the object with respect to the reference point indicated by the reference point position data on the second captured image is obtained as the parallax of the object, and from the obtained parallax A distance measuring device comprising distance calculating means for calculating the distance of the object based on the relationship defined by the reference parameter. A vehicle periphery monitoring device comprising the distance measuring device according to any one of claims 1 to 4,
The vehicle periphery monitoring device, wherein the object extraction area setting means is a means for variably setting the object extraction area in accordance with a traveling environment of the vehicle. The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring method for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object,
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image A reference parameter that defines the relationship and position data representing the position of the reference image area in each captured image are determined in advance,
Variably setting an object extraction region for extracting an object in each of the first captured image and the second captured image;
A positional relationship between the position of the set target object extraction area and the position of the reference image area represented by the position data in each of the first captured image and the second captured image; and the reference parameter Based on the above, when the same object is extracted in the object extraction region set in each of the first captured image and the second captured image, the parallax of the object and the distance of the object Determining the parameters defining the relationship;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the parameter determined by the parameter determination unit corresponding to the object extraction area And a step of calculating the distance of the object from the parallax of the object in the object extraction area based on the relationship defined by The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring device that calculates the distance of the object with respect to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object.
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image A reference parameter that defines the relationship and position data representing the position of the reference image area in each captured image are determined in advance,
Variably setting an object extraction region for extracting an object in each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the object in the object extraction area is based on the relationship defined by the reference parameter. Calculating the distance of the object from the parallax of the object,
The step of setting the object extraction area includes the parallax of the object when the same object is extracted in the object extraction area set in each of the first captured image and the second captured image. The object extraction region is set according to the position of the reference image region represented by the position data so that the relationship with the distance of the object is equal to the relationship defined by the reference parameter. Ranging method. The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring method for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object,
In the case where the same object is extracted in each type of image area for each of a plurality of types of image areas determined in advance in the entire area of each of the first captured image and the second captured image. Parameters that define the relationship between the parallax of the object and the distance of the object are determined in advance in association with each type of image area,
Selectively setting an object extraction area for extracting an object from the plurality of types of image areas for each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, based on the relationship defined by the parameter corresponding to the object extraction area, And a step of calculating a distance of the object from a parallax of the object in the object extraction region. The same object included in each captured image is extracted from the first captured image and the second captured image respectively obtained from the two image capturing apparatuses mounted on the vehicle, and the first object of the target object is extracted. In the distance measuring method for calculating the distance of the object to the vehicle based on the parallax that is the difference between the position on the captured image and the position on the second captured image of the object,
Reference point position data indicating the position of a reference point as a reference of the position of the object in each of the first captured image and the second captured image, and each of the first captured image and the second captured image A reference parameter that defines the relationship between the parallax of the target object and the distance of the target object when the same target object is extracted;
Variably setting an object extraction region for extracting an object in each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set for each of the first captured image and the second captured image, the reference point indicated by the reference point position data on the first captured image The difference between the relative position of the object and the relative position of the object with respect to the reference point indicated by the reference point position data on the second captured image is obtained as the parallax of the object, and from the obtained parallax And a step of calculating a distance of the object based on the relationship defined by the reference parameter. The position on the first captured image of the same target object extracted from the first captured image and the second captured image respectively obtained from two image capturing devices mounted on the vehicle, and the second of the target object A distance measurement program for causing a computer to execute a process of calculating a distance of the object to the vehicle based on a parallax that is a difference from a position on a captured image of
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image A reference parameter that defines the relationship, position data that represents the position of the reference image area in each captured image, and a process of reading from the storage means;
A process of variably setting an object extraction region in which an object is to be extracted in each of the first captured image and the second captured image;
A positional relationship between the position of the set target object extraction area and the position of the reference image area represented by the position data in each of the first captured image and the second captured image; and the reference parameter Based on the above, when the same object is extracted in the object extraction region set in each of the first captured image and the second captured image, the parallax of the object and the distance of the object Processing to determine parameters defining the relationship;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the parameter determined by the parameter determination unit corresponding to the object extraction area A ranging program comprising a function for causing a computer to execute a process of calculating the distance of the object from the parallax of the object in the object extraction region based on the relationship defined by The position on the first captured image of the same target object extracted from the first captured image and the second captured image respectively obtained from two image capturing devices mounted on the vehicle, and the second of the target object A distance measurement program for causing a computer to execute a process of calculating a distance of the object to the vehicle based on a parallax that is a difference from a position on a captured image of
The parallax of the target object and the distance of the target object when the same target object is extracted in a predetermined reference image area within the entire area of each of the first captured image and the second captured image A process of reading from the storage means a reference parameter that defines the relationship and position data representing the position of the reference image area in each captured image;
A process of variably setting an object extraction region in which an object is to be extracted in each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, the object in the object extraction area is based on the relationship defined by the reference parameter. A function of causing a computer to execute processing for calculating the distance of the object from the parallax of the object,
The function of causing the computer to execute the process of setting the target object extraction area is the case where the same target object is extracted in the target object extraction area set in each of the first captured image and the second captured image. The object extraction region is set according to the position of the reference image region represented by the position data so that the relationship between the parallax of the object and the distance of the object is equal to the relationship defined by the reference parameter. A distance measurement program characterized by having a function for causing a computer to execute processing to be performed. The position on the first captured image of the same target object extracted from the first captured image and the second captured image respectively obtained from two image capturing devices mounted on the vehicle, and the second of the target object A distance measurement program for causing a computer to execute a process of calculating a distance of the object to the vehicle based on a parallax that is a difference from a position on a captured image of
In the case where the same object is extracted in each type of image area for each of a plurality of types of image areas determined in advance in the entire area of each of the first captured image and the second captured image. A process for reading from the storage means a parameter that defines the relationship between the parallax of the object and the distance of the object;
A process of selectively setting an object extraction area for extracting an object from the plurality of types of image areas for each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set in each of the first captured image and the second captured image, based on the relationship defined by the parameter corresponding to the object extraction area, A ranging program comprising a function of causing a computer to execute a process of calculating a distance of the object from a parallax of the object in the object extraction region. The position on the first captured image of the same target object extracted from the first captured image and the second captured image respectively obtained from two image capturing devices mounted on the vehicle, and the second of the target object In a distance measurement program that causes a computer to execute a process of calculating a distance of the object to the vehicle based on a parallax that is a difference from a position on a captured image of
Reference point position data indicating the position of a reference point as a reference of the position of the object in each of the first captured image and the second captured image, and each of the first captured image and the second captured image A process of reading from the storage means a reference parameter that defines the relationship between the parallax of the object and the distance of the object when the same object is extracted;
A process of variably setting an object extraction region in which an object is to be extracted in each of the first captured image and the second captured image;
When the same object is extracted in the object extraction area set for each of the first captured image and the second captured image, the reference point indicated by the reference point position data on the first captured image The difference between the relative position of the object and the relative position of the object with respect to the reference point indicated by the reference point position data on the second captured image is obtained as the parallax of the object, and from the obtained parallax A ranging program having a function of causing a computer to execute a process of calculating a distance of the object based on a relationship defined by the reference parameter.

Images