JP2009277021A - Approaching object detection apparatus and approaching object detection method - Google Patents

Abstract

An object of the present invention is to prevent erroneous detection of an approaching object by detecting turning of a vehicle without newly adding hardware such as a sensor.
In an approaching object detection device, an optical flow calculation unit calculates an optical flow for a moving image of a front side of a vehicle photographed by a side camera provided at a front end of the vehicle. . The turning detection unit 24 detects turning of the vehicle by detecting that an optical flow in the same direction as the traveling direction of the vehicle appears in a wide area of the distant view of the moving image with respect to the calculated optical flow. . The approaching object detection unit 25 detects an approaching object based on the calculated optical flow when the turning detection unit 24 does not detect the turning of the vehicle, and detects the approaching object when the turning of the vehicle is detected. Stop detection.
[Selection] Figure 1

Description

  The present invention relates to an approaching object detection device and an approaching object detection method for detecting an approaching object approaching from the front side of a vehicle.

  In recent years, a vehicle such as a passenger car traveling on a road is often provided with a camera at the front end portion of the vehicle, and left and right side images obtained by the camera are displayed on a display device in the vehicle, and a driver at an intersection or the like. An approaching object (vehicle, bicycle, person, etc.) approaching from the left-right side can be easily confirmed. Patent Document 1 discloses an approaching object detection device that automatically detects an approaching object from the video (moving image).

  In the approaching object detection apparatus, an optical flow is used as a basic technique for detecting an approaching object in a moving image. The optical flow represents the apparent moving speed of an object moving in a moving image as a vector. For example, a point P fixed on a moving object in a moving image is at a pixel position (x1, y1) at a certain time (that is, a certain frame), and the pixel position (for example, the next frame) after a unit time ( If it is moved to x2, y2), the optical flow at the pixel position (x1, y1) can be represented by a vector (x2-x1, y2-y1).

  In general, when the vehicle is traveling, an optical flow having a component in the direction opposite to the traveling direction of the vehicle appears in a wide area of the moving image on the front side of the vehicle. Therefore, in the approaching object detection device described in Patent Document 1, in a wide area where the optical flow appears, when a small area of the optical flow having the same direction component as the traveling direction of the vehicle appears, It is determined that there is an approaching object approaching the host vehicle in the small area portion. Thus, an approaching object can be detected by the optical flow obtained from the moving image on the front side.

However, when the vehicle turns (turns right or left) at an intersection or the like, the situation of the optical flow that appears in the moving image changes greatly, so it is not possible to detect an approaching object only from the optical flow. An object is erroneously detected (including a detection omission). Therefore, the approaching object detection device described in Patent Document 1 is provided with a steering angle detection means, and when the turning of the vehicle is detected by the steering angle detection means, detection of the approaching object by the optical flow is stopped, and It is supposed to prevent false detection.
JP 2005-276056 A

  In the case of the approaching object detection device described in Patent Document 1, a steering angle sensor, a gyro device, and the like are required as steering angle detection means for detecting turning. The approaching object detection device requires a connector and an interface circuit for receiving signals from a steering angle sensor, a gyro device, and the like. Therefore, the number of parts of the approaching object detection device is increased, and the approaching object detection device is increased in size, or the manufacturing cost thereof is increased.

  Therefore, an object of the present invention is to provide an approaching object detection device and an approaching object detection method capable of detecting turning of a vehicle and appropriately detecting an approaching object without increasing hardware.

  An approaching object detection device of the present invention includes an optical flow calculation unit that calculates an optical flow based on a moving image of a front side of a vehicle photographed by a camera provided at a front end of the vehicle, and a vehicle based on the optical flow. A turn detection unit for detecting a turn of the vehicle, and an approaching object detection unit for detecting an approaching object approaching from the front side of the vehicle, wherein the turn detection unit is configured to detect a moving image of the front side of the vehicle. When an optical flow in the same direction as the traveling direction of the vehicle appears in a predetermined portion over a wide area, it is determined that the vehicle is turning and the approaching object detection operation by the approaching object detection unit is stopped.

  Usually, in the moving image acquired by the camera, the feature of the distant view does not change so much, so that the optical flow is difficult to appear. On the other hand, when the vehicle turns, the features in the distant view portion move greatly in the direction opposite to the direction in which the vehicle turns. That is, an optical flow in a direction opposite to the direction in which the vehicle turns turns widely in the distant view portion. The turning detection unit uses this phenomenon to monitor the optical flow appearing in the distant view portion and detect turning of the vehicle.

  In this specification, an optical flow refers to a vector of horizontal components of an optical flow vector. The optical flow is generally a two-dimensional vector. However, in this specification, only the horizontal direction (left-right direction) component is targeted, and the vector of the horizontal component of the optical flow is simply referred to as an optical flow. Therefore, the vertical component of the optical flow is ignored, and even if the vertical optical flow appears, the optical flow does not appear.

  According to the present invention, there are provided an approaching object detection device and an approaching object detection method capable of detecting turning of a vehicle and appropriately detecting an approaching object without increasing hardware.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 1 and FIG. 2, the functional block which comprises the approaching object detection apparatus which concerns on embodiment of this invention is demonstrated. Here, FIG. 1 is a diagram illustrating an example of the functional block configuration of the approaching object detection device according to the embodiment of the present invention, and FIG. 2 is a side camera installation position and a vehicle mounted with the approaching object detection device. It is the figure which showed the example of the imaging | photography range of the side camera.

  As shown in FIG. 1, the approaching object detection device 1 includes functional blocks such as two side cameras 4 (4a and 4b), an image processing device 2, a general-purpose data processing device 3, a display device 5, and an audio output device 6. Consists of including.

  Here, the side camera 4 is constituted by a CCD (Charge Coupled Device) camera, for example, and is provided at the left and right front end portions of the vehicle 7 as shown in FIG. And the image | video of the front part side of the vehicle 7 is image | photographed with a moving image, and the imaged moving image is transmitted to the image processing apparatus 2 as an analog or digital video signal, for example.

  Furthermore, as shown in FIG. 2, when the vehicle 7 approaches an intersection, the left-right direction of the road that intersects the road on which the vehicle 7 is traveling is included in the shooting range of the side camera 4. Therefore, the image processing device 2 and the general-purpose data processing device 3 acquire a moving image of the left and right roads photographed by the side camera 4 and display the moving image on the display device 5 or the like. To the vehicle 7 (own vehicle) side is detected.

  As shown in FIG. 1, the image processing apparatus 2 includes functional blocks such as an image acquisition unit 21, an image storage unit 22, an optical flow calculation unit 23, a turning detection unit 24, and an approaching object detection unit 25. .

  In FIG. 1, the image acquisition unit 21 receives a video signal or the like input from the side camera 4, and from the frame image of one frame of the video signal, the digital value (hereinafter simply referred to as RGB luminance signal for each pixel). Image data) and is stored in the image storage unit 22.

  The image storage unit 22 includes a RAM (Random Access Memory) or the like, and temporarily stores image data of the left and right moving images acquired from the side camera 4 via the image acquisition unit 21. In that case, the image storage unit 22 has a storage capacity for temporarily storing frame images of a predetermined number of frames (minimum of 2 frames), and each time the latest frame image is stored, the past frame images are discarded in the oldest order. Is done.

  The optical flow calculation unit 23 calculates an optical flow for each pixel based on image data for at least two frame images stored in the image storage unit 22. The optical flow can be calculated, for example, by comparing two frame images and identifying corresponding points for the same object included in each of the two frame images. Since there is a well-known technique already established for the optical flow calculation method, the description thereof will be omitted.

  The turning detection unit 24 detects turning of the vehicle based on the optical flow of each pixel calculated by the optical flow calculation unit 23. The detection method will be described in detail separately with reference to the drawings.

  The approaching object detection unit 25 detects an approaching object approaching from the front side of the vehicle 7 based on the optical flow of each pixel calculated by the optical flow calculation unit 23. The detection can be performed, for example, according to a known technique described in Patent Document 1 or the like, but in principle, in the optical flow in the direction opposite to the traveling direction that appears when the vehicle 7 travels straight, When a region having an optical flow in the same direction appears, the region is detected as an approaching object.

  The optical flow calculation unit 23, the turning detection unit 24, and the approaching object detection unit 25 may be configured by a dedicated processing circuit using, for example, an FPGA (Field Programmable Gate Array) or the like, and may be a general-purpose signal processing. You may comprise by the control circuit by the program using a processor, a microprocessor, etc.

  The general-purpose data processing device 3 includes at least a CPU (Central Processing Unit) and a memory, and includes a moving image on the side of the vehicle front side from the image processing device 2, a detection result of turning of the vehicle 7 by the turning detection unit 24, The detection result of the approaching object by the approaching object detection unit 25 is acquired, and the acquired information is appropriately processed and output to the display device 5 and the audio output device 6. Here, the display device 5 is configured by an LCD (Liquid Crystal Display) or the like, and the audio output device 6 is configured by a speaker or the like.

  In the present embodiment, it is assumed that the general-purpose data processing device 3, the display device 5, and the audio output device 6 are a data processing device, a display device, and an audio output device of a car navigation device, respectively. And may be independent.

  In addition, when the general-purpose data processing device 3 is provided in a form independent of the car navigation device or the like, there is a margin in its processing capability, so the optical flow calculation unit 23, the turning detection unit 24, and the approaching object in the image processing device 2 The function of the detection unit 25 may be realized by the general-purpose data processing device 3.

  Next, the principle of detecting the turning of the vehicle 7 using the optical flow will be described with reference to FIGS. FIG. 3 is a diagram showing an example of the difference in the appearance of the optical flow due to the difference in the movement of the vehicle, and FIG. 4 is a moving image taken by the right side camera 4b when the vehicle turns right at the intersection. It is the figure which showed the example of the optical flow which appears in. In FIGS. 3 and 4, the direction of the optical flow is represented by the direction of the arrow, and the magnitude of the optical flow is represented by the length of the arrow.

  In general, when the vehicle 7 is traveling straight ahead, in the moving image captured by the side camera 4, the foreground feature moves in a direction opposite to the traveling direction of the vehicle 7. , Don't move much. In the case of the present embodiment, the traveling direction of the vehicle 7 is the direction from right to left in the moving image obtained by the right side camera 4b, and in the moving image obtained by the left side camera 4a, The direction is from left to right.

  Therefore, when the vehicle 7 travels straight at an intersection as shown in FIG. 3A, the moving image photographed by the side camera 4b includes a near ground such as a building as shown in FIG. A large optical flow appears in the object portion in a direction opposite to the traveling direction of the vehicle 7 (from left to right). In addition, an optical flow in a direction opposite to the traveling direction of the vehicle 7 (from left to right) appears in a distant feature such as the houses on the other side of the road. The size of the optical flow is The smaller you go, the smaller. This is because the magnitude of the optical flow that appears when the vehicle 7 (that is, the side camera 4b) moves straight ahead is inversely proportional to the distance to the feature.

  On the other hand, when the vehicle 7 turns, in the moving image captured by the side camera 4, not only the foreground feature but also the distant view feature is the same in the direction opposite to the turning direction of the vehicle 7. Move so big.

  Therefore, when the vehicle 7 turns to the right at an intersection as shown in FIG. 3A (however, here, consider a case where the vehicle 7 turns in a stopped state), a moving image shot by the side camera 4b. As shown in FIG. 3 (c), the image is opposite to the turning direction of the vehicle 7, such as a foreground feature such as a building, or a distant view such as a house on the other side of the road. An optical flow in the direction (from right to left) appears, and the magnitude of the optical flow is almost the same regardless of the foreground portion and the far background portion. This is because the optical flow that appears when the vehicle 7 (that is, the side camera 4b) turns is almost the same regardless of the distance from the side camera 4 to the feature.

  Next, consider the case where the vehicle 7 turns right or left at the intersection, that is, the case where the vehicle 7 turns while traveling at the intersection. Such right turn or left turn traveling of the vehicle 7 includes a straight motion as well as a turning motion.

  Therefore, when the vehicle 7 makes a right turn at an intersection as shown in FIG. 4A, the moving image taken by the side camera 4b is shown in FIG. 3B as shown in FIG. An optical flow is produced by superimposing the optical flow appearing by the straight movement and the optical flow appearing by the turning motion shown in FIG.

  That is, an optical flow in a direction opposite to the traveling direction of the vehicle 7 (from the left to the right) appears in a foreground feature portion such as a building in which the optical flow that appears due to the straight motion is larger than the optical flow that appears due to the turning motion. . On the other hand, in a distant feature such as a house on the other side of the road where the optical flow that appears by the straight movement is smaller than the optical flow that appears by the turning movement, the direction opposite to the turning direction of the vehicle 7 (from right to left) Direction) optical flow appears.

  As described above, when the vehicle 7 makes a right turn, an optical flow in a direction opposite to the turning direction of the vehicle 7 appears in the distant feature portion of the moving image photographed by the side camera 4b, and the near-ground feature In the physical part, an optical flow in the direction opposite to the traveling direction of the vehicle 7 appears. At this time, the direction of the optical flow that appears in the distant view portion on the captured moving image is the direction opposite to the turning direction of the vehicle 7 and is the right-to-left direction, which coincides with the traveling direction of the vehicle 7. The image processing apparatus 2 detects the turning of the vehicle 7 using this fact.

  In other words, when the vehicle 7 is traveling straight ahead, the image processing apparatus 2 detects an optical flow in the opposite direction to the traveling direction in the foreground portion and does not detect an optical flow in the distant portion. If the optical flow in the same direction as the traveling direction is detected in a wide range of the region estimated to be a distant view portion while continuing, the optical flow is determined to be caused by the turning of the vehicle 7. .

  In other words, when the image processing apparatus 2 detects an optical flow in the same direction as the traveling direction in a wide range of a region estimated as a distant portion of the moving image captured by the side camera 4b, the vehicle 7 turns, That is, it determines with turning right.

  In the above description, the distance that becomes the boundary between the near view and the distant view is a value that is determined by the speed of the vehicle 7 that travels straight and the speed of the turn, and cannot always be set to a certain value. However, when the vehicle 7 decelerates at an intersection and turns right or left at a speed of about 10 km / h, for example, a feature that is at a distance farther than 30 to 40 m can be called a distant view.

  Moreover, in the above description, when the vehicle 7 turns right at the intersection, the moving image photographed by the side camera 4b has been described. However, the case where the vehicle 7 turns left at the intersection will be supplemented below.

  When the vehicle 7 makes a left turn at an intersection as shown in FIG. 4 (a), in the moving image taken by the side camera 4b, the optical flow that appears due to the turning motion and the optical flow that appears due to the rectilinear motion are in the same direction. (Both directions from left to right). Therefore, a large optical flow in the same direction appears in the feature portion in the near view and the feature portion in the distant view. In this case, although the direction is different, an optical flow (larger than a predetermined size) appears in the distant feature portion is the same as when the vehicle 7 turns right.

  Therefore, even when the vehicle 7 turns left at the intersection, the turning detection unit 24 of the image processing device 2 has the same direction as the traveling direction of the vehicle over a wide range of the distant portion of the moving image acquired from the side camera 4b. The turning of the vehicle 7 can be detected by detecting the appearance of the optical flow.

  Furthermore, if it supplements, although the above description demonstrated the moving image image | photographed with the side camera 4b, it is "right" by the description also about the moving image image | photographed with the side camera 4a, for example. Can be applied in exactly the same way simply by redrawing the diagrams of FIGS. 3 and 4 symmetrically.

  In the present embodiment, in order to more easily and reliably detect the turning of the vehicle 7 by the optical flow as described above, a distant view portion is included in the moving image acquired from the side camera 4. A portion to be obtained is set in advance as a distant view area. Here, the distant view area is substantially the same as the vanishing point of the moving image obtained from the side camera 4 as shown in FIG. 4B as an area surrounded by a thick line. It shall mean an area that combines an area and an upper area continuous with the area.

  Note that the position of the vanishing point in the moving image and the region for turning detection (the above-mentioned distant view region: see FIG. 4B) are usually set when the approaching object detection device 1 is designed. In this case, a vanishing point position is set by using a test chart board on which a mark indicating the vanishing point direction of each camera is written. That is, the vehicle 7 facing in a predetermined direction is installed at a position separated by a predetermined distance from the position where the test chart plate is provided, and is photographed by the side camera 4 mounted at the predetermined position of the vehicle 7. The installation direction of the side camera 4 is adjusted so that the vanishing point in the image matches the mark indicating the vanishing point direction in the test chart.

  As described above, when the installation position and the installation direction of the side camera 4 are determined, the height of the vanishing point in the moving image is always approximately the same height position unless it is an extremely steep slope. You can think. Therefore, the height position of the vanishing point can be regarded as a constant, and therefore, the distant view area can be set as a fixed area as shown in FIG. 4B.

  In the present embodiment, as shown in FIG. 4B, the distant view area does not include the uppermost portion (for example, a portion such as the sky) of the moving image acquired from the side camera 4, but the distant view The region may include the uppermost portion. In this case, the distant view area is an area where the vanishing point and the height are substantially the same, and all the areas above it.

  Subsequently, the turning detection operation of the vehicle 7 in the image processing apparatus 2 will be specifically described. The turning detection unit 24 determines whether or not the optical flow in the same direction as the traveling direction of the vehicle 7 appears widely in the “distant view area” in the moving image acquired from the side camera 4. Is detected. That is, the turning detection unit 24 determines whether the area of the optical flow in the same direction as the traveling direction of the vehicle 7 that appears in the “distant view area” is equal to or greater than a predetermined area. If it is, it is determined that the vehicle 7 is turning right or left, and turning of the vehicle 7 is detected.

  Here, as can be seen from FIG. 4, when the vehicle 7 is traveling straight ahead, even if it is a predetermined “distant view area”, if there is a large feature such as a building nearby, An optical flow in the direction opposite to the traveling direction of the vehicle 7 appears in the portion. Therefore, when an optical flow appears in the “far view area”, it is identified whether the optical flow appears due to the turning of the vehicle 7 or a large feature near a straight line. There is a need to.

  The identification can be performed by determining the direction of the optical flow for each optical flow obtained based on the moving images acquired from the left and right side cameras 4a and 4b.

  Incidentally, when the vehicle 7 turns to the right, as shown in FIG. 4B, a left-side optical flow appears widely in the distant view area of the moving image acquired from the right side camera 4b, and nearby features A right-facing optical flow appears in the part. When the vehicle 7 turns to the left, a rightward optical flow appears widely in the distant view area of the moving image acquired from the left side camera 4a, and a leftward optical flow appears in the nearby feature portion. The direction of the optical flow that appears widely in the distant view area corresponds to the direction opposite to the direction in which the vehicle 7 turns.

  That is, in the above case, the optical flow that appears due to the turning of the vehicle 7 and the optical flow that appears due to the straight traveling are opposite to each other. be able to.

  Accordingly, the turning detection unit 24 turns the vehicle 7 to the right when the area of the area occupied by the leftward optical flow exceeds a predetermined area in the distant area of the moving image acquired from the right side camera 4b. It is determined that Further, in the distant view area of the moving image acquired from the left side camera 4a, when the area of the area occupied by the rightward optical flow is equal to or larger than a predetermined area, it is determined that the vehicle 7 is turning right.

  When the vehicle 7 makes a right turn, the distant view area of the moving image acquired from the left side camera 4a has the same direction as the optical flow direction that appears in the nearby feature portion when the vehicle 7 travels straight ( An optical flow (right direction) appears. Further, when the vehicle 7 turns to the left, the distant view area of the moving image acquired from the right side camera 4b has the same direction as the optical flow direction that appears in the nearby feature portion when the vehicle 7 travels straight ( A left-facing optical flow appears.

  Therefore, the left turn cannot be determined based on the optical flow of the moving image acquired from the right side camera 4b, and the right turn is determined based on the optical flow of the moving image acquired from the left side camera 4a. Cannot be determined. Therefore, in the present embodiment, the turning detection unit 24 determines only the right turn of the vehicle 7 from the optical flow of the moving image acquired from the right side camera 4b, and is acquired from the left side camera 4a. Only the left turn of the vehicle 7 is determined from the optical flow of the moving image.

  As described above, the turning detection unit 24 independently determines whether the vehicle 7 is to turn right or left from the moving images acquired from the side cameras 4b and 4a. When a right turn or a left turn is determined, turning of the vehicle 7 is detected.

  Then, the procedure of the approaching object detection performed in the approaching object detection apparatus 1 which concerns on embodiment of this invention is demonstrated. FIG. 5 is a diagram illustrating an example of a processing procedure for detecting an approaching object in the image processing device 2 of the approaching object detection device 1 according to the embodiment of the present invention. The processing procedure shown in FIG. 5 is executed every time one frame of the moving image of the front side of the vehicle 7 is captured by the side camera 4.

  As shown in FIG. 5, the image processing apparatus 2 first acquires a frame image of one frame from the side camera 4 as processing of the image acquisition unit 21 (step S1), and the acquired frame image is stored in the image storage unit 22. Next, the image processing apparatus 2 determines whether or not a frame image of a frame before the frame image acquired in Step S1 (hereinafter referred to as a previous frame image) exists in the image storage unit 22 (Step S2). If the previous frame image does not exist (No in step S2), the process returns to step S1 to acquire the latest frame image from the side camera 4 again.

  On the other hand, when the previous frame image exists (Yes in step S2), the image processing apparatus 2 performs the optical flow based on the previous frame image and the frame image acquired in step S1 as the processing of the optical flow calculation unit 23. Is calculated (step S3).

  Next, the image processing apparatus 2 calculates the area of the area occupied by the optical flow appearing in the predetermined distant view area based on the optical flow calculated by the optical flow calculation section 23 as a process of the turning detection section 24 (step S40). S4) Based on the calculated area, turning of the vehicle 7 is determined (step S5).

  Here, the optical flow appearing in the predetermined distant view area indicates an optical flow in the same direction as the traveling direction of the vehicle 7. That is, the optical flow obtained from the moving image acquired from the left side camera 4a is an optical flow directed to the right, and the optical flow obtained from the moving image acquired from the right side camera 4b. Is a left-facing optical flow.

  Next, as a result of the determination in step S5, when the area of the region occupied by the optical flow is larger than a predetermined threshold (Yes in step S6), the image processing apparatus 2 indicates that the vehicle 7 is turning. The process returns to step S1 without performing the processes after step S7 such as detection of an approaching object.

  On the other hand, when the area of the region occupied by the optical flow in the same direction as the traveling direction of the vehicle 7 does not reach the predetermined threshold (No in step S6), it is not determined that the vehicle 7 is turning. In that case, the image processing apparatus 2 detects an approaching object approaching the vehicle 7 based on the optical flow calculated by the optical flow calculation unit 23 as a process of the approaching object detection unit 25 (step S7).

  In step S <b> 7, the image processing apparatus 2 has a small area of the optical flow in the same direction as the traveling direction of the vehicle 7 in the optical flow in the direction opposite to the traveling direction of the vehicle 7 that appears when the vehicle 7 travels straight ahead. When it appears, the area is detected as an approaching object. According to such a detection method, when the vehicle 7 approaches an intersection, the image processing device 2 approaches an approaching object (such as a vehicle) that approaches the intersection on both roads that intersect the road on which the vehicle 7 travels. ) Can be detected.

  Next, when the image processing device 2 detects an approaching object by the determination in step S7 (Yes in step S8), the image processing device 2 is connected to at least one of the display device 5 and the audio output device 6 via the general-purpose data processing device 3. Then, an alarm indicating that there is an approaching object is output (step S9), and the frame image of the frame captured by the side camera 4 and stored in the image storage unit 22 is displayed via the general-purpose data processing device 3. 5 (step S10).

  On the other hand, if the approaching object is not detected by the determination in step S7 (No in step S8), the image processing apparatus 2 is photographed by the side camera 4 without outputting an alarm indicating the approaching object. The frame image of the frame stored in the image storage unit 22 is displayed on the display device 5 via the general-purpose data processing device 3 (step S10). In this case, since no approaching object is detected, step S10 may be skipped and the frame image captured by the side camera 4 may not be displayed on the display device 5.

  In the approaching object detection processing procedure described above, when the image processing apparatus 2 determines turning of the vehicle 7, the vehicle 7 is based on the area of the area occupied by the optical flow in the same direction as the traveling direction of the vehicle 7. However, the turning of the vehicle 7 may be determined based not on the area itself but on the increase rate of the area. That is, in step S6, when the increase rate of the area becomes larger than a predetermined threshold, it may be determined that the vehicle 7 has started to turn. When the turning of the vehicle 7 is determined based on the area increase rate, the turning of the vehicle 7 can usually be detected earlier than the turning determination based on the area itself.

  It should be noted that when the turning of the vehicle 7 is determined based on the area increase rate, it cannot be determined that the vehicle 7 continues to turn. Therefore, in this case, it is necessary to determine turning based on the area itself.

  As described above, according to the present embodiment, the approaching object detection device 1 can detect the turning of the vehicle 7 by the optical flow based on the moving image obtained from the side camera 4, and the approaching object approaching the vehicle 7 can be detected. Further, when the turning of the vehicle 7 is detected, the approaching object detection operation can be stopped. Therefore, in the approaching object detection device 1 according to the present embodiment, it is possible to prevent erroneous detection of an approaching object accompanying the turning of the vehicle 7.

  Further, in the approaching object detection device 1 according to the present embodiment, a signal from a steering angle sensor, a gyro device, or the like for detecting the turning of the vehicle 7 in preventing erroneous detection of an approaching object accompanying the turning of the vehicle 7. An interface circuit including a connector for receiving the signal is not required. Therefore, the approaching object detection device 1 can be reduced in size or the manufacturing cost can be reduced.

  In particular, when the approaching object detection device 1 is retrofitted to an inexpensive car navigation device that does not have autonomous navigation, a connector for receiving a signal from a steering angle sensor, a gyro device, or the like is unnecessary. This means that it is not necessary to modify the hardware of the car navigation device, which is a great advantage in reducing the manufacturing cost of the approaching object detection device 1.

  Next, a modification of the present embodiment will be described with reference to FIG. FIG. 6 is a diagram showing an example of a shooting range by a wide-angle camera when a wide-angle camera is provided at the front end of the vehicle.

  As shown in FIG. 6, the vehicle 7a is provided with a wide-angle camera 4c at the center of the front end thereof. The shooting range of the wide-angle camera 4c exceeds 180 degrees. Here, it is assumed that an image in which peripheral image distortion is corrected by digital image processing is output from the wide-angle camera 4c.

  The image acquisition unit 21 of the approaching object detection device 1 acquires a moving image output from the wide-angle camera 4c, and extracts the left and right images from the side camera 4 for each frame image. An image substantially equivalent to each acquired frame image is generated. By doing so, the processing procedure of the approaching object detection in the approaching object detection device 1 thereafter can be made the same as the processing procedure shown in FIG.

  Therefore, also in the modified example of the present embodiment, the same effect as that of the above-described embodiment can be obtained.

The figure which showed the example of the structure of the functional block of the approaching object detection apparatus which concerns on embodiment of this invention. The figure which showed the example of the installation position of the side camera in the vehicle carrying the approaching object detection apparatus which concerns on embodiment of this invention, and the imaging range of the side camera. The figure which showed the example of the difference in how the optical flow appears by the difference in the motion of a vehicle. The figure which showed the example of the optical flow which appears in the moving image image | photographed with the right side camera when a vehicle turns right at an intersection. The figure which showed the example of the process sequence of the approaching object detection in the image processing apparatus of the approaching object detection apparatus which concerns on embodiment of this invention. The figure which showed the example of the imaging | photography range by the wide angle camera at the time of providing a wide angle camera in the front-end part of a vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Approaching object detection apparatus 2 Image processing apparatus 3 General-purpose data processing apparatus 4, 4a, 4b Side camera 4c Wide-angle camera 5 Display apparatus 6 Audio | voice output apparatus 7, 7a Vehicle 21 Image acquisition part 22 Image storage part 23 Optical flow calculation part 24 Turn detection unit 25 Approaching object detection unit

Claims (8)

An approaching object detection device that detects an approaching object approaching from the front side of the vehicle based on a moving image of the front side of the vehicle taken by a camera provided at a front end of the vehicle,
An image acquisition unit for acquiring a moving image of a front side of the vehicle from the camera;
An image storage unit for storing the acquired moving image;
An optical flow calculation unit that calculates an optical flow based on the moving image stored in the image storage unit;
A turning detection unit for detecting turning of the vehicle based on the optical flow of the predetermined portion of the calculated optical flow;
Based on the calculated optical flow, an approaching object detection unit that detects an approaching object approaching from the front side of the vehicle;
With
The approaching object detection device, wherein when the turning detection unit detects turning of the vehicle, the approaching object detection operation by the approaching object detection unit is stopped. The predetermined portion is a region in the moving image that is a region in which the height of the moving image is substantially the same as the vanishing point of the moving image and an upper region in contact with the region. The approaching object detection device according to 1. The turning detection unit calculates an area of a region where an optical flow in the same direction as the traveling direction of the vehicle appears in the predetermined portion, and when the area becomes larger than a predetermined threshold, the turning of the vehicle The approaching object detection device according to claim 1, wherein the approaching object detection device is detected. The image acquisition unit acquires a left front side moving image and a right front side moving image of the vehicle from the camera, and the optical flow calculation unit includes the acquired left front side moving image and When the optical flow is calculated for each of the moving images on the right front side, the turning detection unit detects turning of the vehicle based on the turning detection result of the vehicle based on the optical flow of at least one of the moving images. The approaching object detection device according to any one of claims 1 to 3, wherein: An approaching object detection method by an approaching object detection device that detects an approaching object approaching from the front side of the vehicle based on a moving image of the front side of the vehicle photographed by a camera provided at a front end of the vehicle Because
The approaching object detection device is
Obtaining a moving image of the front side of the vehicle from the camera;
Storing the acquired moving image in a predetermined image memory;
Based on the moving image stored in the image memory, an optical flow is calculated,
Based on the optical flow of the predetermined part of the calculated optical flow, the turning of the vehicle is detected,
When the turning of the vehicle is not detected, an approaching object approaching from the front side of the vehicle is detected based on the calculated optical flow, and when the turning of the vehicle is detected, the approaching object is detected. An approaching object detection method characterized by stopping. The predetermined portion is a region in the moving image that is a region in which the height of the moving image is substantially the same as the vanishing point of the moving image and an upper region in contact with the region. 5. The approaching object detection method according to 5. The approaching object detection device includes:
An area of a portion where an optical flow in the same direction as the traveling direction of the vehicle appears in the predetermined portion is calculated, and when the area becomes larger than a predetermined threshold, turning of the vehicle is detected. The approaching object detection method according to claim 5 or 6. The approaching object detection device includes:
The left front side moving image and the right front side moving image of the vehicle are acquired from the camera, and optical flows are respectively performed on the acquired left front side moving image and right front side moving image. 8. The turning of the vehicle is detected based on a result of turning detection of the vehicle based on an optical flow of at least one of the moving images when calculated. 8. The approaching object detection method as described.

Images