JP2014013452A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an in-vehicle image processor capable of performing image recognition processing with high accuracy even when a light source, such as the sun is reflected in a captured image.SOLUTION: An image processor includes: an image acquisition part for acquiring image information outputted by capturing outside image and including a shade area shaded by a shading member in a portion of an image captured area, by imaging device; a high luminance pattern detection part for detecting that a prescribed high luminance pattern is included in the shade area from the image information; and an image recognition part for determining whether to be direct light on the basis of the high luminance pattern when the high luminance pattern detection part detects the high luminance pattern, and performing prescribed image recognition from the image information on the basis of a determined result.

Description

  The present invention relates to an image processing apparatus.

  2. Description of the Related Art Conventionally, a technique is known in which a vehicle periphery is photographed by an in-vehicle imaging device, and a light source position such as the sun or other vehicle headlight is estimated from the photographed image. For example, Patent Document 1 discloses a vehicle image processing apparatus that can estimate the luminance change of the sun or a headlight by luminance change estimation means.

JP 2010-086266 A

  The prior art has a problem that the accuracy of image recognition processing such as detection of lane markings is reduced when a light source such as the sun is reflected in a captured image.

  An image processing apparatus according to claim 1, an image acquisition unit that acquires image information output by imaging an outside of the vehicle by an imaging device that includes a light-shielding region shielded by a light-shielding member as a part of the imaging region, and image information Therefore, if a high-intensity pattern detection unit that detects that a predetermined high-intensity pattern is included in the light-shielding area, and the high-intensity pattern detection unit detects a high-intensity pattern, direct light is detected based on the high-intensity pattern. And an image recognition unit that performs predetermined image recognition from image information based on the determination result.

  According to the present invention, image recognition processing can be performed with high accuracy even when a light source such as the sun is reflected in a captured image.

It is a block diagram which shows the structure of the vehicle-mounted vehicle recognition apparatus by one Embodiment of this invention. It is a figure which shows the imaging | photography area | region and light-shielding area | region of a camera. It is a figure which shows the example of the attachment position of a camera. It is a block diagram of each function part with which the control part 2 is provided in order to implement | achieve a light source detection function. It is a figure which shows an example of a picked-up image. It is a flowchart of the light source detection process by the control part. It is a figure which shows an example of the histogram produced by the high-intensity pattern detection part. It is a figure which shows an example of the luminance value of each pixel on the straight line which passes along the center position of a light source. It is a figure which shows the other example of a picked-up image.

  FIG. 1 is a block diagram showing a configuration of an in-vehicle vehicle recognition device 100 according to an embodiment of the present invention. A vehicle-mounted vehicle recognition device 100 shown in FIG. 1 is used by being mounted on a vehicle, and includes a camera 1 having a light shielding plate 1a attached thereto, a control unit 2, an alarm output unit 3, and an operation state notification unit. 4 and an external device control unit 5.

  The camera 1 is installed toward the rear of the vehicle, and images an image in an imaging region including a road surface behind the vehicle at predetermined time intervals. For this camera 1, for example, an image sensor such as a CCD or a CMOS is used. A captured image acquired by the camera 1 is output from the camera 1 to the control unit 2.

  The light shielding plate 1 a is attached to the camera 1 to shield a part of the incident light to the camera 1. FIG. 2 is a diagram showing a shooting region and a light shielding region of the camera 1, and shows a state in which the camera 1 is viewed from the lateral direction. As shown in FIG. 2, a light shielding area is formed by masking an upper part of the photographing area of the camera 1 with a light shielding plate 1a. The camera 1 captures an image including a road surface behind the vehicle in an imaging region other than the light shielding region. Here, the shooting area (view angle) of the camera 1 is set to be relatively wide so that the road surface behind the vehicle can be shot in a sufficiently wide range in the left-right direction. Unnecessary light is also incident on the camera 1. Therefore, in order to block such unwanted incident light on the camera 1, a light shielding region is provided by the light shielding plate 1a.

  FIG. 3 is a diagram illustrating an example of the attachment position of the camera 1. A number plate 21 is installed on the vehicle body 20 at the rear portion of the host vehicle. The camera 1 is mounted obliquely downward at a position directly above the number plate 21, and a light shielding plate 1a is installed thereon. In addition, since the attachment position shown here is an example to the last, you may attach the camera 1 to another position. As long as the road surface behind the vehicle can be photographed within an appropriate range, the attachment position of the camera 1 may be determined in any manner.

  The control unit 2 performs predetermined image processing using the captured image from the camera 1 and performs various controls according to the processing result. By the control performed by the control unit 2, the vehicle-mounted vehicle recognition device 100 has various functions called LDW (Lane Departure Warning), PED (Pedestrian Detection), RSR (Road Sign Recognition), and IMD (Image Diagnosis), for example. Realized. The LDW is a function that outputs a warning when the vehicle is likely to deviate from the traveling lane by detecting a white line (lane boundary line) on the road surface from the photographed image. PED is a function that informs the driver of the presence of a pedestrian on the path of the vehicle by detecting the shape of a person from a captured image. The RSR is a function for recognizing a traffic sign on a road from a photographed image and, for example, warning a driver when the speed exceeds a speed limit sign. The IMD is a function for diagnosing whether a captured image is correctly captured by the camera 1.

  The alarm output unit 3 is a part for outputting an alarm by an alarm lamp, an alarm buzzer or the like to the driver of the vehicle. The operation of the alarm output unit 3 is controlled by the control unit 2. For example, when it is determined in the aforementioned LDW that the host vehicle is about to depart from the lane in which the vehicle is traveling, or when a vehicle that may collide with the host vehicle is detected in the BSW, according to the control of the control unit 2 An alarm is output from the alarm output unit 3.

  The operation state notification unit 4 is a part for notifying the driver of the vehicle of the operation state of the vehicle-mounted vehicle recognition device 100. For example, when the in-vehicle vehicle recognition device 100 is in a non-operating state without satisfying a predetermined operating condition, a lamp installed near the driver's seat of the vehicle as the operating state notifying unit 4 under the control of the control unit 2 Lights up. This notifies the driver that the in-vehicle vehicle recognition device 100 is in a non-operating state.

  The external device control unit 5 controls the external device according to the control from the control unit 2.

  Next, the light source detection function by the vehicle-mounted vehicle recognition device 100 will be described. The light source detection function is a function of detecting the presence and position of a light source such as the sun reflected in the shooting area of the camera 1.

  When a strong light source (direct light) such as the sun is reflected, the amount of light in the entire photographing area is increased by the light source, so the exposure setting of the camera 1 is set to be under. As a result, areas other than the light source are photographed darker than usual, which adversely affects various controls based on the photographed image. In addition, light from a strong light source such as the sun is reflected on the road surface, and erroneous detection of a vehicle boundary line existing on the road surface is induced. Furthermore, since the contrast is reduced in the vicinity of the light source, the detection accuracy of other vehicles and the like deteriorates. The vehicle-mounted vehicle recognition device 100 according to the present embodiment avoids the above-described problem by detecting the presence and position of a light source such as the sun using a light source detection function.

  FIG. 4 is a block diagram of each functional unit provided for the control unit 2 to realize the light source detection function. Each of these functional units is realized by the control unit 2 reading and executing a predetermined control program.

  The present invention provides an image acquisition unit that acquires image information output by an imaging apparatus that includes a light-shielding region shielded by a light-shielding member as a part of an imaging region, and outputs the information from the acquired image information to the light-shielding region. A high-intensity pattern detection unit 11 that detects that a predetermined high-intensity pattern is included, and when the high-intensity pattern detection unit 11 detects a high-intensity pattern, direct sunlight such as sunlight is generated based on the high-intensity pattern. An image recognition unit that determines whether or not it contains light and performs predetermined image recognition from image information based on the determination result.

  The processing area setting unit 10 sets a light shielding area masked by the light shielding plate 1a in the entire imaging area as a processing area to be analyzed by the high luminance pattern detection unit 11.

  The high-intensity pattern detection unit 11 detects that a predetermined high-intensity pattern is included in the processing region (light-shielding region) set by the processing region setting unit 10. Alternatively, it is detected that a predetermined high-intensity pattern is included across both the light-shielding area that is the processing area and the other area (background area) with respect to the imaging area.

  The light source position detection unit 12 detects the position of the light source corresponding to the high luminance pattern when the high luminance pattern detection unit 11 detects the high luminance pattern from the light shielding region or both the light shielding region and the background region.

  FIG. 5 is a diagram illustrating an example of a captured image. As shown in FIG. 5A, the camera 1 includes a light shielding region 31 that is shielded by the light shielding plate 1 a as part of the captured image 30. In addition, a road area 33 and a background area 32 are included in areas other than the light shielding area 31 in the imaging area. For example, white lines such as lane markings and road markings are reflected in the road surface area 33. The background area 32 includes, for example, sky and buildings.

  Since the light shielding region 31 is masked by the light shielding plate 1a, it is usually almost black. However, when a strong light source such as the sun is reflected in the background area 32, light from the light source 35 leaks to the light shielding area 31 as shown in FIG. The vehicle-mounted vehicle recognition device 100 detects the presence of the light source 35 such as the sun from the light 36 leaked into the light shielding region 31.

  In this light source detection, the number of pixels (area) is counted for the amount of leaked light, that is, a pixel having a predetermined luminance value or more, and it is determined that there is a light source if it is more than a predetermined area. To do. Here, since the amount of light leaked into the light-shielding region depends on the lens material and structure and the camera mounting state, the threshold value of the luminance value and the threshold value of the area may be set for each vehicle type.

  Further, in this light source detection, the amount of light that leaks changes according to the state of the lens surface. For example, if water droplets or muddy water adheres to the lens by splashing rain or moisture with a tire or the like, the amount of light leaking increases as the amount of adhesion increases. This is because when a transparent or semi-transparent object adheres to the lens surface, the ambient illumination light is refracted by the lens effect on the surface, and if there is no strong light source such as the sun, the light shielding region 31 that should not be directly incident on the lens. As a result, light is leaked and the possibility of erroneous detection in light source detection increases. Therefore, it is possible to detect that water droplets or muddy water is attached to the lens, and in such a case, the threshold value of the luminance value and the threshold value of the area may be set higher. Regarding the adhesion of water droplets and muddy water, since there is a known technique (for example, JP-A-2011-232193), the details are omitted here.

  FIG. 6 is a flowchart of light source detection processing by the control unit 2. Hereinafter, based on the flowchart of FIG. 6, the specific content of the light source detection process will be described.

  First, in step S90, the control unit 2 acquires image data (image information) from the camera 1. In step S100, the processing area setting unit 10 sets the light shielding area 31 as a processing area. The position and range of the light shielding region 31 are determined when the camera 1 and the light shielding plate 1a are installed in the vehicle, and are input to the control unit 2 in advance. In the next step S110, the high-intensity pattern detection unit 11 aggregates the luminance values in the processing region, and creates a histogram of the number of pixels for each luminance value. In step S120, the high-intensity pattern detection unit 11 detects a predetermined high-intensity pattern in the processing region based on the histogram totalized in step S110.

  FIG. 7 is a diagram illustrating an example of a histogram created by the high brightness pattern detection unit 11. In step S120 of FIG. 6, the high luminance pattern detection unit 11 detects a high luminance pattern in which a pixel having a specific luminance value that is equal to or greater than a predetermined threshold th1 exists.

  For example, in the histogram of FIG. 7, there are th2 or more pixels having a luminance value br1 of th1 or more. The high luminance pattern detection unit 11 detects the luminance value br1 as the high luminance pattern described above.

  In step S <b> 130, the light source position detection unit 12 determines whether a high luminance pattern is detected by the high luminance pattern detection unit 11. If a high luminance pattern is not detected, the process proceeds to step S170, and the light source position detection unit 12 outputs “no light source (no light source detected)”. On the other hand, if a high-luminance pattern is detected, there is a possibility that a strong light source such as the sun may be present, so the process proceeds to step S140 to search for light source coordinates.

  Hereinafter, a method for searching for a light source position (light source coordinates) by the light source position detection unit 12 will be described. First, the light source position detection unit 12 searches for a pixel having the maximum luminance value from a region obtained by combining the light shielding region 31 and the background region 32. The light source position detection unit 12 assumes that the position of the pixel having the maximum luminance value found here is the center of the light source.

  It is considered that the light from the light source spreads concentrically from the center position of the light source. Therefore, the light source position detection unit 12 searches the captured image for a pixel pattern whose luminance value decreases concentrically from the center position. Specifically, first, consider a straight line of a predetermined angle passing through the center position of the light source (for example, a straight line parallel to the shooting screen), and whether or not the luminance of each pixel on the straight line has a predetermined pattern. judge.

  FIG. 8 is a diagram illustrating an example of the luminance value of each pixel on a straight line passing through the center position of the light source. Here, six points A to F are determined. Point C is a point closest to the center position among points where the slope of the luminance value is a predetermined value (positive number) or more on the left side of the center position. Point A is a point closest to the center position among points where the slope of the luminance value is a predetermined value (positive number) or less on the left side of point C. Point B is an intermediate point between points A and C. Point D is a point closest to the center position among points where the slope of the luminance value is equal to or less than a predetermined value (negative number) on the right side of the center position. Point F is the point closest to the center position among points where the slope of the luminance value is equal to or greater than a predetermined value (negative number) on the right side of point D. Point E is an intermediate point between points D and F.

  In step S140 of the flowchart of FIG. 6, the light source position detection unit 12 has substantially the same slope between A and C and the slope between BC and the slope between D and E and the slope between DF. Are determined to be substantially the same. If no such pattern is found, the process proceeds to step S170, and the output "No light source (no light source detected)" is performed.

  On the other hand, when the above pattern is found, concentric circles are continuously detected. If a concentric luminance distribution is seen between point A and point F, it is considered that the center of the light source is surely present at the above-described center position. For example, several brightness values are determined in the range from the brightness value of the point C to the brightness value of the point A, and the photographed image is scanned for the presence of a circle with the brightness value centered on the center position. Make sure. If a circle as described above exists for each of the determined several luminance values, the process proceeds to step S160 in the flowchart of FIG. 6, and the light source position detection unit 12 outputs a result of “with light source”. Further, the center position is output as the center coordinates of the light source.

  The detection result output as described above from the light source position detection unit 12 is used as follows.

  For example, when the control unit 2 realizes LDW, predetermined image recognition is performed on a captured image from the camera 1 to detect a white line on the road surface. At this time, it is considered that light from the light source is reflected on the road surface close to the light source position. Therefore, if normal white line detection is performed on such a road surface, there is a high possibility that a white line will be erroneously detected. Therefore, when the light source position detection unit 12 outputs that the light source exists, the control unit 2 changes the white line detection parameter at a position close to the light source on the entire road surface. Alternatively, the white line is not detected at such a position.

  Further, when the control unit 2 realizes PED, predetermined image recognition is performed on a captured image from the camera 1 to detect a person. At this time, it is considered that the contrast is lowered at a position near the light source. Therefore, similarly to the above-described white line detection, if normal person detection is performed, there is a high possibility of erroneous detection or the like. Therefore, when the light source position detection unit 12 outputs that the light source exists, the control unit 2 changes the person detection parameter according to the position from the light source. For example, since it is considered that the contrast is more severe as the position is closer to the light source, person detection is performed after improving the image quality using a look-up table (LUT) that extends the contrast closer to the light source, and the above contrast. Compensate for the decline.

  Further, when the control unit 2 realizes RSR, predetermined image recognition is performed on the captured image from the camera 1 to detect an elliptical shape. At this time, the elliptical shape of the outer shape is easy to observe because it is backlit at a position close to the light source, but it is considered that the contrast of the image pattern inside is reduced. Therefore, an ellipse shape that is relatively dark inside is detected, luminance histogram normalization is performed on the inner area, the contrast is improved, and then the internal pattern recognition is performed. Contrast reduction compensation suitable for the stage is performed.

According to the vehicle-mounted vehicle recognition device according to the first embodiment described above, the following operational effects can be obtained.
(1) The control unit 2 acquires image information output by the camera 1 including the light-shielding region 31 shielded by the light-shielding plate 1a as a part of the photographing region 30 and imaging the outside of the vehicle. The high brightness pattern detection unit 11 detects from the image information that the light shielding area 31 includes a predetermined high brightness pattern. The light source position detection unit 12 detects the position of the light source 35 corresponding to the high luminance pattern when the high luminance pattern detection unit 11 detects the high luminance pattern from the light shielding region 31. The control unit 2 performs predetermined image recognition from the image information based on the position of the light source 35 detected by the light source position detection unit 12. Since it did in this way, also when light sources, such as the sun, are reflected in the picked-up image, an image recognition process can be performed with high precision.

(2) The high luminance pattern detection unit 11 detects that there are a predetermined number th2 or more of pixels having a specific luminance value equal to or greater than a predetermined threshold th1 in the light shielding region 31. Since it did in this way, the light sources of intensity more than fixed, such as the sun, can be detected appropriately.

(3) The light source position detection unit 12 detects the position of the light source 35 by searching for a pixel pattern whose luminance value decreases concentrically from the center from the region where the light shielding region 31 and the background region 32 are combined. Since it did in this way, the position of a light source can be detected correctly.

(4) The control unit 2 changes a parameter for detecting a white line on the road surface around the vehicle by image recognition according to the position of the light source 35. Since it did in this way, also when light sources, such as the sun, are reflected in a picked-up image, lane recognition can be performed with high precision.

(5) The control unit 2 changes a parameter when detecting a vehicle around the vehicle by image recognition according to the position of the light source 35. Since it did in this way, also when light sources, such as the sun, are reflected in a picked-up image, vehicle recognition can be performed with high precision.

  The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.

(Modification 1)
The shapes and positions of the light shielding plate 1a and the light shielding region 31 are not limited to those shown in FIG. As long as it is installed so that diffracted light from a light source such as the sun appears, it may have any shape at any position.

(Modification 2)
Image recognition based on the position of the light source detected by the light source position detection unit 12 is not limited to the above-described white line, lane, and vehicle recognition, and may be anything.

(Modification 3)
Since the camera and the light shielding plate are installed outside the passenger compartment, raindrops and muddy water may adhere to the light shielding plate due to rain or the vehicle's tire springing up. In such a case, raindrops or muddy water hangs in the vicinity of the lower end of the light shielding region 31, and the luminance value of the light shielding region 31 changes even without a strong light source such as the sun.

  In this case, as shown in FIG. 9, the above-described light source detection defines a light shielding plate upper region 37 that is shifted upward by a predetermined distance with respect to the light shielding region 31, and leakage is applied to the light shielding plate upper region 37. The presence of the light source 35 such as the sun may be detected using the light 36.

  By adopting this configuration, there is an effect that light source detection that is robust against environmental fluctuations can be performed without erroneously detecting water droplet adhesion due to rainfall or splashing.

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

DESCRIPTION OF SYMBOLS 1 ... Camera, 1a ... Light-shielding plate, 2 ... Control part, 3 ... Alarm output part, 4 ... Operation | movement state notification part, 5 ... External apparatus control part 10 ... Processing area setting part, 11 ... High-intensity pattern detection part, 12 ... Light source position detection unit, 100 ... vehicle-mounted vehicle recognition device

Claims (8)

An image acquisition unit that acquires image information output by imaging an outside of the vehicle by an imaging device including a light-shielding region shielded by the light-shielding member as a part of the imaging region;
From the image information, a high-intensity pattern detection unit that detects that the light-shielding region includes a predetermined high-intensity pattern;
When the high-intensity pattern detection unit detects the high-intensity pattern, it is determined whether the light is direct light based on the high-intensity pattern, and predetermined image recognition is performed from the image information based on the determination result. An image recognition unit for performing
An image processing apparatus comprising: The image processing apparatus according to claim 1.
A light source position detection unit that detects a position of a light source corresponding to the high luminance pattern when the high luminance pattern is detected from the light shielding region by the high luminance pattern detection unit;
The image processing apparatus, wherein the image recognition unit performs predetermined image recognition from the image information based on the position of the light source detected by the light source position detection unit. The image processing apparatus according to claim 1.
The high-intensity pattern detection unit detects, as the high-intensity pattern, that a predetermined number or more of pixels having a specific luminance value that is equal to or greater than a predetermined threshold exists in the light-shielding region. apparatus. The image processing apparatus according to claim 3.
The light source position detection unit detects the position of the light source by searching for a pixel pattern whose luminance value decreases concentrically from the center from the light-shielding region and other regions in the imaging region. Image processing device. In the image processing device according to any one of claims 1 to 4,
The image recognizing unit changes a condition for detecting a white line on a road surface around the vehicle by the image recognition according to a position of the light source. In the image processing device according to any one of claims 1 to 4,
The image recognition device, wherein the image recognition unit changes a condition when detecting a vehicle around the vehicle by the image recognition according to a position of the light source. In the image processing apparatus according to any one of claims 1 to 6,
A background area different from the light-shielding area;
The high-intensity pattern detection unit detects whether the high-intensity pattern is included across the light-shielding region and the background region;
When the image recognition unit detects that the high-intensity pattern is included across the light-shielding region and the background region, the image recognition unit determines that the light is direct light, and performs predetermined image recognition from the image information based on the determination result. An image processing apparatus characterized by The image processing apparatus according to claim 7.
An image processing apparatus, wherein the image recognition unit sets a luminance threshold based on the determined result, and performs an image recognition process based on the set luminance threshold.

Images