JP2002046504A - Display device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device for a vehicle to facilitate recognition of an obstacle in front in running environment where visibility is not good and recognition of a map image for navigation are effected by a driver. SOLUTION: A display control device 1 specifies (S51-S55) a running lane region, in which an own vehicle in an infrared image photographed by an infrared ray camera runs based on position information of a running lane, on which an own vehicle runs, obtained by a communicator between a road and a vehicle or an advancing passage estimated based on a steering angle and a car speed. A partial image corresponding to the running lane region and a solid map image produced based on map image information are displayed (S58-S60) in a state to be overlapped with each other on a display screen. In this case, in case a heat source being an obstacle is protruded from the running lane region, a partial image is picked up (S57-S59) such that a whole of the obstacle is contained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device for a vehicle which informs an occupant of the vehicle of a state of an obstacle or the like existing around the vehicle by displaying the information. And a suitable display device.

[0002]

2. Description of the Related Art Hitherto, there has been proposed an apparatus which takes an image of the surroundings of a vehicle using an infrared imaging apparatus and displays the taken image in front of a driver's seat. According to these devices, even if the headlights are turned on, even in a situation around the vehicle (for example, at night, in dense fog, or the like) where it is difficult for the driver to recognize due to human visual characteristics, there is an obstacle that actually exists. The driver can easily grasp the state of things and the like, and can effectively support the driving operation.

[0003] Japanese Patent Application Laid-Open No. 8-263784 discloses a pedestrian in any part of a road ahead by displaying temperature distribution information imaged using an infrared imager on a map screen for navigation. There has been proposed a technique for notifying a driver whether or not a driver exists.

[0004]

According to the above prior art, the driver can confirm the presence of a pedestrian on a map image even in bad weather, which can contribute to safe driving. In the superimposed image, a street light, a building light, an obstacle existing in a direction different from the traveling lane of the own vehicle (or an estimated traveling route), and the like are also displayed. It is difficult to recognize the information displayed on the screen, and the convenience is lacking.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a vehicle display device which enables a driver to easily recognize a forward obstacle and a map image for navigation in a traveling environment having poor visibility.

[0006]

In order to achieve the above object, a vehicle display device according to the present invention has the following features.

That is, an image pickup device provided in a vehicle and taking an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. A display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, wherein the display control means A partial image region corresponding to a road on which the vehicle travels and the vicinity thereof and the map image are superimposed on a display screen and displayed.

In a preferred embodiment, a radiant heat source corresponding to an obstacle is included in a region outside the partial image region in the entire region of the infrared image, and a part of the radiant heat source is When the radiant heat source is included in the partial image area or when the approach speed of the radiant heat source to the traveling lane (or the estimated traveling path) of the vehicle is faster than a predetermined speed, the display control unit also sets the partial image of the radiant heat source to It may be displayed together with the display screen.

Further, for example, the display control means may specify an obstacle in the partial image area of the radiant heat source when the amount of heat of the radiant heat source included in the infrared image is larger than a predetermined value.

[0010] Alternatively, in order to achieve the above object, a vehicle display device according to the present invention has the following configuration.

That is, an image pickup device provided in a vehicle for picking up an image of the front of the vehicle using infrared rays, route guidance means for guiding a route to a set desired destination, and generation based on map image information. A display control means for displaying on a display provided in the vehicle in a display mode in which the obtained three-dimensional map image and the infrared image captured by the imaging device are superimposed on a display screen. A display device,
The display control unit, when estimating a direction to change a course on a branch road existing ahead of the road on which the vehicle travels according to the route guidance route information calculated by the route guidance unit, includes: With respect to the partial image area corresponding to the distance from the branch road based on the position of the vehicle, only the image area extending in the direction in which the course change is performed is superimposed on the map image on the display screen and displayed. Features.

Alternatively, in order to achieve the above object, a vehicle display device according to the present invention has the following configuration.

That is, an image pickup device provided in a vehicle for picking up an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. And a display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, wherein the display control means includes a steering angle of the vehicle. When the value is larger than a predetermined value, only the partial image area corresponding to the turning direction of the vehicle according to the steering angle is displayed on the display screen so as to overlap the map image.

[0014]

According to the present invention, there is provided a vehicle display device in which a driver can easily recognize a forward obstacle and a map image for navigation in a driving environment having poor visibility. I do.

That is, according to the first, fifth, and sixth aspects of the present invention, the map is formed in such a manner that a partial image area which is relatively unnecessary for the driver is cut out of the entire area of the infrared image. Since superimposed display with the image is performed, recognition of obstacles ahead in a driving environment with poor visibility,
The driver can easily recognize a map image for navigation (including guidance display such as a route guidance arrow).

According to the second aspect of the present invention, since the entire outline of the obstacle that is present on the road on which the vehicle is traveling and a place deviated from the vicinity thereof is displayed, the driver can determine the type of the displayed obstacle. Can be easily identified.

According to the third aspect of the present invention, even if an obstacle exists on a road on which the vehicle is traveling and a place deviating from the vicinity thereof, the obstacle approaching the own vehicle is displayed. In addition, a quick and accurate driving operation for the obstacle can be performed.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a display device for a vehicle according to the present invention will be described in detail as an embodiment of a display device mounted on an automobile, which is a typical vehicle, with reference to the drawings.

[First Embodiment] FIG. 1 shows a first embodiment of the present invention.
FIG. 3 is a block diagram showing a system configuration of a display device according to the embodiment.

In FIG. 1, reference numeral 2 denotes a light switch capable of turning on and off a head lamp, a small lamp, and a fog lamp. In the present embodiment, the display on / off condition for displaying the image captured by the infrared camera 3 in a situation where the visibility in the front is not good is determined, for example, by the operation state of the light switch 2 by the driver. However, the method is not limited to this.
The operation state of the wiper switch that can set the operation of the wiper (for example, the display may be turned on in response to the start of the operation of the wiper), a predetermined time period (for example, by referring to the time information and calendar information, Display on at night,
Alternatively, the display may be turned on from 19 o'clock to 5 o'clock in summer and from 17 o'clock to 7 o'clock in winter. Or, weather conditions obtained from outside via communication (for example, display on when it is rainy or dense fog) The determination may be made by using.

Reference numeral 3 denotes an infrared camera which takes an image of the front of the host vehicle by using infrared rays. The infrared camera 3 moves rightward from a central axis extending in the front-rear direction of the vehicle due to the installation space at the head of the vehicle. At a position offset by B (m). Here, the offset amount B
Are stored in the memory of the display control device 1 in advance.

Reference numeral 4 denotes a front obstacle sensor for detecting a relative distance to an obstacle existing in front by using a laser radar, a millimeter wave radar, or the like. It is mounted on a central axis extending in the front-rear direction.

Reference numeral 5 denotes a GPS sensor that detects the current position of the vehicle based on a GPS (global positioning system) signal received from the outside. Reference numeral 6 denotes a navigation main switch capable of turning on / off a navigation function.

Reference numeral 7 denotes a map scroll switch capable of instructing scrolling of a map image displayed on the display device 9A or 9B. Reference numeral 8 denotes a destination setting switch capable of setting a desired destination for which route guidance is desired.

9A and 9B are map images based on map information stored in the map database 10 in advance and / or images captured by the infrared camera 3 (hereinafter, infrared images: FIGS. 8A and 9B). a) a display device, such as a liquid crystal display or a head-up display, which displays a).

Here, the display device 9A (first display) is
A position in front of the driver's seat of the host vehicle, where the driver can easily see the display image without making a large line-of-sight movement when the driver stares ahead, and is a high position (dashboard May be located near the center position of the camera). Further, the display device 9B (second display device) is disposed on a center console which is closer to the outer peripheral portion of the line of sight when the driver gazes forward as compared with the disposition position of the display device 9A.

Reference numeral 11 denotes a road-to-vehicle communication device that communicates with communication equipment on the road side to receive position information (information on lane shape) of the traveling lane of the vehicle and the like. 12
Is a steering angle sensor that detects the steering angle of the host vehicle.

1 controls display of a map image based on map information stored in the map database 10 in advance, and displays current position information detected by the GPS sensor 5 and a map corresponding to the current position information. A control function of a general navigation device for guiding a route to a desired destination set by an occupant (driver) using information and the like, and a display control using an infrared image (details will be described later).
And a display control device that performs the following. The navigation control and the display control by the display control device 1 are realized by executing software stored in a memory in advance by a CPU (not shown).

In this embodiment, the display control device 1
FIG. 8B or FIG.
As illustrated in (b), this type is of a type that allows a driver to perform a stereoscopic view similar to the field of view in front of the driver's seat, and when the route is guided, the vehicle should advance on the display screen as guidance display for the driver. It is assumed that the arrow symbol is displayed. Note that a general method for displaying a three-dimensional map image based on the map information is adopted, and the description in this embodiment is omitted.

Next, in this embodiment, the display control device 1
A specific control process performed by the server will be described.

When displaying a three-dimensional map image by the navigation function, the display control device 1 superimposes and displays an infrared image of a forward obstacle at a position in front of the traveling path of the own vehicle, so that the display control device 1 provides the driver with a display. Even in an environment where visibility is poor, the recognizability of the type and size of an obstacle present ahead and the location (relationship with a traveling road) is improved. At this time, when the entire area of the map image and the entire area of the infrared image are displayed in a superimposed manner, as illustrated in FIG. In some cases, it may not be easy for the driver to recognize information (guidance display such as arrow lines) included in the map image for use in the present embodiment. Only the partial image corresponding to the vicinity is extracted, and the extracted partial image and the entire area of the map image are extracted as shown in FIG.
By superimposing and displaying on the display screen as exemplified in (d), the recognizability of the map image for navigation by the driver is ensured.

Next, the method of adjusting the infrared camera 3 and the display standard of the map image, which must be performed when the map image and the infrared image are superimposed on each other, is described below. The description will be made separately for the case of a lane-only motorway and the case of a multi-lane motorway.

<Case where the traveling road is a one-lane road> First, the road on which the own vehicle travels is basically a one-lane automobile-only road where no human exists on the traveling road. Next, a case will be described in which only a partial image of a vehicle as a front obstacle in an infrared image is superimposed and displayed. As a display mode in this case, an infrared image of a preceding vehicle traveling in the same lane as the host vehicle may be displayed based on the center of the traveling lane of the traveling road.

As a more specific operation, the display control device 1
The display device displays a map image as illustrated in FIG. 8B so as to be positioned substantially at the center of a traveling lane in which the host vehicle is traveling, while gradually scrolling the map image according to a change in the current position. It is displayed on 9A or 9B. At this time, when there is a preceding vehicle ahead, only a partial image of the vehicle among the infrared images of the infrared camera 3 is superimposed so as to be located substantially at the center of the traveling lane, and FIG. )
The image of the display mode as illustrated in FIG.

Here, the shift amount of the infrared image (the road and the partial image in the vicinity thereof) includes the relative position of the preceding vehicle to the own vehicle detected by the forward obstacle sensor 4, the offset amount B, the infrared image, and the map. Set based on the scale of each image. For example, both images have a scale of 1/2
In the case where the preceding vehicle exists at a position shifted A (m) to the left from the center axis of the own vehicle and the offset amount of the infrared camera 3 is B (m) to the right as described above, the shift amount of the infrared image Is
In the right direction, it is (A−B) × １ ／ (m).

The scale 1 / n refers to the lateral direction (horizontal direction) of the driver's eyes with respect to an obstacle existing in front of the host vehicle.
Of the driver's eyes with respect to the obstacle displayed on the display device 9A in the lateral direction (horizontal direction), and is a value obtained by an experiment in advance.

When displaying the map image at the set scale, the display control device 1 enlarges or reduces the entire infrared image in accordance with the scale (if both images have the same scale, they remain the same). The road is shifted in the horizontal direction by the shift amount calculated as described above, a road and a partial image near the road are extracted from the infrared image, and the partial image is displayed on the map image in a superimposed manner. Thus, when the partial image of the preceding vehicle is superimposed on the map image as it is, FIG.
8D, a display screen corresponding to the field of view of the driver from the driver's seat as shown in FIG. 8D can be obtained.

The correspondence between the angle of view of the infrared camera 2 and the detection range of the front obstacle sensor 4 is shown in FIGS.
As described later with reference to, the partial image corresponding to the preceding vehicle (obstacle) can be specified by storing the information in the memory in advance, and the detection result (distance and position with respect to the own vehicle) of the forward obstacle sensor 4 And the corresponding relationship, by detecting from the whole infrared image. Further, a temperature characteristic of a general vehicle as a radiant heat source may be stored in a memory in advance and referred to when specifying a preceding vehicle (an obstacle) included in the infrared image.

<Case where the Traveling Road is a Multiple Lane Road> Next, the road on which the own vehicle travels is basically a multiple lane automobile exclusive road where no human exists on the traveling road, A case where the entire infrared image is superimposed on the image and displayed will be described. As a display mode in this case, by referring to the map information based on the detected current position, the lane in which the host vehicle is traveling is detected from a plurality of lanes, and the preceding vehicle traveling in the same lane as the host vehicle is detected. It is preferable to perform the same display as that of FIG. 6D in which the traveling road is a one-lane road, with reference to the center of the detected lane.

As a more specific operation, the shift amount of the infrared image is set based on the offset amount B and the respective scales of the infrared image and the map image. For example, when the scale of both images is １ ／ and the offset amount of the infrared camera 3 is B (m) to the right as described above, the shift amount of the infrared image is B × １ ／ ( m).

When displaying the map image at the set scale, the display control device 1 enlarges or reduces the entire infrared image in accordance with the scale (if both images have the same scale, they remain the same). The infrared image is shifted in the horizontal direction by the shift amount calculated as described above, and is displayed on the map image in a superimposed manner. (In the case of a plurality of lanes, the road and the vicinity thereof are similar to the case of the one lane. Only the image may be superimposed on the map image.)

In either case of one lane or a plurality of lanes, if the own vehicle is largely deviated from the center of the traveling lane, the infrared image (or a partial image thereof) is shifted as described above. When the vehicle is superimposed and displayed on the position where the host vehicle actually exists, the difference between the display screen and the actual field of view of the driver is large, and it is expected that the driver will feel uncomfortable.

Therefore, the lateral position (deviation amount) of the vehicle with respect to the traveling lane is detected, and when it is determined that the detected lateral position is more than a predetermined amount away from the center of the traveling lane, the deviation amount is determined. However, it is preferable to shift the display position of the obstacle (vehicle ahead) displayed on the map image or to shift the map image. For example, if the detected horizontal position is 1
m, the display position of the obstacle is (1 × scale) m
It may be shifted rightward, or a line deviated rightward by (1 × scale) m from the center of the traveling lane may be displayed substantially at the center of the display screen.

Incidentally, the method of detecting the lateral position with respect to the traveling lane is currently common, and the description thereof will be omitted.

<Display Control Process> Next, the procedure of the display control process performed by the display control device 1 to realize the display mode in the case of one lane or a plurality of lanes will be described.

FIG. 2 is a flowchart showing a display control process by the display control device 1 according to the first embodiment.

In FIG. 5, step S1: input of received data of the road-vehicle communication device 11, detection results of various sensors and operation states of various operation switches are input.

Steps S2 and S3: The operating state of the light switch 2 is detected (step S2).
In 3, the map image by the navigation function and the partial image of the infrared image are superimposed and displayed on the display device 9A as the first display, as in the display mode described above. On the other hand, when the switch has been turned off, the process proceeds to step S4. The processing in step S2 will be described later with reference to FIG.

Step S4: The operation state of the navigation main switch 6 is detected, and if the switch has been operated to the ON state in accordance with the operation state of the switch, step S4 is executed.
The routine proceeds to 5 and returns when the switch is operated in the off state.

Step S5: Based on the current position information detected by the GPS sensor 5, map image information near the current position is read from the map database 10, and a map image corresponding to the map image information is displayed on the second display. In addition to displaying on a certain display device 9B, a symbol representing the current position is displayed on the map image.

Step S6: By detecting the operation state of the destination setting switch 8, it is determined whether or not the driver has set the destination and the route guidance to the destination is being performed. When step S
Go to 7, otherwise return.

Step S7: In the route guidance to the set destination, it is determined whether or not the distance to a point where a lane change or a right / left turn is to be made is shorter than a predetermined distance, and it is time to display guidance. When the guidance display is necessary, the process proceeds to step S8, and otherwise, returns.

Steps S8 to S10: It is determined whether or not the light switch 2 is turned on and the map image and the infrared image are displayed in a superimposed manner on the display device 9A (step S8). Is YES (the switch is on), the screen being displayed on the display device 9A (the display screen on which the map image and the infrared image are superimposed)
Is further superimposed and displayed (step S9), and when NO (the switch is OFF), a map image representing the road ahead is displayed on the display device 9A, and the guidance image is displayed on the map image. Is displayed (step S10). The processing in step S8 will be described later with reference to FIG.

FIG. 3 is a flowchart showing details of steps S3 and S9 in the flowchart of the display control processing shown in FIG.

In the figure, step S51: It is determined from the received data from the road-to-vehicle communication device 11 whether or not the position information of the lane in which the vehicle is traveling is acquired. The process proceeds to S52, and if not acquired, proceeds to step S53.

Step S52: Using the position information (information on lane shape) of the lane in which the vehicle is traveling received from the road-to-vehicle communication device 11, it corresponds to the traveling lane area of the vehicle in the infrared image. Find a partial image.

Specifically, first, the lane position information received from the road-vehicle communication device 11 is associated with the infrared image. FIG. 13 is a diagram showing a group of a plurality of node data (point data) as lane information obtained from the road-vehicle communication device 11. FIG. 14 is a diagram illustrating a state where each node data is perspective-transformed and captured in an infrared image.

In FIG. 13, the coordinate center is the position of the vehicle, the horizontal axis is a direction perpendicular to the direction of travel of the vehicle, and the vertical axis is the direction of travel of the vehicle. As shown in FIG.
Is a collection of position information obtained from transmission / reception facilities provided on the road around the own vehicle, and a plurality of node data (point data) like map information.
It is a group of m1 to m5 and n1 to n5.

Therefore, as shown in FIG. 14, each of the node data m1 to m5 and n1 to n5 is perspective-transformed to the infrared image at the current position of the host vehicle and fetched into the image. Connect node data with a straight line or 2
By approximating by the next curve, the lane information (road edge or lane dividing line R1) obtained from the road-to-vehicle communication device 11 can be associated with the infrared image. A portion surrounded by two trajectories (straight line or curve) representing the lane determined based on the lane can be specified as a partial image of the traveling lane region.

The position of the traveling lane is detected by a magnetic nail sensor mounted on the own vehicle, by detecting magnetism from a plurality of magnetic nails buried in advance along the width direction of the lane on the road. May be specified based on the

Steps S53 and S54: Since the position information of the traveling lane could not be received in the judgment of step S51, it must be obtained by another method. Therefore, based on the steering angle detected by the steering angle sensor 12 and the own vehicle speed detected by a vehicle speed sensor (not shown), the traveling direction of the own vehicle is calculated (step S53), and along the calculated traveling direction. Then, a traveling path (predicted path in which the host vehicle will travel) having a predetermined width (for example, 3 meters) is calculated in the vehicle width direction (step S54).

Here, the traveling path is estimated based on the steering angle and the vehicle speed using, for example, a known method described in Japanese Patent Application Laid-Open No. H10-100820. The position information for the own vehicle is calculated at predetermined intervals (for example, every 10 m).

Step S55: From the entire area of the infrared image, the area of the partial image corresponding to the traveling route calculated in step S54 is specified, and the partial image is set as the traveling lane area. That is, the plurality of pieces of position information calculated in step S54 are combined with the road-vehicle communication device 1 described above in step S52.
As in the case of the node data in the association between the lane information and the infrared image of No. 1, the perspective transformation is performed and the image is taken into the infrared image.

Step S56: It is determined whether or not the heat source included in the infrared image exists inside the partial image area corresponding to the traveling lane area obtained in step S52 or S55, and if this determination is NO (When it does not exist)
Proceeds to step S58, and if YES (if present), proceeds to step S57.

Step S57: It is determined whether or not a heat source protruding from the partial image area corresponding to the traveling lane area obtained in step S52 or S55 exists in the infrared image (for example, the state shown in FIG. 10A). If the determination is NO (if not present), step S58
Proceeds to step S if YES (exists)
Go to 59.

Step S58: As a result of the determination in steps S56 and S57, it is determined that there is no heat source (obstacle) outside the partial image area corresponding to the travel lane area. Are extracted, the partial image area corresponding to the traveling lane area obtained in step S52 or step S55 is extracted.

Step S59: As a result of the determinations in steps S56 and S57, it is determined that there is a heat source (obstacle) protruding from the partial image area corresponding to the travel lane area. When only a partial image region corresponding to the traveling lane region is extracted by extracting a partial image of the traveling lane region and a partial image of the heat source which is an obstacle from the state shown in FIG. However, the display image obtained in step S60 described later can be configured to display the entire shape of the obstacle as shown in FIG. More specifically, a partial image within a predetermined range including a portion with the highest luminance (corresponding to a radiant heat source) is extracted from the entire region of the infrared image by image processing such as general filter processing. good. When setting a predetermined range for a high-luminance portion, the higher the higher-luminance portion is, the farther the vehicle is from the host vehicle (that is, the higher the luminance portion is located at the left or right end or upper portion of the angle of view of the infrared image). It is good to set it to a small size.

Step S60: If the processing in steps S51 to S59 is step S3 in FIG. 2 (no route guidance is performed), the partial image extracted in step S58 or step S59 and the GPS sensor 5 Is displayed in a state of being superimposed on a map image obtained based on the current position detected by the above or the map information of the map database 10 or the like. On the other hand, when the processing of steps S51 to S59 is step S9 of FIG. 2 (in which the route guidance is performed), in addition to the partial image and the map image,
Further, predetermined symbols (arrows and the like) for route guidance are displayed in a superimposed state.

According to the above-described embodiment, a state in which a partial image outside the traveling lane area which is relatively unnecessary for the driver is cut out of the entire infrared image area (protruding from the traveling lane area) If an obstacle is present, the image of the obstacle is included) and the map image is displayed in a superimposed manner. Therefore, recognition of an obstacle ahead in a driving environment with poor visibility and recognition of a map image for navigation are performed. (Including guidance display such as arrows for route guidance) can be easily performed by the driver, and the driving operation of the driver can be efficiently supported.

Also, in combination with the display control processing shown in FIG. 2, the two indicators can be used properly, so that the map image or the infrared image can be used in a situation where the driver does not actively need the driving operation. Is displayed on the display device 9A from the center of the field of view, so that the driver's discomfort (complexity) can be prevented.

[Second Embodiment] Next, a second embodiment based on the above-described vehicle display device according to the first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

In the display control process of FIG. 2, when the display condition of the infrared image (the light switch 2 is turned on) is satisfied, the map image and the infrared image (or a partial image thereof) are always superimposed. Although the display screen is displayed, when the entire infrared image is superimposed on the map image, the contour of an object in the display screen is overlapped, and the display screen lacks sharpness, and it is expected that a burden is imposed on the driver's vision. You.

Therefore, in the present embodiment, even if such display conditions are satisfied, when no obstacle is detected, only the map image corresponding to the current position is displayed.
Only when an obstacle is detected, as in the first embodiment, the map image and the infrared image are displayed in a superimposed manner.

In the present embodiment, the method of calculating the shift amount of the infrared image in the case of one lane or a plurality of lanes is the same, and redundant description will be omitted.

FIG. 4 is a flowchart showing a display control process by the display control device 1 according to the second embodiment.

In the figure, the difference from the flowchart of FIG. 2 in the first embodiment is that when the operation state of the light switch 2 is determined to be ON in step S12, first, in step S13, A map image corresponding to the current position is displayed on the display device 9A as the first display. Then, in step S14, it is determined whether or not the infrared image from the infrared camera 3 includes a heat source representing the preceding vehicle. Only when the preceding vehicle is detected in this determination, the map image and the infrared The image (or a partial image thereof) is superimposed and displayed.

The processing in each step after step S16 is the same as the processing in each step after step S4 in FIG. 2, and in step S15 and step S21,
Each step of the flowchart of FIG. 3 is executed similarly to step S3 and step S9 in the embodiment.

According to the present embodiment, as in the first embodiment, it is possible to improve the recognizability of the type and size of the obstacle present ahead and the location of the obstacle (relation with the traveling road). In addition to being able to efficiently assist the driver's driving operation, when there is no obstacle, only a map image is displayed, and a display screen where a map image and an infrared image are superimposed So you can reduce your chances,
The burden on the driver's vision can be reduced.

[Third Embodiment] Next, a third embodiment based on the vehicular display device according to the above-described second embodiment will be described. In the following description, the same configuration as that of the second embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

In this embodiment, when no obstacle is detected, only a map image corresponding to the current position is displayed, and only when an obstacle is detected, the map image is displayed in the same manner as in the first embodiment. When an image and an infrared image are displayed in a superimposed manner, and a heat source corresponding to the obstacle is included in an area outside the travel lane area as an obstacle to be displayed on the display screen, that heat source is When the speed approaching the lane area is faster than a predetermined speed, a partial image of the heat source is also displayed on the display screen.

Accordingly, the overall configuration of the display control process is the same as that of the second embodiment.
This embodiment is the same as the flowchart of FIG. 4 described in the second embodiment, and differs from the second embodiment in that the processing of the flowchart shown in FIG. 5 described below is performed in step S15.

FIG. 5 is a flowchart showing details of step S15 in the flowchart of the display control processing shown in FIG.

In the configuration shown in FIG.
To S67, S69 to S7
Each step shown in FIG. 1 is substantially the same as each of the steps S51 to S60 in FIG. 3, but in the present embodiment, when the determination in step S66 is NO (the heat source is included in the partial image of the detected traffic lane area). Does not exist), the process does not directly proceed to step S69 for extracting a partial image of the traveling lane region from the infrared image, but instead proceeds to step S6.
In step 8, it is determined whether a heat source (obstacle) outside the travel lane area is approaching the travel lane area at a speed higher than a predetermined speed. If the determination is YES (an obstacle approaching at a speed higher than the predetermined speed) When there is an obstacle, the process proceeds to step S70, and when NO (when there is no obstacle approaching at a speed higher than the predetermined speed), the process proceeds to step S69.

Here, in the determination in step S68, the detection result of the front obstacle sensor 4 is used to accurately specify the heat source (obstacle) to be displayed among the heat sources outside the travel lane area. In this case, it is necessary to previously associate the angle of view of the infrared image with the detection range of the sensor. Therefore, a method of association will be described below.

Next, a specific method of detecting the position of the obstacle by associating the detection signal of the obstacle sensor with the infrared image in step 7 will be described.

FIG. 15 is a diagram showing a display example of an infrared image taken by an infrared camera. FIG. 16 is a diagram showing a state in which the detection range of the front obstacle sensor 4 is viewed from above. In the detection range, the obstacle A corresponding to the image captured by the infrared camera 3 illustrated in FIG. The figure shows a case where (another preceding vehicle) and an obstacle B (pedestrian) are included.

In this embodiment, the front obstacle sensor 4
Of the obstacle A and the obstacle B according to the distance Da of the own vehicle and the obstacle A and the angle θa (≒
0), and the distance Db between the host vehicle and the obstacle B and the angle θb with the center axis.

Accordingly, if the positional relationship between the center axis in the detection range of the front obstacle sensor 4 and the coordinate axis of the imaging plane (angle of view) of the infrared camera 3 is previously corresponded (matched), the front obstacle sensor 4 Distance to obstacle output from
As shown in FIG. 15, the distance D is counted as the number of lines (the number of pixels) from the lower side of the imaging surface on the imaging surface and the angle θ (θa, θa).
b) counts the number of pixels from the left and right center lines of the imaging surface, and the radiant heat source including the position (pixel) in the imaging surface specified by the count is detected by the front obstacle sensor 4 It can be determined that it corresponds to the entire shape of the obstacle that has been performed. Therefore, if such processing is performed in step S68, the display control device 1 can recognize the obstacle included in the image captured by the infrared camera 3 and the distance D to the obstacle in association with each other.

In step S68, the notable obstacle is associated with the heat source in the infrared image as described above.
Based on the moving amount and moving direction of the obstacle per unit time, it is determined whether the obstacle is approaching the traveling lane area at a speed higher than a predetermined speed.

According to the present embodiment, in addition to the same effects as those of the second embodiment, even if an obstacle exists on a traveling road and a place deviating from the vicinity thereof, the vehicle approaches the own vehicle. The displayed obstacle is displayed, so that the driver can perform a quick and accurate driving operation on the obstacle.

[Fourth Embodiment] Next, a fourth embodiment based on the vehicular display device according to the first or second embodiment will be described. In the following description, the same components as those of the first and second embodiments will not be described repeatedly, and will be described focusing on the characteristic portions of the present embodiment.

In the present embodiment, similar to the above-described embodiments, the recognition of the obstacle ahead and the navigation in the driving environment with poor visibility are performed by the superimposed display screen in a display mode different from the above-described embodiments. A vehicle display device that allows a driver to easily recognize a map image for use (including guidance display such as a route guidance arrow) will be described.

In the present embodiment, the overall configuration of the display control process is the same as the flowcharts of FIGS. 2 and 4 described in the first and second embodiments. 4 in that step S9 in FIG. 2 and step S21 in FIG. 4 perform the processing of the flowchart shown in FIG. 6 described below.

FIG. 6 is a flowchart showing details of step S9 (FIG. 2) and step S21 (FIG. 4) in the flowchart of the display control processing shown in FIG. 2 or FIG.

In FIG. 6, in step S81, it is determined whether or not the content of the guidance display for route guidance to be displayed is at least one of a right / left turn or a forked road.
In the case of ES (when the content of the guidance indicates a direction of a right turn or a left turn), the process proceeds to step S89, and in the case of NO (when the content of the guidance indicates straight ahead), the process proceeds to step S82. .

Steps S82 to S86: Steps S51 to S5 described in the first embodiment.
By performing the same processing as in FIG. 5 (FIG. 3), a traveling lane area corresponding to the lane in which the host vehicle is traveling in the infrared image is specified.

Steps S87 and S88: A partial image area corresponding to the travel lane area obtained in step S83 or S86 is extracted from the infrared image (step S87), and the extracted partial image and the GPS sensor are extracted. 5. Current position and map database 10 detected by 5
And a predetermined symbol (e.g., arrow) for route guidance is displayed in a superimposed state.

Step S89: The position in the infrared image of the right or left turn point or the branch road point to be displayed as guidance for route guidance to be displayed is obtained.

Step S90: If the guidance display for route guidance to be displayed is a right / left turn with reference to the position obtained in step S89, the guidance display is located above (corresponding to a far distance) the right / left turn point in the infrared image. If the content of the guidance display is a forked road, a portion that is farther from the forked road point in the infrared image and that is in a direction different from the traveling path of the host vehicle is cut.

Step S91: The partial image (remaining part) obtained in step S90, the current position detected by the GPS sensor 5, the map image obtained based on the map information of the map database 10 and the like, and the route guidance A predetermined symbol (such as an arrow) is displayed in a superimposed state.

According to the present embodiment for performing such processing, FIG. 11A includes an infrared image as illustrated in FIG. 11A and guidance display for route guidance for instructing a right turn.
When superimposing the map image as illustrated in (b) as it is, superimposing the map image as it is, the guidance display becomes difficult to see as shown in FIG.
Since the image farther from the right turn point is cut by the process of 0, the screen displayed in step S91 is the same as that shown in FIG.
As shown in (d), the guidance display for turning right is a display mode that is easy for the driver to see, so that the driver's driving operation can be efficiently supported.

[Fifth Embodiment] Next, a fifth embodiment based on the vehicle display device according to the first or second embodiment and the fourth embodiment will be described. In the following description, the same configurations as those of the first, second, and FIG. 4 embodiments will not be described repeatedly, and will be described focusing on the characteristic portions of the present embodiment.

In the present embodiment, similar to the above embodiments, the recognition of the obstacle ahead and the navigation in the driving environment with poor visibility are performed by the superimposed display screen having a display mode different from those of the above embodiments. A vehicle display device that allows a driver to easily recognize a map image for use (including guidance display such as a route guidance arrow) will be described.

In the present embodiment, the overall configuration of the display control process is the same as the flowcharts of FIGS. 2 and 4 described in the first and second embodiments. 2 and the step S21 of FIG. 4 are different in that the processing of the flowchart shown in FIG. 7 described below is performed.

FIG. 7 is a flowchart showing details of step S9 (FIG. 2) and step S21 (FIG. 4) in the flowchart of the display control processing shown in FIG. 2 or FIG.

In FIG. 7, in step S101, it is determined whether the steering angle detected by the steering angle sensor 12 is larger than a predetermined value. If the determination is YES (the steering angle is equal to or larger than the predetermined value), the process proceeds to step S109, and NO ( When the steering angle is smaller than the predetermined value), the process proceeds to step S102.

Steps S102 to S108: The same processing as steps S82 to S88 (FIG. 6) of the fourth embodiment is performed to extract the extracted partial image, the current position detected by the GPS sensor 5, and the map database. The map image obtained based on the map information and the like and predetermined symbols (arrows and the like) for route guidance are displayed in a superimposed state.

Step S109: In the infrared image,
The portion in the direction opposite to the turning direction (direction in which the steering angle is detected) is cut as the left and right center of the image. At this time, the area to be cut is preferably set to be larger as the steering angle or the vehicle speed is larger, and the cut is preferentially performed from the edge of the screen.

Step S110: The partial image (remaining portion) obtained in step S109, the map image obtained based on the current position detected by the GPS sensor 5, the map information in the map database 10, and the like, and the route guidance A predetermined symbol (such as an arrow) is displayed in a superimposed state.

According to the present embodiment for performing such processing, a diagram including a traveling lane region and a heat source (obstacle) protruding from the traveling lane region from an infrared image as illustrated in FIG. When superimposing a partial image as exemplified in FIG. 12 (b) on a map image (here,
It is assumed that the guidance display such as turning left or right is not performed), and if they are superimposed as they are, obstacles located in a direction opposite to the turning direction of the own vehicle are displayed, making it difficult to see the map image. Step S1
Since the image in the opposite direction is cut by the processing of step 09, the screen displayed in step S110 is
The guidance display for turning right as shown in FIG. 12C is a display mode that is easy for the driver to see, so that the driver's driving operation can be efficiently supported.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a system configuration of a display device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating a display control process performed by the display control device 1 according to the first embodiment.

FIG. 3 is a flowchart showing details of steps S3 and S9 in the flowchart of the display control process shown in FIG. 2;

FIG. 4 is a flowchart illustrating a display control process performed by a display control device 1 according to a second embodiment.

FIG. 5 is a flowchart showing details of step S15 in the flowchart of the display control process shown in FIG. 2 as a third embodiment.

FIG. 6 is a flowchart of a display control process shown in FIG. 2 or FIG. 4 according to the fourth embodiment;
FIG. 2 is a flowchart showing details of step S21 (FIG. 4).

FIG. 7 is a flow chart of the display control processing shown in FIG. 2 or FIG. 4 as the fifth embodiment;
FIG. 2 is a flowchart showing details of step S21 (FIG. 4).

FIG. 8 is a diagram illustrating a display example of a map image or the like on a display device.

FIG. 9 is a diagram illustrating a basic display mode of superimposed display according to the first embodiment.

FIG. 10 is a diagram illustrating a display mode in a case where a heat source to be displayed in the overlap display according to the first embodiment is protruding from a traffic lane area.

FIG. 11 is a diagram illustrating a basic display mode of superimposed display according to a fourth embodiment.

FIG. 12 is a diagram illustrating a basic display mode of superimposed display according to the fifth embodiment.

FIG. 13 is a diagram showing a group of a plurality of node data as lane information obtained from a road-to-vehicle communication device.

FIG. 14 is a diagram showing a state in which each node data is perspective-transformed and captured in an infrared image.

FIG. 15 is a diagram showing a correspondence between an angle of view of an infrared image captured by the infrared camera 3 and a predetermined coordinate system.

FIG. 16 is a diagram showing a state where the detection range of the front obstacle sensor 4 is viewed from above.

[Explanation of symbols]

 1: display control device, 2: light switch, 3: infrared camera, 4: forward obstacle sensor, 5: GPS sensor, 6: navigation main switch, 7: map scroll switch, 8: destination setting switch, 9A, 9B : Display device, 10: map database, 11: road-to-vehicle communication device, 12: steering angle sensor,

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 624 B60R 21/00 624G 628 628B 628C G09B 29/00 G09B 29/00 A F // B60R 11 / 02 B60R 11/02 C (72) Inventor Kenichi Okuda 3-1 Shinchi, Fuchu-cho, Aki-gun, Hiroshima Mazda Co., Ltd. (72) Inventor Hiroki Uemura 3-1 Shinchi, Fuchu-cho, Aki-gun, Hiroshima Mazda (72) Inventor Hidekazu Sasaki 3-1 Shinchi, Fuchu-cho, Aki-gun, Hiroshima Prefecture F-term (reference) in Mazda Co., Ltd.

Claims (6)

[Claims] An image pickup apparatus provided in a vehicle and taking an image of the front of the vehicle using infrared rays, a three-dimensional map image generated based on map image information, and an infrared image taken by the image pickup apparatus. And a display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, wherein the display control means includes: A partial image region corresponding to the road on which the vehicle travels and the vicinity thereof,
A display device for a vehicle, wherein the map image is superimposed and displayed on a display screen. 2. A case in which a radiant heat source corresponding to an obstacle is included in an area deviating from the partial image area in the entire area of the infrared image, and a part of the radiant heat source is included in the partial image area. The vehicle display device according to claim 1, wherein when included, the display control means also displays a partial image of the radiant heat source together with the display screen. 3. A case in which a radiant heat source corresponding to an obstacle is included in an area outside the partial image area in the entire area of the infrared image, and the approach speed of the radiant heat source to a traveling lane of the vehicle The display device for a vehicle according to claim 1, wherein when the speed is higher than a predetermined speed, the display control means also displays a partial image of the radiant heat source together with the display screen. 4. The display control unit according to claim 1, wherein the heat amount of the radiant heat source included in the infrared image is larger than a predetermined value.
The vehicle display device according to claim 2, wherein an obstacle is specified in the partial image area of the radiant heat source. 5. An image pickup apparatus provided in a vehicle for picking up an image of the front of the vehicle using infrared rays, route guidance means for guiding a route to a set desired destination, and generation based on map image information. A display control means for displaying on a display provided in the vehicle in a display mode in which the obtained three-dimensional map image and the infrared image captured by the imaging device are superimposed on a display screen. A display device, wherein the display control unit estimates a direction of changing a course on a branch road existing in front of a road on which the vehicle travels according to the route guidance route information calculated by the route guidance unit. Of the infrared image, with respect to a partial image area corresponding to a distance from the forked road based on the position of the vehicle, only an image area extending in a direction in which the course change is performed is displayed on the display screen by the map image. Vehicle display apparatus characterized by superimposing to display on. 6. An image pickup device provided in a vehicle and taking an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. And a display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, wherein the display control means includes a steering angle of the vehicle. A display device for a vehicle, wherein when the value is larger than a predetermined value, only a partial image area corresponding to a turning direction of the vehicle according to the steering angle is displayed on the display screen so as to overlap the map image.