JP7147590B2 - Driving support device - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driving support device that provides a driver with information regarding traffic restrictions (travel restrictions) on a road on which a vehicle is traveling.

One of such driving assistance devices (hereinafter also referred to as a "conventional device") captures a portion of an image (front image) obtained by photographing an area in the direction in which the vehicle travels, including a road sign. It recognizes and displays the marked portion on the display (see Patent Document 1, for example). According to the conventional device, even if the driver overlooks the road sign, it is possible to acquire information regarding regulations, warnings, etc., related to the road on which the vehicle is traveling. For example, when the conventional device recognizes a no-passing sign (see FIG. 3A) included in the front image, the no-passing sign is displayed on the display, so that the driver of the vehicle equipped with the conventional device can can recognize that the road on which the vehicle is traveling is an overtaking prohibited section.

JP-A-2001-514110

However, if the road sign is dirty or covered with branches and leaves of trees, there is a possibility that the road sign included in the front image cannot be recognized. If the conventional device cannot correctly recognize the overtaking prohibition sign, the driver cannot obtain information on whether or not the road on which the vehicle is traveling is in an overtaking prohibited section through the display of the conventional device.

Therefore, one of the objects of the present invention is to provide information on whether or not the road on which the vehicle is traveling is in an overtaking prohibited section even if the overtaking prohibited sign included in the front image cannot be correctly recognized. An object of the present invention is to provide a driving support device that can be provided to a driver.

A driving support device (hereinafter also referred to as "the device of the present invention") for achieving the above object includes an imaging device, a display device, a lane marking determination section, and an information provision section.

The imaging device (front camera 31) is
It is mounted on a vehicle (10) and acquires a "front image" by photographing an area in front of the vehicle.

The display device (display 33) is
It is disposed at a position visible to the driver of the vehicle.

The lane marking determination unit (driving support ECU 20)
The "roadway center line of the road on which the vehicle is traveling" included in the front image is extracted, and it is determined whether or not the roadway center line is a "yellow solid line".

The information providing unit (driving support ECU 20)
When the lane marking determination unit determines that the roadway center line is a solid yellow line (determined as "Yes" in step 535 in FIG. 5), a predetermined "specific display (overtaking prohibition symbol 52)" is displayed. is displayed on the display device (step 540 in FIG. 5).

On roads where the roadway center line (center line) is a solid yellow line, overtaking traffic for overtaking is prohibited. The device of the present invention causes the display device to display the specific display when the roadway center line included in the front image is a yellow solid line. Therefore, according to the apparatus of the present invention, even if the overtaking prohibition sign included in the forward image cannot be correctly recognized, the driver is provided with information on whether or not the road on which the vehicle is traveling is in an overtaking prohibited section. can be provided to

In the above description, in order to facilitate understanding of the present invention, names and/or symbols used in the embodiments are added in parentheses to configurations of the invention corresponding to the embodiments to be described later. However, each component of the present invention is not limited to the embodiments defined by the names and/or symbols. Other objects, features and attendant advantages of the present invention will be readily understood from the description of embodiments of the present invention described with reference to the following drawings.

1 is a schematic diagram of a vehicle (present vehicle) in which a driving assistance device (present assistance device) according to an embodiment of the present invention is mounted; It is the figure which showed the example of the road sign displayed on the display with which this vehicle is equipped. (A) is a diagram showing an overtaking prohibition sign, and (B) and (C) are diagrams showing an end sign (section end sign) of an overtaking prohibition section. FIG. 2 is a diagram showing a state in which the vehicle enters an overtaking prohibited section and then finishes traveling in the overtaking prohibited section. 4 is a flow chart showing an overtaking prohibition symbol display processing routine executed by the support device;

Hereinafter, a driving support device (hereinafter also referred to as "this support device") according to an embodiment of the present invention will be described with reference to the drawings. This support device is applied to a vehicle 10 shown in FIG. This assistance device includes a driving assistance ECU 20, which is an electronic control unit (ECU).

The driving assistance ECU 20 includes a microcomputer having a CPU, a nonvolatile memory and a RAM as main elements. The CPU sequentially executes predetermined programs (routines) to read data, perform numerical calculations, output calculation results, and the like. The nonvolatile memory is composed of flash memory and stores programs executed by the CPU and lookup tables (maps) referenced when the programs are executed. The RAM temporarily stores data referred to by the CPU.

The driving assistance ECU 20 is connected to a front camera 31 , a vehicle speed sensor 32 and a display 33 .

The front camera 31 is an imaging device that acquires an image (hereinafter, also referred to as a "front image") of a front area of the vehicle 10, and sends a signal representing the front image every time a predetermined time elapses. Output to the ECU 20 . The vehicle speed sensor 32 detects a vehicle speed Vt, which is the travel speed of the vehicle 10, and outputs a signal representing the vehicle speed Vt to the driving support ECU 20. FIG.

The display 33 is a display device (specifically, a liquid crystal display (LCD)) arranged at a position visible by the driver in the vehicle interior of the vehicle 10 (specifically, in front of the driver). . Characters, graphics, and the like displayed on the display 33 are controlled by the driving support ECU 20 .

The driving assistance ECU 20 provides various driving assistance functions including a road sign assist function to assist the driver in driving the vehicle 10 . The road sign assist function is a function of recognizing a road sign in front of the vehicle 10 and displaying a symbol representing the recognized road sign on the display 33 .

In order to provide the road sign assist function, the driving assistance ECU 20 executes "sign recognition processing" and "lane recognition processing". The sign recognition process is a process of detecting (recognizing) road signs included in the forward image by a well-known method. The driving assistance ECU 20 executes a sign recognition process each time a front image is received from the front camera 31 .

On the other hand, the lane recognition processing detects the "roadway centerline (centerline) of the road on which the vehicle 10 is traveling" included in the front image, and furthermore, it is known whether or not the roadway centerline is a yellow solid line. This is the process of determining by the method. The timing at which the driving assistance ECU 20 executes the lane recognition process will be described later with reference to FIG.

In this embodiment, the driving assistance ECU 20 extracts lane markings (i.e., roadway center line, lane boundary line, and roadway outer line) included in the forward image by pattern matching processing when executing lane recognition processing. That is, the driving assistance ECU 20 extracts regions in the forward image that are similar to various pre-stored templates related to lane markings. If the extracted road center line is a solid line, the driving assistance ECU 20 determines whether the road center line is yellow or white.

Specifically, if the difference between the value representing the color of the area of the roadway centerline in the front image and the pre-stored color reference value of the yellow roadway centerline is smaller than a predetermined threshold value, the driving assistance ECU 20 , that the road center line is yellow. In addition, if the difference between the value representing the color of the roadway outer line (that is, the white division line) in the front image and the value representing the color of the area of the roadway center line is greater than a predetermined threshold, the driving assistance ECU 20 determines that the road center line is yellow.

A screen 41, which is an example of a screen displayed on the display 33, is shown in FIG. In FIG. 2, a sign symbol 51 indicates that a speed limit sign has been detected by sign recognition processing. On the other hand, the overtaking prohibited symbol 52 is displayed on the display 33 when the vehicle 10 is traveling in the overtaking prohibited section. The overtaking prohibition symbol 52 is also called a "specific display" for convenience.

The driving support ECU 20 determines whether or not the vehicle 10 is traveling in an overtaking prohibition section (that is, whether or not the overtaking prohibition symbol 52 should be displayed on the display 33) based on the execution results of the sign recognition processing and the lane recognition processing. do. More specifically, the driving assistance ECU 20 starts displaying the overtaking prohibition symbol 52 on the display 33 when the "passing prohibition symbol display start condition" is satisfied. On the other hand, the driving support ECU 20 ends the display of the overtaking prohibition symbol 52 on the display 33 when the "passing prohibition symbol display end condition" is satisfied while the overtaking prohibition symbol 52 is being displayed.

The overtaking prohibition symbol display start condition is a condition that is satisfied when at least one of the following conditions (1a) and (1b) is satisfied.
Condition (1a): A no passing sign was detected by the sign recognition processing.
Condition (1b): The lane recognition process determined that the roadway center line was a solid yellow line.

In the present embodiment, the no passing sign of condition (1a) is the road sign shown in FIG. 3(A).

The overtaking prohibition symbol display end condition is a condition that is satisfied when at least one of the following conditions (2a) to (2c) is satisfied.
Condition (2a): The end sign of the overtaking prohibited section was detected by the sign recognition processing.
Condition (2b): The lane recognition processing determined that the roadway center line was not a solid yellow line.
Condition (2c): Post-detection travel distance Dr, which is the travel distance of vehicle 10 from the point in time when the no overtaking sign was last detected by the sign recognition processing to the present time, is greater than a predetermined distance threshold value Dth (that is, Dr >Dth).

The end sign of the overtaking prohibited section in condition (2a) is hereinafter simply referred to as the "section end sign". In this embodiment, the section end sign is the road sign shown in FIGS. 3(B) and 3(C). In the condition (2c), the distance threshold Dth is set to a distance longer than the "general interval (distance) at which overtaking prohibition signs are installed" in the overtaking prohibition section. The driving assistance ECU 20 acquires (calculates) an integral value of the vehicle speed Vt in a period from the last detection of the no passing sign to the current time as the post-detection traveling distance Dr.

FIG. 4 shows an example in which the display of the overtaking prohibition symbol 52 on the display 33 is started and then the display of the overtaking prohibition symbol 52 is terminated. In FIG. 4 , an arrow Ar represents the travel route of the vehicle 10 .

When the vehicle 10 is at the position indicated by point P0 in FIG. 4, no overtaking prohibition sign is detected, and the roadway center line (that is, the lane marking L1) at the running position of the vehicle 10 is a broken white line. Therefore, neither the condition (1a) nor the condition (1b) is satisfied, so the overtaking prohibition symbol 52 is not displayed on the display 33. FIG.

After that, when the vehicle 10 reaches the traveling position P1, the sign recognition process detects the sign S1, which is an overtaking prohibition sign, and the lane recognition process determines that the roadway center line (that is, the division line L2) is a solid yellow line. be done. In this case, since both the condition (1a) and the condition (1b) are satisfied, the overtaking prohibition symbol display start condition is satisfied. Therefore, the driving assistance ECU 20 starts displaying the overtaking prohibition symbol 52 on the display 33 .

After that, when the vehicle 10 reaches the traveling position P2, the sign S2, which is the second overtaking prohibition sign in the overtaking prohibition section of this example, is detected by the sign recognition processing. Furthermore, other overtaking prohibition signs in this overtaking prohibition section are also detected by the sign recognition processing. For example, the sign S3 detected when the vehicle 10 reaches the traveling position P3 is the last overtaking prohibition sign in this overtaking prohibition section.

When the vehicle 10 reaches the travel position P4, which is the end point of the overtaking prohibited section in this example, the section end sign S4 is detected by the sign recognition processing, and the roadway center line (that is, the white dashed line) is detected by the lane recognition processing. is not a solid yellow line. In this case, since both the condition (2a) and the condition (2b) are satisfied, the overtaking prohibition symbol display end condition is satisfied. Therefore, the driving assistance ECU 20 stops displaying the overtaking prohibition symbol 52 on the display 33 .

Here, it is assumed that the driving assistance ECU 20 cannot detect the sign S1 by the sign recognition processing due to the presence of the foliage of the tree covering the sign S1, or the like. Even in this case, when the vehicle 10 reaches the traveling position P1, the lane recognition process determines that the roadway center line (that is, the lane marking L2) is a solid yellow line (that is, condition (1b) is satisfied). ), as a result, the condition for starting the display of the overtaking prohibition symbol is satisfied.

Similarly, assume that the sign recognition process failed to detect sign S4. Even in this case, when the vehicle 10 reaches the traveling position P4, the lane recognition process determines that the roadway center line (that is, the lane marking L3) is not a yellow solid line (that is, condition (2b) is satisfied). ), as a result, the condition for ending the display of the overtaking prohibition symbol is satisfied.

Alternatively, it is assumed that the sign S4 is not detected by the sign recognition processing even when the traveling position P4 is reached, and that the lane marking L3 is erroneously determined to be a yellow solid line by the lane recognition processing. In this case, neither condition (2a) nor condition (2b) is satisfied. However, when the vehicle 10 reaches the traveling position P5 which is separated from the traveling position P3 by the distance threshold value Dth, the condition (2c) is established, so the prohibition symbol display end condition is established. Therefore, the driving assistance ECU 20 stops displaying the overtaking prohibition symbol 52 on the display 33 .

A vehicle symbol 10a shown in FIG. 2 represents the vehicle 10, and a vehicle symbol 53 represents a vehicle to be followed (preceding of the vehicle 10) by the radar cruise control function, which is one of the driving assistance functions provided by the driving assistance ECU 20. vehicle) is detected. A description of the radar cruise control function is omitted in this specification.

(Specific operation)
Next, specific operations of the driving assistance ECU 20 related to the display of the overtaking prohibition symbol 52 on the display 33 will be described with reference to FIG. Note that a description of processing (for example, display of the sign symbol 51 in the example of FIG. 2) when a road sign other than a no-passing sign is detected by the sign recognition processing is omitted.

The CPU of the driving assistance ECU 20 (hereinafter also simply referred to as "CPU") executes the "overtaking prohibition symbol display processing routine" represented by the flowchart in FIG. 5 each time a predetermined period of time elapses. Therefore, at an appropriate timing, the CPU starts processing from step 500 in FIG. is started and continued).

(Case A)
Assume that vehicle 10 is currently at point P0 in FIG.
In this case, since the overtaking prohibition symbol 52 is not displayed on the display 33, the CPU determines "No" in step 505, proceeds to step 530, and determines whether condition (1a) is satisfied. . That is, the CPU determines whether or not the no passing sign has been detected by the sign recognition processing executed during the period from the time when execution of this routine was last completed to the present time.

According to this assumption, since the vehicle 10 is not traveling in the overtaking prohibited section, no overtaking prohibited sign is detected. That is, condition (1a) is not satisfied. Therefore, the CPU determines "No" in step 530, proceeds to step 535, and determines whether condition (1b) is satisfied. That is, the CPU executes lane recognition processing to determine whether the roadway center line is a solid yellow line.

At this time, the roadway center line of the road on which the vehicle 10 is traveling (that is, the lane marking L1) is a broken white line, so the lane recognition processing does not determine that the roadway center line is a solid yellow line. That is, condition (1b) is not satisfied. Therefore, the CPU makes a "No" determination in step 535, proceeds to step 595, and terminates this routine.

(Case B)
After that, it is assumed that this routine is executed for the first time after the vehicle 10 reaches the position of point P1.
In this case, the sign S1 (that is, no overtaking sign) is detected by the sign recognition processing, so the condition (1a) is established. Therefore, the CPU makes a "Yes" determination in step 530 and proceeds to step 540 to start displaying the overtaking prohibition symbol 52 on the display 33 . The CPU then proceeds to step 595 .

Note that if the lane recognition process determines that the roadway center line (that is, the lane marking L2) is a yellow solid line (that is, the condition (1b) is satisfied Then), the CPU determines “Yes” in step 535 and proceeds to step 540 .

After that, when the process of this routine is newly started, the overtaking prohibition symbol 52 is already displayed on the display 33, so the CPU determines "Yes" in step 505, proceeds to step 510, and sets the condition ( It is determined whether or not 2a) is established. That is, the CPU determines whether or not the section end indicator has been detected by the indicator recognition process executed during the period from the last time execution of this routine was completed to the current time.

At this point in time (that is, when the vehicle 10 is in an overtaking prohibited section starting from the position of the point P1 and is traveling in the vicinity of the point P1), the section end sign is not detected, so the condition (2a) is not established. Therefore, the CPU determines "No" in step 510, proceeds to step 515, and determines whether condition (2b) is satisfied. That is, the CPU executes lane recognition processing to determine whether the roadway center line is a solid yellow line.

At this time, the roadway center line (that is, the lane marking L2) is a solid yellow line, so the condition (2b) is not satisfied. Therefore, the CPU determines "No" in step 515, proceeds to step 520, and determines whether condition (2c) is satisfied. That is, it is determined whether or not the post-detection travel distance Dr is greater than the distance threshold value Dth.

According to the above assumption, the post-detection travel distance Dr is the distance between the position of the point P1 and the current position of the vehicle 10 (that is, the position near the point P1). At this point, the post-detection travel distance Dr is smaller than the distance threshold Dth, so the condition (2c) is not satisfied. Therefore, the CPU determines “No” at step 520 and proceeds directly to step 595 .

(Case C)
After that, it is assumed that this routine is executed for the first time after the vehicle 10 reaches the position of point P4.
In this case, the sign S4 (that is, the section end sign) is detected by the sign recognition processing, so the condition (2a) is established. Therefore, the CPU makes a "Yes" determination in step 510 and proceeds to step 525 to terminate the display of the overtaking prohibition symbol 52 on the display 33 . The CPU then proceeds directly to step 595 .

Note that if the lane recognition process determines that the roadway center line (that is, the lane marking L3) is not a yellow solid line (that is, the condition (2b) is satisfied Then), the CPU determines “Yes” in step 515 and proceeds to step 525 .

(Case D)
During the period from when the vehicle 10 reaches the position of the point P4 to when it reaches the position of the point P5, either condition (2a) or condition (2b) may be changed for some reason (for example, the sign S4 and the lane marking L3 are defaced). It is also assumed that this routine is executed for the first time after the vehicle 10 reaches the position of point P5.
In this case, since the condition (2c) is satisfied, the CPU makes a “Yes” determination in step 520 and proceeds to step 525 .

As described above, the driving assistance ECU 20 displays the overtaking prohibition symbol 52 on the display 33 if the condition (1b) is satisfied even if the condition (1a) is not satisfied. Therefore, according to this support device, even if the overtaking prohibition sign included in the front image cannot be correctly recognized, the road on which the vehicle 10 is traveling is determined based on the roadway center line included in the front image. It is possible to provide the driver with information regarding whether or not the section is a prohibited section.

Although the embodiments of the driving assistance device according to the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications are possible without departing from the object of the present invention. For example, the overtaking prohibition symbol display start condition in the present embodiment is satisfied when at least one of the conditions (1a) and (1b) is satisfied. However, condition (1a) may be omitted. That is, the driving assistance ECU 20 may determine that the overtaking prohibition symbol display start condition is satisfied only when the condition (1b) is satisfied.

In addition, the driving assistance ECU 20 in this embodiment determines that the condition (1b) is satisfied when the lane recognition process determines that the road center line is a yellow solid line. However, the driving assistance ECU 20 may determine that the condition (1b) is established when the state in which the roadway center line is determined to be a yellow solid line by the lane recognition process continues for a predetermined time.

In addition, the driving assistance ECU 20 according to the present embodiment determines that the overtaking prohibition symbol display end condition is satisfied when at least one of the conditions (2a) to (2c) is satisfied. However, one or two of conditions (2a) to (2c) may be omitted. Alternatively, the driving assistance ECU 20 may determine that the condition (2b) is satisfied when the lane recognition process determines that the road center line is not a solid yellow line for a predetermined period of time.

In addition, the driving assistance ECU 20 in this embodiment displays the overtaking prohibition symbol 52 as a specific display on the display 33 when the condition for starting display of the overtaking prohibition symbol is satisfied. However, the specific indication may be different from the no overtaking symbol 52 . For example, the driving support ECU 20 may change the marking line symbol 54 included in the screen 41 of FIG. 2 from white to yellow when the overtaking prohibition symbol display start condition is satisfied. That is, in this case, the marking line symbol 54 colored yellow becomes the specific display.

Reference Signs List 10 Vehicle 10a Vehicle symbol 20 Driving support ECU 31 Forward camera 32 Vehicle speed sensor 33 Display 41 Screen 51 Sign symbol 52 Do not overtake symbol 53 Vehicle symbol 54 … Plot symbol.

Claims (1)

an imaging device that is mounted on a vehicle and acquires a front image by capturing an area in front of the vehicle;
a display device disposed at a position visible to the driver of the vehicle;
a lane marking determination unit that extracts the roadway centerline of the road on which the vehicle is traveling, included in the front image, and executes lane recognition processing for determining whether or not the roadway centerline is a yellow solid line; ,
a sign recognition processing unit that executes sign recognition processing for recognizing road signs included in the forward image;
The condition is that an overtaking prohibition sign is detected by the sign recognition processing of the sign recognition processing unit, and the condition is that the roadway center line is determined to be a solid yellow line by the lane recognition processing of the lane marking determination unit . an information providing unit that causes the display device to display an overtaking prohibition symbol, which is a predetermined specific display, when at least one of the above conditions is satisfied ;
with
The information providing unit
When the prohibition symbol for crossing is displayed on the display device,
a condition that the end sign of the overtaking prohibited section is detected by the sign recognition processing;
A condition that the lane recognition processing determines that the roadway center line is not a yellow solid line
,as well as,
a condition that a traveled distance after detection, which is a traveled distance of the vehicle from the point in time when the no passing sign was last detected by the sign recognition processing to the present time, is greater than a predetermined distance threshold;
When at least one of
configured as
Driving assistance device.

Images