JP7613358B2 - Display device - Google Patents

Description

The present invention relates to a display device.

Patent documents 1 to 3 disclose display devices that display an image that is visually recognized by a user (mainly the driver of the vehicle) as a virtual image that is superimposed on the scenery ahead of the vehicle. The display devices disclosed in patent documents 1 to 3 display an alarm image as a virtual image to notify the user of the presence of a forward object, etc., at a position corresponding to the position of the forward object in front of the vehicle (for example, a position superimposed on the forward object). Each of these display devices performs unique control to prevent the alarm image from shifting from the desired display position due to external factors such as vibrations applied to the vehicle or the behavior of the forward object, causing the user to feel uncomfortable.

JP 2015-80988 A JP 2017-13590 A JP 2017-171226 A

Factors that cause the notification image to shift from the desired display position include not only the external factors mentioned above, but also internal factors such as the load of drawing control and the sensing process of the object ahead. Therefore, there is room for improvement in preventing the user from feeling uncomfortable due to the shift in the display position of the notification image.

The present invention has been made in consideration of the above-mentioned situation, and aims to provide a display device that can suppress the sense of discomfort felt by a user viewing an alarm image.

In order to achieve the above object, the display device according to the present invention comprises:
A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle,
an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object;
a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of the superimposed image based on the object information acquired by the acquisition means; and
a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing,
the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information;
When the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes an image enlargement process to draw the notification image at a size larger than a predetermined standard size, and when the object ahead is a moving body moving away from the vehicle at a predetermined relative speed or greater, does not execute the image enlargement process on the notification image to notify information regarding the moving body.
In order to achieve the above object, a display device according to the present invention comprises:
A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle,
an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object;
a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of the superimposed image based on the object information acquired by the acquisition means; and
a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing,
the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information;
the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size;
The identification means identifies the drawing control load as a cause of the display delay based on at least one of the area, number, and dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image.

According to the present invention, it is possible to prevent the user viewing the notification image from feeling uncomfortable.

1 is a diagram showing an example of a manner in which a head-up display (HUD) device according to a first embodiment of the present invention is mounted on a vehicle. 1A and 1B are diagrams illustrating the relationship between a user's viewpoint position and a virtual image. FIG. 1 is a block diagram of a vehicle display system. 5A to 5C are diagrams illustrating an example of a display of a notification image according to the first embodiment. 11A and 11B are diagrams illustrating an example of occurrence of display deviation. 13A to 13C are diagrams illustrating an example of a display of a notification image when the image enlargement process according to the first embodiment is executed. 11A and 11B are diagrams illustrating an example of occurrence of display deviation. 13A to 13C are diagrams illustrating an example of a display of a notification image when the image enlargement process according to the first embodiment is executed. FIG. 13 is a diagram showing an example of a specific type of notification image. FIG. 13 is a diagram showing an example of a notification image of a non-specific type. 5A to 5C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. 5A to 5C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. 5A to 5C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. 5A to 5C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. 4 is a flowchart of a display control process according to the first embodiment. 13A and 13B are diagrams illustrating a state in which display deviation occurs in a notification image according to the second embodiment. 13A and 13B are diagrams illustrating an example of a display of a notification image when an image position adjustment process according to the second embodiment is executed. 11A and 11B are diagrams for explaining a specific moving state of a forward object; 11A and 11B are diagrams for explaining a specific moving state of a forward object; 13 is a diagram showing an example of the configuration of a movement amount determination table used in the display control process according to the second embodiment; FIG. 10 is a flowchart of a display control process according to a second embodiment. 13A and 13B are diagrams illustrating a state in which a display delay occurs in an alarm image according to the third embodiment. 13A and 13B are diagrams illustrating an example of a display of a notification image when a line width enlargement process according to the third embodiment is executed. 11A and 11B are diagrams illustrating an example of occurrence of display deviation. 13A and 13B are diagrams illustrating an example of a display of a notification image when a line width enlargement process according to the third embodiment is executed. 13 is a diagram showing an example of the configuration of an enlargement amount determination table used in the display control process according to the third embodiment. FIG. 13 is a flowchart of a display control process according to the third embodiment.

A display device according to one embodiment of the present invention will be described with reference to the drawings.

First Embodiment
The display device according to this embodiment is a HUD (Head-Up Display) device 10 included in a vehicle display system 100 shown in Fig. 2. As shown in Fig. 1A, the HUD device 10 is provided inside a dashboard 2 of a vehicle 1 (hereinafter also referred to as the vehicle 1), and notifies a user 4 (mainly the driver of the vehicle 1) of not only information related to the vehicle 1 (hereinafter referred to as vehicle information) but also information other than the vehicle information in an integrated manner. Note that the vehicle information includes not only information about the vehicle 1 itself but also information outside the vehicle 1 related to the operation of the vehicle 1.

The vehicle display system 100 is a system configured within the vehicle 1, and as shown in Figure 2, includes a HUD device 10, a forward object detection unit 40, a viewpoint detection unit 50, an ECU (Electronic Control Unit) 60, and a car navigation device 70.

As shown in Fig. 1A, the HUD device 10 emits display light Q toward the windshield 3 of the vehicle 1. The display light Q reflected by the windshield 3 travels toward the user 4. By placing the user 4's viewpoint within the eye box 5, the user 4 can visually recognize the image represented by the display light Q displayed in front of the windshield 3 as a virtual image A. This allows the user 4 to visually recognize the virtual image A superimposed on the scenery ahead.

As shown in Figure 1B, the HUD device 10 displays a virtual image A on a virtual plane set in front of the vehicle 1. The virtual plane is set at a position a predetermined distance P (e.g., about 5 to 10 m) ahead of the vehicle 1 from the viewpoint position 4a when the user 4 places his/her viewpoint within the eye box 5. The virtual plane corresponds to the display surface of the image on the display unit 20. The virtual plane and the eye box 5 are set based on the size of the display surface and the optical system formed by the various mirrors and windshield 3 within the HUD device 10.

The HUD device 10 comprises a display unit 20 and a control unit 30 shown in Figure 2, and a reflector unit not shown.

The display unit 20 displays a superimposed image that is visually recognized by the user 4 as a virtual image A under the control of the control device 30. The display unit 20 has, for example, a TFT (Thin Film Transistor) type LCD (Liquid Crystal Display) and a backlight that illuminates the LCD from behind. The backlight is configured to include, for example, an LED (Light Emitting Diode). Under the control of the control device 30, the display unit 20 generates display light Q by displaying an image on the LCD illuminated by the backlight. The generated display light Q is reflected by the reflecting unit and then emitted toward the windshield 3. The reflecting unit is composed of, for example, two mirrors, a folding mirror and a concave mirror. The folding mirror folds the display light Q emitted from the display unit 20 and directs it toward the concave mirror. The concave mirror reflects the display light Q from the folding mirror toward the windshield 3 while enlarging it. As a result, the virtual image A visually recognized by the user 4 is an enlarged image displayed on the display unit 20. The type and number of mirrors constituting the reflecting portion can be changed as desired depending on the design.

In the following, the display unit 20 displaying the image that is visually recognized by the user 4 as virtual image A is also referred to as "displaying a superimposed image." Furthermore, the control device 30 controlling the display of the display unit 20 is also referred to as "performing display control of a superimposed image." Furthermore, as long as the display unit 20 can display a superimposed image, it is not limited to one that uses an LCD, and may use a display device such as an OLED (Organic Light Emitting Diodes), a DMD (Digital Micro mirror Device), or an LCOS (Liquid Crystal On Silicon).

The control device 30 is composed of a microcomputer that controls the overall operation of the HUD device 10, and includes a control unit 31 and a memory unit 32. The control device 30 also includes components (not shown) such as a driver for driving the display unit 20 and an input/output circuit for communicating with various systems within the vehicle 1.

The storage unit 32 is composed of a ROM (Read Only Memory) that stores operation programs and various image data in advance, and a RAM (Random Access Memory) that temporarily stores various calculation results. The ROM of the storage unit 32 stores data of operation programs for executing the display control process described below, and data of various tables and formulas used when executing the display control process. The control unit 31 includes a CPU (Central Processing Unit) 31a that executes the operation programs stored in the ROM of the storage unit 32, and a GDC (Graphics Display Controller) 31b that cooperates with the CPU 31a to execute image processing. The GDC 31b is composed of, for example, a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The GDC 31b is capable of both drawing processing using vector images and drawing processing using raster images (bitmap images). The configuration of the control device 30 and the control unit 31 is arbitrary as long as they fulfill the functions described below.

The control unit 31 drives and controls the display unit 20. For example, the control unit 31 drives and controls the backlight of the display unit 20 using the CPU 31a, and drives and controls the LCD of the display unit 20 using the GDC 31b that operates in cooperation with the CPU 31a.

The CPU 31a of the control unit 31 cooperates with the GDC 31b to control the superimposed image based on various image data stored in the ROM of the storage unit 32. The GDC 31b determines the control content of the display operation of the display unit 20 based on a display control command from the CPU 31a. The GDC 31b reads image part data necessary for constituting one screen to be displayed on the display unit 20 from the ROM and transfers it to the RAM of the storage unit 32. The GDC 31b also uses the RAM to create picture data for one screen based on the image part data and various image data received by communication from the outside of the HUD device 10. Then, when the GDC 31b completes one screen of picture data in the RAM, it transfers it to the display unit 20 in accordance with the image update timing (frame rate). As a result, the superimposed image visually recognized by the user 4 as the virtual image A is displayed on the display unit 20. Also, a layer is assigned in advance to each image constituting the image visually recognized as the virtual image A, and the control unit 31 is capable of individual display control of each image.
By such control, the control unit 31 draws the content image within the display area of the virtual image A. Note that the rectangular frame indicated by the virtual image A in Fig. 1B, Fig. 3, etc. indicates the display area of the virtual image A, and the content image is an image that is visually recognized within the display area as part of the virtual image A. This is synonymous with the content image being displayed within the display area of the superimposed image in the display unit 20 that displays the superimposed image visually recognized by the user 4 as the virtual image A.

The content image is any image drawn in the display area of the virtual image A, and includes a notification image C for notifying information about a forward object F present in front of the vehicle 1. The notification image C is displayed at a corresponding position corresponding to the position information (information indicating the position of the forward object F) described later in the display area of the virtual image A under the control of the control unit 31. Here, the corresponding position as the display position of the notification image C in the display area of the virtual image A is a position where the notification image C is superimposed on at least a part of the forward object F or adjacent to the forward object F and is visually recognized by the user 4 in an ideal state in which the display deviation described later does not occur. The corresponding position can be appropriately set according to the type of the forward object F. FIG. 3A is an example in which a notification image C is displayed to notify the presence of a forward object F as a preceding vehicle. Note that the content image includes not only the notification image C but also an image showing vehicle information. For example, the image showing vehicle information is an image A1 showing the vehicle speed and an image A2 showing the speed limit of the driving lane, as shown in FIG. 3B. In addition, the forward object F that is the target of notification by the notification image C is not limited to the preceding vehicle, but may be a pedestrian, a building, part of the road (for example, a dividing line such as a solid or dashed white line, an intersection, or a fork in the road), etc., and can be set arbitrarily.

The control unit 31 also communicates with each of the forward object detection unit 40, the viewpoint detection unit 50, the ECU 60, and the car navigation device 70. For this communication, communication methods such as CAN (Controller Area Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport) (registered trademark), and LVDS (Low Voltage Differential Signaling) can be applied.

The forward object detection unit 40 detects a forward object F located in front of the vehicle 1, and supplies "object information" including at least the position information of the forward object F to the control unit 31 as information related to the detected forward object F. The position information included in the object information is, for example, data of coordinates (x, y, z) in which the traveling direction of the vehicle 1 is the x-axis, the left-right direction of the vehicle 1 is the y-axis, and the height direction of the vehicle 1 is the z-axis, as shown in FIG. 1B. The origin of the coordinates may be any position set, such as a representative point at the tip of the vehicle 1 or the center point of the vehicle 1.

The forward object detection unit 40 is configured to include, for example, one or a combination of various sensors described below, and a processing unit that processes information detected by the sensors.
As a sensor constituting the forward object detection unit 40, for example, LiDAR (Light Detection And Ranging) can be adopted. The forward object detection unit 40 removes unnecessary points from the point cloud data measured by the LiDAR, and then performs clustering processing to detect one or more clusters (point cloud sets). In addition, the forward object detection unit 40 uses a classifier to determine whether or not a cluster that is estimated to have measured a preset forward object F among the detected clusters is a forward object F. For example, Euclidean Clustering in PCL (Point Cloud Library) can be used as the clustering processing. In addition, a support vector machine (SVM) can be used as the classifier. In this way, the forward object detection unit 40 supplies object information including position information (including information indicating the size of the forward object F) and type information indicating the type of the forward object F (leading vehicle, pedestrian, building, etc.) to the control unit 31 for the forward object F detected in this way.
Furthermore, the forward object detection unit 40 may be configured to include an imaging device that captures an image of a forward scene including a road surface on which the vehicle 1 is traveling. The imaging device may be, for example, a stereo camera. The forward object detection unit 40 calculates position information and type information of the forward object F by performing image analysis on the data of the image captured by the imaging device using a known method such as a pattern matching method in a processing unit. The forward object detection unit 40 then supplies this information to the control unit 31 as object information. The forward object detection unit 40 may be configured to include a sonar, an ultrasonic sensor, a millimeter wave radar, or the like that detects the forward object F.

The control unit 31 acquires object information from the forward object detection unit 40 and identifies the position of the forward object F within the display area of the virtual image A based on the acquired object information. In addition, when there are multiple forward objects F detected by the forward object detection unit 40, the control unit 31 assigns identification information (ID) to each of the multiple forward objects F and identifies the position of a specific forward object F based on the position information corresponding to the ID. Note that if the position information (coordinates) of the forward object F is known, the direction of the forward object F relative to the vehicle 1 can be identified. In addition, if the position information is known, the distance from the vehicle 1 to the forward object F can be calculated. By differentiating the distance with respect to time, the relative speed of the forward object F relative to the vehicle 1 can be calculated. The direction of the forward object F and the calculation of the distance and relative speed may be identified by the forward object detection unit 40 or the control unit 31.

The viewpoint detection unit 50 is a known configuration for detecting the viewpoint position (including the line of sight) of the user 4, and includes, for example, an infrared camera that captures an image of the face of the user 4, and a viewpoint position analysis unit that analyzes the image data (imaging data) captured by the infrared camera. The viewpoint position analysis unit identifies the viewpoint position of the user 4 by performing image analysis of the imaging data using a known method such as a pattern matching method, and supplies information indicating the viewpoint position to the control unit 31.

The ECU 60 controls each part of the vehicle 1 and supplies vehicle information related to the vehicle 1 to the control unit 31. The control unit 31 may directly acquire vehicle speed information from various sensors such as a vehicle speed sensor. The vehicle information related to the vehicle 1 includes, for example, vehicle speed, engine RPM, and warning information (low fuel, abnormal engine oil pressure, etc.). The control unit 31 controls the display of the superimposed image based on the vehicle information acquired from the ECU 60, and is also capable of displaying an image showing specified vehicle information within the display area of the virtual image A. In other words, images showing vehicle information other than the alarm image C can also be displayed within the display area of the virtual image A.

The car navigation device 70 includes a GPS controller that calculates the position of the vehicle 1 based on a GPS (Global Positioning System) signal received from an artificial satellite or the like. The car navigation device 70 has a storage unit that stores map data, and based on the position information of the vehicle 1 from the GPS controller, reads map data in the vicinity of the current position from the storage unit and determines a guide route to the destination set by the user 4. The car navigation device 70 then outputs information on the position of the vehicle 1 and the determined guide route to the control unit 31. The car navigation device 70 also references the map data to output information indicating the name and type of the facility ahead of the vehicle 1, the distance between the facility and the vehicle 1, and the like to the control unit 31. In the map data, various information such as road shape information (lanes, road width, number of lanes, intersections, curves, branching roads, etc.), regulation information on road signs such as speed limits, and information on each lane when there are multiple lanes are associated with position data. The car navigation device 70 outputs these various pieces of information to the control unit 31 as navigation data. Here, when the forward object F that is the notification target of the notification image C is a building or an intersection, the navigation data includes object information regarding the forward object F, information for displaying a guide route as the notification image C, and information for the control unit 31 to specify the distance from the vehicle 1 to the forward object F. That is, the control unit 31 may obtain the object information from the car navigation device 70. The notification image C may be an image showing a POI (Point of Interest) such as a destination or a facility that is set in advance by the user 4 in the car navigation device 70. Note that the car navigation device 70 is not limited to being mounted on the vehicle 1, and may be realized by a mobile terminal (such as a smartphone or a tablet PC (Personal Computer)) that communicates with the control unit 31 by wire or wirelessly and has a car navigation function.

The control unit 31 calculates the display position of the notification image C within the display area of the virtual image A based on the position information of the forward object F identified based on the object information from the forward object detection unit 40 and information on the viewpoint position of the user 4 input from the viewpoint detection unit 50, and controls the display operation of the display unit 20. As a result, the control unit 31 displays the notification image C at a desired position (the aforementioned corresponding position) relative to the forward object F in the real scene.

In addition, the control unit 31 may not only display the content image including the notification image C in two dimensions (2D), but may also display it in 3D using three-dimensional (3D) image data pre-stored in the ROM of the storage unit 32. In addition, the control unit 31 may perform warping to pre-distort the image displayed on the display surface of the display unit 20 (the screen when a configuration is adopted in which the display light Q is projected onto a screen) in the opposite direction by the amount of distortion caused by the optical meter after the display surface, as a process for correcting the distortion of the virtual image A mainly caused by the curved shape of the windshield 3. Furthermore, the control unit 31 may perform dynamic warping, which is warping according to the viewpoint position of the user 4. Dynamic warping uses parameters set for each of multiple areas obtained by dividing the rectangular area of the eye box 5 into a grid, and realizes distortion correction optimized according to the viewpoint position.

Here, Fig. 3A shows an example of the display of a notification image C for notifying the driver of a forward object F. Fig. 3A shows an example in which a notification image C is displayed at a desired display position, and an elliptical notification image C that is visually recognized superimposed on a part of the forward object F is displayed on the road surface side of the forward object F, which is a preceding vehicle.

The notification image C displayed according to the position of the forward object F may be displayed with a delay from the desired display timing due to the drawing control load (hereinafter also referred to as the drawing load) and the like. When the display delay of the notification image C occurs, depending on the relative speed between the vehicle 1 and the forward object F, a "display shift" occurs in which the notification image C is shifted from the desired display position, which gives a sense of discomfort to the user. FIG. 3B shows an example in which a display shift occurs in the notification image C due to an increase in the number of content images displayed in the display area of the virtual image A and an increase in the drawing load. FIG. 3B shows an example in which a preceding vehicle F1 in the driving lane of the vehicle 1, a preceding vehicle F2 in the lane next to the driving lane, and buildings F3 and F4 located in front of the vehicle 1 are detected as the forward object F, and a display shift occurs in the notification image C (notification images C1 to C4) that notifies each of these four forward objects F.
In order to reduce the discomfort felt by the user due to such display misalignment, the control unit 31 according to the first embodiment executes image enlargement processing.

From here, we will explain the main functions of the control unit 31 that executes the image enlargement process. The control unit 31 mainly functions as an acquisition means, a drawing control means, and a specification means.

The acquisition means acquires the above-mentioned object information from the forward object detection unit 40. The drawing control means draws, in the display area of the superimposed image, an alarm image C that is visible to the user 4 at a corresponding position that corresponds to the position (position information) of the forward object F, based on the object information acquired by the acquisition means. The correspondence between the position information and the corresponding position (the display position of the alarm image C in the display area of the virtual image A) is stored in advance in the ROM of the storage unit 32.

In addition, the identification means identifies a display delay factor that causes the notification image C, which is drawn based on the object information acquired by the acquisition means at a predetermined timing, to be displayed with a delay from the predetermined timing. Specifically, the control unit 31 as the identification means identifies the display delay factor as follows.

(Identifying the cause of display delays)
In identifying the cause of the display delay, first, the control unit 31 refers to a predicted drawing load determination table TA1 shown in Fig. 7A and determines a predicted drawing load (predicted drawing load L) when the drawing prediction means (mainly GDC 31b) draws a content image (including notification image C) to be displayed in the display area of the virtual image A. The predicted drawing load determination table TA1 is referenced for one content image, for example, and is configured so that a combination of the content type (2D or 3D) and the drawing area S corresponds to the predicted drawing load L. The predicted drawing load determination table TA1 is stored in advance in the ROM of the storage unit 32.

The control unit 31 refers to the predicted drawing load determination table TA1 to determine the predicted drawing load L for one content image (hereinafter referred to as the predicted target image) among images for one screen created by the GDC 31b and transferred to the display unit 20 at a specified update timing (hereinafter referred to as the predicted target timing) from the next time onwards at the frame rate. For example, if the type of the predicted target image is "2D" and its drawing area S is "S1≦S<S2", the control unit 31 refers to the predicted drawing load determination table TA1 to determine the predicted drawing load L to be "10%".

When one screen's worth of picture data to be drawn at the prediction target timing contains multiple prediction target images, the control unit 31 determines a predicted drawing load L equal to the number of prediction target images. The control unit 31 then sums up the determined predicted drawing loads L to obtain a total predicted drawing load ΣL. For example, as shown in FIG. 3B, when the display area of virtual image A contains six content images, namely, four notification images C (C1 to C4), an image A1 representing the vehicle speed, and an image representing the speed limit of the driving lane, the control unit 31 determines a predicted drawing load L for each of the six content images and adds them up to obtain a total predicted drawing load ΣL.

That is, the control unit 31 as the identification means identifies the drawing control load as a cause of the display delay based on the area, number, and dimension of the content image (including the notification image C). Note that the control unit 31 may identify the drawing control load as a cause of the display delay based on at least one of the area, number, and dimension of the content image.

The control unit 31, which has calculated the total predicted drawing load ΣL, refers to a predicted delay time determination table TA2 shown in Fig. 7B to predict a delay time in displaying a content image caused by the drawing load (determines a predicted delay time T _L ). The predicted delay time determination table TA2 is configured so that the total predicted drawing load ΣL corresponds to the predicted delay time T _L , and is stored in advance in the ROM of the storage unit 32. The control unit 31 refers to the predicted delay time determination table TA2 to determine a predicted delay time T _L corresponding to the calculated total predicted drawing load ΣL.

The display delay of the content image is also caused by system configuration other than the drawing load. For example, if the timing when the forward object detection unit 40 senses a predetermined forward object F is t0, the notification image C is drawn based on the object information sensed at t0 at a timing t1 after t0. In this embodiment, a sensing delay time T _S , which is a delay time mainly caused by sensing, is considered as a display delay factor other than the drawing load. The sensing delay time T _S is mainly a transmission delay of object information from the forward object detection unit 40 to the control unit 31, and is stored in advance as data in the ROM of the storage unit 32.
The control unit 31 specifies the sum of the predicted delay time T _L caused by the drawing load determined as described above and the sensing delay time T _S as the display delay factor (T=T _L +T _S ).

The predicted delay time T _L may be set in advance taking into consideration not only the drawing load but also the display control and display operation of the display unit 20. The sensing delay time T _S may be determined taking into consideration the processing delay when the forward object detection unit 40 calculates object information. Furthermore, the processing load when the control unit 31 executes the above-mentioned warping or dynamic warping may be taken into consideration as a display delay factor caused by the system configuration. The processing load when performing dynamic warping can be predicted in consideration of the number of viewpoint positions set in advance, the setting of the number of grids when dividing the eye box 5 into multiple regions, and the like, or can be obtained by experiment.

(Image enlargement processing)
As described above, when the cause of the display delay is identified by the identification means, the control unit 31, as a drawing control means, executes an image enlargement process to draw the notification image C at a size larger than the reference size if the value indicated by the identified display delay cause is greater than or equal to a predetermined value.

The reference size is the size of the notification image C at a predetermined distance P (see Figure 1B) determined according to the type of notification image C. The reference size is not a fixed size within the display area of the virtual image A, but varies according to reference information (mathematical formula data or table data) that specifies the correspondence between the distance from the vehicle 1 to the forward object F and the size of the reference size. The reference information is stored in advance in the memory unit 32, and is set, for example, using the law of perspective, so that the reference size becomes smaller as the distance to the forward object F of the notification target becomes longer. The control unit 31 draws the notification image C at the reference size based on the distance from the vehicle 1 to the forward object F and the reference information.

When performing image enlargement processing, the control unit 31, which serves as a drawing control means, determines the magnification relative to the reference size as follows.

The control unit 31 refers to the first magnification determination table TA3 shown in Fig. 8A to determine a magnification (hereinafter referred to as the first magnification) corresponding to the display delay factor obtained as described above. The first magnification determination table TA3 is configured so that the display delay factor corresponds to the first magnification, and is stored in advance in the ROM of the storage unit 32.

Furthermore, when the control unit 31 has determined the first magnification and the relative speed between the vehicle 1 and the approaching forward object F is equal to or greater than a predetermined speed, the control unit 31 refers to the second magnification determination table TA4 shown in Fig. 8B to determine a magnification (hereinafter referred to as the second magnification) corresponding to the relative speed. The second magnification determination table TA4 is configured so that the relative speed corresponds to the second magnification, and is stored in advance in the ROM of the storage unit 32.

The control unit 31, which has determined the second magnification, multiplies the first magnification and the second magnification and determines the magnification in the image enlargement process as the product of the first magnification and the second magnification. For example, if the determined first magnification is "1.1" and the second magnification is "1.2", the control unit 31 determines 1.32 as the magnification in the image enlargement process, since 1.1 x 1.2 = 1.32. In this case, the control unit 31 draws an alarm image C that is 1.32 times the standard size according to the above-mentioned reference information. For example, the image enlargement process is performed while maintaining the aspect ratio of the alarm image C of the standard size. Note that the shape of the alarm image C is not limited to the ellipse shown in the figure and may be any shape, such as a rectangle, a circle, a triangle, or a polygon. In addition, the configuration of the alarm image C may be any shape as long as it can notify information about the forward object F, and may be, for example, a character, a symbol, a figure, an icon, or a combination of these.

Figure 4 shows an example of the display of notification image C when image enlargement processing is performed. When the value indicated by the display delay factor is equal to or greater than a predetermined value, image enlargement processing is performed to enlarge notification image C of a standard size at the magnification determined as described above, thereby making the deviation from the desired display position less noticeable and reducing the sense of discomfort felt by the user.

In particular, when the control unit 31, which functions as a drawing control means, executes an image enlargement process when the vehicle 1 and the forward object F become relatively closer in a specified direction, it draws the notification image C so that it is enlarged and viewed toward the vehicle 1, based on an end B of the notification image C that is viewed at a position far from the vehicle 1 in the specified direction.

For example, in the state shown in Figure 4, when the vehicle 1 and the forward object F approach each other relatively in the fore-and-aft direction, the image enlargement process is performed by drawing the notification image C so that it is enlarged and viewed toward the vehicle 1, based on the end B (the upper end in Figure 4) that is viewed at a position farther from the vehicle 1 in the fore-and-aft direction. In this way, even if a display shift occurs in which the notification image C is shifted above the desired display position due to a display delay, the display shift can be effectively made inconspicuous.

Also, FIG. 5A shows an example in which the forward object F is a road marking (a white line in the example of the figure), and the notification image C notifying the forward object F is displayed superimposed on the marking to emphasize the presence of the marking. In this case, the forward object F is an immovable object with respect to the ground, but as the vehicle 1 approaches the marking, a display shift of the notification image C may occur as shown in FIG. 5A. In this way, when the vehicle 1 and the forward object F relatively approach each other in the lateral direction, as shown in FIG. 5B, when the image enlargement process is performed, the notification image C is drawn so that it is enlarged and viewed toward the vehicle 1 side, based on the end B (the right end in FIG. 5B) that is viewed at a position far from the vehicle 1 in the lateral direction. In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be made less noticeable.

(Display control process)
From here on, the display control process executed by the control unit 31 will be described mainly with reference to Fig. 9. The display control process is executed continuously while the ignition of the vehicle 1 is turned on, for example.

When the display control process is started, the control unit 31 first determines whether or not a forward object F has been detected (step S101). When object information is supplied from the forward object detection unit 40, the control unit 31 determines that a forward object F has been detected (step S101; Yes) and proceeds to step S102. On the other hand, when a forward object F has not been detected (step S101; No), the control unit 31 waits in step S101.

Next, the control unit 31 calculates the relative speed between the vehicle 1 and the forward object F as described above based on the object information obtained from the forward object detection unit 40 (step S102).

Next, the control unit 31 determines whether the notification image C for notifying the user of the detected forward object F is a specific type of image (step S103).

Here, the specific type of image is an image that is acceptable for making the notification even if the notification image C for notifying information about the forward object F is a non-moving object such as a road and is viewed shifted in the traveling direction of the vehicle 1 from the desired display position (the corresponding position described above), and the type is stored in the ROM of the storage unit 32 in advance. For example, as shown in FIG. 6A, when the forward object F is the road surface in front of the vehicle 1 and the notification image C is simply a guide display indicating going straight, even if a display delay occurs in the notification image C, the user 4 is unlikely to recognize that it is displayed shifted from the forward road surface. Therefore, in this way, the notification image C, such as a guide display indicating going straight, is stored in advance as a specific type, and when the notification image C is an image of a specific type, the image enlargement process described later is not executed. On the other hand, as shown in FIG. 6B, when the forward object F is an intersection and the notification image C is a guide display indicating a left turn, the notification image C is not set to a specific type. 6B is viewed out of the direction of travel of the vehicle 1, the guidance will be affected. In addition, when the notification image C provides guidance in a direction other than going straight ahead, the notification image C is not set to a specific type.

In step S103, if the notification image C is a specific type of image as shown in Fig. 6B (step S103; Yes), the control unit 31 executes normal image control to display a notification image C of a standard size according to the above-mentioned standard information at a corresponding position corresponding to the position information included in the object information (step S104). Note that, if multiple forward objects F are detected in step S101, the control unit 31 displays a notification image C corresponding to each of the multiple forward objects F.

In step S103, if the notification image C is not a specific type of image (step S103; No), the control unit 31 determines whether the forward object F is moving away at a predetermined relative speed or more based on the relative speed calculated in step S102 (step S103A). Specifically, if the forward object F identified by the object information acquired in step S101 is a moving object such as a preceding vehicle, and the moving object is moving away from the vehicle 1 at a predetermined relative speed or more previously set in the ROM of the storage unit 32 (step S103A; Yes), the control unit 31 executes normal image control (step S104). This is because the moving object is considered to be of low importance to the user 4 at the present time. Note that a moving object is an object that moves relative to the ground, such as a preceding vehicle or a pedestrian.

In step S103A, if the forward object F is not moving away at a relative speed equal to or greater than a predetermined speed (step S103A; No), the control unit 31 identifies the cause of the display delay (step S105). As described above, the control unit 31 determines the predicted delay time T _L based on the type (2D or 3D) of the notification image C to be displayed, the predicted drawing load determination table TA1, and the predicted delay time determination table TA2. The control unit 31 then determines the sum of the determined predicted delay time T _L and the sensing delay time T _S as the display delay cause (T=T _L +T _S ).

Next, the control unit 31 determines whether the value T indicating the display delay factor identified in step S105 is equal to or greater than a predetermined value T1 previously defined in the ROM of the storage unit 32 (step S106). If the value T indicating the display delay factor is less than the predetermined value T1 (step S106; No), the control unit 31 executes normal image control (step S104).

On the other hand, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S106; Yes), the control unit 31 determines the magnification for the reference size (step S107). Here, as described above, the control unit 31 refers to the first magnification determination table TA3 to determine the magnification (first magnification) corresponding to the display delay factor identified in step S105. Also, if the relative speed V between the vehicle 1 and the forward object F approaching relatively is equal to or greater than a predetermined speed (V1 in the example of FIG. 8B), the control unit 31 refers to the second magnification determination table TA4 to determine the magnification (second magnification) corresponding to the relative speed V. Then, the control unit 31 determines the product of multiplying the first magnification and the second magnification as the magnification in the image enlargement process. Note that, if the relative speed V is less than the predetermined speed V1, the first magnification is determined as it is as the magnification in the image enlargement process.

Next, the control unit 31 executes an image enlargement process to draw the notification image C with the reference size enlarged at the magnification determined in step S107 (step S108). As described above, when the vehicle 1 and the forward object F are relatively approaching each other in a predetermined direction, the notification image C is drawn so as to be enlarged and viewed toward the vehicle 1 side, based on the end B of the notification image C that is viewed at a position far from the vehicle 1 in the predetermined direction, when the image enlargement process is executed. Note that, when the vehicle 1 and the forward object F are not relatively approaching each other in the predetermined direction, the control unit 31 may enlarge the notification image C based on an arbitrary position. In addition, when multiple forward objects F are detected in step S101, the control unit 31 executes the image enlargement process for the image of the notification image C with the magnification determined in step S107 among the multiple forward objects F that are displayed corresponding to each of the multiple forward objects F.

After executing step S108 or step S104, the control unit 31 executes the process again from step S101. The above display control process is continuously executed until, for example, the HUD device 10 is turned off. This concludes the description of the first embodiment.

From here on, other embodiments will be described in which some of the processes in the display control process are different from the first embodiment. In the following embodiments, the configuration of the vehicle display system 100 is similar to that of the first embodiment. Therefore, below, the same components as in the first embodiment will be described using the same reference numerals as in the first embodiment, and the differences from the first embodiment will be mainly described.

Second Embodiment
In the second embodiment as well, the control unit 31 displays an alarm image C for notifying a user of a forward object F, for example, as shown in Fig. 3A. The control unit 31 according to the second embodiment executes an image position adjustment process, which will be described later, in order to reduce the sense of discomfort felt by the user due to the occurrence of display misalignment as shown in Fig. 3B and Fig. 10A. The control unit 31, which executes a display control process including the image position adjustment process, mainly functions as an acquisition means, a drawing control means, and an identification means. The identification means according to the second embodiment identifies the cause of the display delay in the same manner as in the first embodiment.

(Image position adjustment process)
When the identification means identifies the cause of the display delay, if the value indicated by the identified cause of the display delay is equal to or greater than a predetermined value, the control unit 31 as the drawing control means executes an image position adjustment process to draw an alert image C that is visually recognized at an adjusted position obtained by moving the desired display position (the corresponding position described above) by a predetermined movement amount in the direction in which the forward object F moves relative to the vehicle 1. When executing the image position adjustment process, the control unit 31 as the drawing control means determines the movement amount of the alert image C as follows.

When the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 sets the movement amount to a predetermined value M (hereinafter, M is also referred to as a reference movement amount). Furthermore, the control unit 31 determines the relative speed V between the vehicle 1 and the forward object F, and determines the movement amount corresponding to the determined relative speed V by referring to the movement amount determination table TA5 shown in FIG. 12. The movement amount determination table TA5 is configured so that the relative speed and the movement amount correspond to each other, and is stored in the ROM of the storage unit 32 in advance. For example, when the magnitude (absolute value) of the calculated relative speed V is 0≦V<V1, the control unit 31 sets the movement amount of the notification image C as M as it is. Also, when the magnitude of the calculated relative speed V is V1≦V<V2, the control unit 31 sets the movement amount of the notification image C to α×M (α>1). In other words, the control unit 31 increases the movement amount when the relative speed V is a second value greater than the first value, compared to when the relative speed V is a first value.

The control unit 31 may variably control the reference movement amount M itself so that the movement amount of the notification image C increases as the value indicated by the display delay factor increases. The control unit 31 may also determine a coefficient (α or β in the example of FIG. 12) according to the determined relative speed V, and multiply the determined coefficient by the reference movement amount M to determine the movement amount of the notification image C according to the relative speed V. The relationship between the relative speed V and the movement amount of the notification image C may be stored in advance in the ROM of the storage unit 32 as data of a function (e.g., a linear function or a quadratic function), and the control unit 31 may determine the movement amount of the notification image C based on the determined relative speed V and the function.

Figure 10B shows an example of the display of the notification image C when the image position adjustment process is executed. For example, when the display delay factor increases and the forward object F approaches the vehicle 1, a display shift as shown in Figure 10A occurs. In order to suppress the discomfort felt by the user 4 due to this display shift, the control unit 31 according to the second embodiment executes an image position adjustment process in which, when the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 draws the notification image C that is visible at an adjusted position that is moved a predetermined amount from the desired display position (the corresponding position described above) in the direction in which the forward object F moves relative to the vehicle 1 (downward in the example of Figure 10).

In particular, the control unit 31, which functions as a drawing control means, corrects the amount of movement when the forward object F is in a specific moving state in which it is approaching the vehicle 1 from a position more than a specific distance away from the vehicle 1 in an intersecting direction (e.g., y direction) that intersects with the vehicle 1's traveling direction (e.g., x direction) so that the amount of movement when performing the image position adjustment process is larger than when the forward object F is not in the specific moving state.
Here, the specific movement state of the forward object F will be described with reference to Figs. 11A and 11B. Fig. 11A is an example in which the forward object F approaches the vehicle 1 in a straight line with a relative velocity vector of magnitude v. On the other hand, Fig. 11B is an example in which the forward object F approaches the vehicle 1 in an oblique manner with a relative velocity vector of magnitude v. For the user 4 (mainly the driver), it is more difficult to grasp the sense of distance in the case shown in Fig. 11B than in the case shown in Fig. 11A, even if the magnitude of the relative velocity in both cases is the same at v. Taking this into consideration, for example, the control unit 31 determines that the forward object F is in a specific movement state based on the position information (x, y, z) included in the acquired object information when the forward object F is approaching the vehicle 1 and the y coordinate of the forward object F is at a position away from the vehicle 1 by a specific distance or more. The specific distance is stored in advance in the ROM of the storage unit 32 as a distance in the y direction.
If the movement amount determined as described above is Mf, when the control unit 31 determines that the forward object F is in a specific movement state, it adds a correction amount ΔM to Mf and sets the movement amount of the notification image C to (Mf+ΔM). In this way, the notification image C can be displayed closer to the vehicle 1, so that the presence of the forward object F in a specific movement state, which is assumed to be difficult for the user 4 to grasp, can be properly notified.

As shown in FIG. 5A, the image position adjustment process according to the second embodiment can be executed even when the forward object F is a road dividing line and the notification image C emphasizes the dividing line to notify the driver. As the vehicle 1 approaches the dividing line, a display shift of the notification image C may occur as shown in FIG. 5A. However, by executing the image position adjustment process according to the second embodiment and drawing the notification image C to be viewed at an adjusted position moved by a determined amount in the direction in which the forward object F moves relative to the vehicle 1 (leftward in FIG. 5A), the display shift can be made less noticeable. In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be made less noticeable.

(Display control process)
From here, the display control process according to the second embodiment will be described with reference to Fig. 13. The control unit 31 executes the processes of steps S201 to S206 in the same manner as steps S101 to S106 described in the first embodiment.

In step S206, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S206; Yes), the control unit 31 determines the amount of movement from the corresponding position of the notification image C to the adjustment position (step S207). Here, as described above, the control unit 31 refers to the movement amount determination table TA5 to determine the amount of movement corresponding to the relative speed V calculated in step S202.

Next, the control unit 31 determines whether or not the vehicle 1 is in a specific moving state in which the forward object F is approaching the vehicle 1 from an oblique angle, as described above (step S207A).

If the forward object F is not in a specific movement state (step S207A; No), the control unit 31 executes an image position adjustment process to draw the notification image C at an adjustment position moved from the corresponding position by the movement amount determined in step S207 (step S208). Note that if step S207A is No and the process transitions to step S208, the control unit 31 may execute the image position adjustment process even if the forward object F is far away from the vehicle 1.

If the forward object F is in a specific movement state (step S207A; Yes), the control unit 31, assuming that the movement amount determined in step S207 is Mf, corrects the movement amount of the alarm image C to (Mf + ΔM) by adding a correction amount ΔM to Mf (step S207B), and performs image position adjustment processing with the corrected movement amount (step S208).

After executing step S208 or step S204, the control unit 31 executes the process again from step S201. This completes the description of the second embodiment.

Third Embodiment
In the third embodiment, the control unit 31 displays an alert image C composed of a linear image (an image that is visually recognized as a line) as an image for alerting a forward object F, for example, as shown in FIG. 14A.

The notification image C is drawn, for example, by a vector image, and as shown in Fig. 14A, includes a specific linear image Cs that is viewed along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other (the up-down direction in Fig. 14A), and a predetermined linear image Cn that is viewed as extending from each of both ends of the specific linear image Cs toward the front of the vehicle 1. Note that the predetermined linear image Cn in this embodiment is an image that is not subject to the line width expansion process described below.

The notification image C shown in Figure 14A is an example of a state in which the tips of the two predetermined linear images Cn are tilted toward each other, creating a sense of perspective and allowing the image to be viewed along the road surface ahead. Note that Figure 14A shows an example in which the notification image C is displayed shifted from the desired display position due to the above-mentioned display shift.

The control unit 31 according to the third embodiment executes a line width enlargement process described below in order to reduce the discomfort felt by the user due to the occurrence of a display misalignment as shown in Fig. 14A. The control unit 31, which executes a display control process including the line width enlargement process, mainly functions as an acquisition means, a drawing control means, and an identification means. The identification means according to the third embodiment identifies the cause of the display delay in the same manner as in the first and second embodiments.

(Regarding line width expansion processing)
When the display delay factor is identified by the identification means, the control unit 31 as the drawing control means executes a line width enlargement process for drawing the notification image C so that the line width of the specific linear image Cs is enlarged by a predetermined enlargement amount in the direction in which the forward object F moves relative to the vehicle 1 and is visually recognized when the value indicated by the identified display delay factor is equal to or greater than a predetermined value. The line width enlargement process is executed by enlarging a predetermined reference line width by a determined enlargement amount. The reference line width is the width of the linear image (including the specific linear image Cs) at a predetermined distance P (see FIG. 1B ) determined according to the type of the notification image C. The reference line width does not need to be fixed within the display area of the virtual image A, and may be variable according to reference line width information (formula data or table data) that specifies the correspondence between the distance from the vehicle 1 to the forward object F and the reference line width. This reference line width information is stored in advance in the storage unit 32, and is set so that the reference line width becomes narrower as the distance to the forward object F to be notified becomes longer, for example, by using the law of perspective. The control unit 31 draws the notification image C, which is composed of a linear image, with the reference line width based on the distance from the vehicle 1 to the forward object F and the reference line width information.

When the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 sets the enlargement amount to a predetermined value E (hereinafter, E is also referred to as a reference movement amount). Furthermore, the control unit 31 obtains the relative speed V between the vehicle 1 and the forward object F, and determines the enlargement amount corresponding to the obtained relative speed V by referring to the enlargement amount determination table TA6 shown in FIG. 16. The enlargement amount determination table TA6 is configured so that the relative speed and the enlargement amount correspond to each other, and is stored in the ROM of the storage unit 32 in advance. For example, when the magnitude (absolute value) of the calculated relative speed V is 0≦V<V1, the control unit 31 sets the enlargement amount of the specific linear image Cs as E as it is. Also, when the magnitude of the calculated relative speed V is V1≦V<V2, the control unit 31 sets the enlargement amount of the specific linear image Cs to α×E (α>1). In other words, the control unit 31 increases the enlargement amount when the relative speed V is a second value greater than the first value, compared to when the relative speed V is a first value.

The control unit 31 may variably control the reference enlargement amount E itself so that the enlargement amount of the specific linear image Cs increases as the value indicated by the display delay factor increases. The control unit 31 may also determine a coefficient (α or β in the example of FIG. 16) according to the determined relative speed V, and multiply the determined coefficient by the reference enlargement amount E to determine the enlargement amount of the specific linear image Cs according to the relative speed V. The relationship between the relative speed V and the enlargement amount of the specific linear image Cs may be stored in advance in the ROM of the storage unit 32 as data of a function (e.g., a linear function or a quadratic function), and the control unit 31 may determine the enlargement amount of the specific linear image Cs based on the determined relative speed V and the function.

Figure 14B shows an example of the display of the notification image C when the line width enlargement process is executed. For example, when the display delay factor increases and the forward object F approaches the vehicle 1, a display shift as shown in Figure 14A occurs. In order to suppress the discomfort felt by the user 4 due to this display shift, the control unit 31 according to the third embodiment executes a line width enlargement process to draw the notification image C so that the line width of the specific linear image Cs is visually enlarged by a determined enlargement amount in the direction in which the forward object F moves relative to the vehicle 1 (downward in the example of Figure 14) when the value indicated by the display delay factor is equal to or greater than a predetermined value.

In particular, the control unit 31, which functions as a drawing control means, corrects the enlargement amount when executing the line width enlargement process so that when the forward object F is in a specific moving state in which it is approaching the vehicle 1 from a position more than a specific distance away from the vehicle 1 in an intersecting direction (e.g., y direction) that intersects with the traveling direction (e.g., x direction) of the vehicle 1, the enlargement amount is larger than when the forward object F is not in the specific moving state, as described in the second embodiment.
If the enlargement amount determined as described above is Ef, when the control unit 31 determines that the forward object F is in a specific moving state, it adds a correction amount ΔE to Ef and sets the enlargement amount of the specific linear image Cs to (Ef+ΔE). In this way, the specific linear image Cs in the notification image C can be enlarged further toward the vehicle 1, so that the presence of the forward object F in a specific moving state in which it is assumed that it is difficult for the user 4 to grasp the sense of distance can be properly notified.

15A shows an example in which the forward object F is a road division line (a white line in the example of the figure), and an alert image C for alerting the forward object F is displayed superimposed on the division line to emphasize the presence of the division line. In the example of FIG. 15A, the alert image C is configured in a substantially rectangular enclosing shape, and of two linear images visually recognized along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other (the direction in which the division line, which is the forward object F in FIG. 15A, extends), the one closer to the vehicle 1 is set as a specific linear image Cs. Among the alert images C configured as linear images, the ones other than the specific linear image Cs are set as predetermined linear images Cn.
As the vehicle 1 approaches the lane marking, a display shift of the notification image C may occur as shown in Fig. 15A, but the display shift can be made less noticeable by executing a line width enlargement process for drawing the notification image C so that the line width of the specific linear image Cs is visually enlarged by a predetermined amount in the direction in which the forward object F moves relative to the vehicle 1 (leftward in Fig. 15B). In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be made less noticeable.

(Display control process)
From here on, the display control process according to the third embodiment will be described with reference to Fig. 17. The control unit 31 executes the processes of steps S301 to S306 in the same manner as steps S101 to S106 described in the first embodiment.

In step S306, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S306; Yes), the control unit 31 determines whether the forward object F is approaching the vehicle 1 (step S306A). If the forward object F is not approaching the vehicle 1 (step S306A; No), the control unit 31 does not execute the line width enlargement process and draws the notification image C by normal image control (step S304). As a result, for the notification image C in which the forward object F is not approaching the vehicle 1 and is considered to be of low importance to the user 4, the control load can be reduced without executing the line width enlargement process.
If the forward object F is approaching the host vehicle 1 (step S306A; Yes), the control unit 31 refers to the enlargement amount determination table TA6 and determines the enlargement amount corresponding to the relative speed V obtained in step S302 (step S307).

Next, the control unit 31 determines whether the vehicle is in a specific moving state in which the forward object F is approaching the vehicle 1 from an oblique angle (step S307A), similar to step S207A in the second embodiment.

If the forward object F is not in a specific moving state (step S307A; No), the control unit 31 executes a line width enlargement process to enlarge the line width of the specific linear image Cs in the alarm image C by the enlargement amount determined in step S307 (step S308).

If the forward object F is in a specific moving state (step S307A; Yes), the control unit 31 corrects the enlargement amount of the notification image C to (Ef + ΔE) by adding a correction amount ΔE to the enlargement amount determined in step S307, when the enlargement amount is Ef (step S307B), and performs line width enlargement processing with the corrected enlargement amount (step S308).

After executing step S308 or step S304, the control unit 31 executes the process again from step S301. This concludes the explanation of the third embodiment.

Note that the present invention is not limited to the above-described embodiments and drawings. Modifications (including the deletion of components) may be made as appropriate within the scope that does not change the gist of the present invention.

In the above, an example has been shown in which the value indicated by the display delay factor is the delay time T obtained by adding up the predicted delay time T _L caused by the drawing load and the sensing delay time T _S , but this is not limited thereto. The display delay factor may be specified based on at least one of the drawing control load by the drawing control means and the transmission delay of the object information. For example, the value indicated by the display delay factor may be the predicted drawing load L shown in FIG. 7A. The control unit 31 may determine the magnification according to the first embodiment, the movement amount according to the second embodiment, and the enlargement amount according to the third embodiment according to the predicted drawing load L. In addition, in the above, an example has been shown in which the value indicated by the display delay factor is determined using table data, but the control unit 31 may calculate the value indicated by the display delay factor using data of a formula. The data of the formula may be stored in advance in the ROM of the storage unit 32, and may be configured to be capable of calculating the value indicated by the display delay factor from various parameters such as the type, area, and number of content images, for example.

The control unit 31 may also execute a combination of the image enlargement process according to the first embodiment, the image position adjustment process according to the second embodiment, and the line width enlargement process according to the third embodiment. For example, the control unit 31 may determine the magnification when executing the image enlargement process and the movement amount when executing the image position adjustment process according to the identified display delay factor, execute the image enlargement process to enlarge the notification image C from the reference size, and execute the image position adjustment process to move the display position of the notification image C by the determined movement amount. In addition, when both the notification image C consisting of a linear image and the notification image C that is not a linear image (for example, a filled-in image, etc.) are displayed within the display area of the virtual image A, the control unit 31 may be able to execute at least one of the image enlargement process and the image position adjustment process, and the line width enlargement process.

The notification image C described in the third embodiment may have any shape or form as long as it is composed of a linear image including a specific linear image Cs. For example, the notification image C may be a ring-shaped linear image. In this case, any part of the ring-shaped linear image that is visible along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other can be set as the specific linear image Cs, and the other part can be set as the specified linear image Cn.

The specific linear image Cs in the notification image C described in the third embodiment may be a horizontal linear image extending in the left-right direction as viewed from the user 4. In addition, in the line width enlargement process, at least a part of the specific linear image Cs whose width has been enlarged relative to the reference line width may be blurred. For example, the part of the specific linear image Cs whose width has been enlarged from the reference line width may be blurred, or the entire enlarged specific linear image Cs may be blurred. In this way, for example, the strange appearance caused by the enlargement of the specific linear image Cs can be reduced, and the superimposition on the forward object F can be improved. In addition, in the line width enlargement process, the line width of the specific linear image Cn (for example, a vertical line image extending in the approximately up-down direction as viewed from the user 4) may be enlarged in addition to the enlargement of the line width of the specific linear image Cs. The process of enlarging the line width of the specific linear image Cn in this way together with the enlargement of the line width of the specific linear image Cs is useful, for example, when the forward object F moves diagonally relative to the vehicle 1.

Although an example of a configuration in which the forward object detection unit 40 senses the forward object F from the host vehicle 1 has been described above, the present invention is not limited to this.
The forward object detection unit 40 may be configured to receive object information related to a forward object F from outside the vehicle 1 and supply the received object information to the control unit 31. The forward object detection unit 40 configured as such may be configured from various modules that enable communication between the vehicle 1 and a wireless network (V2N: Vehicle To Cellular Network), communication between the vehicle 1 and other vehicles (V2V: Vehicle To Vehicle), communication between the vehicle 1 and pedestrians (V2P: Vehicle To Pedestrian), and communication between the vehicle 1 and roadside infrastructure (V2I: Vehicle To Roadside Infrastructure). For example, (i) the forward object detection unit 40 may include a communication module that can directly access a WAN (Wide Area Network), a communication module for communicating with an external device (such as a mobile router) that can access the WAN, an access point of a public wireless LAN (Local Area Network), or the like, and may perform Internet communication, or may include a GPS (Global Positioning System) controller that calculates the position of the vehicle 1 based on a GPS signal received from an artificial satellite or the like. These configurations enable communication by V2N. (ii) The forward object detection unit 40 may also include a wireless communication module that complies with a predetermined wireless communication standard and perform communication by V2V or V2P. (iii) The forward object detection unit 40 may also have a communication device that wirelessly communicates with roadside infrastructure, and may acquire forward information of the vehicle 1 from a base station of a driving safety support system (DSSS) via a roadside wireless device installed as infrastructure. This enables communication by V2I.

In other words, the forward object detection unit 40 may be configured to enable communication by V2X (Vehicle To Everything) between the vehicle 1 and the outside of the vehicle 1. The forward object detection unit 40 may be configured to be composed of a combination of multiple devices from among the above-mentioned imaging devices, sensor groups, and various modules that enable communication by V2X, and to supply object information from each of the multiple devices to the control unit 31.

By tilting the display surface of the display unit 20 (the display surface of an LCD, or a screen when a DMD or LCOS is used), the virtual surface on which the virtual image A is displayed may be tilted forward relative to the vertical direction of the vehicle 1.

The projection target of the display light Q is not limited to the windshield 3, but may be a combiner composed of a plate-shaped half mirror, a hologram element, etc. Furthermore, the display device that executes the display control process described above is not limited to the HUD device 10. The display device may be configured as a head mounted display (HMD) worn on the head of a user 4 of the vehicle 1. Then, in an image displayed as a virtual image by the HMD, the display control of the notification image C may be executed in a manner similar to that described above. In other words, the display device is not limited to one mounted on the vehicle 1, but may be one that is used in the vehicle 1.

The HUD device 10, which is an example of the display device described above, displays a superimposed image that is visually recognized by the user 4 as a virtual image A that overlaps with the scenery ahead of the vehicle 1. The HUD device 10 includes a control unit 31 that functions as an acquisition means, a drawing control means, and a specification means. The acquisition means acquires object information that is information about a forward object F that exists ahead of the vehicle 1, and includes at least position information of the forward object F. The drawing control means draws an alarm image C for notifying information about the forward object F at a corresponding position that corresponds to the position information in the display area of the superimposed image based on the object information acquired by the acquisition means. The specification means specifies a display delay factor that causes the alarm image C, which is drawn based on the object information acquired at a specified timing, to be displayed with a delay from the specified timing. The display delay factor is specified based on at least one of the drawing control load by the drawing control means and the transmission delay of the object information.

(1-1) In the HUD device 10 according to the first embodiment, when the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes an image enlargement process to draw the notification image C at a size larger than a predetermined reference size.
According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

(1-2) Specifically, when performing image enlargement processing, the drawing control means draws the notification image C at a magnification relative to a reference size determined based on the display delay factor.

(1-3) In addition, when the drawing control means executes an image enlargement process when the vehicle 1 and the forward object F become relatively closer in a specified direction, the drawing control means draws the notification image C so that it is enlarged and viewed toward the vehicle 1, based on an end B of the notification image C that is viewed at a position far from the vehicle 1 in the specified direction.
According to this configuration, even if a display delay causes a display deviation in which the notification image C is shifted from a desired display position, the display deviation can be effectively made inconspicuous.

(1-4) In addition, when the vehicle 1 and the object ahead F are approaching each other relatively and the relative speed V between the vehicle 1 and the object ahead F is equal to or greater than a predetermined speed, the drawing control means draws the notification image C at a magnification relative to a reference size determined based on the display delay factor and the relative speed V when executing the image enlargement process.
According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

(1-5) In addition, when the forward object F is an immovable object relative to the ground, if the notification image C for notifying information about the immovable object is a predetermined specific type of image (an image that is acceptable for making the notification even if it is viewed shifted in the direction of travel of the vehicle 1 from the desired display position), the drawing control means does not perform image enlargement processing on the specific type of image.
According to this configuration, it is not necessary to execute image enlargement processing for the specific type of notification image C, and therefore the control load can be reduced.

(1-6) In addition, when the forward object F is a moving object moving away from the vehicle 1 at a relative speed greater than or equal to a predetermined speed, the drawing control means does not execute image enlargement processing on the notification image C for notifying information about the moving object.
According to this configuration, it is not necessary to perform image enlargement processing on the notification image C that is considered to be of low importance to the user 4, and therefore the control burden can be reduced.

(1-7) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimensions of a content image, which is an image drawn by the drawing control means within the display area of the superimposed image and includes an alarm image C.

(2-1) In the HUD device 10 according to the second embodiment, when the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes an image position adjustment process to draw an alarm image C that is visible at an adjusted position where the corresponding position (desired display position) is moved a predetermined amount relative to the vehicle 1 in the direction in which the forward object F is moving.
According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

(2-2) Specifically, when performing image position adjustment processing, the drawing control means determines the amount of movement based on the display delay factor.

(2-3) In addition, the drawing control means controls the amount of movement when performing the image position adjustment process in accordance with the relative speed V between the vehicle 1 and the forward object F, and increases the amount of movement when the relative speed V is a second value greater than the first value compared to when the relative speed V is a first value.
According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

(2-4) In addition, when the forward object F is in a specific movement state in which it is approaching vehicle 1 from a position that is a specific distance or more away from vehicle 1 in an intersecting direction that intersects the direction of travel of vehicle 1, the drawing control means corrects the amount of movement when performing the image position adjustment process so that it is greater than the amount of movement when the forward object F is not in the specific movement state.
According to this configuration, the notification image C can be displayed closer to the vehicle 1, thereby effectively notifying the user 4 of the presence of a forward object F in a specific moving state, which is assumed to be difficult for the user 4 to grasp the sense of distance.

(2-5) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimensions of a content image, including an alarm image C, which is an image drawn by the drawing control means within the display area of the superimposed image.

(3-1) In the HUD device 10 according to the third embodiment, the notification image C is configured from linear images and includes a specific linear image Cs that is visually recognized along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other. When the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes a line width enlargement process to draw the notification image C such that the line width of the specific linear image Cs is visually enlarged by a predetermined enlargement amount in the direction in which the forward object F moves relative to the vehicle 1.
According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

(3-2) Specifically, when performing line width enlargement processing, the drawing control means determines the amount of enlargement based on the display delay factor.

(3-3) In addition, the drawing control means controls the amount of enlargement when executing the line width enlargement process in accordance with the relative speed V between the vehicle 1 and the forward object F, and increases the amount of enlargement when the relative speed V is a second value greater than the first value compared to when the relative speed V is a first value.
According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

(3-4) In addition, when the forward object F is in a specific movement state, the drawing control means corrects the enlargement amount when executing the line width enlargement process so that the enlargement amount is larger than when the forward object F is not in a specific movement state.
According to this configuration, the notification image C can be displayed closer to the vehicle 1, thereby effectively notifying the user 4 of the presence of a forward object F in a specific moving state, which is assumed to be difficult for the user 4 to grasp the sense of distance.

(3-5) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimensions of a content image, including an alarm image C, which is an image drawn by the drawing control means within the display area of the superimposed image.

In the above explanation, explanations of well-known technical matters have been omitted as appropriate in order to facilitate understanding of the present invention.

100...vehicle display system 10...HUD device 20...display unit 30...control device, 31...control unit, 32...storage unit 40...forward object detection unit, 50...viewpoint detection unit, 60...ECU, 70...car navigation device Q...display light, A...virtual image, C...alert image, F...forward object Cs...specific linear image, Cn...predetermined linear image TA1...predicted drawing load determination table TA2...predicted delay time determination table TA3...first magnification determination table TA4...second magnification determination table TA5...movement amount determination table TA6...magnification amount determination table 1...vehicle, 2...dashboard, 3...windshield, 4...user, 5...eye box

Claims (7)

A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle,
an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object;
a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of the superimposed image based on the object information acquired by the acquisition means; and
a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing,
the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information;
the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size, and , when the forward object is a moving object moving away from the vehicle at a relative speed equal to or greater than a predetermined relative speed, does not execute the image enlargement process for the notification image for notifying information about the moving object.
Display device. A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle,
an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object;
a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of the superimposed image based on the object information acquired by the acquisition means; and
a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing,
the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information;
the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size;
The determination means determines the drawing control load as the cause of the display delay based on at least one of an area, a number, and a dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image.
Display device. When executing the image enlargement process, the drawing control means draws the notification image at a magnification ratio with respect to the reference size determined based on the cause of the display delay.
The display device according to claim 1 . and when the vehicle and the forward object are relatively approaching each other in a predetermined direction, the drawing control means, when executing the image enlargement process, draws the notification image so that the notification image is enlarged and viewed toward the vehicle side based on an end of the notification image that is viewed at a position far from the vehicle in the predetermined direction.
The display device according to claim 1 . when the vehicle and the forward object are relatively approaching each other and a relative speed between the vehicle and the forward object is equal to or greater than a predetermined speed, the drawing control means draws the notification image at a magnification ratio with respect to the reference size determined based on the cause of the display delay and the relative speed when executing the image enlargement process.
The display device according to claim 1 . the drawing control means, when the forward object is an immovable object relative to the ground, and when the notification image for notifying information regarding the immovable object is an image of a predetermined specific type, does not execute the image enlargement process for the image of the predetermined specific type.
The display device according to claim 1 . The determination means determines the drawing control load as the cause of the display delay based on at least one of an area, a number, and a dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image.
The display device according to claim 1 .

Images