JP2007141223A - Information processing apparatus and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more improve safety or convenience of a driver. <P>SOLUTION: An operation control section 18 controls an audio output control section 20 or a display control section 22 so as to present to a driver 2 information for supporting driving during a travel or to present to the driver 2 information for supporting operation of a device to be supported in the direction where the face of the driver 2 is turned during stop on the basis of the direction of the face of the driver 2 detected by a face direction detection section 13 and a traveling state of an automobile detected by a traveling state detection section 14 or a driving state of the driver 2 detected by a driving state detection section 15. The present invention may be applicable to support systems for supporting driving of drivers. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and method, a recording medium, and a program, and more particularly, to an information processing apparatus and method, a recording medium, and a program that can further improve the safety or convenience of a driver. .

  In recent years, in order to provide drivers with a more comfortable driving environment, technology that supports driving to improve safety and supports operations of various in-vehicle devices installed in automobiles has attracted attention. Yes. Among them, conventionally, only when the car is going straight ahead, when it is determined that the driver is looking aside, an alarm is output, and the car is in another state (for example, driving on a curve, going backward, When the vehicle is stopped), an aside look determination device has been proposed that does not output an alarm (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 06-243367

  However, the invention described in Patent Document 1 cannot provide assistance for improving the safety or convenience of the driver in a state other than the state where the automobile is traveling straight ahead.

  The present invention has been made in view of such a situation, and is intended to further improve the safety or convenience of a driver.

  An information processing apparatus according to an aspect of the present invention includes a direction detection unit that detects the direction of a driver's face or line of sight, a traveling state detection unit that detects a traveling state of a vehicle, and the traveling of the vehicle when the vehicle is traveling. Based on the state and direction of the driver's face or line of sight, the driver is presented with information to assist driving, and if the car is parked, it is mounted on the car based on the driver's face or line of sight. Presentation control means for controlling the presentation of information so as to present information for supporting the operation of the existing device to the driver.

  In the information processing apparatus according to one aspect of the present invention, the direction of the driver's face or line of sight is detected, the driving state of the car is detected, and when the car is running, the driving state of the car and the driver's face or line of sight are detected. Based on the direction, the driver is presented with information for assisting driving, and when the vehicle is stopped, the operation of the device mounted on the vehicle is supported based on the direction of the driver's face or line of sight. The presentation of information is controlled so as to present this information to the driver.

  Therefore, information according to the driving state of the automobile and the direction of the driver's face or line of sight can be presented. In addition, the safety or convenience of the driver can be further improved.

  The direction detection means, the running state detection means, and the presentation control means are configured by, for example, a CPU (Central Processing Unit).

  When presenting the vehicle, the presenting control means presents the driver with information for alerting the driver based on the traveling state of the vehicle and the direction of the driver's face or line of sight. The presentation of information can be controlled.

  Thereby, the safety of the driver can be further improved.

  The information processing apparatus further includes target device detection means for detecting a state in which the driver's face or line of sight is directed to a predetermined device, and the presentation control means includes a driver's face when the vehicle is stopped. Information presentation can be controlled so that information supporting the operation of the device in the direction of the face or line of sight is presented to the driver.

  Thereby, the convenience of the driver can be further improved.

  An information processing method, a program, or a program recorded on a recording medium according to an aspect of the present invention includes a direction detection step for detecting the direction of a driver's face or line of sight, and a driving state detection step for detecting a driving state of an automobile. If the car is running, information to assist driving is presented to the driver based on the driving state of the car and the driver's face or line of sight. If the car is parked, the driver's face Or a presentation control step of controlling presentation of information so as to present information for assisting the operation of the device mounted on the vehicle to the driver based on the direction of the line of sight.

  In the information processing method, program, or program recorded on the recording medium according to one aspect of the present invention, the direction of the driver's face or line of sight is detected, the driving state of the car is detected, and the car is running If the vehicle is parked, it presents information to assist the driver based on the driving state of the car and the driver's face or line of sight, and if the car is parked, based on the driver's face or line of sight, The presentation of information is controlled so that information for supporting the operation of the device mounted on the vehicle is presented to the driver.

  Therefore, information according to the driving state of the automobile and the direction of the driver's face or line of sight can be presented. In addition, the safety or convenience of the driver can be further improved.

  This direction detection step is configured by, for example, a step of detecting the direction of the driver's face or line of sight by the CPU, and this driving state detection step is configured by, for example, a step of detecting the driving state of the automobile by the CPU. For example, when the vehicle is running, the control step presents the driver with information for assisting driving based on the driving state of the vehicle and the direction of the driver's face or line of sight, and the vehicle is stopped. Based on the direction of the driver's face or line of sight, the CPU is configured to control the presentation of information so as to present the driver with information for supporting the operation of the device mounted on the automobile.

  As described above, according to one aspect of the present invention, it is possible to present information according to the driving state of the automobile and the direction of the driver's face or line of sight. In addition, according to one aspect of the present invention, the safety or convenience of the driver can be further improved.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a support system to which the present invention is applied. A support system 1 to which the present invention is applied is a system that is mounted on a vehicle and improves the safety or convenience of a driver by supporting driving or operating a vehicle-mounted device.

  The support system 1 includes cameras 11 and 12, a face direction detection unit 13, a running state detection unit 14, a driving state detection unit 15, a support target device detection unit 16, a communication unit 17, an operation control unit 18, an operation data storage unit 19, The audio output control unit 20, the audio output unit 21, the display control unit 22, and the display unit 23 are included.

  The camera 11 is, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or a logarithmic conversion image sensor (for example, HDRC (High Dynamic Range CMOS (Complementary Metal Oxide Semiconductor)) (registered). And the performance necessary for detecting the orientation of the face of the driver 2 (for example, the resolution is 640 pixels wide × 480 pixels high, the number of gradations is 256 gradations, and the frame rate is every second) 30 frames). The camera 11 is installed at a position where the face of the driver 2 can be photographed, and supplies image data including the face of the driver 2 obtained by photographing to the face direction detection unit 13 and the operation control unit 18.

  The camera 12 is, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or a logarithmic conversion type image sensor (for example, HDRC (High Dynamic Range CMOS (Complementary Metal Oxide Semiconductor)) (registered). Trademark)). The camera 12 is installed at a position where the front of the vehicle can be photographed, and supplies image data obtained by photographing to the display control unit 22.

  The face direction detection unit 13 detects the face direction of the driver 2 based on the image data supplied from the camera 11. The method by which the face direction detection unit 13 detects the face direction is not limited to a specific method, and a method that can more quickly and accurately detect the face direction is desirable. The face direction detection unit 13 supplies the detected face direction information indicating the face direction of the driver 2 to the driving state detection unit 15, the support target device detection unit 16, and the operation control unit 18.

  The traveling state detection unit 14 acquires vehicle speed data indicating a vehicle speed from a vehicle speed sensor 31 provided in the automobile, and a steering angle of a steering wheel (handle) of the automobile from a snake angle detection unit 32 installed in the automobile. The rudder angle data indicating is acquired. The traveling state detection unit 14 detects the traveling state of the automobile (for example, start, stop, vehicle speed, traveling direction, etc.) based on the vehicle speed data and the steering angle data. The traveling state detection unit 14 supplies traveling state information indicating the detected traveling state to the driving state detection unit 15 and the operation control unit 18.

  The driving state detection unit 15 detects the driving state of the driver 2 based on the face direction information and the driving state information. The operation state detection unit 15 supplies operation state information indicating the detected operation state of the driver 2 to the operation control unit 18.

  The support target device detection unit 16 detects a state in which the face of the driver 2 faces the direction of the support target device set in the operation data stored in the operation data storage unit 19 based on the face direction information. To do. As will be described later, the support target device is a device for which operation support is performed by the support system 1. For example, an on-vehicle device mounted on a vehicle such as a car navigation device, a car audio device, or an air conditioner. A device such as a mobile phone, a PDA (Personal Digital Assistant), etc., which is placed at a predetermined position of the automobile while riding. When the support target device detection unit 16 detects a state in which the face of the driver 2 is continuously facing the direction of the support target device for a predetermined time or more, the support target device information indicating the support target device is displayed as the operation control unit 18. To supply.

  The communication unit 17 performs wired or wireless communication with the support target device, receives information and data processed by the support target device from the support target device, and transmits information and data processed by the support system 1 to the support target device. To do. The communication unit 17 supplies the information and data received from the support target device to the operation control unit 18 or the display control unit 22 as necessary.

  The operation control unit 18 sets the operation mode of the support system 1 to the running mode when the vehicle is running, and sets the operation mode of the support system 1 to the stopped mode when the vehicle is stopped.

  In FIG. 1, the driver 2 is a person driving a car provided with the support system 1, and is not included in the elements constituting the support system 1.

  As will be described later with reference to FIG. 2, when the operation mode is the traveling mode, the operation control unit 18 supports driving based on the face direction information, the traveling state information, or the driving state information. Control presentation of information to present information. In addition, when the operation mode is the stop mode, the operation control unit 18 operates the support target device indicated in the support target device information based on the face direction information or the information or data acquired from the support target device. Control the presentation of information to assist. The operation control unit 18 generates voice information related to information presented to the driver 2 by voice or display information related to information presented to the driver 2 by display of an image or text, and the voice information is output to the voice output control unit 20. The display of information is controlled by supplying display information to the display control unit 22. Further, the operation control unit 18 supplies information indicating the display position of the window indicating the detection result of the face direction of the driver 2 to the display control unit 22.

  The operation data storage unit 19 stores operation data indicating operations performed by the support system 1. The operation data includes, for example, the operation content corresponding to the driving state of the driver 2 detected by the driving state detection unit 15, the direction to be warned when the face of the driver 2 faces while driving (hereinafter, the warning target). The operation contents when the face of the driver 2 faces the warning target direction, the direction of each support target device with respect to the driver 2, and the direction of each support target device It contains data indicating the content of actions to be performed when the face is facing. Note that the warning target direction is, for example, a direction in which a device (for example, a car navigation device, a car audio device, or a mobile phone) that interferes with safe driving when operated or viewed while traveling is placed. .

  The audio output control unit 20 generates audio data based on the audio information supplied from the operation control unit 18 and supplies the generated audio data to the audio output unit 21.

  The audio output unit 21 includes, for example, a speaker, a buzzer, a siren, and the like, and outputs audio based on audio data supplied from the audio output control unit 20.

  The display control unit 22 generates image data based on the display information supplied from the operation control unit 18 and supplies the generated image data to the display unit 23. In addition, the display control unit 22 superimposes the foreground image data of the automobile photographed by the camera 12 on the generated image data as necessary, and supplies the superimposed image data to the display unit 23.

  The display unit 23 is, for example, a head-up display system that projects an image formed at a point at infinity on the windshield of an automobile, and displays an image based on the image data supplied from the display control unit 22. By configuring the display unit 23 with a head-up display system, the driver 2 can display an image displayed on the display unit 23 while looking at the foreground of the automobile, that is, while keeping the line of sight while driving. Can see.

  Next, a driver support process executed by the support system 1 of FIG. 1 will be described with reference to the flowchart of FIG. This process is started, for example, when an automobile engine is turned on.

  FIG. 3 is a diagram schematically illustrating an example of an installation position of the camera 51 and the display 51 of the display unit 23 and an example of a support target device when the front of the automobile is viewed from slightly behind the seat of the driver 2. is there. Hereinafter, the driver support processing will be described using the example shown in FIG. 3 as appropriate.

  In the example shown in FIG. 3, the camera 11 is installed between the display 51 and the steering wheel (handle) 52 so that the face of the driver 2 can be photographed. The display 51 is, for example, a display of a head-up display system configured by performing high-reflectance coding on a part of the surface of a windshield of an automobile.

  Further, in the example shown in FIG. 3, three of the support target devices set in the operation data are a car navigation device 53, a car audio device 54, and a mobile phone 55. The car navigation device 53 is installed on the upper surface of a dashboard of an automobile (not shown) and on the left side of the display 51 when viewed from the driver 2. In addition, the car audio device 54 is installed under the dashboard of a car (not shown) and almost directly below the car navigation device 53. The mobile phone 55 is placed in a holder for a mobile phone (not shown) on the right side of the handle 52 when viewed from the driver 2.

  In step S <b> 1, the camera 11 starts photographing the face of the driver 2. Further, the camera 11 starts supplying image data including the face of the photographed driver 2 to the face direction detection unit 13.

  In step S <b> 2, the face direction detection unit 13 starts detecting the face direction of the driver 2 based on the image data supplied from the camera 11. The face direction detection unit 13 starts supplying the detected face direction information indicating the face direction of the driver 2 to the driving state detection unit 15, the support target device detection unit 16, and the operation control unit 18.

  In step S3, the driving state detection unit 15 starts detecting the driving state. Specifically, the running state detection unit 14 is based on the vehicle speed data supplied from the vehicle speed sensor 31 and the steering angle data supplied from the steering angle detection unit 32, such as the vehicle speed and the traveling direction. Start detection. The driving state detection unit 15 starts supplying the driving state information indicating the detected driving state to the driving state detection unit 15 and the operation control unit 18.

  The driving state detector 15 starts detecting the driving state of the driver 2 based on the face direction information and the driving state information. When the vehicle is moving forward at a predetermined speed or more (the steering angle of the steering wheel 52 is less than a predetermined angle), the driving state detection unit 15 causes the face of the driver 2 to move forward of the vehicle for a predetermined time or more. When it is detected that the driver 2 is not facing, it is determined that the driver 2 is looking aside, and driving state information indicating that the driver 2 is looking aside is supplied to the operation control unit 18. In addition, when the speed of the automobile is equal to or higher than a predetermined speed, the driving state detection unit 15 indicates a state in which the face of the driver 2 continuously faces the warning target direction set in the operation data for a predetermined time or more. When detected, the driving state information indicating that the face of the driver 2 is facing the warning target direction is supplied to the operation control unit 18.

  In step S4, the support target device detection unit 16 starts detecting a state in which the face of the driver 2 faces the direction of the support target device based on the face direction information. When the support target device detection unit 16 detects a state in which the face of the driver 2 is continuously facing the direction of the support target device for a predetermined time or more, the support target device information indicating the support target device is displayed as the operation control unit 18. To supply.

  In step S <b> 5, the display unit 23 starts displaying the detection result of the face orientation of the driver 2. Specifically, the operation control unit 18 displays information indicating a position for displaying the window 61 indicating the face direction of the driver 2 illustrated in FIG. 4 according to the detected face direction of the driver 2. 22 is supplied. The display control unit 22 generates image data including a window 61 and a window 62 for notifying that the face orientation of the driver 2 is continuously detected (tracked), and the generated image data is displayed on the display unit 23. To supply. Based on the image data, the display unit 23 displays a window 61 at a position corresponding to the orientation of the face of the driver 2 and displays the window 62 on the top of the display 51, for example, as shown in FIG.

  Note that the windows 61 and 62 are translucent, and the foreground of the automobile seen through the windshield can be seen through the windows 61 and 62. Therefore, the driver 2 can check the display contents of the window 61 or 62 while viewing the foreground of the automobile.

  The window 61 is displayed in any one of a total of nine areas obtained by dividing the display 51 into approximately three equal parts in the vertical direction and approximately three equal parts in the horizontal direction according to the face direction of the driver 2. The driver 2 moves in accordance with the movement of the face of the driver 2. FIG. 4 shows an example in which the face of the driver 2 faces the front of the automobile indicated by the arrow A1 and is substantially horizontal with respect to the road surface, and the window 61 faces the face of the driver 2. Accordingly, it is displayed in the central area of the display 51 among the nine areas. Note that the window 61 is not displayed when the face of the driver 2 faces right or left more than a predetermined angle, or upwards or downwards more than a predetermined angle.

  In addition, a message such as “Tracking your face orientation” or “Starting driver monitoring” is displayed on the window 62 to notify the driver 2 that the face orientation is being monitored. Thus, the driver 2 who has started driving can be motivated to turn his face or line of sight in a direction that must be seen during driving.

  The display of the detection result of the face orientation of the driver 2 is not limited to the above example. For example, the display area of the display 51 is divided more finely and the face orientation of the driver 2 is more detailed. You may make it show. Further, the detection result of the face direction of the driver 2 may not be displayed.

  In step S6, the operation control unit 18 determines whether or not the traveling state of the automobile has changed. Specifically, when the driving state information indicates that the start or stop of the vehicle has been detected, the operation control unit 18 determines that the driving state of the vehicle has changed, and the process proceeds to step S7.

  In step S7, the operation control unit 18 changes the operation mode. Specifically, the operation control unit 18 sets the operation mode of the support system 1 to the running mode when the start of the automobile is detected. On the other hand, when the stop of the automobile is detected, the operation control unit 18 sets the operation mode of the support system 1 to the stopping mode.

  If it is determined in step S6 that the traveling state of the automobile has not changed, that is, if the automobile is running or stopped, the processing in step S7 is skipped, and the processing proceeds to step S8.

  In step S8, the operation control unit 18 determines whether or not the traveling mode. If it is determined that the traveling mode is set, the process proceeds to step S9.

  In step S9, the operation control unit 18 determines whether or not the automobile is traveling straight on the basis of the traveling state information. If it is determined that the vehicle is traveling straight, the process proceeds to step S10.

  In step S10, the operation control unit 18 determines whether or not the driver 2 is looking aside. When the driving state information indicating that the driver 2 is looking aside is not supplied from the driving state detection unit 15, the operation control unit 18 determines that the driver 2 is not looking aside, and the process proceeds to step S11. move on.

  If it is determined in step S9 that the vehicle is not moving straight, for example, if the vehicle is moving backward or turning a curve, the process of step S10 is skipped, and the process proceeds to step S11.

  In step S <b> 11, the operation control unit 18 determines whether the face of the driver 2 is facing the warning target direction. The operation control unit 18 determines that the face of the driver 2 is facing the warning target direction when the driving state information indicating that the face of the driver 2 is facing the warning target direction is supplied from the driving state detection unit 15. Then, the process proceeds to step S12.

  In step S <b> 10, when the information indicating that the driver 2 is looking aside is supplied from the driving state detection unit 15, the operation control unit 18 determines that the driver 2 is looking aside, and the process of step S <b> 11 is skipped. Then, the process proceeds to step S12.

  In step S <b> 12, the support system 1 issues a warning to the driver 2. Specifically, when it is detected that the driver 2 is looking aside, the operation control unit 18 displays voice information and display information to be presented when a driver is looking aside based on the operation data. Generate. The operation control unit 18 supplies audio information to the audio output control unit 20 and supplies display information to the display control unit 22.

  The audio output control unit 20 generates audio data based on the audio information and supplies the generated audio data to the audio output unit 21. The audio output unit 21 outputs audio based on audio data. The display control unit 22 generates image data based on the display information, and supplies the generated image data to the display unit 23. The display unit 23 displays an image based on the image data. For example, as shown in FIG. 5, when a state in which the face of the driver 2 is facing the direction indicated by the arrow A2 different from the traveling direction of the car continues for a predetermined time or longer, a message for calling attention to a side look is displayed. A window 63 for notification is displayed at the top of the display 51, and a voice message or a warning sound for calling attention to an aside is output.

  Thereby, the driver 2 can be alerted to the driver 2 quickly and surely, and the driver 2 can be supported to perform safe driving. That is, the driving safety of the driver 2 is improved.

  When the state in which the driver 2 is facing the warning target direction is detected, the motion control unit 18 detects the state in which the face of the driver 2 is facing the warning target direction based on the motion data. The voice information and display information to be presented are generated. The operation control unit 18 supplies audio information to the audio output control unit 20 and supplies display information to the display control unit 22.

  The audio output control unit 20 generates audio data based on the audio information and supplies the generated audio data to the audio output unit 21. The audio output unit 21 outputs audio based on audio data. The display control unit 22 generates image data based on the display information, and supplies the generated image data to the display unit 23. The display unit 23 displays an image based on the image data.

  For example, in the operation data, the direction of the mobile phone 55 indicated by the arrow A3 in FIG. 6 is set as the warning target direction, and the driver 2 faces the direction of the mobile phone 55 as the ringtone of the mobile phone 55 rings. 6 is displayed on the upper part of the display 51 to notify that the call by the mobile phone while traveling is dangerous, as shown in FIG. A voice message or a warning sound is output to notify that the telephone call is dangerous.

  Thereby, the driver 2 can be alerted promptly and reliably to the direction of the warning target, and the driver 2 can be supported to perform safe driving. That is, the driving safety of the driver 2 is improved.

  Note that the windows 63 and 64 are translucent windows, and the foreground of the automobile seen through the windshield can be seen through the windows 63 and 64. Therefore, the driver 2 can check the display contents of the window 63 or 64 while looking at the foreground of the automobile.

  Thereafter, the process proceeds to step S15.

  If it is determined in step S11 that the face of the driver 2 is not facing the warning target direction, the process of step S12 is skipped, and the process proceeds to step S15.

  If it is determined in step S8 that the vehicle is in the stop mode, the process proceeds to step S13.

  In step S <b> 13, the operation control unit 18 determines whether the face of the driver 2 is facing the support target device. When the support target device information is supplied from the support target device detection unit 16, the motion control unit 18 determines that the face of the driver 2 is facing the support target device, and the process proceeds to step S14.

  In step S14, the support system 1 performs operation support. Specifically, the operation control unit 18 sends information necessary for supporting operation of the support target device indicated in the support target device information from the support target device via the communication unit 17 as necessary. get. The motion control unit 18, based on the motion data or the information acquired from the support target device, the voice information to be presented when the state in which the face of the driver 2 faces the direction of the support target device is detected. Generate display information. The operation control unit 18 supplies audio information to the audio output control unit 20 and supplies display information to the display control unit 22.

  The audio output control unit 20 generates audio data based on the audio information and supplies the generated audio data to the audio output unit 21. The audio output unit 21 outputs audio based on audio data. The display control unit 22 acquires image data necessary for assisting the operation of the support target device from the support target device via the communication unit 17 as necessary. The display control unit 22 generates image data for supporting the operation of the support target device based on the display information or the image data acquired from the support target device, and supplies the generated image data to the display unit 23. . The display unit 23 displays an image based on the image data.

  For example, as shown in FIG. 7, when it is detected that the face of the driver 2 is facing the direction of the car navigation device 53 that is the support target device (the direction indicated by the arrow A4 in the figure), the display The map displayed on the car navigation device 53 is enlarged and displayed in the window 65 at the center of 51. Unlike the example shown in FIG. 7, the window 65 is actually translucent, and behind the window 65, the foreground of the automobile seen through the windshield can be seen through. Accordingly, the driver 2 can check the map while viewing the foreground of the automobile without operating the car navigation device 53.

  Further, for example, as shown in FIG. 8, when a state is detected in which the face of the driver 2 faces the direction of the car audio device 54 that is the support target device (the direction indicated by the arrow A5 in the drawing). In the window 66 at the center of the display 51, information (for example, title, performance time, etc.), lyrics, and the like related to the currently reproduced song are displayed. The window 66 is translucent, and the foreground of the automobile seen through the windshield can be seen behind the window 66. As a result, the driver 2 can check the information related to the song being played while operating the car audio device 54 while looking at the foreground of the car.

  Furthermore, for example, as shown in FIG. 9, when it is detected that the face of the driver 2 is facing the direction of the mobile phone 55 that is the support target device (the direction indicated by the arrow A6 in the figure) A reception history of the incoming e-mail on the mobile phone 55 is displayed in a window 67 at the center of the display 51, and a ring tone is output from the voice output unit 21. The window 67 is translucent, and the foreground of the automobile seen through the windshield can be seen behind the window 67. As a result, the driver 2 can check the reception history of the e-mail while looking at the foreground of the automobile without operating the mobile phone 55. When the mobile phone 55 has a videophone function, for example, a videophone screen may be displayed on the display 51.

  Although not shown, for example, when the face of the driver 2 faces the direction of the PDA that is the support target device, the schedule set in the PDA is displayed on the display 51. May be. Furthermore, for example, when a state in which the face of the driver 2 faces the direction of various meters of the automobile that is the support target device, the display of the meter may be displayed on the display 51.

  In addition, since the process of step S14 is performed only in the stationary mode and not performed in the traveling mode, that is, operation assistance is performed even if the driver 2 faces the support target device while the vehicle is traveling. Therefore, it is possible to prevent the driver 2 from being distracted by performing unnecessary operation support.

  Thereafter, the process proceeds to step S15.

  If it is determined in step S13 that the driver 2 is not facing the direction of the support target device, the process of step S14 is skipped, and the process proceeds to step S15.

  That is, when it is determined that the driver 2 is not facing the support target device, the support of the support target device is not supported. Thereby, for example, even when the driver 2 is temporarily stopped by a signal or the like, operation support is not performed unless the driver 2 is directed to the support target device. It is prevented from feeling annoying or distracting attention.

  In step S15, the operation control unit 18 determines whether or not the automobile engine is turned off. If it is determined that the automobile engine is not turned off, the process returns to step S6, and the processes of steps S6 to S15 are repeatedly executed until it is determined in step S15 that the automobile engine is turned off. In other words, depending on the face direction of the driver 2 and the driving state of the automobile, a warning to the driver 2 or support for operation of the support target device is performed.

  If it is determined in step S15 that the automobile engine has been turned off, the driver assistance process ends.

  As described above, during driving of the automobile, information for assisting the driver's driving is presented based on the orientation of the driver's face and the driving state of the automobile, so that the safety of the driver is improved. In addition, since the information for assisting the operation of the support target device is presented based on the direction of the driver's face while the vehicle is stopped, the convenience of the driver is improved. Furthermore, since the information for supporting the operation of the support target device is not presented while the vehicle is running, it is possible to prevent the driver's attention from being reduced.

  In addition, as described above, even if the driver does not perform any special operation, the operation mode is automatically changed according to the driving state of the vehicle, and processing corresponding to each operation mode is performed, so that the driver is presented. Information to be automatically switched. Therefore, accurate information can always be quickly presented to the driver according to the driving state of the automobile and the driving state of the driver.

  For example, with the transition to the driving mode, that is, with the transition to a state where the driver should concentrate on driving and pay attention to the situation around the vehicle, the monitoring of the driving state is automatically started, and the driving state of the driver Accordingly, a warning is given to the side-view driving or a warning not to face the warning target direction, and the driver is motivated not to look away from the direction to be seen during driving. In addition, presentation of information not related to driving is stopped so as not to hinder driving.

  Also, for example, when the support target device is operated while the vehicle is stopped, or when the vehicle starts with the line of sight directed toward the support target device (for example, the car audio device 54 is When the vehicle turns on, it turns into a green light, and the operation is continued and the vehicle is rushed to start.) The driver can be motivated not to do so.

  Furthermore, for example, with the transition to the stop mode, that is, with the transition to a state where it is not necessary to pay much attention to the situation around the vehicle, the monitoring of the driving state is automatically stopped, and Operation support is started. Accordingly, it is possible to prevent the driver from being unnecessarily warned about the driving state even when the vehicle is not in operation, and the driver can be prevented from feeling bothered. In addition, presentation of information not related to driving, which has not been presented because it interferes with driving during driving, is started.

  For example, when the vehicle is temporarily stopped due to a red light or traffic jam, it automatically shifts to the stopping mode, so even if the vehicle stops for a short time, the driver can promptly and efficiently provide operational support for the support target device with the line of sight. Can be received. For example, the driver simply turns his gaze toward the car navigation device 53 during the pause, and the screen of the car navigation device 53 is displayed on the display 51. Although it is not directly related to the driving operation at that time, such as regulation information and traffic jam information, more information that affects the subsequent driving can be acquired in a shorter time. In addition, since it is not necessary to operate the device to acquire information during the pause, it is possible to safely acquire information while concentrating on the driving operation.

  In addition, if it is determined in step S11 that the vehicle is in the warning target direction, a warning is given to the driver, so that the automobile does not travel straight and does not have to look ahead of the vehicle. Even in the determined state, the driver can be warned when looking at a direction that the driver should not see while driving (for example, the direction of the car navigation device 53 or the mobile phone 55 in the vehicle). Therefore, in the conventional device that warns aside, a warning is given when the left and right sides are confirmed on a curve or the like, and in order to prevent the driver from feeling annoyed, the aside is detected only while traveling straight ahead. However, the driver can be warned accurately in a wider range than that.

  Furthermore, when a side look is determined based only on the movement of the face and line of sight in the left-right direction, for example, when the automobile is traveling straight, the driver's face is almost in front of the driver in the left-right direction and the driver's face in the vertical direction. It is assumed that the driver's side-by-side cannot be detected when facing the direction of the in-vehicle device in the downward direction, but when the driver is facing the warning target direction, the driver is warned. Even when looking at the in-vehicle device that is almost in front of the camera, it is possible to reliably warn to look aside.

  Further, as described above with reference to FIGS. 7 to 8, by making the information displayed on the display 51 translucent when the vehicle is in the stop mode, it is possible to check the presented information while looking at the situation ahead. it can.

  Note that, in the above, an example is shown in which information that supports driving is presented when the operation mode is the traveling mode, and information that supports the operation of the support target device is presented when the operation mode is the stationary mode. However, in the embodiment of the present invention, in a state where the driver needs to concentrate on driving (for example, during driving), only the information for assisting driving is presented and it is not necessary to concentrate on driving It is desirable to present information including information having a purpose different from driving assistance (for example, when the vehicle is stopped). In addition, for example, when it is necessary to concentrate on driving, information that is not related to driving such as incoming calls and e-mails should not be presented in a presentation method that takes time to check, and driving operations and monitoring ahead When it is not necessary to perform the operation, it may be possible to present information irrelevant to the operation or to operate a device irrelevant to the operation. In this way, it is necessary to concentrate on driving while improving the convenience of the driver by switching the information presentation method or supporting the operation of various devices according to the degree of necessity to concentrate on driving In a certain state, it is possible to effectively prevent a dangerous state from being caused by operating a device that is not related to driving such as operation of a mobile phone.

  In addition, in step S6 of FIG. 2 described above, for example, when it is detected that the speed of the automobile is zero for a predetermined time or longer, it is determined that the automobile has stopped, thereby easily stopping the automobile. Although it can be detected, for example, if necessary, it may be determined that the automobile has stopped when the following states are detected.

  (1) When the state where the speed of the car is 0 and the engine is stopped, or the state where the engine that was operating after the speed of the car becomes 0 is stopped is detected.

  (2) When it is detected that the vehicle cannot travel until an operation that enables traveling is performed. For example, regardless of whether the engine is operating or not, when the vehicle speed is 0, the auxiliary brake is in operation, the AT (automatic transmission) selector is set to parking, Or, if any state of the state where the shift lever of the car is in neutral is detected.

  (3) When it is detected that an operation performed while the vehicle is stopped or an operation indicating that the vehicle is stopped, such as a parking light turned on, a door opened, or a trunk opened.

  (4) When a state where the vehicle is not traveling is detected based on the detection result of the sensor indicating the traveling state of the vehicle. For example, when a state in which acceleration in the front-rear, left-right, and vertical directions is almost zero for 40 seconds or longer is detected.

  (5) When it is detected that the driver has performed a preparatory action for getting out of the car and parking, such as removing the seat belt.

  (6) When the speed is 0 and a state in which the seat is tilted backward (for example, to take a rest) is detected.

  (7) When a state in which the driving operation is not continuously performed for a predetermined time or more is detected. For example, if a state where both hands are released from the steering wheel is detected for 30 seconds or more, or if the vehicle is not driving automatically, a state where the feet are completely released from the accelerator pedal and the brake pedal is 30. When detected continuously for more than a second.

  (8) When a state in which the vehicle is stopped is detected by an external detection means such as a device that automatically activates the parking stop of the parking lot.

  (9) When it is detected based on traffic information that the vehicle is incapable of moving, for example, by being involved in a traffic jam.

  (10) When it is detected that the vehicle cannot travel or is prohibited from traveling due to an abnormality such as a failure or theft.

  (11) In addition, when a state that is not detected during traveling is detected.

  Conversely, it may be determined that the vehicle has started when it is detected from information other than the speed of the vehicle that it can be determined that the vehicle is to be moved from the stopped state.

  Note that the more accurate the determination of the running mode and the stopped mode is, the more accurate information can be presented to the driver, and the convenience and safety for the driver are improved.

  Furthermore, in the above description, the direction of the driver's face is detected. However, instead of the face direction, the direction of the line of sight is detected, and the driver support process is performed using the direction of the line of sight. Good. Further, both the driver's face direction and line-of-sight direction may be used. Note that the method of detecting the direction of the line of sight is not limited to a specific method, and a method capable of detecting the direction of the line of sight more quickly and accurately is desirable.

  In addition, as a method for detecting the direction of the warning target or the direction of the support target device, a method of detecting using the vertical and horizontal directions of the face, a method of detecting using the vertical and horizontal directions of the line of sight For example, a method of detecting the line of sight and the direction of the face in the vertical and horizontal directions can be considered. For example, it may be determined that the driver's line of sight is facing the direction of the support target device by detecting that the line of sight of the driver is facing an angle within a predetermined range including each support target device.

  In the above, an example in which the display 51 is a head-up display has been described. However, other displays such as a display of a car navigation device may be used. For example, when the display of the car navigation device is used, the windows 61 to 67 shown in FIGS. 4 to 9 are displayed superimposed on the foreground of the car photographed by the camera 12. When the display 51 is a head-up display, the driver can see the foreground of the car while looking at the display 51. Therefore, the camera 12 for photographing the front of the car can be omitted. .

  Further, for example, the operation mode may be classified based on whether or not it is in operation, or may be classified on the basis of main power, that is, whether or not the engine is in operation.

  Further, the operation mode may be classified more finely. For example, the running mode is classified based on the vehicle speed, such as the low-speed running mode, the high-speed running mode, or the like, or the car-only road running mode, the city running mode, the living road running mode, etc. You may make it classify | categorize based on the place.

  Also, for example, stop mode, temporary stop mode by red traffic light, temporary stop mode by traffic jam, parking light lit stop mode, hazard lamp lit stop mode, stop after accident report You may make it classify | categorize based on the cause of having stopped, the state in stopping, etc. like the mode in the state in which the mode and the failure were detected. Further, the stop mode is classified in detail based on whether or not the side brake is applied, the state of the automatic transmission (AT) (for example, drive, reverse, parking, etc.), and whether or not the engine is operating. You may do it.

  By finely classifying the operation modes and performing the operation according to each operation mode, the driver can be supported more finely. Specifically, for example, the determination method of whether or not the side-by-side determination is necessary in step S9, the side-by-side determination method in step S10, the type and determination method of the warning target in step S11, the warning method to the driver in step S12, the support in step S13 By changing the type and determination method of the target device and the content and method of operation support in step S14 according to the detailed operation mode, the driver can be supported more finely.

  For example, riding mode, pre-running mode, high-speed running mode, low-speed running mode, stopping mode due to traffic jam, temporarily stopping mode, parking mode (excluding idling stop when engine is stopped), getting-off preparation mode, abnormal It is conceivable that the operation modes are classified into occurrence modes (for example, failure, accident, etc.) and the driver is finely supported according to each operation mode.

  For example, depending on the operation mode classified in detail, the content, type, presentation method, display conditions, etc. for supporting driving (for example, the number of warning target directions, judgment conditions for warning, etc.) are changed. You may make it do. For example, if the mode is set to indicate a state that does not hinder the temporary warning direction, such as a temporarily stopped mode or a stopped mode due to traffic jams, the warning to the driver is started for a predetermined time. If it is set to the high-speed driving mode, the driver can be started more quickly, or the warning can be given with a more strict expression and method to the driver more accurately. Can be warned. Even when the vehicle is stopped, a warning may be given to the driver according to the operation modes classified in detail.

  Furthermore, for example, the content, type, presentation method, display condition, or support method of information for supporting the operation of the support target device may be changed according to the operation mode classified in detail. For example, when the traveling mode with a speed of 0 or higher is set, it is not desirable to operate the cellular phone. Therefore, if there is an incoming call, the cellular phone is controlled so as to reject the response, and the temporarily stopped mode is set. When the transition is made, the caller's information is displayed on the display 51, and the mobile phone is controlled to respond when the driver sees the direction of the mobile phone, so that the incoming operation can be performed based on the driving state and the driver's intention. As a result, the safety and convenience of the driver are improved.

  In addition, when the driver sees a predetermined support target device (for example, a device that supports rearward confirmation such as a room mirror, a door mirror, and a back monitor, a sun visor, etc.) in the running mode, You may make it show with an apparatus.

  Furthermore, in the traveling mode, various information is presented by a driving support display device such as a head-up display and a back monitor that displays an image captured by the rear monitoring camera so as not to interfere with driving. Various types of information may be presented by the support target device.

  In the above description, an example is shown in which the operation of the support target device is not supported in the traveling mode. However, the operation of the support target device is supported as necessary even in the traveling mode. It may be. For example, if it is determined in step S11 that the warning target direction is not turned, the same processing as in steps S13 and S14 is performed before the processing in step S15, so that the support target device not in the warning target direction is performed. When the user is directed in the direction, the operation of the support target device may be supported.

  For example, as a support target device that is not in the warning target direction, a room mirror, a door mirror, and a back monitor are set, and when facing the direction of these devices, information displayed on the devices is displayed on the display 51. You may do it.

  In addition, as a support target device that is not in the warning target direction and is a support target device for presenting information for supporting driving, instruments such as a vehicle speed meter and a fuel meter are set, for example, these instruments When the information is displayed on the display 51, the driver's driving assistance is controlled by turning the direction of those instruments and changing the display contents and the display size. May be. For example, when the fuel of an automobile decreases during traveling and the warning screen is displayed on the display 51, when the direction of the fuel gauge is viewed, the warning screen is turned off or the display size is reduced, Despite confirming the information, the strong warning is not continuously given to the driver, and the driver is prevented from feeling bothersome.

  Further, for example, even when the direction of the support target device in the warning target direction is turned in the traveling mode, the driver is warned and, if necessary, the operation of the support target device is supported. May be. For example, when the car navigation device 53, the car audio device 54, and the air conditioner that are the operation target modes are turned in the traveling mode, the driver is warned and information about the devices (for example, the current vehicle status) The position, the title of the music being played, the set temperature of the air conditioner, etc.) are displayed on the display 51, so that the driver can be motivated not to face the direction of the support target device while traveling, and the support target device The information regarding can be easily acquired.

  In the above description, the example in which the driver assistance process is terminated when the vehicle engine is turned off in step S15 has been described. However, the condition for terminating the driver assistance process is not limited to this example. However, if it is determined that the use of the automobile has been stopped, the driver support process may be terminated. For example, when the accessory power is turned off, the key is turned to the handle lock state, the key is removed from the key cylinder, the door is opened or closed, and there are no people in the car due to the sensor etc. When the door is locked from outside the vehicle, a suspicious activity on the parked vehicle is detected by a sensor, etc., and the vehicle security device is set in a warning state. The driver assistance process may be terminated when an action to be performed or an event caused by the action is detected by a sensor or a vehicle control system. As a result, after the use of the automobile is stopped, the driver is not warned and the operation of the support target device is not supported, and the driver is prevented from feeling bothered by unnecessary warning and support. In addition, it is possible to prevent the support target device from performing a useless operation.

  Furthermore, for example, in a vehicle having an idling stop function for stopping the engine when the vehicle stops for a predetermined time or more by applying a determination condition other than the engine stop, a driver assist process is performed when the driver does not desire. Is prevented from ending.

In addition to the driver's face or line of sight, by combining other conditions,
You may make it determine whether the warning with respect to a driver is performed. For example, when the driver is facing the direction of the rearview mirror in the warning target direction so as to detect the inter-vehicle distance from the preceding vehicle based on the image taken by the camera 12, the approach to the preceding vehicle is not detected. A warning may be given to the driver when it is detected, and a warning to the driver may not be given when approach to the vehicle ahead is not detected. As a result, the driver is warned more accurately, and the troublesomeness that the driver feels about the warning can be reduced.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 10 is a block diagram showing an example of the configuration of a personal computer that executes the above-described series of processing by a program. A CPU (Central Processing Unit) 901 executes various processes according to a program stored in a ROM (Read Only Memory) 902 or a recording unit 908. A RAM (Random Access Memory) 903 appropriately stores programs executed by the CPU 901, data, and the like. The CPU 901, ROM 902, and RAM 903 are connected to each other by a bus 904.

  An input / output interface 905 is also connected to the CPU 901 via the bus 904. The input / output interface 905 is connected to an input unit 906 made up of a keyboard, mouse, microphone, etc., and an output unit 907 made up of a display, a speaker, etc. The CPU 901 executes various processes in response to a command input from the input unit 906. Then, the CPU 901 outputs the processing result to the output unit 907.

  The recording unit 908 connected to the input / output interface 905 includes, for example, a hard disk, and stores programs executed by the CPU 901 and various data. A communication unit 909 communicates with an external device via a network such as the Internet or a local area network.

  Further, the program may be acquired via the communication unit 909 and stored in the recording unit 908.

  A drive 910 connected to the input / output interface 905 drives a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the program or data recorded therein. Get etc. The acquired program and data are transferred to and stored in the recording unit 908 as necessary.

  As shown in FIG. 10, a program recording medium for storing a program that is installed in a computer and is ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, a removable medium 911 which is a package medium composed of a semiconductor memory, a ROM 902 in which a program is temporarily or permanently stored, and a recording unit 908 It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 909 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the order described, but is not necessarily performed in time series. Or the process performed separately is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows one Embodiment of the assistance system to which this invention is applied. It is a flowchart for demonstrating the driver assistance process of the assistance system of FIG. It is a figure which shows the example of installation of the assistance system of FIG. 1, and the example of a assistance object apparatus. It is a figure which shows the example of a display of the detection result of a face direction. It is a figure for demonstrating the example of the warning operation | movement with respect to a driver. It is a figure for demonstrating the other example of warning operation | movement with respect to a driver. It is a figure for demonstrating the example of the operation assistance operation | movement with respect to a driver. It is a figure for demonstrating the other example of the operation assistance operation | movement with respect to a driver. It is a figure for demonstrating the further another example of the operation assistance operation | movement with respect to a driver. And FIG. 11 is a block diagram illustrating an example of a configuration of a personal computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Support system 2 Driver 11 Camera 12 Camera 13 Face direction detection part 14 Running state detection part 15 Driving | running state detection part 16 Support object apparatus detection part 17 Communication part 18 Operation control part 19 Operation | movement data storage part 20 Audio | voice output control part 21 Audio | voice output Unit 22 display control unit 23 display unit 31 vehicle speed sensor 32 rudder angle detection unit 51 display 61 to 67 window 901 CPU
902 ROM
903 RAM
908 Recording unit 910 Drive 911 Removable media

Claims (6)

Direction detection means for detecting the direction of the driver's face or line of sight;
Driving state detection means for detecting the driving state of the automobile;
When the vehicle is running, based on the driving state of the vehicle and the direction of the driver's face or line of sight, the driver is presented with information for assisting driving, and when the vehicle is stopped, An information processing unit that controls presentation of information so as to present to the driver information for supporting operation of a device mounted on the vehicle based on the face or line of sight of the driver. apparatus. The presenting control means is configured to present information for alerting the driver to the driver based on a traveling state of the vehicle and a direction of the driver's face or line of sight when the vehicle is traveling. The information processing apparatus according to claim 1, wherein the information processing apparatus controls presentation of the information. A target device detecting means for detecting a state in which the face or line of sight of the driver faces the direction of a predetermined device;
The presentation control means controls the presentation of information so as to present to the driver information supporting the operation of the device in the direction of the driver's face or line of sight when the vehicle is stopped. The information processing apparatus described in 1. A direction detection step for detecting the direction of the driver's face or line of sight;
A driving state detecting step for detecting a driving state of the automobile;
When the vehicle is running, based on the driving state of the vehicle and the direction of the driver's face or line of sight, information to assist driving is presented to the driver, and when the vehicle is stopped, A presentation control step for controlling presentation of information so as to present to the driver information for assisting operation of a device mounted on the vehicle based on the direction of the face or line of sight of the driver. Method. A direction detection step for detecting the direction of the driver's face or line of sight;
A driving state detecting step for detecting a driving state of the automobile;
When the vehicle is running, based on the driving state of the vehicle and the direction of the driver's face or line of sight, the driver is presented with information for assisting driving, and when the vehicle is stopped, Based on the direction of the driver's face or line of sight, the computer executes a presentation control step for controlling the presentation of information so as to present the driver with information for supporting the operation of the device mounted on the automobile. Program to make.   A recording medium on which the program according to claim 5 is recorded.

Images