JP2013042402A - On-vehicle camera controller, on-vehicle camera control system, and on-vehicle camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an in-vehicle camera control device or the like capable of monitoring an object to be monitored around a vehicle while suppressing power consumption during parking.
An in-vehicle camera control device uses an image acquisition unit 110 that acquires infrared image data and visible image data, and uses any one of a plurality of processing modes including a traveling mode and a security mode as a processing process. A mode setting unit 130 that sets the mode and a processing unit 120 are included. When the travel mode is set as the usage processing mode, the processing unit 120 sets the visible image capturing unit 220 to the normal driving state, and generates notification data based on the visible image data and the infrared image data. When the security mode is set as the usage processing mode, the visible image capturing unit 220 is set to the power saving state in which the power consumption is lower than the operation off state or the normal driving state, and is converted into the infrared image data. Based on the monitoring object detection processing.
[Selection] Figure 1

Description

  The present invention relates to an in-vehicle camera control device, an in-vehicle camera control system, an in-vehicle camera system, and the like.

  In recent years, in order to cope with the worsening and more sophisticated crime, crime prevention measures have been strengthened and various crime prevention items have been desired.

  For example, with regard to crime prevention in residences, it is described in Patent Document 1 that, when unauthorized opening / closing by a suspicious person to a door or window of a residence is detected, an alarm such as an imaging by an infrared camera or a buzzer is given to notify the resident. There is such a conventional technology.

  However, simply increasing the number of crime prevention items increases the cost of the crime prevention system as the crime prevention system increases, resulting in an increase in the cost burden on the user.

  On the other hand, a vehicle-mounted night vision device that captures and displays a near-infrared image in front of the vehicle body with an in-vehicle infrared camera, detects a pedestrian from the captured image, and displays the detection result superimposed on the near-infrared image. Has been put to practical use. As a pedestrian detection technique used in such a night vision device, there is a conventional technique described in Patent Document 2.

  In the prior art of Patent Document 2, a search is made as to whether or not a part having the same shape as the shape of the predetermined part in the detection target is continuously present in the target, and the part having the same shape is continuously present. In this case, it is determined that the object is an artificial object, and a pedestrian is recognized from the remaining objects excluding the object.

  Under such circumstances, the present applicant is examining a method of using a vehicle-mounted camera for both intruder prevention use in a residence and pedestrian detection use on a road.

JP 2002-150438 A JP 2005-352974 A

  When the vehicle is traveling, the driving power for the imaging process and the image generation process of the in-vehicle camera is supplied from the car battery, so that a problem hardly occurs. However, when the vehicle is parked, the imaging process and the image generation process are performed by the battery of the vehicle or the battery held by the in-vehicle camera, so that it is necessary to reduce the power consumption of the in-vehicle camera as much as possible.

  According to some aspects of the present invention, an in-vehicle camera control device, an in-vehicle camera control system, an in-vehicle camera system, and the like that can monitor a monitoring object around the vehicle while suppressing power consumption during parking. Can be provided.

  One aspect of the present invention is an image acquisition unit that acquires infrared image data from an infrared image capturing unit, acquires visible image data from a visible image capturing unit, and a plurality of processing modes including a running mode and a security mode. A mode setting unit that sets any one of the processing modes as a usage processing mode; and a processing unit that performs processing according to the set usage processing mode, wherein the processing unit has the travel mode as the usage processing mode. If set, the visible image capturing unit is set in a normal driving state, and processing including generation processing of notification data is performed based on the visible image data and the infrared image data, and the use processing When the security mode is set as a mode, the visible image capturing unit is in an operation-off state or a power saving state in which power consumption is lower than that in the normal driving state. Set, related to the vehicle camera controller for processing including the process of detecting the monitored object based on the infrared image data.

  In one aspect of the present invention, different processing can be performed when the use processing mode is set to the traveling mode and when the use mode is set to the crime prevention mode. Thereby, it becomes possible to monitor the monitoring object according to the situation of the vehicle and the surroundings of the vehicle while suppressing power consumption.

  Moreover, in one aspect of the present invention, the mode setting unit may acquire traveling state information of a vehicle and perform the use processing mode setting process based on the traveling state information.

  This makes it possible to switch the usage processing mode depending on the traveling state of the vehicle. For example, when the vehicle is in a non-traveling state, it is possible to switch to the crime prevention mode to suppress power consumption.

  In the aspect of the invention, the mode setting unit obtains a vehicle speed based on the traveling state information of the vehicle, and when the traveling of the vehicle is detected based on the vehicle speed, the use processing mode is set to the traveling process mode. When the process of setting to the mode is performed, and the travel of the vehicle is not detected by the vehicle speed, the process of setting the use processing mode to the crime prevention mode may be performed.

  Thereby, for example, when the vehicle travel is detected by the vehicle speed, it becomes possible to set the travel mode and monitor the danger in front of the vehicle, and the vehicle travel is not detected by the vehicle speed. It is possible to set a crime prevention mode and monitor a residence or the like.

  Further, according to an aspect of the present invention, the image processing apparatus includes a storage unit that stores the infrared image data, and the processing unit includes the infrared ray at a given time interval when the crime prevention mode is set as the use processing mode. Image data may be stored, and monitoring processing of the monitoring area in the crime prevention mode may be performed based on the stored infrared image data.

  As a result, when the crime prevention mode is set and the visible image capturing unit is in the operation-off state, for example, it is possible to compare only the time difference component of the infrared image capturing unit, etc. While performing detection of an object, it is possible to greatly reduce the amount of calculation of image processing and, in turn, reduce power consumption for image processing.

  Moreover, in one aspect of the present invention, the processing unit performs the monitoring process by using first infrared image data and the first infrared image from among the plurality of infrared image data stored in the storage unit. The second infrared image data having a different acquisition timing from the data is specified, the first infrared image data and the second infrared image data are compared, and the monitoring is performed based on the result of the comparison processing. An object detection process may be performed, and security alarm data generation processing may be performed based on the result of the detection process.

  Thereby, when the crime prevention mode is set, the monitoring object is detected using only the infrared image data, and when the monitoring object is detected, the alarm notification data is generated and the residence is sent to the residence. It is possible to repel any intruder or notify the user of the presence of the intruder.

  Moreover, in one aspect of the present invention, the processing unit sets the first infrared image data as the monitoring process when the crime prevention mode is set as the use processing mode and the vehicle speed is not detected. And the second infrared image data are compared, the monitoring object is detected based on the result of the comparison, and the crime prevention notification data is generated based on the detection result. May be performed.

  As a result, it is possible to avoid erroneous detection that occurs when the user moves the vehicle while the crime prevention mode is set.

  In one aspect of the present invention, when the security mode is set, the processing unit sets a frame rate of the infrared image capturing unit to be lower than a frame rate at the time of setting the traveling mode. May be.

  Thereby, it is possible to suppress the consumption of power used for driving the infrared image capturing unit and image processing of infrared image data.

  Further, in one aspect of the present invention, when the security mode is set, the processing unit is configured such that at least one of the direction and the position of the infrared image capturing unit is set to the travel mode. Differently, at least one of the direction and the position of the infrared image capturing unit may be controlled.

  As a result, when the crime prevention mode is set, the position and direction of the infrared image capturing unit is changed and the monitoring object is monitored without parking the vehicle so that it faces the door or window of the residence. Etc. becomes possible.

  In one aspect of the present invention, the processing unit controls at least one of a direction and a position of the infrared image capturing unit based on input information from a user when the security mode is set. May be.

  This makes it possible to change the position and direction of the infrared image capturing unit according to the user's needs when setting the crime prevention mode.

  In one aspect of the present invention, when the travel mode is set, the processing unit sets a direction of the infrared image capturing unit to a front direction of the vehicle, and the security mode is set. In the case of being present, the direction of the infrared image capturing unit may be set to a direction different from the forward direction of the vehicle.

  This makes it possible to change the direction in which the infrared image capturing unit captures an image when the traveling mode is set and when the crime prevention mode is set.

  Moreover, in one aspect of the present invention, the processing unit performs detection processing of an object based on information obtained from the infrared image data when the traveling mode is set, and the visible image The visible image represented by the data may be processed.

  Thereby, when the traveling mode is set, the user can be alerted to the object.

  Further, in one aspect of the present invention, when the processing unit detects the object in the detection process, the processing unit, at the position on the visible image corresponding to the position where the object exists, as the processing process, You may perform the process which superimposes an alert image.

  Thereby, when the traveling mode is set, it is possible to call attention to the object in a format that is easier for the user to understand.

  Another aspect of the present invention provides a plurality of processing modes including an image acquisition unit that acquires infrared image data from an infrared image imaging unit and acquires visible image data from a visible image imaging unit, and a traveling mode and a security mode. Including a mode setting unit that sets the processing mode as a usage processing mode, and a processing unit that performs processing according to the set usage processing mode, and the processing unit includes the processing mode as the usage processing mode. When the driving mode is set, the visible image capturing unit is set in a normal driving state, and based on the visible image data and the infrared image data, processing including generation processing of notification data is performed, When the crime prevention mode is set as the use processing mode, the visible image capturing unit is configured to save power that is lower than the operation-off state or the normal drive state. Set the force condition, relating to the vehicle camera control system that performs processing including the process of detecting the monitored object based on the infrared image data.

  Another aspect of the present invention relates to a vehicle-mounted camera system including the vehicle-mounted camera control device, the infrared image capturing unit, and the visible image capturing unit.

The system block diagram of this embodiment. FIG. 2A is a plan view illustrating the attachment positions of the infrared image capturing unit and the visible image capturing unit, and FIG. 2B is a side view illustrating the attachment positions of the infrared image capturing unit and the visible image capturing unit. Hardware configuration diagram of this embodiment 4 (A) to 4 (D) are explanatory diagrams of processing when the travel mode is set. FIG. 5A to FIG. 5C are explanatory diagrams of processing when the crime prevention mode is set. 6A and 6B are explanatory diagrams of alert images. The flowchart explaining the process of this embodiment.

  Hereinafter, this embodiment will be described. First, a system configuration example of the present embodiment will be described, and then features of the present embodiment will be described. Finally, the flow of processing of this embodiment will be described using a flowchart. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. System Configuration Example First, FIG. 1 shows a configuration example of an in-vehicle camera control device of this embodiment and an in-vehicle camera system including the same.

  The in-vehicle camera control device 100 includes an image acquisition unit 110, a processing unit 120, a mode setting unit 130, and a storage unit 140. Moreover, the function of the vehicle-mounted camera control apparatus 100 may be implement | achieved by the vehicle-mounted camera control system containing a some information processing apparatus, for example. Furthermore, examples of the in-vehicle camera system 200 including the in-vehicle camera control device 100 include an in-vehicle camera system including an infrared image capturing unit 210, a visible image capturing unit 220, a presentation unit 230, a sensor 240, and the like. The in-vehicle camera control device 100 and the in-vehicle camera system 200 including the same are not limited to the configuration in FIG. 1, and various modifications such as omitting some of these components or adding other components. Implementation is possible. For example, some or all of the functions of the in-vehicle camera control device 100 may be realized by the infrared image capturing unit 210 or the visible image capturing unit 220. Further, some or all of the functions of the in-vehicle camera control device 100 according to the present embodiment may be realized by a server connected by communication. The same applies to the in-vehicle camera control system.

  Next, processing performed in each unit will be described.

  First, the image acquisition unit 110 acquires infrared image data representing an infrared image from the infrared image capturing unit 210 and acquires visible image data representing a visible image from the visible image capturing unit 220.

  Then, the processing unit 120 performs various processes using data obtained from the storage unit 140 according to the usage processing mode set by the mode setting unit 130.

  Furthermore, the mode setting unit 130 sets any one of a plurality of processing modes including a traveling mode and a crime prevention mode as a usage processing mode.

  The functions of the image acquisition unit 110, the processing unit 120, and the mode setting unit 130 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program, and the like.

  The storage unit 140 stores a database and serves as a work area for the processing unit 120 and the like, and its function can be realized by a memory such as a RAM or an HDD (hard disk drive).

  The infrared image capturing unit 210 captures an infrared image. The infrared image includes a far-infrared image and a mid-infrared image. When the human body is included in the detection target, the infrared imaging unit 210 may be sensitive to a far infrared wavelength region of 8 to 12 μm, and it is necessary to obtain sensitivity at a longer distance (approximately 200 m away). If there is, it is desirable to use a pyroelectric infrared sensor camera as the infrared image capturing unit 210. Using a pyroelectric sensor camera, thermal images (infrared images) from a long distance to a short distance can be acquired with a single sensor film, and the body temperature of the human body and the boundary line of other stationary objects can be clearly displayed. Therefore, it is optimal for human body extraction, and costs are significantly lower than other infrared sensors.

  Further, in the present embodiment, the infrared image capturing unit 210 is used for crime prevention purposes such as prevention of intrusion of a house. In this case, the monitoring area is an area including a window and a door of the house. Therefore, in order to detect an intruder into the monitoring area, it is desirable that the infrared imaging unit 210 has sensitivity in the far infrared wavelength region of 8 to 12 μm as described above. Therefore, the same infrared image pick-up part can also be shared at the time of driving | running | working and a house crime prevention. Furthermore, compared to detecting pedestrians while the vehicle is running, when detecting an intruder during crime prevention in a residence, the background image does not change in time, and only the presence or absence of the intruder needs to be detected. Therefore, image processing is very simple as compared to when traveling. In addition, since the time for the intruder to pass through the imaging area is longer than that for the pedestrian, it is possible to detect the intruder while setting the frame rate low and performing thinning imaging. It is possible to reduce the amount.

  On the other hand, the visible image capturing unit 220 captures a visible image. The visible image capturing unit 220 may be a camera using a CCD or CMOS as an image sensor, and may be a monochrome camera or a color camera. When a monochrome camera is used, a grayscale image is captured as a visible image, and when a color camera is used, a color image is captured as a visible image.

  The infrared image capturing unit 210 and the visible image capturing unit 220 may include a device (processor) used for image processing or the like. In the present embodiment, infrared image data or visible image data is output as it is to the in-vehicle camera control device 100, but the present invention is not limited to this. For example, a part of the processing unit of the in-vehicle camera control device 100 may be provided in the infrared image capturing unit 210 or the visible image capturing unit 220. In this case, information after image processing is performed on the infrared image or the visible image is output to the in-vehicle camera control device 100.

  Next, FIG. 2A and FIG. 2B show examples of attachment positions of the infrared image capturing unit, the visible image capturing unit, and the like.

  As shown in FIGS. 2A and 2B, the infrared image capturing unit 210 has a pan angle formed by the horizontal component direction HLL1 of the optical axis of the lens and the vehicle body centerline CL1 as PA1, and the optical axis of the lens. It is mounted in the bumper or grill of the vehicle body so that the tilt angle formed by the horizontal line HL1 between the vertical component direction PLL1 and the ground is TA1. Here, the infrared image capturing unit 210 may include an electric motor that swings the infrared image capturing unit 210 vertically and horizontally. In this case, the rotation angle can be freely adjusted so that the road ahead or the side of the road can be imaged.

  As shown in FIGS. 2A and 2B, the visible image capturing unit 220 has PA2 as the pan angle formed by the horizontal component direction HLL2 of the optical axis of the lens and the vehicle body centerline CL2, and the optical axis of the lens. It is attached to the upper part of the front window so that the tilt angle formed by the horizontal line HL2 between the vertical component direction PLL2 and the ground is TA2. Here, similarly to the infrared image capturing unit 210, the visible image capturing unit 220 may include an electric motor that swings the visible image capturing unit 220 up and down and left and right. In this case, the rotation angle can be freely adjusted so that the road ahead or the side of the road can be imaged.

  In FIG. 2A, PA1 and PA2 are 0 ° or more for easy understanding. However, when imaging the front in the vehicle body traveling direction, PA1 = PA2 = 0 ° and TA1 = TA2. Is desirable. Further, the infrared image capturing unit 210 and the visible image capturing unit 220 may be installed on the side surface of the vehicle body, or a plurality of each of the image capturing units may be installed.

  Further, the infrared image capturing unit 210 and the visible image capturing unit 220 are provided with a selector switch in the vehicle so that the driver can arbitrarily determine the shooting direction and the shooting position. 210 or the driving of the visible image capturing unit 220 can be turned ON / OFF, and the positions and directions of the infrared image capturing unit 210 and the visible image capturing unit 220 can be changed. Note that the selector switch is omitted in FIGS. 1, 2A, and 2B.

  Next, the presentation unit 230 shows the processing result of the in-vehicle camera control device 100 to the user. The presentation unit 230 may be an alarm device or a display.

  When the presenting unit 230 is an alarm device, the alarm device issues an alarm (voice) based on the output signal obtained from the processing unit 120 when the presence of a pedestrian or an intruder in a residence is detected. If the horn and front light of the vehicle body are used as an alarm device, the horn can be sounded or the front light can be blinked when an intruder is detected in the residence, and there is no need to install new equipment. There is.

  Further, when the presentation unit 230 is a display, the display displays an image indicating these positions when the processing unit 120 detects a preceding vehicle, a pedestrian, or an intruder into the residence, or detects An image generated by superimposing a contour image of an object on a visible image may be acquired from the processing unit 120 and displayed.

  Finally, the sensor 240 may be a vehicle speed sensor that obtains the vehicle speed from the rotational speed of the front wheels of the vehicle, or may be an acceleration sensor or a GPS sensor.

  FIG. 3 shows a hardware configuration diagram of this embodiment. First, the in-vehicle camera control device 100 of FIG. 1 can be realized by the CPU 450 and the memory 470 of FIG. Here, the image acquisition unit 110, the processing unit 120, and the mode setting unit 130 in FIG. 1 can be realized by the CPU 450 in FIG. 3, and the storage unit 140 in FIG. 1 can be realized by the memory 470 in FIG.

  1 includes the infrared camera 410, the visible camera 430, the CPU 450, and the memory 470 shown in FIG. Here, the infrared image capturing unit 210 in FIG. 1 is realized by the infrared camera 410 in FIG. 3 and includes an electric motor 420. Further, the visible image capturing unit 220 in FIG. 1 is realized by the visible camera 430 in FIG. 3 and includes an electric motor 440.

  1 can be realized by the alarm device 480 and the display 490 of FIG. The sensor 240 in FIG. 1 can be realized by the vehicle speed sensor 460 in FIG.

2. In the above embodiment, the image acquisition unit 110 that acquires infrared image data from the infrared image capturing unit 210 and acquires visible image data from the visible image capturing unit 220, the traveling mode, and the security mode are described. A mode setting unit 130 that sets any one of a plurality of processing modes as a use processing mode and a processing unit 120 that performs processing according to the set use processing mode are included. Then, when the traveling mode is set as the use processing mode, the processing unit 120 sets the visible image capturing unit 220 to the normal driving state, and based on the visible image data and the infrared image data, the notification data Processing including generation processing is performed. On the other hand, when the crime prevention mode is set as the use processing mode, the processing unit 120 sets the visible image capturing unit 220 to the power saving state in which the power consumption is lower than the operation off state or the normal driving state, and Processing including detection processing of a monitoring object based on image data is performed.

  Thereby, a different process can be performed by the case where use processing mode is set to driving mode, and the case where crime prevention mode is set.

  Here, the processing mode is a generic name of the state or state of the in-vehicle camera control device 100 set by the mode setting unit 130, and a series of processing performed by the in-vehicle camera control device 100 is determined by the set processing mode. Say what you can. Furthermore, the processing mode actually used when controlling the in-vehicle camera control device is referred to as a usage processing mode. In the present embodiment, at least one of the traveling mode and the crime prevention mode can be selected as the usage processing mode, and the processing mode can be switched to another processing mode.

  Here, the traveling mode is a processing mode that is set to the usage processing mode when it is determined that the vehicle is in a traveling state, for example. The crime prevention mode is determined that the vehicle is in a non-driving state, for example. Sometimes the processing mode is set to the usage processing mode.

  When the travel mode is set, the target object is a pedestrian or a preceding vehicle, and an optimal process is performed for monitoring these. On the other hand, when the crime prevention mode is set, the monitored object becomes an intruder into a residence or the like, and performs an optimal process for these monitoring. That is, it is possible to monitor an object according to the situation of the vehicle such as when the vehicle is traveling or parked.

  In addition, when the driving mode is set, the visible image data and the infrared image data are acquired, so that the object according to the situation around the vehicle such as there are many pedestrians or many other vehicles. It is possible to monitor.

  For example, FIG. 4A shows a state where the vehicle CA1 is traveling and the traveling mode is set. At this time, each of the infrared image capturing unit 210 and the visible image capturing unit 220 provided in the vehicle CA1 captures the front of the vehicle, and captures the preceding vehicle CA2, the pedestrian WP1, the person WP2 riding the bicycle, and the like. To do.

  Here, the infrared image data is information representing an infrared image (thermal image) captured by the infrared image capturing unit 210. The infrared image is as described above. For example, FIG. 4C illustrates an infrared image IIM captured by the infrared image capturing unit 210 provided in the vehicle CA1 illustrated in FIG. 4A as an example of the infrared image. In the infrared image IIM, a pedestrian WP1, a preceding vehicle CA2, a person WP2 riding a bicycle, and the like are projected darkly at portions where the temperature is high. An infrared image is not suitable for identifying a landscape or the like, but it is easy to identify a temperature distribution or the like, and is useful for distinguishing between a background and an object.

  The visible image data is information representing a visible image captured by the visible image capturing unit 220. The visible image is as described above. For example, as an example of the visible image, a visible image VIM captured by the visible image capturing unit 220 provided in the vehicle CA1 illustrated in FIG. 4A is illustrated in FIG. The visible image VIM also shows a pedestrian WP1, a preceding vehicle CA2, a person WP2 riding a bicycle, and the like. Since the visible image VIM has almost no difference from the scenery seen by the user, it is easy to grasp the entire image including the scenery at a glance. Conversely, it is difficult to distinguish a landscape from an object using only a visible image. Therefore, in the present embodiment, the object is distinguished using the infrared image having the characteristics as described above.

  Further, when the driving mode is set, the visible image capturing unit is set to the normal driving state, and the notification data is generated based on the visible image data and the infrared image data, which is presented to the user. It is possible to call attention to.

  Here, the normal drive state refers to a normal state in which the visible image capturing unit is activated and is not a power saving state. On the other hand, the power saving state is a state where the visible image capturing unit is activated, and means a state where the power consumption is lower than that in the normal driving state. Furthermore, the operation off state is a state where the visible image capturing unit is stopped. It is desirable that the visible image capturing unit can switch between at least the operation-off state and the normal drive state, and can also switch to the power saving state.

  The notification data is information for notifying the user of the presence of an object such as a pedestrian or a preceding vehicle when the traveling mode is set. For example, as described above, the notification data is image data generated by superimposing an image representing a pedestrian or a preceding vehicle on a visible image, or a warning notifying that the distance between the preceding vehicle and the preceding vehicle has been shortened. Such as sound.

  For example, FIG. 4D shows an image CIM represented by the notification data. When generating the image CIM, for example, based on the characteristics of the temperature distribution represented by the infrared image IIM shown in FIG. 4C, template matching processing as will be described later is performed, so that people (WP1 and WP2) And other than people (background and preceding car CA2). Then, an alert image (ALIM1 and ALIM2), which will be described later, is superimposed on the visible image so as to surround a portion where a person is located, and an image CIM is generated. By presenting such an image, the user is warned to pay attention to the pedestrian.

  On the other hand, when the security mode is set, the visible image capturing unit 220 and the visible image data are set in order to set the visible image capturing unit 220 to the power saving state in which the power consumption is lower than the operation off state or the normal drive state. It is possible to suppress the power used for the image processing.

  In this case, for example, as shown in FIG. 5A, only the infrared image capturing unit 210 is driven to monitor a residence or the like. FIG. 5 (A) shows a state of processing when the crime prevention mode is set, and shows a state where the vehicle CA is parked on the parking in front of the residence. In FIG. 5A, of the infrared image capturing unit 210 and the visible image capturing unit 220 provided in the vehicle CA, the operation of the visible image capturing unit 220 is turned off and only the infrared image capturing unit 210 is driven. The door DR and the window WD that are highly likely to be invaded by an intruder are captured. And the detection process of the monitoring target object is performed based on the acquired infrared image data.

  Therefore, even when the vehicle is in a non-traveling state, the surroundings of the vehicle can be monitored using the in-vehicle camera control device used in the traveling state.

  As described above, the monitoring target according to the situation of the vehicle and the surroundings of the vehicle while performing different processing and suppressing power consumption when the traveling mode is set and when the crime prevention mode is set. Can be monitored.

  Moreover, the mode setting part 130 may acquire the driving | running | working condition information of a vehicle, and may perform the setting process of the said use process mode based on driving | running | working condition information.

  Here, the traveling state information is information relating to the traveling state of the vehicle. For example, whether the vehicle is traveling, whether the vehicle body is vibrated, whether there is vibration, the magnitude of vibration, and the vehicle speed are This refers to information such as whether there is a vehicle speed, temperature information inside and outside the vehicle body, and travel time. The traveling state information may be acquired by the sensor 240, or may be obtained by comparing an infrared image obtained from the infrared image capturing unit 210 or a visible image obtained from the visible image capturing unit 220.

  This makes it possible to switch the usage processing mode depending on the traveling state of the vehicle. For example, when the vehicle is in a non-traveling state, it is possible to switch to the crime prevention mode to suppress power consumption.

  In addition, the mode setting unit 130 obtains the vehicle speed based on the vehicle traveling state information, and when the vehicle traveling is detected based on the vehicle speed, the mode setting unit 130 performs a process of setting the use processing mode to the traveling mode. If no is detected, a process for setting the use processing mode to the crime prevention mode may be performed.

  Thereby, for example, the vehicle speed is obtained from the rotational speed of the front wheels of the vehicle, and when the vehicle travel is detected based on the vehicle speed, it is possible to set the travel mode and monitor the danger ahead of the vehicle, etc. When vehicle travel is not detected by the vehicle speed, it is possible to set a crime prevention mode and monitor a residence or the like. Then, by switching the use processing mode, useless operations are not performed, and power consumption can be suppressed.

  Moreover, in this embodiment, you may include the memory | storage part 140 which memorize | stores infrared image data. Then, when the security mode is set as the usage processing mode, the processing unit 120 stores the infrared image data at a given time interval, and based on the stored infrared image data, the monitoring unit in the security mode A monitoring process may be performed.

  Here, the monitoring area is an area that needs to be monitored in order to detect the monitoring target of the in-vehicle camera system 200. For example, the monitoring area is captured by the infrared image capturing unit 210 or the visible image capturing unit 220. It means the range to be done. In the example of FIG. 5A, the range includes the door DR and the window WD of the residence imaged by the infrared image capturing unit 210.

  The monitoring process refers to, for example, a comparison process of infrared image data and a generation process of crime prevention notification data. These processes will be described later.

  Accordingly, when the security mode is set and the visible image capturing unit 220 is in the operation-off state, for example, it is possible to compare only the time difference component of the infrared image capturing unit 210, and so on. It is possible to significantly reduce the amount of calculation of image processing while detecting the monitoring object, and to reduce power consumption for image processing.

  In addition, as the monitoring process, the processing unit 120 selects the second infrared image data from the plurality of infrared image data stored in the storage unit 140 and the second infrared image data having different acquisition timings. The infrared image data is specified, the first infrared image data and the second infrared image data are compared, the monitoring target is detected based on the comparison processing result, and the detection processing result is detected. Then, the process for generating crime prevention alarm data may be performed.

  Here, the first infrared image data and the second infrared image data refer to infrared image data representing infrared images captured at different timings. For example, infrared image data for a certain timing 1 is acquired, this is used as first infrared image data, and infrared image data acquired at every predetermined period thereafter is used as second infrared image data for comparison processing. Also good. In addition, the infrared image data acquired at the timing next to the first infrared image data is the second infrared image data, and the infrared image data acquired at the timing next to the second infrared image data is the first infrared image data. Comparison processing may be performed assuming that the image data is infrared image data.

  For example, in the example of FIG. 5A, the infrared image captured by the infrared image capturing unit 210 at a certain timing t is IIM1 illustrated in FIG. 5B, and the infrared image captured at timing (t + 1) is illustrated in FIG. It is referred to as IIM2 shown in (C). In this case, first, the infrared image data indicating the infrared image IIM1 is stored in the storage unit as the first infrared image data, and when the infrared image data indicating the infrared image IIM2 is acquired, the infrared image data of the IIM2 is acquired. Is compared with the first infrared image data as the second infrared image data. As a result, in this example, it is assumed that the monitoring target object is not reflected in the infrared image IIM1, but the appearance of an intruder IV is confirmed in the infrared image IIM2. In this case, in FIG. 5A, it is assumed that at timing t, the person IV who was at the position P1 outside the monitoring area moved to the position P2 and entered the monitoring area before the timing (t + 1). This person IV is recognized as an intruder. Then, crime prevention notification data is generated.

  Also, the crime prevention data means that when the crime prevention mode is set, when the presence of a surveillance object such as an intruder into a residence is detected, the surveillance object is threatened or notified to a user or the like. This information is used for this purpose. For example, in order to notify the user that an intruder has been detected, as an example of crime prevention notification data, command information for sounding a car horn, command information for lighting a vehicle headlight, etc. Characters and image data.

  Thereby, when the crime prevention mode is set, the monitoring object is detected using only the infrared image data, and when the monitoring object is detected, the alarm notification data is generated and the residence is sent to the residence. It is possible to repel any intruder or notify the user of the presence of the intruder.

  Here, when a difference is detected in the infrared image data in the state where the crime prevention mode is set, there is a possibility that something has entered the monitoring area. Alternatively, the user may have moved the vehicle while the crime prevention mode is set, and the background around the vehicle may have changed. Originally, the purpose of the crime prevention mode is to detect only the former case.

  Therefore, when the crime prevention mode is set as the use processing mode and the vehicle speed is not detected, the processing unit 120 performs a comparison process between the first infrared image data and the second infrared image data as a monitoring process. It is possible to perform the detection processing of the monitoring object based on the result of the comparison processing, and perform the generation processing of the crime prevention alarm data based on the result of the detection processing.

  As a result, it is possible to avoid erroneous detection that occurs when the user moves the vehicle while the crime prevention mode is set.

  Here, it is known that an intruder into a residence or the like often stays at a certain place in order to hear the state of the residence, and the passing time of the monitoring area is longer than that of a pedestrian or the like. Further, when the vehicle is parked, the background image does not change with time, so image processing is easier than when the travel mode is set, and there is no need to set the frame rate high.

  Therefore, when the security mode is set, the processing unit 120 may set the frame rate of the infrared image capturing unit 210 to be lower than the frame rate at the time of setting the traveling mode.

  Accordingly, it is possible to suppress the consumption of power used for driving the infrared image capturing unit 210 and image processing of infrared image data.

  In addition, when the crime prevention mode is set, the processing unit 120 is configured so that at least one of the direction and the position of the infrared image capturing unit 210 is different from that when the traveling mode is set. At least one of the direction and the position may be controlled.

  Thus, when the crime prevention mode is set, the position and direction of the infrared image capturing unit 210 are changed and the monitoring target is monitored without parking the vehicle so as to face the door or window of the residence. It becomes possible.

  Further, when the security mode is set, the processing unit 120 may control at least one of the direction and the position of the infrared image capturing unit 210 based on input information from the user.

  Here, the input information from the user is information about the position and direction of the infrared image capturing unit 210 specified by the user using a selector switch or the like.

  For example, the user selects an appropriate combination from combinations of positions and directions of the infrared image capturing unit 210 prepared in advance, and operates the selector switch to input to the in-vehicle camera control apparatus 100. Then, the processing unit 120 notifies the infrared image capturing unit 210 of control information that changes at least one of the position and direction of the infrared image capturing unit 210 based on the input information.

  This makes it possible to change the position and direction of the infrared image capturing unit 210 according to the user's needs when the crime prevention mode is set.

  Further, the processing unit 120 sets the direction of the infrared image capturing unit 210 to the forward direction of the vehicle when the traveling mode is set, and the infrared image capturing unit when the crime prevention mode is set. The direction 210 may be set to a direction different from the front direction of the vehicle.

  Here, the front direction of the vehicle is not only the direction from the center point of the vehicle body to the center point of the vehicle width that is in front of the vehicle body from the center point of the vehicle body (that is, the direction directly in front of the vehicle body), The direction shifted in the direction is also included. For example, in FIG. 2A, the right direction of the vehicle center line CL1 is the forward direction of the vehicle. Moreover, the predetermined angle is an angle of about ± 10 °.

  Thereby, it is possible to change the direction in which the infrared image capturing unit 210 captures an image when the traveling mode is set and when the crime prevention mode is set. Therefore, it is possible to use one infrared image capturing unit 210 for both running and non-running.

  Further, the processing unit 120 performs processing for detecting a target object based on information obtained from infrared image data and processing a visible image represented by visible image data when the traveling mode is set. May be performed.

  Here, the processing refers to, for example, generating a new image by superimposing a contour image of an object on a visible image. After performing the processing, the processed image may be displayed on an in-vehicle display or the like. Specifically, as an example of the processing process, there is a process of generating the image CIM shown in FIG. In FIG. 4D, an image CIM is generated by superimposing ALIM1 and ALIM2, which are images that emphasize a pedestrian WP1 or a person riding a bicycle, on the visible image VIM.

  Thereby, when the traveling mode is set, the user can be alerted to the object.

  Further, when the processing unit 120 detects an object in the detection process, the processing unit 120 may perform a process of superimposing an alert image on a position on the visible image corresponding to a position where the object exists as a processing process.

  Here, the alert image is an image for informing the user of the presence of the target object. Examples of the alert image include an exclamation mark, a schematic diagram showing a pedestrian and a preceding vehicle, and an emphasis character indicating that a danger is imminent. Further, a warning sound or the like may be generated simultaneously with the alert image.

  Specifically, images ALIM1 and ALIM2 shown in FIG. 4D correspond to alert images. In addition, on the image CIM1 shown in FIG. 6A, an exclamation mark such as ALIM3 displayed on the head of the pedestrian WP1, or on the image CIM2 shown in FIG. There are an image ALIM4 for emphasizing the outline of the person WP1, and a band image such as ALIM5 for emphasizing that the danger is imminent using characters and symbols.

  Thereby, when the traveling mode is set, it is possible to call attention to the object in a format that is easier for the user to understand.

  Note that the functions of the in-vehicle camera control device and the like of this embodiment may be realized by a program. In this case, a function such as the in-vehicle camera control device of the present embodiment is realized by a processor such as a CPU executing a program. Specifically, a program stored in the information storage medium is read, and a processor such as a CPU executes the read program. Here, the information storage medium (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes of this embodiment based on a program (data) stored in the information storage medium. That is, the information storage medium includes a program (a program for causing the computer to execute the processing of each unit) for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment. Remembered.

3. Processing Flow Hereinafter, the processing flow of this embodiment will be described with reference to the flowchart of FIG.

  First, initial parameter settings are performed (S0). Next, in order to grasp the state of the vehicle, the vehicle speed is obtained based on the rotational speed of the front wheels (S1). In step S1, if the vehicle speed is greater than 0, it is determined that the vehicle is traveling, and if the vehicle speed is 0, it is determined that the vehicle is parked or stopped.

  If it is determined that the vehicle is running, the power of each electric motor used when rotating the infrared image capturing unit and the visible image capturing unit is turned on (S2). As a result, the motor is excited at a fixed position. As a result, when there was a slight vibration, the vibration was absorbed by the motor excitation, and the positions and directions of the infrared image capturing unit and the visible image capturing unit were not shifted, and there was a large vibration during the running. In this case, the infrared image capturing unit and the visible image capturing unit can be quickly returned to the original positions by the excitation force.

  On the other hand, if it is determined in step S1 that the vehicle is stopped and parked, the electric motor that rotates the infrared image capturing unit and the visible image capturing unit is turned off (S13). Thereby, the positions of the infrared image capturing unit and the visible image capturing unit can be manually rotated. The processing in this case will be described later.

  Next, after the process of step S2, the front of the vehicle body traveling direction is imaged using the infrared image capturing unit to obtain a thermal image (infrared image) (S3). Then, the visible image capturing unit captures the front in the vehicle body traveling direction, and acquires a visible image (S4). In addition, as for the order of step S3 and step S4, as long as the space | interval between steps is short, whichever is first. In some cases, both may be processed in parallel.

  Here, in order to acquire an image without image blur when the vehicle is traveling, it is necessary to first detect vibration of the in-vehicle camera and perform image blur suppression according to the detected value. In general, the vibration of the traveling vehicle body usually has a frequency of 1 Hz to 20 Hz unless there is a special step. In this example, such vibration is detected based on the amount of change such as acceleration, angular acceleration, angular velocity, and angular displacement. Then, based on this detection information, the photographing optical axis is decentered to suppress image blur.

  Next, a region having a temperature equal to or higher than a given temperature range is extracted from the thermal image obtained in step S3 as a candidate region for the detection target (S5).

  For example, the case where the detection target is a preceding vehicle or a pedestrian (person) will be described below. First, when the detection target is a preceding vehicle, a given temperature range is set to 80 ° C. or higher by utilizing the fact that the muffler region becomes a high temperature of about 100 ° C. Then, a region having a temperature of 80 ° C. or higher is searched from the thermal image, and position information and temperature information of the region are detected.

  On the other hand, when the object to be detected is a pedestrian, the given temperature range is set to 28 ° C. or higher and lower than 37 ° C. by utilizing the fact that the surface temperature of the exposed face is about 30 ° C. To do. Then, a region having a temperature of 28 ° C. or higher and lower than 37 ° C. is searched from the thermal image, and position information and temperature information of the region are detected.

  Here, among the results detected in each region, the position information may be, for example, the barycentric coordinate position of the candidate region, and the temperature information may be, for example, the average temperature of the candidate region.

  In addition, when the detection object is a thing other than the preceding person and the pedestrian, the given temperature range is set based on the temperature information that the detection object is assumed to have, and The detection process may be performed as follows.

  And the position and temperature list which matched the positional information and temperature information of each candidate area | region obtained by step S5 are produced (S6).

  Next, it is determined whether there is a candidate area for the detection target in the list created in step S6 (S7). If no candidate area for the detection target is extracted from the list, there is no preceding vehicle or pedestrian that is the detection target in the thermal image captured by the current infrared image capturing unit. It returns to step S3 for the next process.

  On the other hand, if there are one or more candidate areas for the detection target in the list, each candidate area is superimposed on the visible image based on the list created in step S6 (S8).

  Next, template matching processing is performed for each candidate region superimposed on the visible image in step S8, and it is determined whether or not the candidate region is a region representing a detection target.

  In the template matching process, first, a template to be used for the template matching process is selected based on the temperature information in the list created in step S6 (S9). For example, a template in which the difference between the temperature information pre-assigned to the template and the average temperature of the candidate area is within a given threshold (± ΔT) is selected as the template to be used. Note that the template may represent a tendency of the temperature distribution of the detection target.

  As a specific example, for example, when temperature information of 100 ° C. is assigned to the preceding vehicle template and the given threshold value ΔT is ± 3 ° C., the average temperature of the candidate region is 100 ± 3 ° C. In addition, the preceding vehicle template is selected as a template to be used.

  On the other hand, for example, temperature information of 28 ° C. is assigned to the template for the pedestrian, and when the given threshold ΔT is ± 3 ° C., when the average temperature of the candidate region is 28 ± 3 ° C., It is selected as a template to be used by the pedestrian template. In addition, regarding the high temperature part, it is necessary to consider that the temperature changes within a predetermined range even for the same object, considering that it is affected by the outside air temperature. In this example, the object having medium temperature information is a pedestrian. For example, in order to distinguish it from an animal (such as a dog) that is a similar object having medium temperature information, an animal template is used in this step. Can also be selected as the template to use.

  Then, the template selected in step S9 is compared with the candidate area to determine whether the candidate area is an area representing the detection target (S10). For example, when the candidate region has a similar temperature distribution tendency to the template, or when the error between the contour of the shape of the detection target assigned to the template and the contour of the candidate region is smaller than a predetermined threshold, etc. The candidate area is determined to be related to the same detection target as the template, and the position of the detection target is acquired.

  As another method of template matching processing, the density value (gray value) in the candidate area superimposed on the visible image is obtained based on the visible image, and the correlation with the density value (gray value) of the selected template is obtained. It may be determined by calculation whether or not the candidate region is a region representing a detection target. In addition, by preparing an edge image as a template, detecting edge information of the candidate area, and performing a correlation calculation of the edge information, it is determined whether or not the candidate area is an area representing the detection target. You may judge.

  Then, it is determined whether template matching processing has been performed for all candidate regions in the list created in step S6 (S11). If the template matching process has not been completed for all candidate areas in the list, the process returns to step S8. If the template matching process has been completed for all candidate areas in the list, the detection result is output to the external device (S12).

  Specifically, in order to notify the presence or absence of a pedestrian or the danger of a collision, the information is output to a notification device by an alarm or an email with an image. Moreover, in order to display the position of the preceding vehicle or pedestrian as a detection result, the detection result of this process may be output to the presentation unit (in-vehicle display).

  Finally, the process returns to step S1 in order to perform the subsequent processing.

  Next, in step S1, when the vehicle stops at the residence and the vehicle speed sensor detects zero, the electric motors that rotate the infrared image capturing unit and the visible image capturing unit are turned off (S13). . Thereby, it is possible to arbitrarily set the positions and directions of the infrared image capturing unit and the visible image capturing unit.

  At the same time as detecting that the vehicle speed has become zero, the drive sources of the visible image capturing unit and the infrared image capturing unit are stopped (S14). At this time, when the user turns on the selector switch installed in the vehicle (S15), only the infrared image capturing unit is driven to enter the image capturing state (S16). At this time, the counter i is initialized and an infrared image (thermal image) is taken at time t. Also, by operating the selector switch while displaying the infrared image on the presentation unit (in-vehicle display), the optimal camera position can be determined from several types of imaging positions and shooting directions of the infrared image capturing unit stored in advance in the storage unit. It is also possible to specify the direction and drive the electric motor (S17).

  Next, after a predetermined time has elapsed since the selector switch was turned on, the infrared image capturing unit captures the front and acquires a thermal image (S18). Here, the sensitivity of the infrared image capturing unit may be reduced. This is because the distance between the vehicle body and the monitoring object (window / door) is short, and there is little outline blur due to the overlap of the background portion and the image of the intruder (monitoring object).

  And after predetermined time progress, the counter i is updated, the front is imaged by an infrared image imaging part, and a new thermal image is acquired (S19).

  Here, when the vehicle speed is detected during the acquisition of the thermal image (S20), the process shifts to the traveling mode, and the processes after step S2 are performed. At this time, the infrared image capturing unit is moved so that the installation position of the infrared image capturing unit is the front position.

  On the other hand, when the vehicle speed is not detected during the acquisition of the thermal image, the temperature feature quantity of the thermal image obtained in step S18 and the thermal image obtained in step S19 is compared (S21).

  As a result of the temperature feature amount comparison process, if there is no difference in temperature feature amount between the two thermal images, it is determined that the monitoring target has not been detected, and the process returns to S19.

  On the other hand, as a result of the temperature feature amount comparison processing, if there is a difference in temperature feature amount between the two thermal images, there is a possibility that a monitoring object has been detected. Therefore, in this case, for example, when detecting an intruder, the temperature information of about 30 ° C. is obtained from the thermal image using the fact that the surface temperature of the exposed person's face is about 30 ° C. A region having the temperature information is searched, the presence or absence of corresponding temperature information is detected, and the presence or absence of an intruder is determined.

  If an intruder is detected in step S21, the horn of the vehicle body or the blinking of the front light is output as an alarm (S22).

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configurations and operations of the in-vehicle camera control device, the in-vehicle camera control system, and the in-vehicle camera system are not limited to those described in the present embodiment, and various modifications can be made.

100 on-vehicle camera control device, 110 image acquisition unit, 120 processing unit,
130 mode setting unit, 140 storage unit, 200 in-vehicle camera system,
210 infrared image capturing unit, 220 visible image capturing unit, 230 presenting unit,
240 sensors, 410 infrared cameras, 420 electric motors,
430 Visible Camera, 440 Electric Motor, 450 CPU,
460 Vehicle speed sensor, 470 memory, 480 alarm device, 490 display

Claims (14)

An image acquisition unit that acquires infrared image data from an infrared image capturing unit and acquires visible image data from a visible image capturing unit;
A mode setting unit for setting any one of a plurality of processing modes including a traveling mode and a crime prevention mode as a usage processing mode;
A processing unit that performs processing according to the set use processing mode;
Including
The processor is
When the travel mode is set as the use processing mode, the visible image capturing unit is set in a normal driving state, and notification data generation processing is performed based on the visible image data and the infrared image data. Process including
When the security mode is set as the use processing mode, the visible image capturing unit is set to an operation-off state or a power saving state in which power consumption is lower than the normal drive state, and the infrared image data An on-vehicle camera control device that performs processing including detection processing of a monitoring object based on the on-vehicle camera. In claim 1,
The mode setting unit
A vehicle-mounted camera control device that acquires travel status information of a vehicle and performs setting processing of the use processing mode based on the travel status information. In claim 2,
The mode setting unit
Obtaining the vehicle speed based on the travel status information of the vehicle,
If the vehicle travel is detected by the vehicle speed, a process for setting the use processing mode to the travel mode is performed,
An in-vehicle camera control device that performs processing for setting the use processing mode to the crime prevention mode when the vehicle speed is not detected based on the vehicle speed. In any one of Claims 1 thru | or 3,
A storage unit for storing the infrared image data;
The processor is
When the security mode is set as the use processing mode,
An in-vehicle camera control device that stores the infrared image data at given time intervals and performs monitoring processing of a monitoring area in the crime prevention mode based on the stored infrared image data. In claim 4,
The processor is
As the monitoring process, out of the plurality of infrared image data stored in the storage unit, the first infrared image data and the second infrared image data having different acquisition timings from the first infrared image data; The first infrared image data and the second infrared image data are compared, and the monitoring target is detected based on the comparison processing result. A vehicle-mounted camera control device that performs generation processing of crime prevention alarm data based on the above. In claim 5,
The processor is
When the crime prevention mode is set as the use processing mode and the vehicle speed is not detected, the comparison processing between the first infrared image data and the second infrared image data is performed as the monitoring processing. An in-vehicle camera control device that performs the detection process of the monitoring object based on the result of the comparison process and generates the crime prevention alarm data based on the result of the detection process. In any one of Claims 1 thru | or 6.
The processor is
When the crime prevention mode is set, the in-vehicle camera control device is characterized in that the frame rate of the infrared image capturing unit is set lower than the frame rate at the time of setting the travel mode. In any one of Claims 1 thru | or 7,
The processor is
When the security mode is set, at least one of the direction and position of the infrared imaging unit is different from the direction and position of the infrared imaging unit so that the traveling mode is set. An on-vehicle camera control device that controls at least one of them. In claim 8,
The processor is
An on-vehicle camera control device that controls at least one of a direction and a position of the infrared image capturing unit based on input information from a user when the security mode is set. In any one of Claims 1 thru | or 9,
The processor is
When the traveling mode is set, the direction of the infrared image capturing unit is set to the front direction of the vehicle,
When the security mode is set, the in-vehicle camera control device is characterized in that the direction of the infrared image capturing unit is set to a direction different from the front direction of the vehicle. In any one of Claims 1 thru | or 10.
The processor is
When the travel mode is set, the object detection process is performed based on the information obtained from the infrared image data, and the visible image processing process represented by the visible image data is performed. An in-vehicle camera control device. In claim 11,
The processor is
When the object is detected in the detection process, as the processing process, a process of superimposing an alert image on a position on the visible image corresponding to a position where the object exists is performed. Camera control device. An image acquisition unit that acquires infrared image data from an infrared image capturing unit and acquires visible image data from a visible image capturing unit;
A mode setting unit for setting any one of a plurality of processing modes including a traveling mode and a crime prevention mode as a usage processing mode;
A processing unit that performs processing according to the set use processing mode;
Including
The processor is
When the travel mode is set as the use processing mode, the visible image capturing unit is set in a normal driving state, and notification data generation processing is performed based on the visible image data and the infrared image data. Process including
When the security mode is set as the use processing mode, the visible image capturing unit is set to an operation-off state or a power saving state in which power consumption is lower than the normal drive state, and the infrared image data An in-vehicle camera control system that performs processing including detection processing of a monitoring object based thereon. The in-vehicle camera control device according to any one of claims 1 to 12,
The infrared imaging unit;
The visible image capturing unit;
An in-vehicle camera system characterized by including: