JP2012060291A - Image pickup system - Google Patents

Abstract

An image pickup system capable of preventing loss of a function as an image pickup apparatus even in a suspend mode.
When a face detection LSI that detects a face from an image captured by a CMOS sensor detects a face from an image captured by a CMOS sensor, the host computer and the DSP are returned from a low current consumption mode. .
[Selection] Figure 1

Description

  The present invention relates to an image capturing system having an image capturing apparatus that can acquire an image captured by an image sensor and can shift to a low current consumption mode such as a suspend mode.

  There are a digital camera, a digital video camera, and the like as an image pickup device that takes an image using an image pickup device. Recently, a personal computer (PC) includes an image pickup device, and a CPU (Central Processing Unit) in the personal computer. There is also an image capturing system configured to be controllable from an application program or the like.

  FIG. 3 shows a conventional image capturing system. The image capturing system 101 illustrated in FIG. 3 includes a host computer 102 and an image capturing device 110.

  The host computer 102 includes a CPU, a ROM / RAM, and the like, and controls the image capturing apparatus 110 connected via the USB interface by an OS (Operating System) or an application program that operates on the CPU.

  The image capturing apparatus 110 includes a DSP 103 and a CMOS sensor 104.

  The DSP 103 includes a sensor interface control unit 1031, a data control unit 1032, a USB control unit 1033, a CPU 1034, a ROM / RAM 1035, a PLL circuit 1036, a clock control unit 1037, an IO 1038, an external oscillator 1039, It has.

  The sensor interface control unit 1031 performs sample hold on the image signal output from the CMOS sensor 104 so as to maintain a good S / N (Signal / Noise) ratio by CDS (Correlated Double Sampling) processing, and further performs AGC. The gain is controlled by (Auto Gain Control) processing, A / D conversion is performed, and digital image data is output to the data control unit 1032. When the CMOS sensor 104 has an A / D conversion function, the sensor interface control unit 1031 generates an interface control signal between the CMOS sensor 104 / DSP 103, latches image data, and sends the data to the data control unit 1032. Output. The data control unit 1032 subjects the image data output from the sensor interface control unit 1031 to compression encoding processing according to an encoding method such as image quality correction processing or JPEG (Joint Photographic Experts Group), and outputs the result to the USB control unit 1033. To do. The USB control unit 1033 controls data communication in accordance with the USB standard with the host computer 102 connected via a USB (Universal Serial Bus) interface.

  The CPU 1034 reads and executes a control program stored in a ROM (Read Only Memory) section of the ROM / RAM 1035 to control the entire DSP 103. The ROM / RAM 1035 stores a control program and various data to be executed by the CPU 1034 in a ROM (Read Only Memory) unit, and the RAM (Random Access Memory) unit functions as a work area when the CPU 1034 operates.

  The PLL circuit 1036 is a phase-locked loop, and multiplies the clock signal output from the external oscillator 1039 that is input via the IO 1038 by a predetermined magnification and outputs it to the clock control unit 1037. To do. The clock control unit 1037 distributes the clock signal output from the PLL circuit 1036 into the DSP 103.

  The CMOS sensor 104 functions as an image sensor, images a subject, and outputs an image signal to the sensor interface control unit 1031. The CMOS sensor 104 is supplied with a clock signal for operation from the DSP 103.

  In addition, as an image pickup apparatus, an apparatus having a function of detecting the face of a subject has been proposed as in the image pickup apparatuses described in Patent Documents 1 and 2.

  In the imaging system 101 as shown in FIG. 3, reduction of power consumption has been an issue in recent years, and it is essential for the DSP 103 and the like to have a low current consumption mode (suspend mode).

  In the image pickup apparatus 110 shown in FIG. 3, the DSP 103 put into the suspend mode stops the source clock output from the PLL circuit 1036 and suppresses the current consumption of the DSP 103 itself. Naturally, in that case, the function as the image pickup apparatus 110 is completely lost.

  When the function of the image pickup apparatus 110 is restored, the DSP 103 needs to return from the suspend mode. The PLL circuit 1036 in the DSP 103 is restored by a return signal from the host computer 102 or the application program, and the clock signal is supplied. It needs to be resumed.

  That is, there is a problem that without the human operation for operating the host computer 102, the function of the image capturing 110 itself cannot be obtained, and the necessity and application of the image capturing apparatus 110 in the suspend mode are reduced. It was.

  Even in the imaging devices described in Patent Documents 1 and 2, the operation in the suspend mode is not touched. Therefore, even if a person passes in front of the imaging device in the suspend mode, face detection does not function, and imaging is performed. It still loses its function as a device.

  The present invention aims to solve such problems.

  That is, an object of the present invention is to provide an image pickup system that can prevent the function of the image pickup apparatus from being lost even in the suspend mode.

  In order to solve the above-mentioned problems, the invention described in claim 1 is an image pickup control that can be set to a low current consumption mode for acquiring an image pickup device and an image taken by the image pickup device and reducing an internal current consumption. An image imaging system comprising: an image imaging device including a means; and a host computer that controls the image imaging device, wherein the image imaging device includes a face detection unit that detects a face from an image captured by the image sensor. When the face detection unit detects a face from the image captured by the image sensor, the face detection unit transmits a signal indicating face detection to the host computer in the low current consumption mode, and the host computer An imaging system, wherein the imaging control means is returned from the low current consumption mode.

  According to a second aspect of the present invention, in the first aspect of the present invention, the imaging control means includes a PLL circuit that multiplies an input clock signal and supplies it to the inside, and the PLL circuit multiplied A clock control circuit for supplying either the clock signal or the input clock signal to the imaging device, and the PLL circuit stops operating in the low current consumption mode and clocks into the internal circuit The supply of the signal is stopped, and the clock control circuit supplies the input clock signal to the imaging device in the low current consumption mode.

  According to a third aspect of the present invention, in the second aspect of the present invention, the clock control circuit receives either the clock signal multiplied by the PLL circuit or the input clock signal to the face detection means. In the low power consumption mode, the input clock signal is supplied to the face detecting means.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the image pickup device includes a switching unit that switches operation or non-operation of the face detection unit. It is characterized by this.

  According to a fifth aspect of the present invention, in the invention according to any one of the second to fourth aspects, the operation of the imaging control unit is controlled by firmware stored in a storage unit provided therein. Further, the switching of the clock control circuit is performed by the firmware.

  The invention described in claim 6 is the invention described in any one of claims 1 to 5, wherein the host computer detects the face in the low power consumption mode when the face detecting means detects a face. Then, after returning from the low power consumption mode, the imaging control unit is caused to acquire an image for one frame from the imaging element, and after acquiring the image for the one frame from the imaging control unit, the imaging control unit is Switching to the low current consumption mode is performed again.

  The invention described in claim 7 is the invention described in any one of claims 1 to 6, wherein the imaging control means and the host computer are connected by a USB interface. To do.

  According to the first aspect of the present invention, if the face detecting means for detecting the face from the image captured by the image sensor detects the face from the image captured by the image sensor, the host computer is in the low current consumption mode. A signal indicating face detection is sent to the image sensor, and the host computer returns the imaging control means from the low current consumption mode, so that the image sensor can operate in the suspend mode, and the current consumption is lower than in the normal operation. Face detection is possible. In addition, since the face detection function can be used even in the suspend mode, when an object (person / face) is detected before the image capturing device, the face detection means triggers the host computer, Since the host computer restores the imaging control means, the entire imaging system can function automatically.

  According to the second aspect of the present invention, the imaging control means includes a PLL circuit that multiplies the input clock signal and supplies it to the inside, and either the clock signal multiplied by the PLL circuit or the input clock signal. And a clock control circuit for supplying an image sensor to the imaging device, and the PLL circuit stops operating in the low current consumption mode and stops supplying the clock signal to the inside, and the clock control circuit operates in the low current consumption mode. Since the input clock signal is supplied to the image pickup device at times, the image pickup control means stops the PLL circuit and stops the clock signal to reduce current consumption, and the image pickup device is supplied with a low-speed clock before multiplication. It can be supplied and operated with lower current consumption than during normal operation.

  According to the invention of claim 3, the clock control circuit supplies either the clock signal multiplied by the PLL circuit or the input clock signal to the face detection means, and is input in the low current consumption mode. Since the clock signal is supplied to the face detection means, it is not necessary for the face detection means to have a dedicated oscillating element, and the low-current clock is supplied in the low current consumption mode, so the face detection is performed with a lower current consumption than in normal operation. The function can be activated.

  According to the invention described in claim 4, since the switching means for switching the operation or non-operation of the face detection means is provided, the face detection operation can be arbitrarily switched ON or OFF. In addition, it is possible to completely stop the image pickup element and the face detecting means in the low current consumption mode so that the current consumption is minimized.

  According to the fifth aspect of the present invention, the imaging control unit is configured to be controlled by the firmware stored in the storage unit provided therein, and the switching of the clock control circuit is performed by the firmware. As a result, the operation can be changed more flexibly than hardware.

  According to the sixth aspect of the present invention, when the face detection unit detects a face in the low power consumption mode, the host computer returns to the imaging control unit for one frame from the image sensor after returning from the low power consumption mode. After the image is acquired and the image for the one frame is acquired from the imaging control unit, the imaging control unit is switched to the low current consumption mode again, so that the image captured by the image imaging apparatus while maintaining the low current consumption mode is maintained. The processing for face detection security that performs only security processing (alarms, locking, etc.) by detecting only the image where the face is photographed can be performed by an application program on the host computer.

  According to the invention described in claim 7, since the imaging control means and the host computer are connected via the USB interface, the suspend function is defined in the USB standard. Easy to design.

1 is a block diagram of an image capturing system according to an embodiment of the present invention. It is a block diagram of the image pick-up system concerning other embodiments of the present invention. It is a block diagram of the conventional image pick-up system.

  Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram of an image capturing system according to an embodiment of the present invention. FIG. 2 is a block diagram of an image capturing system according to another embodiment of the present invention.

  FIG. 1 shows an image capturing system 1 according to an embodiment of the present invention. The image capturing system 1 illustrated in FIG. 1 includes a host computer 2 and an image capturing apparatus 10.

  The host computer 2 includes a CPU, a ROM / RAM, and the like, and controls the image capturing apparatus 10 connected via the USB interface by an OS, an application program, or the like that operates on the CPU.

  The image capturing apparatus 10 includes a DSP 3, a CMOS sensor 4, and a face detection LSI 5.

  The DSP 3 as the imaging control means includes a sensor interface control unit 31, a data control unit 32, a USB control unit 32, a CPU 34, a ROM / RAM 35, a PLL circuit 36, a clock control unit 37, an IO 38, and an external device. And an oscillator 39.

  The sensor interface control unit 31 performs sample and hold on the image signal output from the CMOS sensor 4 so as to maintain a good S / N ratio by CDS processing, and further controls the gain by AGC processing. Conversion is performed to output digital image data to the data control unit 32. If the CMOS sensor 4 has an A / D conversion function, the sensor interface control unit 31 generates an interface control signal between the CMOS sensor 4 and the DSP 3, latches image data, and sends the data to the data control unit 32. Output.

  The data control unit 32 subjects the image data output from the sensor interface control unit 31 to compression encoding processing according to an image quality correction process or an encoding method such as JPEG, and outputs the result to the USB control unit 33.

  The USB control unit 33 performs data communication and control of the host computer 2 connected via the USB interface in accordance with the USB standard.

  The CPU 34 reads and executes a control program (firmware) stored in the ROM portion of the ROM / RAM 35 to control the entire DSP 3.

  A ROM / RAM 35 serving as a storage means stores a control program (firmware) and various data to be executed by the CPU 34 in the ROM section, and the RAM section functions as a work area when the CPU 34 operates.

  The PLL circuit 36 is a phase-locked loop, and receives a clock signal (low-speed clock signal) output from an external oscillator 39 that is input via an IO 38 that is an input terminal of the DSP 3. The signal is multiplied by the magnification and output to the clock controller 37 as a high-speed clock signal. The PLL circuit 36 stops the multiplication operation in the suspend mode described later.

  A clock control unit 37 as a clock control circuit distributes the high-speed clock signal output from the PLL circuit 36 into the DSP 3. The clock control unit 37 includes a selector 36a, and a high-speed clock signal output from the PLL circuit 36 by the CPU 34 and a low-speed clock signal directly input from the IO 38 without passing through the PLL circuit are selected, and a CMOS sensor 4 described later. Is output.

  Further, the DSP 3 described above can be shifted to the suspend mode (low current consumption mode) under the control of the host computer 2.

  The CMOS sensor 4 as an imaging device is a solid-state imaging device using a CMOS (Complementary Metal Oxide Semiconductor), which captures an image of a subject and outputs an image signal to the sensor interface control unit 31 and the face detection. Output to LSI 5. The CMOS sensor 4 is supplied with a clock signal for operation from the DSP 3. Further, although not shown, an optical component such as a lens that collects light from the subject, a driving mechanism for focusing, and the like are provided in the preceding stage of the CMOS sensor 4.

  The face detection LSI 5 serving as a face detection unit detects a human face from the image signal input from the CMOS sensor 4 and outputs a detection signal to the host computer 2. For face detection, for example, a plurality of weak classifiers that are linearly combined in cascade by AdaBoost learning described in Patent Document 2 described above are created as classifiers, and a Hair-Like feature value is calculated based on the classifiers. A known method such as a method of calculating and detecting a face may be used. The face detection LSI 5 has an external oscillator 5a without being supplied with a clock signal from the DSP 3, and the clock signal is supplied from the external oscillator 5a.

  In the suspend mode, the DSP 3 of the image pickup apparatus 10 having the above-described configuration stops the PLL circuit 36, stops the supply of the clock signal to each block inside the DSP 3, and supplies the clock signal only to the CMOS sensor 4. As the source clock supplied at this time, a low-speed clock from the external oscillator 39 is selected. In this selection, the firmware that operates on the CPU 34 immediately before the transition to the suspend mode sets the register and switches the selector 36a of the clock control unit 37 to the low-speed clock side. The DSP 3 itself enters the suspend mode after the selector 36a is switched.

  Further, the face detection LSI 5 operates by receiving a clock signal independently from the external oscillator 5a, acquires an image signal from the CMOS sensor 4, and is in a state where the DSP 3 functions even in the suspend mode. In the suspend mode, face data is discriminated and evaluated by the face detection LSI 5 from the image signal acquired by the CMOS sensor 4. When face data is detected, the face detection LSI 5 outputs a detection signal to the host computer 2 to trigger it. The host computer 2 confirms the trigger from the face detection LSI 5 by using an installed application program or the like, and returns the DSP 3 to the normal operation via the USB interface.

  According to the present embodiment, when the face detection LSI 5 that detects a face from an image captured by the CMOS sensor 4 detects a face from an image captured by the CMOS sensor 4 in the suspend mode, the host computer 2 sends a signal to the DSP 3. Thus, the CMOS sensor 4 can be operated in the suspend mode, and the face can be detected with a lower current consumption than in the normal operation. Further, since the face detection function can be used even in the suspend mode, the face detection LSI 5 triggers the host computer 2 when an object (person / face) is detected in front of the image capturing apparatus 10. Thus, since the host computer 2 restores the DSP 3, the entire image pickup system 1 can automatically function.

  The DSP 3 also multiplies an input clock signal and supplies it to the inside of the PLL circuit 36, and clock control for supplying either the clock signal multiplied by the PLL circuit 36 or the input clock signal to the CMOS sensor 4. The PLL circuit 36 stops its operation in the suspend mode and stops supplying the clock signal to the inside, and the clock control unit 37 receives the input clock signal in the CMOS sensor in the suspend mode. 4, the DSP 3 stops the PLL circuit 36 and stops the clock signal to reduce the current consumption, and the CMOS sensor 4 is supplied with a low-speed clock before multiplication, which is lower than that during normal operation. The CMOS sensor 4 can be operated with current consumption.

  In addition, since the switching of the clock control unit 37 is performed by the firmware, the operation can be changed more flexibly than hardware.

  In addition, since the DSP 3 and the host computer 2 are connected via a USB interface, the suspend function is defined in the USB standard, so that the hardware design can be facilitated by using this function.

  As a specific example of the image capturing system 1 having the above-described configuration, a system including a PC and a Web camera or a monitoring camera connected to the PC via a USB interface, or a node-type PC provided with the camera Etc. In particular, a battery-driven node type PC is preferable because it reduces wasteful consumption of the battery by reducing current consumption of the image pickup device (camera) portion.

  In the above-described embodiment, the face detection LSI 5 has the external oscillator 5a independently. However, as shown in FIG. 2, the face detection LSI 5 may be supplied with a clock signal from the DSP 3 like the CMOS sensor 4. By doing so, the face detection LSI 5 does not need to have a dedicated external oscillator 5a, and the low-speed clock is supplied in the suspend mode, so that the face detection function is operated with lower current consumption than in the normal operation. be able to.

  Further, the operation or non-operation of the face detection LSI 5 may be switched to the image capturing apparatus 10. For example, this may be achieved by providing a switch as a switching means so as to be switched from the outside, or by outputting a signal for enabling or disabling the DSP 3 to the face detection LSI or stopping the supply of the clock signal. May be. That is, the DSP 3 may function as switching means. In this way, the face detection operation can be arbitrarily switched ON or OFF. In addition, it is possible to completely stop the imaging device and the face detection means in the suspend mode so that the current consumption is minimized.

  Further, when the face detection LSI 5 detects a face while the DSP 3 is suspended, the host computer 2 causes the DSP 3 to acquire an image for one frame from the CMOS sensor 4 after returning to the normal operation, and the DSP 3 acquires the one frame from the DSP 3. After acquiring the minute image, the DSP 3 may be switched to the suspend mode again. In this way, for the face detection security that detects only the image of the face captured from the image captured by the image capturing apparatus 10 while maintaining the suspend mode, and performs security processing (alarm, locking, etc.). Processing can be performed by an application program on the host computer 2 or the like.

  In the above-described embodiment, the face detection LSI 5 is configured as an LSI (Large Scale Integration). However, the present invention is not limited thereto, and any configuration that operates independently of the DSP 3 such as a microprocessor and software that operates on the microprocessor can be used. good.

  The present invention is not limited to the above embodiment. That is, various modifications can be made without departing from the scope of the present invention.

1 image pickup system 2 host computer 3 DSP (image pickup control means)
4 CMOS sensor (image sensor)
5 Face detection LSI (face detection means)
35 ROM / RAM (storage means)
36 PLL circuit 37 Clock control unit (clock control circuit)

JP 2010-88093 A Japanese Patent No. 4315234

Claims (7)

An image pickup device including an image pickup device and an image pickup control unit capable of acquiring an image taken by the image pickup device and setting a low current consumption mode for reducing an internal current consumption; and a host computer for controlling the image pickup device; In an imaging system having:
The image capturing apparatus includes a face detecting unit that detects a face from an image captured by the image sensor,
When the face detection unit detects a face from an image captured by the image sensor, the face computer transmits a signal indicating face detection to the host computer in the low current consumption mode, and the host computer captures the image. An image capturing system, wherein the control means is returned from the low current consumption mode. The imaging control means includes a PLL circuit that multiplies an input clock signal and supplies the same to the inside, and a clock that supplies either the clock signal multiplied by the PLL circuit or the input clock signal to the imaging element. And a control circuit,
The PLL circuit stops operating in the low current consumption mode and stops supplying the clock signal to the inside,
The image capturing system according to claim 1, wherein the clock control circuit supplies the input clock signal to the image sensor in the low current consumption mode.   The clock control circuit supplies the face detection means with either the clock signal multiplied by the PLL circuit or the input clock signal, and the face detection means detects the input clock signal in the low current consumption mode. The image pickup system according to claim 2, wherein the image pickup system is supplied to the means.   The image capturing system according to claim 1, wherein the image capturing apparatus includes a switching unit that switches between operation and non-operation of the face detection unit.   The imaging control unit is configured to be controlled by firmware stored in a storage unit provided therein, and the clock control circuit is switched by the firmware. Item 5. The image capturing system according to any one of Items 2 to 4.   When the face detection unit detects a face in the low power consumption mode, the host computer causes the imaging control unit to acquire an image for one frame from the image sensor after returning from the low power consumption mode. The image according to any one of claims 1 to 5, wherein after the image for the one frame is acquired from the imaging control unit, the imaging control unit is switched to the low current consumption mode again. Imaging system.   The image capturing system according to claim 1, wherein the image capturing control unit and the host computer are connected by a USB interface.

Images