JP2009118430A - Imaging device, driving method of the imaging device, image generation device, and image generation method - Google Patents

Abstract

Provided are an imaging device that includes an imaging device having white pixels in which a luminance filter is arranged, and that can expand a dynamic range, a driving method of the imaging device, an image generation device, and an image generation method.
Signal charges are respectively received from an R pixel having a red filter, a G pixel having a green filter, a B pixel having a blue filter, and a W pixel having a luminance filter arranged in a matrix. An imaging device that acquires and generates a color image, in a region where the amount of image signal charge from the W pixel is compared with a predetermined threshold, and in the region where the amount of signal charge detected from the W pixel is equal to or less than the threshold , Obtain signal charges from the R, G, B, and W pixels to generate a color image, and obtain signal charges from the W pixels in areas where the amount of signal charges detected from the W pixels exceeds the threshold. And a drive unit that controls to acquire the signal charges of the R pixel, the G pixel, and the B pixel to generate an image.
[Selection] Figure 5

Description

  The present invention relates to an imaging apparatus, a driving method of the imaging apparatus, an image generation apparatus, and an image generation method.

  The image pickup apparatus shown in Patent Document 1 below acquires signal charges for each pixel by an image pickup unit including an image pickup element such as a CCD or a CMOS, and the image pickup element has a filter and luminance information for acquiring RGB color information. There is provided a color filter that arranges a luminance filter for obtaining. According to such an image sensor having a color filter array, the sensitivity of a pixel in which a luminance filter is arranged, that is, a so-called white pixel is about twice as large as that of an RGB pixel corresponding to a normal color filter. By using an image processing algorithm using RGB and W (white) pixels, there is an effect that luminance noise can be significantly reduced.

US Patent Application Publication No. 2007/0024931 JP 2003-199117 A

  When imaging is performed with the imaging apparatus having the above configuration, a scene to be captured may include both a high-luminance portion and a low-luminance portion. At this time, when the amount of light in the high-luminance part generates electrons that exceed any one of the colors of the image sensor or the photodiode capacity of the white pixel, the color of the high-luminance part is fixed to “white”. Clip processing is performed. Then, the dynamic range of the scene shot at that time is an area where white pixels are not clipped. White clip processing is not desirable for imaging with high image quality, and there has been a demand for enlargement in order to generate an image including a high-luminance portion with higher image quality.

  The above-mentioned Patent Document 2 is an image pickup device having a gray pixel in which a gray filter whose visible light sensitivity is smaller than that of a blue filter is arranged, and is configured to calculate each RGB signal using an output signal of the gray pixel. No means for expanding the dynamic range in a configuration using highly sensitive white pixels is described.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image pickup apparatus that includes an image pickup element having a white pixel in which a luminance filter is arranged and can expand a dynamic range, and a method for driving the image pickup apparatus. Another object is to provide an image generation apparatus and an image generation method.

The above object of the present invention is achieved by the following configurations.
(1) a color filter in which red, green, and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An imaging device for generating
Saturation detecting means for detecting a signal charge amount from the W pixel at the time of imaging, and comparing the signal charge amount with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charges of the R pixel, the G pixel, and the B pixel are acquired. An image pickup apparatus comprising a drive unit that controls to generate an image.
(2) a color filter in which red, green and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. A method of driving an imaging device that generates
At the time of imaging, a signal charge amount is detected from the W pixel, and the signal charge amount is compared with a predetermined threshold value.
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charges of the R pixel, the G pixel, and the B pixel are acquired. And a method for driving the imaging apparatus, wherein control is performed so as to generate an image.
(3) a color filter in which red, green and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An image generation device for generating
Saturation detection means for detecting a signal charge amount from the W pixel and comparing the signal charge amount with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the R pixel, the G pixel, and the B pixel is used without using the signal charge from the W pixel. An image generation apparatus comprising a digital signal processing unit for generating an image.
(4) a color filter in which red, green and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An image generation method for generating
A signal charge amount is detected from the W pixel, and the signal charge amount is compared with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not used, and the signal charge of the R pixel, the G pixel, and the B pixel is used. A method for generating an image, characterized in that control is performed to generate the image.

  According to the present invention, in the output image, in a region where the signal charge amount detected from the W pixel is equal to or less than the threshold value, the signal charge is acquired from the R pixel, the G pixel, the B pixel, and the W pixel to generate a color image. At the same time, in the region where the amount of signal charge detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charges of the R pixel, G pixel, and B pixel are acquired to generate an image. If the threshold is set to the saturation signal amount of W pixels, the dynamic range can be expanded when an image is generated using an image sensor including W pixels.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which has an image pick-up element which has the white pixel by which the luminance filter is arrange | positioned, and can expand a dynamic range, the drive method of this imaging device, an image generation apparatus, and an image generation method can be provided.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. In the present embodiment, a configuration of a digital camera will be described as an example of an imaging apparatus. 1 includes an imaging unit 1, an analog signal processing unit 2, an A / D conversion unit 3, a driving unit 4, a strobe 5, a digital signal processing unit 6, and a compression / decompression processing unit 7. A display unit 8, a system control unit 9, an internal memory 10, a media interface 11, a recording medium 12, and an operation unit 13. The digital signal processing unit 6, compression / decompression processing unit 7, display unit 8, system control unit 9, internal memory 10, and media interface 11 are connected to a system bus 14.

  The imaging unit 1 shoots a subject using an optical system such as a photographic lens and an imaging element 20 such as a CCD or CMOS, and outputs an analog imaging signal. The imaging unit 1 also includes a mechanical shutter that controls the blocking of light incident on the imaging device 20. The analog signal processing unit 2 performs predetermined analog signal processing on the imaging signal obtained by the imaging unit 1. The A / D conversion unit 3 converts the analog signal processed by the analog signal processing unit 2 into a digital signal. The output of the A / D conversion unit 3 is sent to the digital signal processing unit 6 as so-called RAW image data. The RAW image data is digital image data that has been digitized in the form of the imaging signal from the imaging unit 1.

  At the time of shooting, the optical system, the mechanical shutter, and the image sensor 20 are controlled via the drive unit 4. The image sensor 20 is activated by the drive unit 4 at a predetermined timing triggered by S2 on of a two-stage release switch (not shown) by operating a two-stage release button (not shown) that is a part of the operation unit 13. It is driven by a drive signal from an included timing generator (denoted as TG in FIG. 1). The drive unit 4 outputs a predetermined drive signal by the system control unit 9.

  The digital signal processing unit 6 performs digital signal processing corresponding to the operation mode set by the operation unit 13 on the digital image data from the A / D conversion unit 3. The processing performed by the digital signal processing unit 6 includes black level correction processing (OB processing), linear matrix correction processing, white balance adjustment processing, gamma correction processing, image synthesis processing, synchronization processing, Y / C conversion processing, and the like. included. The digital signal processing unit 6 is configured by a DSP, for example.

  The compression / decompression processing unit 7 performs compression processing on the Y / C data obtained by the digital signal processing unit 6 and also performs decompression processing on the compressed image data obtained from the recording medium 12.

  The display unit 8 includes an LCD display device, for example, and displays an image based on image data that has been photographed and has undergone digital signal processing. An image is also displayed based on the image data obtained by decompressing the compressed image data recorded on the recording medium 12. It is also possible to display a through image at the time of shooting, various states of the digital camera, information on operations, and the like.

  The internal memory 10 is, for example, a DRAM and is used as a work memory for the digital signal processing unit 6 and the system control unit 9, as well as a buffer memory and a display for temporarily storing captured image data recorded on the recording medium 12. It is also used as a buffer memory for display image data to the unit 8. The media interface 11 inputs / outputs data to / from a recording medium 12 such as a memory card.

  The system control unit 9 is mainly configured by a processor that operates according to a predetermined program, and controls the entire digital camera including a photographing operation.

  The operation unit 13 performs various operations when the digital camera is used, and includes a release button and the like.

  FIG. 2 is a diagram illustrating a pixel array of the image sensor according to the present embodiment. In the imaging device 20, photoelectric conversion units such as photodiodes that generate signal charges by photoelectrically converting incident light are arranged in a matrix on a semiconductor substrate (not shown) such as silicon. A color filter is provided on the light incident side of the semiconductor substrate. The color filter includes a filter that transmits light of R (red) wavelength, a filter that transmits light of G (green) wavelength, a filter that transmits light of B (blue) wavelength, and luminance information. In order to obtain this, a luminance filter that transmits light of all wavelengths is provided, and each filter is arranged in a two-dimensional direction parallel to the light receiving surface of the semiconductor substrate. In the present embodiment, one photoelectric conversion unit and one filter that transmits light incident on the photoelectric conversion unit constitute one pixel. Hereinafter, a pixel having an R filter is referred to as an R pixel, a pixel having a G filter is referred to as a G pixel, a pixel having a B filter is referred to as a B pixel, and a pixel having a W filter is referred to as a W pixel.

  Specifically, the pixels are arranged in the vertical direction (up and down direction in FIG. 2) and the horizontal direction (left and right direction in FIG. 2) in the light receiving region of the semiconductor substrate of the image sensor 20. Of the pixel columns arranged in the vertical direction, the first pixel column (the rightmost column in FIG. 2) is a B pixel with a W pixel sandwiched between W pixel, B pixel, W pixel, G pixel,. Pixels and G pixels are alternately arranged. The second pixel column adjacent to the first pixel column is a B pixel, a W pixel, a G pixel, a W pixel,... Are arranged alternately. Here, the first pixel column is arranged so that the position of each pixel is shifted by one pixel in the vertical direction with respect to the second pixel column. Further, in the third pixel column adjacent to the second pixel column, the G pixel and the R pixel are alternately arranged across the W pixel, such as a W pixel, a G pixel, a W pixel, an R pixel,. ing. The fourth pixel column adjacent to the third pixel column includes G pixel, R pixel, W pixel,..., G pixel, R pixel,. Are arranged alternately. Here, the position of each pixel is shifted by one pixel in the vertical direction with respect to the fourth pixel column in the third pixel column. As shown in FIG. 2, when viewed in a 4 × 4 pixel array, four pixel groups each including 2 pixels × 2 pixels are arranged, one pixel group including B pixels and W pixels, and R pixels And one W pixel group, and two G pixel groups and W pixel groups.

FIG. 3 is a diagram illustrating a control system of the imaging apparatus according to the present embodiment. FIG. 4 is a diagram illustrating an example of captured image data during imaging.
At the time of imaging, first, image data D obtained by imaging a subject by the imaging unit 1 is acquired. Since the image data D is used to detect the signal amount of the W pixel, the image data D may be a monochrome image formed by acquiring signal charges only from the W pixel, and from the RGB pixel and the W pixel as usual. It may be a color image formed by acquiring signal charges.

  In the image data D shown in FIG. 4, a portion where the signal charge amount of the W pixel is saturated may occur in a region with a large amount of light such as sunlight. In the region where the amount of light is small, the signal charge amount of the W pixel is less than the saturation amount. FIG. 5 is a graph showing changes in signal charges accumulated in each pixel with respect to exposure time or incident light quantity. As shown in FIG. 5, the region where the W pixel is not saturated does not exceed the signal charge accumulated in the R pixel, G pixel, B pixel, and W pixel, and therefore does not cause deterioration in image quality. However, on the other hand, when the signal charges accumulated in the R pixel, G pixel, B pixel, and W pixel are read out in the region where the W pixel is saturated, the RGB pixel is changed to the RGB pixel in the W pixel where the signal charge is easily generated. In comparison, the saturation signal amount is reached in an earlier exposure time, and the image quality deteriorates. However, in order to prevent saturation of the signal charge amount of the W pixel in a region with a large amount of light, if an image is formed only with RGB pixels without using the signal charge of the W pixel, the region with a small amount of light is sufficient. It is conceivable that the luminance information cannot be obtained and the image quality is deteriorated.

  In the present embodiment, the signal charge amount of the W pixel is detected based on the image data D acquired by the imaging unit 1, and the signal charge amount of the W pixel is input to the saturation detection unit 31. The saturation detection unit 31 compares a predetermined threshold value set in advance (here, a value corresponding to the saturation amount of the W pixel) with the acquired signal charge amount of the W pixel. Then, control information including the result of comparing the signal charge amount of the W pixel and the threshold value is input to the drive unit 4.

  In the region where the signal charge amount detected from the W pixel is equal to or less than the threshold value, the driving unit 4 performs the R pixel, G pixel, and B pixel in a predetermined region including the W pixel, as in normal still image capturing. The imaging unit 1 is controlled to acquire a signal charge from the W pixel and generate a color image. On the other hand, when the signal charge amount detected from the W pixel exceeds the threshold, the signal charge from the W pixel in a predetermined region including the W pixel is not acquired, and the R pixel, the G pixel, and the B pixel The image pickup unit 1 is controlled so as to generate the image by acquiring the signal charges. In the present embodiment, without reading out signal charges from W pixels, only RGB pixels are transferred and output from an output amplifier. However, when the driving unit 4 controls to read out only the signal charges of RGB, for example, all the pixels including the W pixel are read out, and only the signal charges of the W pixel are not output and discarded. Pixel addition may be performed in the transfer area.

  That is, as shown in FIGS. 4 and 5, in the imaging apparatus of the present embodiment, in the region where the W pixel is saturated, an image is formed with RGB pixels without using the signal charge of the W pixel, and the W pixel is saturated. In an area where no image is generated, an image is formed using signal charges acquired from normal RGB pixels and W pixels.

  FIG. 6 is a diagram schematically illustrating an arrangement of pixels in a region using only signal charges of RGB pixels without using signal charges of W pixels. At this time, when a pattern of 4 pixels × 4 pixels is viewed, a pixel group having 2 pixels of B pixels and a pixel group having 2 pixels of R pixels in a 2 pixel × 2 pixel region, each of 2 pixels And a pixel group having G pixels.

  According to the configuration of the imaging device of the present invention, in the output image, in the region where the signal charge amount detected from the W pixel is equal to or less than the threshold, the signal charge is acquired from the R pixel, the G pixel, the B pixel, and the W pixel. While generating a color image, in a region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charge of the R pixel, the G pixel, and the B pixel is acquired. Is generated. If the threshold is set to the saturation signal amount of W pixels, the dynamic range can be expanded when an image is generated using an image sensor including W pixels. For example, as in the pixel arrangement of the above embodiment, when the W pixel has twice the sensitivity with respect to the RGB pixel, the dynamic range can be doubled.

  Further, the present invention is not limited to the image pickup apparatus having the configuration of the above embodiment. In a region where the signal charge amount detected from the W pixel exceeds the threshold value, the R pixel and the G pixel are not used without using the signal charge from the W pixel. An image generating apparatus that forms a color image by a digital signal processing unit that generates an image using signal charges of pixels and B pixels can be provided.

1 is a diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. It is a figure which shows the pixel arrangement | sequence of an image pick-up element. It is a figure which shows the control system of an imaging device. It is a figure which shows an example of the image data imaged at the time of imaging. It is a graph which shows the change of the signal charge accumulate | stored in each pixel with respect to exposure time or incident light quantity. It is a figure which shows typically the arrangement | sequence of the pixel of the area | region which used only the signal charge of RGB pixel, without using the signal charge of W pixel.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up part 20 Image pick-up element 31 Saturation detection part B Blue (B) pixel G Green (G) pixel R Red (R) pixel

Claims (4)

A color filter in which red, green, and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An imaging device for generating
Saturation detecting means for detecting a signal charge amount from the W pixel at the time of imaging, and comparing the signal charge amount with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charges of the R pixel, the G pixel, and the B pixel are acquired. An image pickup apparatus comprising a drive unit that controls to generate an image. A color filter in which red, green, and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. A method of driving an imaging device that generates
At the time of imaging, a signal charge amount is detected from the W pixel, the signal charge amount is compared with a predetermined threshold value,
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not acquired, and the signal charges of the R pixel, the G pixel, and the B pixel are acquired. And a method for driving the imaging apparatus, wherein control is performed so as to generate an image. A color filter in which red, green, and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An image generation device for generating
Saturation detection means for detecting a signal charge amount from the W pixel and comparing the signal charge amount with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the R pixel, the G pixel, and the B pixel is used without using the signal charge from the W pixel. An image generation apparatus comprising a digital signal processing unit for generating an image. A color filter in which red, green, and blue filters and luminance filters are arranged in a matrix;
A signal charge is obtained from each of the R pixel having the red filter, the G pixel having the green filter, the B pixel having the blue filter, and the W pixel having the luminance filter to obtain a color image. An image generation method for generating
A signal charge amount is detected from the W pixel, and the signal charge amount is compared with a predetermined threshold;
In the output image, in a region where the signal charge amount detected from the W pixel is equal to or smaller than the threshold value, a color image is generated by acquiring signal charges from the R pixel, the G pixel, the B pixel, and the W pixel. In addition, in the region where the signal charge amount detected from the W pixel exceeds the threshold value, the signal charge from the W pixel is not used, and the signal charge of the R pixel, the G pixel, and the B pixel is used. A method for generating an image, characterized in that control is performed to generate the image.

Images