JP2015226299A - Image input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image input device that can enhance S/N irrespective of a compact configuration and obtain a high-quality reconstructed image.SOLUTION: A color filter has plural filter areas corresponding to a facet optical system. Filter elements of different two colors which are alternately arranged two-dimensionally are contained in the plural filter areas, and at least a pair of filter areas having different color combinations of filter elements are contained in the plural filter areas. Accordingly, a subject image which is formed while passing through one filter area containing filter elements of two colors contains color information of two colors, and a subject image which is formed while passing through a filter area containing other filter elements of two colors contains color information of two colors whose color combination is different from the above color information. Therefore, the color information is increased irrespective of a small number of filter areas, and a high-quality reconstruction image can be formed without increasing the number of facet optical systems.

Description

  The present invention is small and thin, and is used to form a subject image using a plurality of lenses on a solid-state imaging device such as a CCD (Charge Coupled Device) type image sensor or a CMOS (Complementary Metal Oxide Semiconductor) type image sensor. The present invention relates to an image input device.

  In recent years, in mobile terminals such as smartphones, in order to improve design, downsizing of a camera mounted thereon has been promoted, and accordingly, a demand for a low-profile optical system mounted on the camera has been increased. On the other hand, it is also required to improve the performance of the optical system so that a captured image has high image quality.

  In response to these requirements, multiple imaging optical systems (single-eye optical systems) arranged with different optical axes are used to form multiple subject images on the imaging surface of a solid-state imaging device, and correspond to each subject image Thus, a small and thin compound-eye camera using a so-called super-resolution technique for reconstructing one image has been developed by processing an image signal to be processed. In such a compound eye camera, a high-pixel secondary image (reconstructed image) can be created from a low-pixel primary image by reconstructing an image obtained from each of a plurality of single-eye optical systems. The number of pixels in the image area corresponding to each single-eye optical system may be small, and the single-eye optical system can be configured with a small lens. As a result, it is possible to provide a high-resolution camera while realizing a significantly lower profile than existing optical systems.

  In such a compound-eye camera, if filters of different colors are arranged in the optical path of each single-eye optical system and a primary image having a low pixel is formed from the transmitted light, these can be synthesized in the same manner. Despite the ability to form secondary images with high pixels, the wavelength band assigned to each single-eye optical system is narrowed, so the performance to cover the entire wavelength band becomes unnecessary, and the required performance required for the optical system is reduced. There is an advantage that the degree of freedom of design is improved.

  One example of a compound eye camera having filters of different colors for each of a plurality of optical systems is disclosed in Patent Document 1. The camera shown in Patent Document 1 has two or more lens pairs, and can form an image by transmitting different wavelength bands for each lens pair.

JP 2013-44893 A

  By the way, when a subject is imaged for each different wavelength band using a single-eye optical system in which a plurality of color filters having different wavelength transmission characteristics are arranged, the transmittance is generally different for each wavelength band. When imaging is performed with the same exposure time in the band, for example, in the wavelength band corresponding to blue, the output signal of the solid-state imaging device becomes low, and the S / N ratio may deteriorate, leading to a reduction in the quality of the primary image. In addition, when the number of divisions in the wavelength band is increased and the number of colors is increased, the number of single-eye optical systems is increased correspondingly, resulting in a problem that the compound-eye camera is increased in size. Furthermore, in a compound-eye camera, parallax occurs due to the optical axes of a plurality of single-eye optical systems being different. Therefore, when this is to be corrected by image processing, distance information to the subject is required. Such distance information can be acquired by the principle of a stereo camera. In this case, information of the same wavelength band is required through two single-eye optical systems, and a single-eye optical system that is twice the number of necessary colors is required. In addition, in order to obtain highly accurate distance information, it is preferable that the two single-eye optical systems in the same wavelength band be as far apart as possible, which causes a further increase in size of the compound-eye camera.

  With respect to such a problem, Patent Document 1 does not mention anything about improving the S / N ratio and increasing the number of colors without increasing the number of single-eye optical systems.

  The present invention has been made in view of such problems, and provides an image input device that can improve the S / N ratio while having a compact configuration and obtain a high-quality reconstructed image. With the goal.

The image input device of the present invention includes:
A plurality of individual optical systems with different optical axes,
An image sensor including a plurality of photoelectric conversion regions in which one subject image is formed by the plurality of single-eye optical systems;
A color filter disposed between the plurality of single-eye optical systems and the plurality of photoelectric conversion regions;
An image processing circuit that forms a reconstructed image by synthesizing a plurality of image signals output from the plurality of photoelectric conversion regions;
The color filter has a plurality of filter regions corresponding to the plurality of single-eye optical systems, and the plurality of filter regions include two different color filter elements arranged alternately in two dimensions, The filter element may include at least one set of filter regions having different color combinations.

  According to the present invention, the color filter includes a plurality of filter regions corresponding to the single-eye optical system, and the plurality of filter regions include two different color filter elements that are alternately arranged two-dimensionally. And at least one set of filter regions in which the color combinations of the filter elements are different from each other. Accordingly, the subject image formed by passing through one filter area including two color filter elements includes color information of two colors and passes through the filter area including the other two color filter elements. Since the subject image formed in this way contains two color information of different combinations, the color information increases despite the small number of filter areas, and without increasing the number of individual eye optical systems. A high-quality reconstructed image can be formed.

  According to one aspect of the present invention, filter elements having the same shape of two different colors are arranged in a checkered pattern in the filter region including the filter elements of the two colors.

  According to one aspect of the present invention, one color of a filter element included in at least one filter region of at least one set of filter regions including the two color filter elements is included in at least one other filter region. The image processing circuit adds the signals from the plurality of photoelectric conversion regions that have received the subject light that has passed through the filter element of the same color in different filter regions. An image signal having the same color information is generated.

  According to an aspect of the present invention, the image processing circuit receives subject light that has passed through the filter element of the first color in a filter area in which filter elements of different first and second colors are arranged. By subtracting the signal from the photoelectric conversion area and the signal from the photoelectric conversion area that has received the subject light that has passed through the filter element of the second color, the first color and the second color differ from each other. An image signal having information of three colors is formed.

  According to an aspect of the present invention, the image sensor further includes an exposure control circuit that adjusts a charge accumulation time independently for each photoelectric conversion region by controlling the image sensor.

  According to an aspect of the present invention, the exposure control circuit adjusts the charge accumulation time according to the color of the filter element through which subject light passes.

  According to one aspect of the present invention, one color of a filter element used in at least one filter region among the plurality of filter regions is the same as one color of a filter element used in at least one other filter region, The exposure control circuit is configured to generate a signal between the photoelectric conversion regions of the at least one set of filter regions based on signals from a plurality of photoelectric conversion regions that have received subject light that has passed through the filter elements of the same color in at least one set of filter regions. Adjust the difference in charge accumulation time.

  According to an aspect of the present invention, there is provided a lens array in which a plurality of lenses corresponding to the plurality of single-eye optical systems are integrally formed.

  According to one aspect of the present invention, the plurality of photoelectric conversion regions are integrally formed on the same substrate.

  According to one aspect of the present invention, one color of a filter element used in at least one filter region among the plurality of filter regions is the same as one color of a filter element used in at least one other filter region, The image processing circuit has a plurality of corresponding to the at least one set of filter regions based on signals from a plurality of photoelectric conversion regions that have received subject light that has passed through the filter elements of the same color in at least one set of filter regions. A parallax correction unit configured to correct parallax of a subject image generated between the single-eye optical systems;

  According to the present invention, it is possible to provide an image input device that can improve the S / N ratio while having a compact configuration and obtain a high-quality reconstructed image.

It is a figure which shows typically the imaging device concerning this Embodiment. It is sectional drawing which shows the specific structure of the imaging device DU. It is a block diagram of imaging device DU. 1 is a diagram illustrating a circuit configuration of an image sensor I. FIG. It is a figure which shows the filter area | region in the comparative example of a color filter. It is a figure which shows the filter area | region of the color filter concerning this Embodiment. It is a figure which shows the filter area | region of the color filter concerning 2nd Embodiment. It is a figure which shows wavelength distribution. It is a figure which shows the filter area | region of the color filter concerning 3rd Embodiment. It is a figure which shows the filter area | region of the color filter concerning 4th Embodiment. It is a figure which shows the filter area | region of the color filter concerning 5th Embodiment. It is a figure which shows the modification of the imaging device concerning this Embodiment. It is a figure which shows the modification of the filter area | region of the color filter concerning this Embodiment.

  Hereinafter, a compound eye optical system according to the present invention and an imaging device (image input device) using the same will be described. The compound-eye optical system is an optical system in which a plurality of lenses are arranged in an array with respect to, for example, one image sensor, and each lens system captures substantially the same field of view, and each lens has It is usually divided into a field division type that performs imaging of different fields of view. Hereinafter, in the present embodiment, description will be made assuming that a super-resolution compound eye optical system is used.

  FIG. 1 schematically shows an imaging apparatus according to the present embodiment. In FIG. 1, the imaging device DU includes nine individual optical systems IL arranged in 3 rows and 3 columns, and nine photoelectric conversion regions Ia in which one subject image is formed by each of the single optical systems IL. And an image sensor I provided, and a color filter CF disposed between the nine single-eye optical systems IL and the photoelectric conversion region Ia. The photoelectric conversion region of the image sensor may exist in an island shape so as to correspond to the single-eye optical system, or a plurality of parts of one photoelectric conversion region of a large image sensor are associated with the single-eye optical system. May be used. In the former case, crosstalk can be easily suppressed, and in the latter case, formation of the photoelectric conversion region is facilitated. The color filter CF is divided into nine filter regions according to the photoelectric conversion region Ia, and each has a plurality of filter elements.

  FIG. 2 is a cross-sectional view illustrating a specific configuration of the imaging device DU. In FIG. 2, a first light shielding member SH1, a first lens array LA1, a second light shielding member SH2, a second lens array LA2, a third light shielding member SH3, and a fourth light shielding member SH4 are arranged in this order from the object side. These are held in a lens frame HLD. The lens frame HLD includes a peripheral wall HLLa and an object side wall HLDb connected to the object side. The first light-shielding member SH1 made of a plate material forms a plurality of (here, 9 pieces) aperture stops S arranged in 3 rows and 3 columns.

  The first lens array LA1 is formed by integrally forming a plurality of (here, 9 lenses arranged in 3 rows and 3 columns) first lenses (single-lens lenses) L1 and a flange portion L1f that connects the first lenses L1. Yes. In addition, the second lens array LA2 is integrally formed with a plurality of (here, 9 lenses arranged in 3 rows and 3 columns) second lenses (single lens) L2 and a flange portion L2 that connects the second lenses L2. doing. The first lens array LA1 and the second lens array LA2 are injection molded using polycarbonate and acrylic resin, respectively, and are integrally formed of these materials. The optical axes of the respective aperture stops S and the corresponding first lens L1 and second lens L2 coincide with each other, thereby constituting a single-eye optical system IL. The first lens array LA1 and the second lens array LA2 may be an integrally molded glass mold, or may be a lens part integrally formed on a glass flat plate. By forming an array lens obtained by integrally molding a plurality of lenses using a mold made of resin or glass, an inexpensive and highly accurate array lens can be obtained. However, a lens may be provided for each photoelectric conversion region.

  Between the fourth light shielding member SH4 and the image sensor I, an IR cut filter F and a cover glass CG covering the imaging surface of the image sensor I are arranged in this order from the object side. Each of the light shielding members SH2 to SH4 has an opening corresponding to each of the individual lenses, and functions as a diaphragm. A color filter is formed on the imaging surface of the image sensor I. Since the photoelectric conversion region is integrally formed on the same substrate, the photoelectric conversion region can be easily formed.

  Although details of the color filter will be described later, as shown in FIG. 1, the filter has nine filter regions, and filter elements having the same shape are arranged for each pixel. A plurality of filter elements in two different transmission wavelength regions arranged in a checkered pattern are included therein. Further, filter elements having the same transmission wavelength band (that is, the same color) are arranged in at least two filter regions. As a result, in the subject image formed through one filter region, resolution degradation is suppressed, and a high-quality image can be obtained. At this time, it is preferable to dispose filter elements having adjacent transmission wavelength bands in the same filter region while dividing the wavelength band, because there is not much difference in spectral sensitivity of the solid-state imaging device in the filter region. In FIG. 1, the size of the filter element is schematically shown for easy understanding.

  FIG. 3 is a circuit block diagram of the imaging device DU. FIG. 4 is a diagram illustrating a circuit configuration of the image sensor I. At the time of imaging a subject, as shown in FIG. 4, the image signal from each photoelectric conversion area Ia is output as a RAW image via the output unit OT. Each photoelectric conversion area Ia is supplied with a clock, a synchronization signal, a control signal, and the like from an exposure control circuit ECC, which will be described later, and can independently control the exposure time (charge accumulation time), that is, start of charge accumulation. (Reset) and charge readout timing can be performed independently.

  A more specific operation will be described. In FIG. 3, a plurality of subject images incident through the single-eye optical system IL are respectively formed on the photoelectric conversion region Ia of the image sensor I. The formed subject image is photoelectrically converted in each photoelectric conversion area Ia and converted into an image signal. An image signal is obtained for each small image (primary image) formed by each eye optical system IL.

  Consider this primary image as an individual image. At this time, an image signal having single color information can be obtained by adding together image signals obtained by photoelectrically converting subject images that have passed through filter elements of the same color among the nine primary images. . As a result, even if the filter element has a color corresponding to a wavelength band in which the sensitivity of the solid-state imaging device is low, such as blue, the signal from the photoelectric conversion region that has received the subject light that has passed through the same blue filter element can be obtained. By performing the addition process, a signal with improved S / N and resolution can be obtained, so that a high-quality image can be obtained. The image signal output from the photoelectric conversion region Ia is input to an image processing circuit PRC including a demosaic unit DM, a signal level correction unit SS, a parallax detection unit PD, a parallax correction unit PS, and an image synthesis unit PR. More specifically, it is input to the parallax detection unit PD and the demosaic unit DM as will be described later. In the parallax detection unit PD, the parallax of the subject image generated between the single-eye optical systems (that is, the position shift of the same subject image formed for each single-eye optical system) is calculated. At this time, it is preferable to perform parallax correction using an image signal obtained by photoelectrically converting a subject image that has passed through filter elements of the same color at the same position in different filter regions. The parallax detection result is output to the parallax correction unit PS.

  On the other hand, the image signal output from the photoelectric conversion region Ia via the output unit OT is also output to the exposure control circuit ECC. The exposure evaluation calculation unit EC in the exposure control circuit ECC calculates an exposure evaluation value for each photoelectric conversion region Ia from the input image signal. At this time, the exposure evaluation value is preferably calculated based on the luminance value obtained from each image signal. Since the optimum exposure amount in each photoelectric conversion region Ia is obtained from the exposure evaluation value, the exposure determining unit ED outputs a control signal based on this, and the drive unit TC to which the control signal is input is provided for each photoelectric conversion region Ia. The charge accumulation time is set so as to optimize the exposure amount. For example, the exposure control circuit ECC arbitrarily changes the exposure time of the corresponding photoelectric conversion region for each filter region. In this way, a signal having a good S / N can be obtained as a whole, and a high-quality image can be obtained. The exposure control circuit ECC may adjust the charge accumulation time according to the color of the filter element. For example, compare the exposure time of a photoelectric conversion area that receives subject light that has passed through a filter element with a low sensitivity, such as blue, to the exposure time of a photoelectric conversion area that receives subject light that has passed through a filter element of other wavelengths. And set it longer. In this way, it is possible to obtain a signal with a balanced exposure amount for each color and to obtain a high-quality image with a good S / N. Further, when adjusting the exposure time, when the exposure time is optimally adjusted for each eye, the adjustment value may be unknown. In this case, if one color of the filter element used in at least one filter area is the same as one color of the filter element used in the other filter areas, the exposure time adjustment is performed using this. It can be carried out. That is, if the exposure control circuit ECC adjusts so that the signals from the photoelectric conversion areas that have received subject light that has passed through the filter elements of the same color in different filter areas are equalized, the photoelectric conversion corresponding to each of the different filter areas The difference in the exposure time of the area can be obtained and adjusted. By giving the same relationship to each pair of all the filter regions, it is possible to adjust the exposure amount of all the photoelectric conversion regions using a common color.

  On the other hand, the image signal output from the photoelectric conversion region Ia is input to the image processing circuit PRC including the demosaic unit DM, the signal level correction unit SS, the parallax detection unit PD, the parallax correction unit PS, and the image synthesis unit PR. . In the image processing circuit PRC, first, in the demosaic unit DM, demosaic processing is performed and separated for each color, and after color separation, the signal level is corrected (gain adjustment) by the signal level correction unit SS, and the parallax correction unit PS After the parallax correction is performed, the image is combined by the image combining unit PR and output as a reconstructed image (secondary image). Thereby, parallax correction can be performed with high accuracy. The parallax can be calculated by general template matching (SSD, SAD, etc.). A plurality of primary images are aligned based on a parallax calculated based on a signal from a photoelectric conversion region that has received subject light that has passed through filter elements of the same color in different filter regions, and a high-resolution secondary image is obtained. Can be generated. If information on the focal length of each individual optical system, image center, lens distortion coefficient, and positional relationship (translation, rotation) between the individual optical systems is obtained in advance, triangulation is performed from these coefficients and parallax values. This enables the distance measurement to the object. These coefficients may be calculated by a general stereo camera calibration method (for example, Zhang's method). It is also possible to correct the signal level for each wavelength before synthesizing the image and then synthesize the image after correcting if there is parallax. Alternatively, it can be output as independent image data for each wavelength.

  Next, a specific example of the color filter will be described. FIG. 5 is a diagram illustrating filter regions F (1) to F (9) of the color filter according to the comparative example. In this example, each filter region has a filter element Fe of one kind of color. Specifically, the filter regions F (1) and F (9) have a filter element Fe having a blue transmission wavelength band, and the filter regions F (2) and F (8) have a blue transmission wavelength band. The filter regions F (3) and F (7) have a filter element Fe whose transmission wavelength band is infrared light, and the filter regions F (4) to F (6) have a green color. It has a filter element Fe as a transmission wavelength band.

  In the case of this comparative example, the filter elements Fe of the same color are arranged symmetrically, which is convenient for parallax correction. However, there is a problem that it is difficult to increase the number of colors to four or more colors for nine photoelectric conversion regions.

  On the other hand, FIG. 6 shows filter areas F (1) to F (9) of the color filter according to the first embodiment. Each filter region has filter elements of one type or two types of colors, and the two types of filter elements are arranged in a checkered pattern. Specifically, the filter regions F (1) and F (9) have filter elements Fe whose transmission wavelength bands are blue and blue-green, and the filter regions F (2) and F (8) are blue-green and A filter element Fe having a green transmission wavelength band and filter regions F (3) and F (7) have a filter element Fe having a transmission wavelength band of red and yellow, and filter regions F (4) and F ( 6) has a filter element Fe whose transmission wavelength bands are green and yellow. The central filter region F (5) has only the filter element Fe whose transmission wavelength band is green. That is, the color filter of the present embodiment has a plurality of filter regions corresponding to a plurality of single-eye optical systems, and the plurality of filter regions have two different color filter elements that are alternately arranged two-dimensionally. And at least one set of filter regions having different filter element color combinations.

  In this example, five color filter elements Fe can be used for the nine photoelectric conversion regions, and although the size is small, the wavelength bands of visible and near infrared are passed through different color filters, and a plurality of lens elements can be used. Similar to a so-called multispectral camera that performs spectral imaging with a camera, an image with better color reproduction can be provided. The wavelengths covered by the respective colors are blue, blue-green, green, yellow, and red from the short wavelength side, and two color filter elements having adjacent transmission wavelengths are provided in the same filter region. Further, since the filter regions having the filter elements Fe of the same color are arranged at point-symmetric positions, it is effective in parallax correction.

  In addition, in the present embodiment, in the adjacent filter regions, one color of the filter element Fe is the same color. For example, the filter elements Fe (1), F (2), F (8), and F (9) have the same blue-green filter element Fe, so that the filter areas F (1), F (2), F ( 8) and the resolution is obtained by adding the image signals output from the photoelectric conversion areas that have received the light beams that have passed through the blue-green filter elements Fe of the two photoelectric conversion areas that have received the subject light that has passed F (9). It is possible to synthesize an image having a high S / N ratio. The more images of the same color, the more advantageous is the restoration of images with high resolution in the super-resolution processing. Further, for example, since the green of the filter element Fe is common between the filter regions F (2) and F (5), exposure correction can be performed, and the filter element Fe of the filter regions F (5) and F (6) can be corrected. Since green is common, exposure correction can be performed in the same manner, and since the yellow of the filter element Fe is common between the filter regions F (6) and F (3), exposure correction can be performed in the same manner. By correcting the level difference between signals of the same wavelength in this way, the exposure amount can be corrected in order, so that a balanced image signal can be obtained as a whole.

  Next, FIG. 7 shows filter regions F (1) to F (9) of the color filter according to the second embodiment. Each filter region has filter elements of two kinds of colors, and the filter elements are arranged in a checkered pattern. Specifically, the filter regions F (1), F (3), F (5), F (7), and (9) have a filter element Fe having total transmission (through) and yellow as a transmission wavelength band. The filter regions F (2) and F (8) have filter elements Fe whose transmission wavelength bands are yellow and red, and the filter regions F (4) and F (6) transmit red and infrared light. It has a filter element Fe as a band.

  In this example, four color filter elements Fe are used for nine photoelectric conversion regions, but an object can be imaged from the visible light region to the infrared region. The wavelengths covered by the respective colors are as shown in FIG. 8, and are all transmitted (W), yellow (Y), red (R), and infrared (Ir) from the short wavelength side, and the transmitted wavelengths are adjacent to each other. Two color filter elements are provided in the same filter region.

  The wavelength band of FIG. 8 is obtained from the photoelectric conversion region that receives the light beam that has passed through all the transmission filter elements Fe in the filter regions F (1), F (3), F (5), F (7), and F (9). An image signal Ws corresponding to W is obtained, and yellow filters in filter regions F (1), F (3), F (5), F (7), F (9) and F (2), F (8) An image signal Ys corresponding to the wavelength band Y of FIG. 8 is obtained from the photoelectric conversion region that receives the light beam that has passed through the element Fe, and the filter regions F (2), F (8), F (4), F ( The image signal Rs corresponding to the wavelength band R in FIG. 8 is obtained from the photoelectric conversion region that receives the light beam that has passed through the red filter element Fe in 6), and the red filters in the filter regions F (4) and F (6). An image corresponding to the wavelength band Ir in FIG. 8 is obtained from the photoelectric conversion region that receives the light flux that has passed through the element Fe. No. Irs is obtained. Similar to the above-described embodiment, since the same color image can be obtained more than the case where the filter area is a single color, an image with good resolution and S / N can be obtained, and the filter element of the same color can be obtained. Since the filter regions having Fe are arranged symmetrically, this is also effective in parallax correction.

In the present embodiment, RGB signals can be calculated from the image signals Ws, Ys, Rs, Irs by the following calculation formula.
B = Ws-Ys
G = Ys−Rs
R = Rs-Irs

  Further, as is apparent from the above calculation formula, RGB signals can be obtained as the difference between these signals by subtracting the signals in the two wavelength regions of the subject light that have all passed through the same filter region. In addition, parallax correction is unnecessary, the calculation speed is high, and problems such as false colors that can occur during parallax correction can be suppressed.

  When there are multiple filter areas with combinations of filter elements of the same color, it is possible to obtain images with a higher dynamic range by shooting with different exposure times and shooting multiple images with different brightness. it can.

  Next, FIG. 9 shows filter regions F (1) to F (9) of the color filter according to the third embodiment. Each filter region has filter elements of two kinds of colors, and the filter elements are arranged in a checkered pattern. Specifically, the filter regions F (1), F (3), F (7), and (9) have filter elements Fe whose transmission wavelength bands are blue and green, and the filter regions F (2), F (5) and F (8) have filter elements Fe whose transmission wavelength bands are green and red, and filter regions F (4) and F (6) are filters whose transmission wavelength bands are red and infrared light. It has element Fe.

  In this example, four color filter elements Fe are used for nine photoelectric conversion regions, but an object can be imaged from the visible light region to the infrared region. The wavelengths covered by the respective colors are blue, green, red, and infrared from the short wavelength side, and two color filter elements having adjacent transmission wavelengths are provided in the same filter region. Since many images of the same color are obtained as compared with the case where the filter region is a single color, an image with good resolution and S / N can be obtained, and the filter region having the filter element Fe of the same color is symmetrically arranged. Since it is arranged, it is also effective in parallax correction. Furthermore, since the color of the filter element Fe is common between different filter regions, the exposure level can be appropriately corrected as in the above-described embodiment. Furthermore, when there is a specific color whose S / N is low at the time of photoelectric conversion of the solid-state imaging device, the total number of filter elements of the specific color is compared with the filter elements of other colors without increasing the number of filter regions. By increasing the number, the color information can be obtained with sufficient sensitivity.

  FIG. 10 shows filter regions F (1) to F (9) of the color filter according to the fourth embodiment. Each filter region has filter elements of one type or two types of colors, and the two types of filter elements are arranged in a checkered pattern. Specifically, the filter regions F (1) and F (9) have filter elements Fe whose transmission wavelength bands are blue and purple, and the filter regions F (2) and F (8) are red and orange. The filter regions F (3) and F (7) have filter elements Fe whose transmission wavelength bands are yellow and green, and the filter regions F (4) and F (6 ) Has a filter element Fe whose transmission wavelength band is light blue and blue-green. Note that the central filter region F (5) has only the filter element Fe having the full transmission wavelength band.

  In this example, filter elements Fe of eight colors can be used for nine photoelectric conversion regions, and the color reproduction is excellent in the same manner as a multispectral camera, although it is small (small number of eyes). Can provide images. Two color filter elements having adjacent transmission wavelengths are provided in the same filter region. Further, since the filter areas having the filter elements Fe of the same color are arranged symmetrically, it is also effective in parallax correction. Since the central filter region F (5) transmits all wavelengths, it is preferable to perform parallax correction, exposure control, and the like with reference to an image signal obtained by photoelectrically converting subject light that has passed therethrough.

  FIG. 11 shows filter regions F (1) to F (4) of the color filter according to the fifth embodiment. In this example, single-lens lenses are arranged in two rows and two columns, which is suitable for an imaging device that images a subject at a position that can be regarded as infinitely far away, for example. When the subject is at a position that can be regarded as infinity, parallax correction is often unnecessary. In such a case, it is not necessary to provide a pair of filter regions having the same combination of filter elements. Further, the present invention can be applied to a case where an optical system that can divide subject light without parallax and form the same image in a plurality of imaging regions is used.

  In the present embodiment, the filter region F (1) has a filter element Fe whose transmission wavelength band is blue and purple, and the filter region F (2) is a filter element Fe whose transmission wavelength band is yellow and green. The filter region F (3) has a filter element Fe having light blue and blue-green transmission wavelength bands, and the filter region F (4) has a filter element Fe having red and orange transmission wavelength bands.

  In this example, filter elements Fe of eight colors can be used for the four photoelectric conversion regions, and the color reproduction is more excellent as in the case of the multispectral camera while being small (small number of eyes). Can provide images.

  FIG. 12 is a schematic diagram of an imaging apparatus according to a modification. In the example shown in FIG. 12, independent single-eye optical systems IL are arranged in 3 rows and 3 columns. In addition, independent solid-state imaging elements IE are arranged in 3 rows and 3 columns on the substrate ST. That is, the image sensor of the present embodiment includes a plurality of independent solid-state imaging elements IE corresponding to the single-eye optical system IL. Note that the filter region is formed on each solid-state image sensor IE. That is, the color filter is composed of a plurality of filter regions formed in individual elements.

  Each of the above embodiments has a single-row optical system with 3 rows and 3 columns or 2 rows and 2 columns. However, the present invention is not limited to this, and has more than 3 rows or 3 columns. You may do it. For example, it may be 3 rows and 5 columns, or 5 rows and 5 columns. FIG. 13 shows an example of an array of filter elements in the case of 5 rows and 5 columns. In the example shown in FIG. 13, filter regions F (1) to F (25) are provided in the color filter. Each filter region has filter elements of one type or two types of colors, and the two types of filter elements are arranged in a checkered pattern. Specifically, the filter regions F (1), F (7), F (19), and F (25) have filter elements Fe whose transmission wavelength bands are blue and blue-green, and the filter region F (2). F (4) and F (22) to F (24) have filter elements Fe whose transmission wavelength bands are blue green and green, and filter regions F (5), F (9), F (17), F (21) has a filter element Fe whose transmission wavelength bands are red and yellow, and has filter regions F (6), F (10), F (11), F (15), F (16), F ( 20) has a filter element Fe whose transmission wavelength bands are green and yellow. The filter regions F (8), F (12) to F (14), F (18) provided in a cross shape at the center have only the filter element Fe whose transmission wavelength band is green.

    Further, instead of the checkerboard pattern, filter elements having the same opening shape of two different colors may be arranged adjacent to each other in each filter region. The filter elements are preferably arranged corresponding to each pixel in the photoelectric conversion region, but may be arranged so as to be alternately replaced for each of a plurality of pixels within a range not impairing the image quality.

CF color filter CG cover glass DM demosaic unit DU imaging device EC exposure evaluation calculation unit ED exposure determination unit F IR cut filter Fe filter element HLD mirror frame HLDb peripheral wall HLDb object side wall I image sensor Ia photoelectric conversion region IL single-eye optical system L1 1st lens L1f Flange part L2 2nd lens L2f Flange part LA1 1st lens array LA2 2nd lens array OT Output part PD Parallax detection part PR Image composition part PS Parallax correction part SH1 1st light shielding member SH2 2nd light shielding member SH3 2nd 3 light shielding member SH4 4th light shielding member SS signal level correction | amendment part ST board | substrate TC drive part ECC exposure control circuit PRC image processing circuit

Claims (10)

A plurality of individual optical systems with different optical axes,
An image sensor including a plurality of photoelectric conversion regions in which one subject image is formed by the plurality of single-eye optical systems;
A color filter disposed between the plurality of single-eye optical systems and the plurality of photoelectric conversion regions;
An image processing circuit that forms a reconstructed image by synthesizing a plurality of image signals output from the plurality of photoelectric conversion regions;
The color filter has a plurality of filter regions corresponding to the plurality of single-eye optical systems, and the plurality of filter regions include two different color filter elements arranged alternately in two dimensions, An image input apparatus comprising at least one set of filter regions having different combinations of colors of the filter elements.   The image input apparatus according to claim 1, wherein filter elements having the same shape of two different colors are arranged in a checkered pattern in a filter region including the filter elements of the two colors.   One color of a filter element included in at least one filter region of at least one set of filter regions including the two color filter elements is the same as one color of a filter element included in at least one other filter region. And the image processing circuit adds the signals from the plurality of photoelectric conversion regions that have received the subject light that has passed through the filter elements of the same color in different filter regions, thereby processing the image signals having the same color information. The image input device according to claim 1, wherein the image input device generates the image input device.   The image processing circuit includes, for a filter region in which filter elements of different first and second colors are arranged, a signal from a photoelectric conversion region that has received subject light that has passed through the filter element of the first color, and By subtracting the signal from the photoelectric conversion region that has received the subject light that has passed through the filter element of the second color, an image signal having information on a third color different from the first color and the second color is obtained. The image input device according to claim 1, wherein the image input device is formed.   The image input apparatus according to claim 1, further comprising an exposure control circuit that adjusts a charge accumulation time independently for each photoelectric conversion region by controlling the image sensor.   The image input apparatus according to claim 5, wherein the exposure control circuit adjusts a charge accumulation time according to a color of the filter element through which subject light passes.   One color of filter elements used in at least one filter area among the plurality of filter areas is the same as one color of filter elements used in at least one other filter area, and the exposure control circuit includes at least one set. The difference in charge accumulation time between the photoelectric conversion regions of the at least one set of filter regions is adjusted based on signals from a plurality of photoelectric conversion regions that have received subject light that has passed through the filter elements of the same color in the filter regions of The image input apparatus according to claim 5, wherein the image input apparatus is an image input apparatus.   The image input apparatus according to claim 1, further comprising a lens array in which a plurality of lenses corresponding to the plurality of single-eye optical systems are integrally formed.   The image input device according to claim 1, wherein the plurality of photoelectric conversion regions are integrally formed on the same substrate.   Of the plurality of filter regions, one color of a filter element used in at least one filter region is the same as one color of a filter element used in at least one other filter region, and the image processing circuit includes at least one set. Generated between the plurality of single-eye optical systems corresponding to the at least one set of filter regions based on signals from the plurality of photoelectric conversion regions that have received subject light that has passed through the filter elements of the same color in the filter region The image input device according to claim 1, further comprising a parallax correction unit configured to correct parallax of the subject image.

Images