JP2009267555A - Imaging apparatus, and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology capable of easily photographing an image with a wide sensitivity range. <P>SOLUTION: First or third color image data are generated which are different in exposure time. Each image data is generated in a form of the color image data expressed by one of the components of RGB by each pixel. Such color image data are treated as the image data of the single component. Thus, after the data are compressed and synthesized to be single component color image data, the obtained synthetic color image data are output as the color image data of a subject. Consequently, the color image data photographed three exposure times are easily output as the case of normal color image data, thereby easily outputting the image with the wide sensitivity range. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for capturing an image by an electronic method, and more particularly to a technique for capturing an image having a wide dynamic range by combining images captured at a plurality of exposure times.

  An image sensor capable of taking out an image as an electronic signal has been developed, and is now widely used by being incorporated in a digital camera or a video camera. The sensitivity range (dynamic range) of this image sensor is still very narrow compared to the sensitivity range of human vision, so even a subject that can be easily recognized by human vision is bright when photographed with a digital camera or the like. There is a case where the image is a white portion and the dark portion is a black image. In such a subject, if the exposure time is shortened so that the bright part is projected, the dark part becomes an all-black image, and conversely, if the exposure time is lengthened so that the dark part is projected, the bright part Will all appear white.

  Therefore, by setting the exposure time in multiple stages (for example, three stages, long, medium, and short) and combining images taken at each exposure time, an image with a wide sensitivity range that can be compared with human vision is obtained. A technique to be obtained has been proposed (Patent Document 1). In addition, in the technique for compositing images obtained with multiple exposure times in this way, if a subject with motion is photographed, the subject will appear blurred in an image with a long exposure time. It will be blurred. Therefore, it is determined whether or not the subject is moving in an image with a long exposure time, and if it is determined that the subject is moving, the combination of the images is abandoned and only an image with a short exposure time is used. This technique has also been proposed (Patent Document 2).

Japanese Patent No. 2522015 JP 2000-050151 A

  However, there has been a problem that a technique for photographing an image with a wide sensitivity range by combining images having such multi-stage exposure times is not an imaging technique that is sufficiently easy to use. This is because when shooting a moving subject, if the image is shot with multiple exposure times in order to expand the sensitivity range, the subject will be blurred in an image with a long exposure time, so a good composite image is obtained. I can't. To avoid this, it is determined whether or not the subject is blurred, and if it is blurred, only an image with a short exposure time is adopted. And the sensitivity range cannot be expanded. Therefore, the current technology must allow the image quality to deteriorate due to subject blurring, or otherwise allow the image sensitivity range to remain narrow. There was a problem that the technology was not sufficiently easy to use.

  The present invention has been made in order to solve the above-described problems of the prior art, and aims to provide an imaging technique that can capture an image with a wide sensitivity range and is sufficiently easy to use. To do.

In order to solve at least a part of the problems described above, the imaging apparatus of the present invention employs the following configuration. That is,
An image of a subject is exposed on an imaging surface in which a plurality of photoelectric elements that convert received light into charges and accumulate are arranged in a matrix, and the R component, G component, and B component that constitute the three primary colors of light An imaging device that captures a color image of the subject by reading a corresponding charge amount,
The first color image data, the second color image data, and the third color image data having different exposure times are obtained by photographing the subject with different exposure times on the imaging surface in a plurality of stages. Color image data generating means for generating;
Synthesized color image data generating means for generating synthesized color image data by synthesizing the first color image data, the second color image data, and the third color image data into one color image data;
Color image data output means for outputting the synthesized color image data,
The color image data generation unit reads each charge amount corresponding to any one of the R component, the G component, and the B component set differently for each photoelectric element on the imaging surface. Means for generating the first to third color image data in an expression format in which a pixel has any one component of RGB;
The composite color image data generation means generates the composite color image data by replacing the first to third color image data with single-component image data and compressing the data as one color image data. It is a summary.

In addition, the imaging method of the present invention corresponding to the imaging device described above,
An image of a subject is exposed on an imaging surface in which a plurality of photoelectric elements that convert received light into charges and accumulate are arranged in a matrix, and the R component, G component, and B component that constitute the three primary colors of light An imaging method for capturing a color image of the subject by reading a corresponding charge amount,
The first color image data, the second color image data, and the third color image data having different exposure times are obtained by photographing the subject with different exposure times on the imaging surface in a plurality of stages. A first step of generating;
A second step of generating combined color image data by combining the first color image data, the second color image data, and the third color image data into one color image data;
A third step of outputting the combined color image data,
In the first step of generating the color image data, a charge amount corresponding to any one of the R component, G component, and B component set differently for each photoelectric element on the imaging surface is calculated. A step of generating the first to third color image data in an expression format in which each pixel has any one component of RGB by reading;
In the second step of generating the composite color image data, the first to third color image data are respectively read as single-component image data and compressed as one color image data. The gist is that it is a process of generating data.

  In the imaging apparatus and imaging method of the present invention, the first to third color image data having different exposure times are generated, and then these three color image data are synthesized into one color image data, and then obtained. The combined color image data is output as color image data of the subject. Here, the first to third color image data includes each component of RGB, but when viewed for each pixel, the image data has only one component. Therefore, the first to third color image data can be handled as image data of RGB components constituting the RGB color image data and compressed as one color image data. The compressed color image data obtained in this way is output as synthesized color image data.

  By doing this, it is possible to output color image data photographed with three exposure times as one compressed composite color image data. Since the color image data for each exposure time has only one component for each pixel, the amount of data is the same as that of single-component image data even if it is color image data. For this reason, even the entire color image data photographed with three exposure times has only the same data amount as general color image data having three RGB components for each pixel. In addition, since image data can be handled in the same way as single-component image data, it can be compressed in the same way as general color image data compression methods. Further, the data amount can be further reduced, and at the same time, the three color image data can be handled as one image data. As a result, it is possible to output color image data photographed at three exposure times with the same ease as normal color image data, so that an image with a wide sensitivity range can be easily output.

  In the image pickup apparatus of the present invention, when the first to third color image data is compressed to generate the combined color image data, the following may be performed. First, color image data with the shortest exposure time is read as G component image data, color image data with the longest exposure time is read as B component image data, and the remaining color image data is read as R component image data. . Then, these three color image data may be handled as one color image data and compressed.

  The shorter the exposure time, the darker the image. In human vision, the G color tends to feel brightest, and then the R color and B color tend to feel brighter. Therefore, color image data captured with the shortest exposure time is treated as a G component, color image data captured with an intermediate exposure time is treated as an R component, and color image data captured with the longest exposure time is defined as a B component. If handled, even when they are displayed as normal RGB color image data, a certain color component can be recognized as a relatively natural image without being emphasized over other color components. For this reason, the content of the photographed image can be easily confirmed, which is preferable.

  In the above-described imaging apparatus of the present invention, the first to third color image data obtained at each exposure time may be compressed as follows. First, since the first to third color image data is image data having any one of RGB components for each pixel, the first to third color image data can be obtained by rearranging the pixel positions so that the same components are gathered together. Alternatively, the third color image data is converted into each. Subsequently, the first to third color image data in which the pixel positions are rearranged may be compressed as one color image data by treating it as if it were single component image data.

  The first to third color image data are image data containing a lot of high-frequency components because data of different color components are recorded in adjacent pixels. Such high-frequency components are likely to be lost when color image data is compressed, and accordingly, the image quality tends to deteriorate. On the other hand, if the pixel positions are rearranged, the data recorded in the adjacent pixels becomes the data of the same color component, so that the high frequency component included in the image data can be greatly reduced. As a result, it is possible to compress and output color image data without deteriorating image quality.

  In the imaging apparatus of the present invention described above, the first to third color image data may be compressed as follows. First, G component image data is generated by collecting G component pixels from the first to third color image data. Similarly, for the R component and the B component, R component or B component pixels are collected from the first to third color image data to generate R component image data and B component image data. The G component image data, R component image data, and B component image data generated in this way may be treated as the G component, R component, and B component of one color image data and compressed.

  In this way, for the following reason, it is possible to obtain a high-quality color image while minimizing deterioration in image quality when color image data is compressed. First, since the same components are collected, high frequency components included in the image data can be reduced, so that deterioration in image quality due to compression of the image data can be suppressed. In addition, it is known that the G component has the largest influence on the image quality among the G component, the R component, and the B component. In correspondence with this, the technology for compressing color image data There is a tendency to compress the information contained in the G component so as to preserve it as much as possible. Therefore, if the G component that has the greatest influence on the image quality is extracted from the first to third color image data and compressed as the G component of the color image data, the deterioration of the image quality is minimized. Thus, the first to third color image data can be compressed.

Furthermore, the present invention can be realized using a computer by causing a computer to read a program for realizing the above-described imaging method and executing a predetermined function. Therefore, the present invention also includes the following aspects as a program. That is, the program of the present invention corresponding to the imaging method described above is
An image of a subject is exposed on an imaging surface in which a plurality of photoelectric elements that convert received light into charges and accumulate are arranged in a matrix, and the R component, G component, and B component that constitute the three primary colors of light A program for realizing a method of taking a color image of the subject by reading a corresponding charge amount using a computer,
The first color image data, the second color image data, and the third color image data having different exposure times are obtained by photographing the subject with different exposure times on the imaging surface in a plurality of stages. A first function to generate;
A second function for generating combined color image data by combining the first color image data, the second color image data, and the third color image data into one color image data;
A third function for outputting the synthesized color image data and a computer, and
The first function of generating the color image data is to calculate a charge amount corresponding to any one of the R component, G component, and B component set differently for each photoelectric element on the imaging surface. A function of generating the first to third color image data in an expression format in which each pixel has any one component of RGB by reading;
The second function of generating the composite color image data is to replace the first to third color image data with single-component image data and compress the single color image data as one color image data. The gist is that it is a function to generate data.

  If this program is read into a computer and the above functions are realized, an image with a wide sensitivity range can be easily taken.

Hereinafter, in order to clarify the contents of the present invention described above, examples will be described in the following order.
A. Device configuration:
B. Imaging processing:
C. Development processing:
D. Variations:
D-1. First modification:
D-2. Second modification:

A. Device configuration :
FIG. 1 is an explanatory diagram showing a rough configuration of the imaging apparatus 100 of the present embodiment. As illustrated, the imaging apparatus 100 of the present embodiment includes an optical system 10 including a plurality of lens groups, and an imaging unit 20 that converts an image of a subject formed by the optical system 10 into an electrical signal. An image processing unit 30 that generates color image data by receiving an electrical signal obtained by the imaging unit 20 and performs predetermined image processing, an image output unit 40 that outputs the generated color image data, and the like Has been.

  The imaging unit 20 is equipped with an image sensor 24 in which fine photoelectric elements that convert received light into electric charges and store them are arranged in a two-dimensional matrix. A color filter array 22 is provided on the front side of the image sensor 24. The color filter array 22 is configured by arranging R (red) fine color filters, G (green) fine color filters, and B (blue) fine color filters in a predetermined arrangement. Each of the photoelectric elements of the image sensor 24 can receive only light of one color component of R color, G color, and B color. Therefore, the image sensor 24 obtains mosaic color image data in which RGB color components are arranged in a predetermined arrangement and each pixel has any one color component. The arrangement of the RGB color components will be described later.

  The image processing unit 30 mounted on the imaging apparatus 100 of the present embodiment generates color image data of a captured image by performing predetermined image processing on the color image data obtained by the image sensor 24. To do. As illustrated, the image processing unit 30 includes an image reading unit 32 that reads an image from the image sensor 24. As will be described in detail later, the image reading unit 32 can read out three images with different exposure times (charge accumulation times in the image sensor 24). The first memory 34, the second memory 35, and the third memory 36 are each temporarily stored. Subsequently, these three images are read out by the image compression unit 38 and compressed as one color image data. The image compression unit 38 of this embodiment uses a JPEG compression module that compresses color image data (RGB color image data) having three RGB components, and the image in the first memory 34 is RGB color image data. As the G component, the image in the second memory 35 is read as the R component, and the image in the third memory 36 is read as the B component to the image compression unit 38 and compressed as one color image data.

  The JPEG-compressed color image data is supplied to the image output unit 40 and output to the external output terminal 42 or the memory card 44. In the imaging apparatus 100 according to the present exemplary embodiment, the image compression unit 38 is described as being configured by a JPEG compression module. However, if it is possible to compress three RGB components as one color image data, JPEG is possible. Not only compression but also a module that performs compression by other methods can be used.

  FIG. 2 is an explanatory diagram showing a state in which fine RGB color filters are arranged in a predetermined arrangement on the color filter array 22. In the color filter array 22, a portion with fine oblique lines is a B color filter, a portion with coarse oblique lines is an R color filter, and a portion not shaded is a G color filter. It is. As shown in the drawing, the RGB color filters are repeatedly arranged in a fixed pattern in the color filter array 22. The size of each color filter is the same as that of each photoelectric element constituting the image sensor 24, and the position of each color filter is configured to match the position of each photoelectric element. . Therefore, the image sensor 24 obtains image data in which RGB components are arranged in a mosaic pattern with the same pattern as the arrangement of the RGB color filters.

  Since the mosaic image data obtained in this way has only one of RGB components for each pixel, it can be handled in the same manner as image data of monochrome components (monochrome image data). it can. In addition, image data that can be converted into RGB color image data by interpolating the color components that are missing in each pixel from surrounding pixels. In the imaging apparatus 100 of the present embodiment, paying attention to this point, three color images photographed by changing the exposure time in three stages are stored in the form of mosaic image data, and these are stored as R component image data. By compressing as G component image data and B component image data, an image with a wide sensitivity range can be taken. In addition, there is no need to worry about the deterioration of image quality due to blurring of the subject and the fact that the sensitivity range may not be expanded. For this reason, it is a technology that is very easy to use. Hereinafter, a process for capturing an image using the imaging apparatus 100 of the present embodiment and a process for developing the captured image will be described.

B. Imaging processing:
FIG. 3 is a flowchart illustrating the flow of an imaging process performed when an image is captured by the imaging apparatus 100 according to the present exemplary embodiment. Such a process is a process executed by a CPU (not shown) that controls the entire imaging apparatus 100. In the imaging process, first, an exposure time setting is acquired (step S100). This exposure time is an exposure time set according to the brightness of the image to be shot, and may be set by the user of the imaging device 100 operating an operation button (not shown). Alternatively, the imaging apparatus 100 may automatically set.

  Next, a three-stage exposure time is determined according to the acquired exposure time (step S102). That is, the exposure time set in step S100 according to the image to be photographed is determined as an intermediate exposure time (medium exposure time), and a predetermined ratio of the exposure time to the intermediate exposure time is set to the short exposure time. Time and long exposure time are determined.

  Then, when the short exposure time has elapsed since the shutter was pressed, the image data is read from the image sensor 24 and stored in the first memory 34 (step S104). Subsequently, the image data read when the medium exposure time has elapsed is stored in the second memory 35 (step S106), and the image data read when the long exposure time has elapsed is stored in the third memory 36 (step S108). . As described above with reference to FIG. 2, the image data read out in this way is color image data in which the RGB components are combined in a mosaic pattern, and each pixel has only one component. Not. For this reason, it can be handled in the same manner as image data of a single color component.

  Therefore, the three color image data stored in the first memory 34 to the third memory 36 are regarded as the G component, R component, and B component of the RGB color image data, respectively, and JPEG compression is performed (step S110). This processing is performed by supplying the image data of the first memory 34 to Gch of the module for JPEG compression of RGB color image data, the image data of the second memory 35 to Rch, and the image data of the third memory 36 to Bch. Can be easily realized.

  After JPEG compression of the short exposure time, medium exposure time, and long exposure time color image data, the obtained JPEG color image data is output to the external output terminal 42 or the memory card 44 as color image data of the photographed image. (Step S112), and the imaging process of FIG. The color image data output as described above includes three color image data with different exposure times. Each color image data is image data in which RGB components are arranged in a mosaic pattern. Therefore, the three color image data has only the data amount of one normal color image data. In addition, since JPEG compression is performed, the data amount is further reduced, and the three color image data can be handled as integrated image data. If necessary, it is possible to obtain a color image with a wide sensitivity range by performing development processing as described below.

C. Development processing:
FIG. 4 is a flowchart showing a flow of processing for developing color image data captured by the imaging apparatus 100 of the present embodiment and outputting a color image. Such processing may be performed using a personal computer or the like, or may be executed in the printer using a computer mounted on the printer.

  In the development process, first, JPEG color image data photographed by the imaging apparatus 100 of the present embodiment is read (step S200). The JPEG color image data may be read directly from the external output terminal 42 of the imaging apparatus 100, or data stored in the memory card 44 or the like may be read.

  Subsequently, the read JPEG color image data is expanded (step S202). JPEG color image data can be immediately developed by using an existing dedicated module. As described above with reference to FIG. 3, the JPEG color image data output from the imaging apparatus 100 treats short exposure time image data as G component image data, and medium exposure time image data as R component data. This is JPEG-compressed image data that is treated as image data and image data having a long exposure time is treated as B-component image data. Accordingly, in the color image data obtained by developing in step S202, the G component corresponds to the image data with the short exposure time, the R component corresponds to the image data with the medium exposure time, and the B component has the long exposure time. It corresponds to data. Also, these image data are color image data in which any one component of RGB is arranged in a mosaic pattern.

  Next, the color image data obtained by the expansion is displayed on a monitor screen of a personal computer or a printer (step S204). The RGB components of the color image data obtained by the development are mosaic color image data photographed at different exposure times. The short exposure time image data is the G component, and the medium exposure time image data is the image data. If the image data of the R component and the long exposure time are handled as the B component, it is possible to confirm the content of the image just by displaying it on the monitor in the same way as normal RGB color image data.

  Then, the user confirms whether or not to output the image displayed on the monitor (step S206), and when the user selects to output (step S206: yes), the color image output described below is performed. Processing is started (step S250). On the other hand, when it is selected not to output (step S06: no), the color image output process is skipped.

  FIG. 5 is a flowchart showing the flow of color image output processing. In the color image output process, the G component of the color image data obtained by the development is stored as short exposure time image data (step S252), and the R component is stored as medium exposure time image data (step S254). , B component is stored as image data of a long exposure time (step S256). The image data stored here is mosaic color image data in which any one component of RGB is stored in each pixel.

  Next, a composite image is generated by combining the image data of these exposure times (step S258). A well-known method can be used to synthesize image data for each exposure time to obtain a composite image. In the present embodiment, image data is synthesized as follows. First, the image data corresponding to the long exposure time is converted by multiplying the image data of the short exposure time by a proportional coefficient of (long exposure time / short exposure time). Similarly, the medium exposure time image data is multiplied by a proportional coefficient of (long exposure time / medium exposure time) to be converted into image data corresponding to the long exposure time. Next, the converted short exposure time image data, the converted medium exposure time image data, and the long exposure time image data are added.

  The image data of the composite image obtained in this way is image data with an expanded sensitivity range. This is because even if the image data of the short exposure time is crushed in black and the gradation information is lost, if the gradation information remains in the image data of the medium exposure time or the long exposure time, these image data The gradation information in this area can be taken over by adding. Similarly, even if the image data of the long exposure time is white and the gradation information does not remain, if the gradation information remains in the image data of the short exposure time or the medium exposure time, these image data are By adding, the gradation information in this area can be taken over. Eventually, it is possible to obtain image data in which gradation information is stored over a wide sensitivity range, from a bright area that must be displayed with a short exposure time to a dark area that must be displayed with a long exposure time. Become.

  However, since the maximum gradation value exceeds the gradation value 255 in the image data obtained by such addition, the gradation value is finally normalized so that it falls within the gradation value 255. The gradation value of the image data obtained by the addition becomes maximum when the image data of any exposure time has a gradation value of 255. Therefore, the maximum gradation value is 255 × (long exposure time. / Short exposure time) + 255 × (long exposure time / short exposure time) +255. Therefore, the image data obtained by adding the images for each exposure time is divided by the maximum gradation value, and then multiplied by the gradation value 255. Thus, the image data normalized in the range of gradation values from 0 to 255 is used as the image data of the composite image.

  FIG. 6 is an explanatory diagram conceptually showing how a short exposure time image, a medium exposure time image, and a long exposure time image are combined to generate a combined image having a wide sensitivity range. In an image with a short exposure time, a bright outdoor video is clearly shown, but a dark indoor video is crushed black. Moreover, in the image of the medium exposure time, the image of the person who receives the light from the window is shown, but the other indoor images remain crushed in black, and the outdoor image has been blown halfway. Furthermore, in the image with a long exposure time, a dark room image is shown, but a person and an outdoor image are flying white. By synthesizing these images by the method described above, it is possible to obtain a synthesized image in which a video is clearly reflected from a bright outdoor room to a dark room.

  In step S258 of the color image output process shown in FIG. 5, a process for generating composite image data from the image data obtained in the short exposure time, the medium exposure time, and the long exposure time as described above is performed. As described above, the short exposure time image data, the medium exposure time image data, and the long exposure time image data obtained by developing the JPEG color image data have a mosaic of any component of RGB in each pixel. It is image data arranged in a shape. Therefore, the image data obtained by combining the image data of each exposure time is also mosaic color image data in which only one of RGB components is set for each pixel. Therefore, for each pixel of the combined image data, the missing color component is calculated by interpolating from the surrounding pixel components (step S260). As a result, RGB color image data in which RGB color components are set for all pixels can be obtained.

  A color image is output using the color image data thus obtained (step S262). As a mode of outputting a color image, various modes such as a mode of displaying on a monitor screen and a mode of printing with a printer can be used.

  Subsequently, the output color image is confirmed, and whether or not to refer to the pre-combination image is specified for an image device (for example, a personal computer or a printer) performing the development process. For example, when a moving subject is photographed and the subject is blurred, it is specified that the pre-combination image is referred to. Then, the image device that has received this designation determines whether or not it is necessary to refer to the image before composition (step S264). If it is determined that it is not necessary to refer to the image before composition (step S264: no). ), The color image output process of FIG. 5 is exited, and the process returns to the development process of FIG.

  On the other hand, when it is determined that it is necessary to refer to an image before synthesis (step S264: yes), an image having an exposure time to be referred to is designated (step S266). That is, the JPEG color image data handled in the present embodiment includes color image data obtained at each exposure time. By developing the JPEG color image data, color image data having a short exposure time, medium exposure can be obtained. Time color image data and long exposure time color image data are obtained. Therefore, it is designated which color image data of which exposure time is to be referred to.

  Subsequently, the missing color component is interpolated for the color image data of the designated exposure time (step S268). That is, image data for each exposure time obtained by developing JPEG color image data is mosaic image data in which only one color component of RGB is set for each pixel. Therefore, the missing color component for each pixel is interpolated from surrounding pixels and converted to RGB color image data in which each RGB color component is aligned. Then, based on the obtained RGB color image data, for example, after outputting a color image on a monitor screen or a printing medium (step S270), the color image output process of FIG. 5 is exited, and the development process of FIG. Return to. Of course, the color image output in this way may be confirmed so that an image with a different exposure time can be designated. As described above, in this embodiment, the color image data obtained at each exposure time is compressed into the JPEG color image data, so that the image obtained by the synthesis is not a satisfactory image. It is possible to select a satisfactory image from the exposure time images used for the composition.

  As shown in FIG. 4, in the development process, when returning from the color image output process described above, the user is asked whether or not to develop another image (step S208). If the user designates that another image is to be developed (step S208: yes), the process returns to step S200 again, and after reading new JPEG color image data, the above-described series of processing is performed. On the other hand, when it is designated not to develop another image (step S208: no), the developing process shown in FIG. 4 is terminated.

  As described above, when an image is captured using the imaging apparatus 100 of the present embodiment, color image data having a short exposure time, a medium exposure time, and a long exposure time is automatically captured, and one of these is captured. It is collected in color image data and output in a compressed state. The color image data for each exposure time has RGB components, but only one component is set for each pixel. Therefore, the three image data obtained for each short, medium and long exposure time are set. Even if the color image data is combined, the amount of data is only one normal RGB color image data. Moreover, since the three color image data obtained at each exposure time are regarded as the R component, G component, and B component of the color image data and compressed as one color image data, the data amount is It becomes even smaller. In addition, since the existing compression module used for JPEG compression or the like can be used as it is, compression can be performed easily and quickly without using special parts. As a result, it is possible to output color image data as easily as normal color image data, even though three color image data having different exposure times are included.

  Further, when developing such color image data photographed using the imaging apparatus 100, first, the compressed color image data is developed, and a short exposure time, a medium exposure time, and a long exposure time are developed. To color image data. Although the three color image data are compressed into one image data, a general method can be used for the compression method itself. Therefore, a general decompression module is used for decompressing the color image data. And can be done easily and quickly. Thereafter, an image having a wide sensitivity range is synthesized using color image data of short exposure time, medium exposure time, and long exposure time obtained by thawing. At this time, even if the subject is moving and becomes a blurred image, the image data of the short, medium, and long exposure times remains as they are, so that appropriate image data may be used from the image data. .

  As described above, the imaging apparatus 100 according to the present exemplary embodiment can shoot without worrying about whether or not the subject is blurred due to movement or the like. For this reason, it is possible to easily shoot an image with a wide sensitivity range without any difference from the case of shooting a general image.

  Further, as described above, in the imaging apparatus 100 according to the present embodiment, when compressing the image data of the short, medium and long exposure times, the image data of the short exposure time is regarded as the image data of the G component, and the medium exposure time. These image data are regarded as R component image data, and image data with a long exposure time is regarded as B component image data and compressed. In this way, it is possible to confirm the outline of the captured image simply by expanding the compressed color image data and then displaying it on the monitor as it is. For this reason, it is possible to avoid a situation in which an image in which a subject is blurred is synthesized unnecessarily.

D. Modified example:
There are several variations of the imaging apparatus 100 described above. Below, the imaging device 100 of these modified examples is demonstrated easily.

D-1. First modification:
In the above-described embodiments, image data having a short exposure time, image data having a medium exposure time, and image data having a long exposure time are described as being subjected to JPEG compression as they are. However, the image data for each exposure time is image data in which the RGB components are arranged in a mosaic as shown in FIG. Naturally, the gradation value changes abruptly between adjacent pixels, and the image data contains a lot of high frequency components. However, in a compression technique such as JPEG, high frequency components are easily lost, so that there is a possibility that the image quality deteriorates when compressing image data obtained in each short, medium and long exposure time. Therefore, instead of compressing the image data of each exposure time as it is, the image data may be compressed after rearranging the pixel positions so that the same color components are collected.

  FIG. 7 is an explanatory diagram showing a first modification example in which image data is compressed after collecting pixels having the same color component by rearranging pixel positions. As shown in the figure, the image data for each exposure time is configured by arranging G component (and g component) pixels, R component pixels, and B component pixels in a fixed pattern. Therefore, G component pixels are collected in the upper left half area, B component pixels are collected in the upper right half area, R component pixels are gathered in the lower left half area, and g component pixels are gathered in the lower right half area. I will collect it in the area. In this way, since pixels of the same color component are continuous in each region, the high frequency component of the image can be greatly reduced. The image data obtained by rearranging the pixel positions in this way is compressed. As described above, since the high frequency components are greatly reduced by rearranging the pixel positions, the image data can be compressed without degrading the image quality.

  Further, when developing the compressed image data, the pixel positions may be rearranged in the opposite direction to that at the time of compression. That is, when the compressed image data is expanded, the image data immediately before compression (image data in which the G component, B component, R component, and g component are collected in the respective regions) shown in FIG. 7 is obtained. Therefore, if the pixels in each region are rearranged in the direction opposite to that at the time of compression, thereafter, it is possible to develop a color image in the same manner as the development processing shown in FIG. As a result, it is possible to easily output a high-quality color image with a wide sensitivity range.

D-2. Second modification:
Further, it is known that the G component has the largest influence on the image quality among the RGB color components. For this reason, the compression technique tends to compress the G component information as much as possible. Therefore, image data obtained by collecting G component, R component, and B component from image data for each exposure time may be compressed as G component, R component, and B component of color image data, respectively.

  FIG. 8 is an explanatory view showing a second modification in which image data extracted for each component from image data at each exposure time is compressed as G component, R component, and B component of color image data. In the illustrated example, first, by rearranging the pixel positions in the image data of each exposure time, the G component pixel, the B component pixel, the R component pixel, and the g component pixel are respectively displayed in the respective regions. To collect. In the example shown in FIG. 8, the image data of the short exposure time includes the G component image data area indicated as G_s, the B component image data area indicated as B_s, and the R data indicated as R_s. It is rearranged in the area of the component image data. Similarly, image data of medium exposure time or long exposure time is rearranged in each component area indicated as G_m, B_m, R_m in the figure, or in each component area indicated as G_l, B_l, R_l in the figure. Has been.

  From each image data rearranged in each component in this way, G component (ie G_s, G_m, G_l) region data, R component (ie R_s, R_m, R_l) region data, B component (ie, B_s, B_m, and B_l) data are extracted to generate three pieces of image data. The image data collected from the G component, the image data collected from the R component, and the image data collected from the B component are compressed as the G component, R component, and B component of the color image data, respectively.

  As described above, it is known that the G component has a larger influence on the image quality than the R component and the B component. Correspondingly, the compression of color image data preserves G component information as much as possible. Often executed as Therefore, if the G component that greatly affects the image quality is extracted from the image data obtained at the short, medium, and long exposure times, and is compressed as the G component of the color image data, information that is lost along with the compression is extracted. The image data can be compressed while being suppressed as much as possible.

  Also in the second modified example, when developing the compressed image data, it is only necessary to perform processing in the opposite direction to that at the time of compression. That is, the G component image data obtained by developing the compressed image data is image data obtained by collecting the G components of the short, medium, and long exposure time image data. Similarly, R component image data obtained by developing compressed image data is image data obtained by collecting R components of image data of short, medium, and long exposure times. The B component image data obtained by the development is image data obtained by collecting the B components of the image data of short, medium and long exposure times. Therefore, these G component, R component, and B component image data are distributed to short exposure time image data, medium exposure time image data, and long exposure time image data. As a result, image data having a short exposure time, image data having a medium exposure time, and image data having a long exposure time are obtained. Thereafter, a color image can be developed in the same manner as the development processing shown in FIG. It becomes possible. In the G component of the image data for each exposure time, the loss of information due to compression is suppressed as much as possible, and the G component greatly affects the image quality compared to other color components, so the sensitivity range is wide and the image quality is high. It is possible to output a simple color image.

  In the above description, the pixel positions are rearranged in the image data of the short, medium, and long exposure times, and the G component pixel, the B component pixel, the R component pixel, and the g component pixel are assigned to each region. In the above description, the data of each component is collected from the image data after being converted into the image data collected. However, the image data of each component is directly extracted by extracting the data of the G component, the B component, and the R component from the image data of the short, medium and long exposure times without rearranging the pixel positions in the image data. It may be generated.

  The various imaging devices 100 have been described above, but the present invention is not limited to all the embodiments described above, and can be implemented in various modes without departing from the scope of the invention.

It is explanatory drawing which showed the rough structure of the imaging device of a present Example. It is explanatory drawing which showed the color filter array comprised by arrange | positioning the RGB fine color filter by the predetermined arrangement | positioning. It is the flowchart which showed the flow of the imaging process performed at the time of imaging | photography with the imaging device of a present Example. 3 is a flowchart showing a flow of processing for developing color image data captured by the imaging apparatus of the present embodiment and outputting a color image. It is a flowchart which shows the flow of a color image output process. It is explanatory drawing which showed notionally the mode that the image of a short-medium-long exposure time was synthesize | combined and the composite image which has a wide sensitivity range is produced | generated. It is explanatory drawing which showed the 1st modification which compresses image data after performing rearrangement of a pixel position. It is explanatory drawing which showed the 2nd modification compressed after extracting the image data for every component from the image data of each exposure time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Optical system, 20 ... Imaging part, 22 ... Color filter array,
24 ... Image sensor, 30 ... Image processing unit, 34 ... First memory,
35 ... 2nd memory, 36 ... 3rd memory, 38 ... Image compression part,
40 ... Image output unit, 42 ... External output terminal, 44 ... Memory card,
100: Imaging device

Claims (5)

An image of a subject is exposed on an imaging surface in which a plurality of photoelectric elements that convert received light into charges and accumulate are arranged in a matrix, and the R component, G component, and B component that constitute the three primary colors of light An imaging device that captures a color image of the subject by reading a corresponding charge amount,
The first color image data, the second color image data, and the third color image data having different exposure times are obtained by photographing the subject with different exposure times on the imaging surface in a plurality of stages. Color image data generating means for generating;
Synthesized color image data generating means for generating synthesized color image data by synthesizing the first color image data, the second color image data, and the third color image data into one color image data;
Color image data output means for outputting the synthesized color image data,
The color image data generation unit reads each charge amount corresponding to any one of the R component, the G component, and the B component set differently for each photoelectric element on the imaging surface. Means for generating the first to third color image data in an expression format in which a pixel has any one component of RGB;
The composite color image data generation means generates the composite color image data by replacing the first to third color image data with single-component image data and compressing the data as one color image data. An imaging device. The imaging apparatus according to claim 1,
The composite color image data generation means includes a G component for the color image data having the shortest exposure time and a B component for the color image data having the longest exposure time among the first to third color image data, and the remaining color image. An imaging apparatus which is means for generating the composite color image data by replacing the data with R component image data. The imaging apparatus according to claim 1 or 2,
When receiving color image data expressed by any one of RGB components for each pixel, color image data conversion means for converting the pixel positions to rearrange color image data by rearranging the same components so as to gather together Prepared,
The color image data generation means is means for supplying the color image data conversion means when generating the first to third color image data,
The composite color image data generation means is an image pickup apparatus which receives the first to third color image data whose pixel positions are rearranged by the color image data conversion means and generates the composite color image data. . The imaging apparatus according to any one of claims 1 to 3,
The composite color image data generation means collects G component pixels from the first to third color image data to form G component image data, and collects other component pixels to collect R component images. An imaging apparatus which is means for generating the composite color image data from the image data of each component after constructing the data and the image data of the B component. An image of a subject is exposed on an imaging surface in which a plurality of photoelectric elements that convert received light into charges and accumulate are arranged in a matrix, and the R component, G component, and B component that constitute the three primary colors of light An imaging method for capturing a color image of the subject by reading a corresponding charge amount,
The first color image data, the second color image data, and the third color image data having different exposure times are obtained by photographing the subject with different exposure times on the imaging surface in a plurality of stages. A first step of generating;
A second step of generating combined color image data by combining the first color image data, the second color image data, and the third color image data into one color image data;
A third step of outputting the combined color image data,
In the first step of generating the color image data, a charge amount corresponding to any one of the R component, G component, and B component set differently for each photoelectric element on the imaging surface is calculated. A step of generating the first to third color image data in an expression format in which each pixel has any one component of RGB by reading;
In the second step of generating the composite color image data, the first to third color image data are respectively read as single-component image data and compressed as one color image data. An imaging method which is a process of generating data.

Images