JP2009124405A - Image sensor, electronic camera, image processing apparatus, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a color reproduction suitable for a subject by dealing with a state in which color reproduction is difficult for a plurality of color pixels.
The imaging device according to the present invention has wavelengths of spectral sensitivity characteristics of at least first to third color pixels and at least first to third color pixels having different spectral sensitivity characteristics. A fourth color pixel having substantially all of the band as spectral sensitivity characteristics. An electronic camera according to the present invention includes any one of the above-described imaging elements and a recording unit that records image data generated by the imaging elements.
[Selection] Figure 1

Description

  The present invention relates to an image pickup device, an electronic camera equipped with the image pickup device, an image processing apparatus that performs image processing on image data generated by the image pickup device, and image processing for image data generated by the image pickup device using a computer. The present invention relates to an image processing program for realization.

  In many cases, a digital camera or the like has an image sensor that can obtain a signal of any one of a plurality of colors (for example, R, G, B) per pixel. Then, the signals of the channels other than the pixel are calculated and generated by interpolation processing based on the signals of the surrounding pixels, and an image composed of a plurality of color channels per pixel is obtained.

  An image signal obtained from such an image sensor may be limited to the upper limit value of the signal by saturation when the corresponding pixel portion of the subject has high luminance and high saturation, for example. In addition, when the corresponding pixel part of the subject has low luminance and low saturation, the image signal obtained from the image sensor may be limited to the lower limit value, and this case is also caused by the discontinuity of gradation. Malfunctions may occur. In such a state where color reproduction is difficult, there may be a problem caused by the discontinuity of gradation, for example, coloring occurs when an achromatic subject is photographed.

Therefore, in the invention of Patent Document 1, the first pixel to the third pixel in the Bayer array have three different relative spectral sensitivity characteristics, and the fourth pixel has an absolute sensitivity different from that of the first pixel. Have the same relative spectral sensitivity characteristics. When the pixel signal is large, a signal having a lower absolute sensitivity is employed, and when the pixel signal is small, a signal having a higher absolute sensitivity is employed to cope with a state in which color reproduction is difficult.
JP 2000-253413 A

  However, in the above-described invention, for example, when the colors of the first pixel to the third pixel are G, R, and B, respectively, and the color of the fourth pixel is G, the G channel signal is difficult to reproduce. However, the R channel and B channel cannot be handled.

  An object of the present invention is to cope with a state in which color reproduction is difficult with respect to a plurality of color pixels and to realize suitable color reproduction of a subject.

  The imaging device of the present invention relates to a wavelength band of spectral sensitivity characteristics of at least first to third color pixels having spectral sensitivity characteristics different from each other and the at least first color pixels to third color pixels. And a fourth color pixel having substantially all of the spectral sensitivity characteristics.

  Preferably, the first color pixel to the third color pixel are an R pixel, a G pixel, and a B pixel, respectively, and the first color pixel to the third color pixel and the fourth color pixel, respectively. May be arranged by replacing part of the G pixels in the Bayer array with the fourth pixel.

  Preferably, the combination of the at least first to third color pixels and the fourth color pixel may be periodically arranged as one combination.

  An electronic camera according to the present invention includes any one of the above-described imaging elements and a recording unit that records image data generated by the imaging element.

  Note that, preferably, the spectral sensitivity characteristic of the fourth pixel is set to the combined spectral sensitivity characteristic obtained by multiplying each of the spectral sensitivity characteristics of the at least first to third color pixels by a coefficient. You may further provide the memory | storage part which memorize | stores the said coefficient at the time of approximating.

  Preferably, the coefficient may be calculated based on a plurality of image data generated by the image sensor.

  Preferably, the recording unit may record the image data and the coefficient in association with each other.

  Preferably, the signal value of the digital signal of any one of the color pixels from the first color pixel to the third color pixel is preferably a conversion unit that converts the output from the image sensor into a digital signal. A calculation unit that calculates an estimated value based on signal values of the digital signals of surrounding color pixels and the fourth color pixel when the value is a limit value, and the recording unit estimates the limit value The image data replaced with values may be recorded.

  Preferably, the signal value of the digital signal of any one of the color pixels from the first color pixel to the third color pixel is preferably a conversion unit that converts the output from the image sensor into a digital signal. A calculation unit that calculates an estimated value based on signal values of the digital signals of surrounding color pixels and the fourth color pixel when the value is a limit value; and the image data in which the limit value is replaced with the estimated value An image processing unit that performs image processing on the image processing unit, and the recording unit may record the image data subjected to the image processing.

  In addition, an image processing apparatus that performs image processing on the image data generated by the above-described imaging device, and image processing for the image data generated by the above-described imaging device are realized by a computer. What is expressed by being converted into an image processing program is also effective as a specific aspect of the present invention.

  According to the present invention, it is possible to cope with a state in which color reproduction is difficult with respect to a plurality of color pixels, and to realize suitable color reproduction of a subject.

  Hereinafter, embodiments of the present invention will be described. In the present embodiment, an electronic camera will be described as an example.

  FIG. 1 is a functional block diagram of an electronic camera 1 according to an embodiment of the present invention.

  An electronic camera 1 includes an imaging unit 11 including an imaging element 10 such as a photographing lens (not shown), a CCD, an image processing unit 12 that performs image processing on image data generated by the imaging unit 11, and a liquid crystal monitor that displays an image. A display unit 13 including a recording medium 14 for recording an image, a control unit 15 for controlling each unit, a release button (not shown) and the like, an operation unit 16 for receiving user operations, a computer and a printer The connection part 17 which can be mutually connected with external apparatuses, such as, is provided. The imaging unit 11, the image processing unit 12, the recording unit 14, the control unit 15, and the connection unit 17 are connected to each other via a bus. The outputs of the control unit 15 and the image processing unit 12 are also connected to the display unit 13 via a bus. The output of the operation unit 16 is connected to the control unit 15.

  The imaging unit 11 photoelectrically converts the subject image via the photographing lens by the imaging element 10, further performs A / D conversion, and outputs image data. Details of the image sensor 10 will be described later. The image processing unit 12 performs processing such as white balance adjustment, color processing, and gamma processing on the image data acquired by the imaging unit 11. The display unit 13 displays the image generated by the image processing unit 12 and the image recorded in the recording unit 14, and also displays a through image for composition confirmation, and is also used as a viewfinder. The control unit 15 includes a memory 18 therein, and stores a program for controlling each unit in advance. And the control part 15 controls each part according to this program.

  In addition, the control unit 15 detects the state of the operation unit 16 and acquires information about the external device connected to the connection unit 17 via the connection unit 17.

  In the electronic camera 1 having the configuration described above, when an imaging instruction is given by the operation unit 16, the control unit 15 detects this, photoelectrically converts a subject image via the imaging lens of the imaging unit 11 by the imaging device 10, Further, A / D conversion is performed. Then, the control unit 15 performs processing such as white balance adjustment, color processing, and gamma processing via the image processing unit 12 and records them in the recording unit 14.

  FIG. 2 is a diagram illustrating a pixel array of the image sensor 10. As shown in FIG. 2, the image sensor 10 includes R, G, B, and A channels. Each channel of R, G, B, and A is a combination of R, G, B, and A, and the combinations are periodically arranged in the vertical and horizontal directions. The R, G, B, and A channels have an R filter, a G filter, a B filter, and an A filter, respectively. Hereinafter, each channel including each filter is referred to as a “pixel”.

  The A pixel is a pixel having substantially all of the spectral bands of the spectral sensitivities of R, G, and B as spectral sensitivity characteristics. FIG. 3 is a diagram illustrating the spectral sensitivity characteristics of each pixel of the image sensor 10. The horizontal axis in FIG. 3 indicates the wavelength, and the vertical axis indicates a value obtained by normalizing the output of each pixel from 0 to 1.

  The following relationships are established for the pixels of each color shown in FIG.

sA (λ) ≈k _R · sR (λ) + k _G · sG (λ) + k _B · sB (λ) (Formula 1)
In Equation 1, sA (λ) indicates the spectral sensitivity of the A pixel, sR (λ) indicates the spectral sensitivity of the R pixel, sG (λ) indicates the spectral sensitivity of the G pixel, and sB (λ) is The spectral sensitivity of the B pixel is shown. Further, k _R , k _G , and k _B in Equation 1 are coefficients that are determined in advance by a method that will be described later in accordance with the spectral sensitivity of the image sensor 10. The coefficients k _R , k _G , and k _B are values less than 1 as is apparent from FIG. The coefficients k _R , k _G , and k _B are the spectral sensitivity characteristics of the A pixel with respect to the combined spectral sensitivity characteristics obtained by multiplying the spectral sensitivity characteristics of the R, G, and B pixels by the coefficients. It is a coefficient when approximated.

Next, a method for calculating the coefficients k _R , k _G , and k _B will be described. When the color light C (λ) having uniform spectral sensitivity characteristics is imaged so that all signals are not clipped in the four pixel regions of R, G, B, and A in the image sensor 10, the above equation 1 shows that The signal value Da can be expressed by the following relational expression.

Da≈∫C (λ) [k _R · sR (λ) + k _G · sG (λ) + k _B · sB (λ)] dλ (Formula 2)
Further, if the signal values of the R pixel, G pixel, B pixel, and A pixel are Dr, Dg, and Db, respectively, the following relational expression is established.

Da = k _R · Dr + k _G · Dg + k _B · Db (Formula 3)
Therefore, each of the pixels included in the image sensor 10 is obtained by actually performing imaging by the image sensor 10 for at least three independent color lights, and substituting the obtained signal values Da, Dr, Dg, and Db into Equation 3. In addition, the coefficients k _R , k _G , and k _B can be calculated. Note that independent color light is color light in which one color light is not a constant multiple of the other color light.

Further, more than three independent color lights are actually picked up by the image pickup device 10 respectively, and the obtained signal values Da, Dr, Dg, Db are substituted into Equation 3 to optimize the least square method or the like. The coefficients k _R , k _G , and k _B may be calculated by

In this way, by calculating the coefficients k _R , k _G , and k _B based on the result of actual imaging with the image sensor 10, there is no need for precise adjustment of the spectral sensitivity characteristics of the A pixel. For this reason, for example, the coefficients k _R , k _G , and k _B are calculated in consideration of individual differences of products at the time of shipment, and the coefficients k _R , k _G , and k _B are updated in accordance with the influence of secular change. Easy. This is also effective when the lens of the electronic camera 1 is replaced. In addition, since the coefficients k _R , k _G , and k _B are calculated for each pixel included in the image sensor 10, higher-definition processing (details will be described later) compared to the case where a constant coefficient is used regardless of the pixel. ) Can be realized.

The imaging for calculating the coefficients k _R , k _G , and k _B is preferably performed using a dedicated color chart or the like, but can also be performed by imaging a normal subject. In this case, it is desirable that the subject area corresponding to the signal value used for calculation is sufficiently uniform.

Further, the coefficients k _R , k _G , and k _B can be calculated without performing imaging by the image sensor 10. For example, the coefficients k _R , k _G , and k _B may be calculated by measuring the spectral sensitivity characteristics of R, G, B, and A pixels and analyzing and optimizing the measurement results.

In any case, the function F (λ) of the wavelength λ can be expressed by the following equation using the calculated coefficients k _R , k _G , and k _B.

F (λ) = k _R · sR (λ) + k _G · sG (λ) + k _B · sB (λ) (Formula 4)
Therefore, this function F (λ) is very close to the spectral sensitivity sA (λ) of the A pixel.

The control unit 15 stores the above-described coefficients k _R , k _G , and k _B in the memory 18 in advance. The coefficients k _R , k _G , and k _B can be updated at a timing desired by the user as described above.

  FIG. 4 is a flowchart showing the operation of the control unit 15 when acquiring image data.

  In step S <b> 1, the control unit 15 controls the imaging unit 11 to acquire image data. In accordance with an instruction from the control unit 15, the imaging unit 11 performs photoelectric conversion on the subject image via the photographing lens by the imaging device 10, performs A / D conversion, and outputs image data.

  In step S2, the control unit 15 detects the pixel whose signal value is the upper limit value for the image data acquired in step S1. For example, in the A / D conversion in the imaging unit 11, for example, when quantization is performed with 12 bits, the data range is 0 to 4095. Therefore, the upper limit value of the signal value is 4095. The control unit 15 detects a pixel whose signal value is the upper limit value (4095) from the image data acquired in step S1.

  In step S3, the control unit 15 calculates an estimated value. The control unit 15 calculates the estimated value by calculating a signal value in an extended data range when a certain pixel is saturated. The upper limit value is 4095 as described above, but the control unit 15 estimates how much the signal value actually becomes.

  Here, the case where the signal value of a certain R pixel is the upper limit value will be described as an example. Further, it is assumed that the spectral radiance of the subject portion is equal for the R pixel described above and the surrounding G pixel, B pixel, and A pixel. And let this spectral radiance be C (λ). At this time, if the signal values of the G pixel, B pixel, and A pixel are Dg, Db, and Da, they can be expressed by the following equations.

Dg = ∫C (λ) sG (λ) dλ (Formula 5)
Db = ∫C (λ) sB (λ) dλ (Expression 6)
Da = ∫C (λ) sA (λ) dλ
≈∫ [C (λ) · (k _R · sR (λ) + k _G · sG (λ) + k _B · sB (λ))] dλ (Expression 7)
In addition, for the signal value of the R pixel whose signal value is the upper limit value, the estimated value Er that is the signal value in the extended data range can be expressed by the following equation.

Er = ∫C (λ) sR (λ) dλ (Equation 8)
Then, from the equations 8 and 3, the following equation can be derived.

Er≈ (Da−k _G · Dg−k _B · Db) / k _R (Equation 9)
In Expression 9, since Dg and Db, which are signal values of surrounding pixels, and coefficients k _R , k _G , and k _B are known data, Dg, Db, coefficients k _R , k _G , and k are expressed in Expression 9. By substituting _B , the estimated value Er can be calculated.

  The calculation of the estimated value described above is the same for the estimated value Eg when the signal value of the G pixel is the upper limit value and the estimated value Eb when the signal value of the B pixel is the upper limit value. The estimated value Eg of the G pixel can be calculated from the signal value of the R pixel, the signal value of the B pixel, and the signal value of the A pixel. Further, the estimated value Eb of the B pixel can be calculated from the signal value of the R pixel, the signal value of the G pixel, and the signal value of the A pixel.

  In step S4, the control unit 15 replaces the upper limit value of the pixel detected in step S2 with the estimated value calculated in step S3.

  In step S5, the control unit 15 determines whether there is another pixel that is the upper limit value in the image data acquired in step S1. If the control unit 15 determines that there is another pixel that is the upper limit value, the control unit 15 returns to step S2 and detects the pixel that is the next upper limit value. On the other hand, if it determines with having detected all the pixels which are upper limit values, the control part 15 will progress to step S6.

  In step S <b> 6, the control unit 15 performs image processing via the image processing unit 12 on the image data in which the upper limit value is replaced with the estimated value. The control unit 15 performs white balance adjustment, matrix processing, gradation conversion processing, compression processing (for example, Jpeg compression processing) and the like on the image data in which the upper limit value is replaced with the estimated value. The details of each process are the same as in the known technique.

  When the storage format of the image data is a so-called “RAW format” that is a recording format without performing the above-described processes, the control unit 15 proceeds to step S7 without performing the process of step S6. move on.

  In step S7, the control unit 15 records the image data subjected to the image processing in step S6 on the recording unit 14, and ends a series of processing.

  In addition, when recording image data, it records in consideration of the bit depth. The estimated value calculated in step S3 is a value larger than the upper limit value. Therefore, when recording image data in which the upper limit value is replaced with the estimated value, it is convenient to record the number of bits that can represent the maximum estimated value. For example, consider a case where the number of A / D conversion bits is 12 bits (0-4095) and the maximum estimated value is 60000. In this case, after the number of bits of the image data to be recorded is set to 16 bits, the saturated pixel is replaced with the estimated value, and the pixel for which the estimated value does not need to be calculated is also converted into 16-bit image data. Record it.

  Moreover, it is good also as a structure which records the information regarding an estimated value as incidental information of the image data acquired by step S1 instead of replacing an upper limit with an estimated value in step S4. In this case, the estimated value and the address of the pixel may be associated with each other as supplementary information, or the order in which the estimated values are calculated and the estimated value may be associated with each other as supplementary information.

<Modification 1>
In the present embodiment, the case where each color component has a coefficient (coefficients k _R , k _G , k _B ) has been described as an example, but a configuration having a single coefficient k for each color component may be employed. For example, when there is a single coefficient k for each color component instead of the R component coefficient k _R , the G component coefficient k _G , and the B component coefficient k _B described in the present embodiment, the above equation 1 Should be replaced by the following equation.

sA (λ) ≈k · (sR (λ) + sG (λ) + sB (λ)) (Equation 10)
Moreover, what is necessary is just to replace the above-mentioned formula 9 with the following formula.

Er≈Da / k− (Dg + Db) (Formula 11)
<Modification 2>
In the present embodiment, when a certain pixel is saturated, an example of calculating a signal value in a data range expanded with respect to the upper limit value is shown. However, for a lower limit value (in this embodiment, “0”) The present invention can also be applied when calculating signal values in the extended data range. In this case, the coefficients k _R , k _G , and k _B may be 1 or more. FIG. 5 is a diagram showing the spectral sensitivity characteristics of each pixel of the image sensor 10 when the coefficients k _R , k _G , and k _B are values of 1 or more. The horizontal axis in FIG. 5 indicates the wavelength, and the vertical axis indicates a value obtained by normalizing the output of each pixel from 0 to 1. Coefficient k _R, k _G, or a value less than one and k _B, for either factor k _R, k _G, and k _B with one or more values, and addressed to the address and the lower limit value to an upper limit It may be determined depending on which of these is prioritized. Also, both of the pixels whose coefficients k _R , k _G , k _B are less than 1 and the pixels whose coefficients k _R , k _G , k _B are 1 or more, and R, G, B If the pixels are combined as one combination and the combinations are periodically arranged in the vertical direction and the horizontal direction, both the upper limit value and the lower limit value can be dealt with.

<Modification 3>
A part of the processing described in the present embodiment may be configured to be performed in an external device. For example, the electronic camera 1 and a computer may be connected via the connection unit 17 so that a part or all of the processing after step S2 in the flowchart of FIG. In this case, the electronic camera 1 records the image data acquired in step S1 in association with the coefficients k _R , k _G , and k _B. On the other hand, a computer stores a program for realizing the processing of the flowchart of FIG. By configuring in this way, the computer can perform the processing after step S2 in the flowchart of FIG. 4 based on the coefficients k _R , k _G , and k _B. If the spectral sensitivity characteristics of the A pixel of the image sensor 10 are recorded in the computer, it is not necessary to obtain the coefficients k _R , k _G , and k _B from the electronic camera 1. The same applies to other external devices such as a printer that can be connected to the electronic camera 1 via the connection unit 17.

  As described above, according to the imaging device and the electronic camera of the present embodiment, at least the first color pixel to the third color pixel and the first color pixel to the third color having different spectral sensitivity characteristics. A fourth color pixel having substantially all of the spectral bandwidth as a spectral sensitivity characteristic with respect to the wavelength band of the spectral sensitivity characteristics of the pixel. Therefore, by performing appropriate image processing on the image data obtained by the image sensor, it is possible to cope with a state in which color reproduction is difficult with respect to a plurality of color pixels and to realize suitable color reproduction of the subject.

  Further, according to the imaging device of the present embodiment, the first to third color pixels to the fourth color pixel are combined as one combination, and the combinations are periodically arranged. Therefore, by using the output of the surrounding pixels, it is possible to realize suitable color reproduction for the pixels that have reached the limit value.

  According to the present embodiment, the spectral sensitivity characteristic of the fourth pixel is compared with the combined spectral sensitivity characteristic obtained by multiplying each of the spectral sensitivity characteristics of the first color pixel to the third color pixel by a coefficient. Is stored as a coefficient. Therefore, by performing image processing using this coefficient, it is possible to realize suitable color reproduction for the pixels that have reached the limit value.

  Further, according to the present embodiment, the above-described coefficients are calculated based on a plurality of image data generated by the image sensor. Therefore, it is possible to calculate the coefficient in consideration of individual differences of products, and it is also easy to update the coefficient in accordance with the influence due to secular change.

  Further, according to the electronic camera of this embodiment, when the signal value of the digital signal of any one of the first to third color pixels is a limit value, the surrounding color pixels and A calculation unit that calculates an estimated value based on the signal value of the digital signal of the fourth color pixel, and the recording unit records image data in which the limit value is replaced with the estimated value. Therefore, it is possible to cope with a state in which color reproduction is difficult with respect to a plurality of color pixels, and to realize suitable color reproduction of a subject.

  Further, according to the electronic camera of this embodiment, the image processing unit that performs image processing on the image data in which the limit value is replaced with the estimated value is provided, and the recording unit records the image data that has been subjected to the image processing. . Therefore, image processing can be performed on suitable data in which the limit value is replaced with the estimated value.

  Moreover, according to the electronic camera of this embodiment, image data and a coefficient are recorded in association with each other. Therefore, suitable image processing for image data generated by the imaging device can be realized by a computer.

  In the present embodiment, the arrangement similar to the Bayer arrangement shown in FIG. 2 has been described as an example, but the present invention is not limited to this example. For example, the present invention can be similarly applied to an array similar to another array such as a stripe array.

  In this embodiment, the R, G, B, and A pixels are described as examples. However, the present invention can be similarly applied to a case having other color components. That is, the imaging device has a plurality of color pixels having different spectral sensitivity characteristics and pixels (corresponding to the A pixel) having substantially all of the spectral sensitivity characteristics in the wavelength band as spectral sensitivity characteristics. If it is, the effect similar to this invention can be acquired. Furthermore, the present invention can be similarly applied to an image sensor provided with a complementary color filter.

  Further, the electronic camera 1 of the present embodiment may be an electronic camera that captures a still image or an electronic camera that captures a moving image. Furthermore, if the image sensor similar to the image sensor 10 is provided and an image can be generated, the present invention can be similarly applied to an input device such as a scanner.

It is a functional block diagram of electronic camera 1 of an embodiment of the present invention. 2 is a diagram illustrating a pixel array of the image sensor 10. FIG. Is a diagram showing spectral sensitivity characteristics of each pixel of the coefficients k _R, k _G, the imaging device 10 when k _B is a value less than 1. It is a flowchart which shows operation | movement of the control part 15 at the time of image data acquisition. Is a diagram showing spectral sensitivity characteristics of each pixel of the coefficients k _R, k _G, the imaging device 10 when k _B is one or more values.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 10 ... Imaging device, 11 ... Imaging part, 12 ... Image processing part, 14 ... Recording part, 15 ... Control part, 17 ... Connection part, 18 ... Memory

Claims (13)

Substantially all of the wavelength bands of the spectral sensitivity characteristics of at least the first to third color pixels having different spectral sensitivity characteristics and the at least first to third color pixels are spectral sensitivity characteristics. An image sensor comprising: a fourth color pixel having: The imaging device according to claim 1,
The first color pixel to the third color pixel are R pixel, G pixel, and B pixel, respectively.
The imaging element, wherein the first color pixel to the third color pixel and the fourth color pixel are arranged by replacing a part of a G pixel in a Bayer array with the fourth pixel. The imaging device according to claim 1,
An imaging device, wherein the at least first to third color pixels and the fourth color pixel are combined as one combination, and the combinations are periodically arranged. The image sensor according to claim 1,
An electronic camera comprising: a recording unit that records image data generated by the imaging device. The electronic camera according to claim 4,
The spectral sensitivity characteristic of the fourth pixel is approximated to the combined spectral sensitivity characteristic obtained by multiplying each of the spectral sensitivity characteristics of the at least first to third color pixels by a coefficient. An electronic camera, further comprising a storage unit that stores a coefficient. The electronic camera according to claim 5,
The electronic camera, wherein the coefficient is calculated based on a plurality of image data generated by the image sensor. The electronic camera according to claim 5,
The electronic camera, wherein the recording unit records the image data and the coefficient in association with each other. The electronic camera according to claim 4,
A converter that converts the output from the image sensor into a digital signal;
When the signal value of the digital signal of any one of the first color pixel to the third color pixel is a limit value, the surrounding color pixel and the digital signal of the fourth color pixel A calculation unit that calculates an estimated value based on the signal value of the signal,
The electronic camera, wherein the recording unit records the image data in which the limit value is replaced with the estimated value. The electronic camera according to claim 4,
A converter that converts the output from the image sensor into a digital signal;
When the signal value of the digital signal of any one of the first color pixel to the third color pixel is a limit value, the surrounding color pixel and the digital signal of the fourth color pixel A calculator that calculates an estimated value based on the signal value of the signal;
An image processing unit that performs image processing on the image data in which the limit value is replaced with the estimated value;
The electronic camera, wherein the recording unit records the image data subjected to the image processing. An image processing apparatus that performs image processing on image data generated by the imaging device according to claim 1,
An acquisition unit for acquiring the image data;
Among the first color pixel to the third color pixel, when the signal value of the digital signal of any one of the color pixels is a limit value, the digital signal of the surrounding color pixel and the digital signal of the fourth color pixel A calculator that calculates an estimated value based on the signal value;
An image processing unit that performs image processing on the image data in which the limit value is replaced with the estimated value;
An image processing apparatus comprising: a recording unit that records the image data subjected to the image processing. An image processing apparatus that performs image processing on image data generated by the imaging device according to claim 1,
A spectral sensitivity characteristic of the fourth pixel is obtained with respect to the combined spectral sensitivity characteristic obtained by multiplying the image data and each of the spectral sensitivity characteristics of the at least first to third color pixels by a coefficient. An acquisition unit for acquiring the coefficients when approximated;
When the signal value of the digital signal of any one of the first color pixel to the third color pixel is a limit value, the digital signal of the surrounding color pixel and the digital signal of the fourth color pixel A calculation unit that calculates an estimated value based on a signal value and the coefficient;
An image processing unit that performs image processing on the image data in which the limit value is replaced with the estimated value;
An image processing apparatus comprising: a recording unit that records the image data subjected to the image processing. An image processing program for realizing image processing on image data generated by the image sensor according to claim 1 by a computer,
An acquisition procedure for acquiring the image data;
When the signal value of the digital signal of any one of the first color pixel to the third color pixel is a limit value, the digital signal of the surrounding color pixel and the digital signal of the fourth color pixel A calculation procedure for calculating an estimated value based on the signal value;
An image processing procedure for performing image processing on the image data in which the limit value is replaced with the estimated value;
An image processing program comprising: a recording procedure for recording the image data subjected to the image processing. An image processing program for realizing image processing on image data generated by the image sensor according to claim 1 by a computer,
A spectral sensitivity characteristic of the fourth pixel is obtained with respect to the combined spectral sensitivity characteristic obtained by multiplying the image data and each of the spectral sensitivity characteristics of the at least first to third color pixels by a coefficient. An acquisition procedure for acquiring the coefficients when approximated;
When the signal value of the digital signal of any one of the first color pixel to the third color pixel is a limit value, the digital signal of the surrounding color pixel and the digital signal of the fourth color pixel A calculation procedure for calculating an estimated value based on a signal value and the coefficient;
An image processing procedure for performing image processing on the image data in which the limit value is replaced with the estimated value;
An image processing program comprising: a recording procedure for recording the image data subjected to the image processing.

Images