JP2001086523A - Signal generating method and device and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a stripe or granular artifact from taking place in the case of generating a high frequency signal from an image signal obtained from an image pickup device such as a single board CCD. SOLUTION: A low frequency chrominance signal generating means 2 generates low frequency chrominance signals RL, GL, BL at each pixel position from an image pickup signal S0 obtained from a single board CCD 1. A low frequency luminance color difference signal generating means 3 generates a low frequency luminance signal YL and color difference signals CrL, CbL. A hue saturation information generating means 4 generates a hue Hue and a saturation Sat on the basis of the color difference signals CrL, CbL, and a variation calculation means 5 calculates a variation V of the hue Hue and the saturation Sat. A high frequency chrominance signal generating means 6 generates high frequency chrominance signals R1, G1, H1 at each pixel position depending on the hue Hue, the saturation Sat and the variation V and a high frequency signal S1 is obtained from them.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a method of disposing a plurality of types of photoelectric conversion elements having different spectral sensitivities on a single surface.
A signal generation method and apparatus for generating a high-frequency color signal representing high-frequency color information at each pixel position corresponding to a photoelectric conversion element based on a color imaging signal obtained by an imaging device such as a so-called single-chip CCD, and a signal generation method And a computer-readable recording medium on which a program for causing a computer to execute the program is recorded.

[0002]

2. Description of the Related Art As an imaging device such as a CCD used in a digital camera, a device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are alternately arranged on the same plane is known (hereinafter, referred to as an imaging device). Single board CCD). Here, in the case of a single-chip CCD in which photoelectric conversion elements having spectral sensitivities in R, G, and B, that is, photoelectric conversion elements of R, G, and B channels are alternately arranged, continuous R, G, and B are used.
A set of three photoelectric conversion elements of the G and B channels forms one pixel. However, in such a single-chip CCD, since the R, G, and B values of each pixel cannot be obtained at the same pixel position, a color shift or a false color may occur. Further, since the number of photoelectric conversion elements in each channel is smaller than the total number of elements constituting the single CCD, a high-resolution image cannot be obtained. For example, R, G, B
Single-plate CC in which photoelectric conversion elements of each channel are alternately arranged
In D, since the number of photoelectric conversion elements in each channel is only 1/3 of the total number of elements, the resolution is reduced to 1/3 as compared with a monochrome imaging device having the same number of elements. For this reason,
A method has been proposed in which a signal value in a portion where the photoelectric conversion element of each of the R, G, and B channels does not exist is obtained by an interpolation process. May occur. In this case, the occurrence of color misregistration can be prevented by performing the smoothing process, but there is a problem that the resolution deteriorates when the smoothing process is performed.

Here, human visual characteristics are more sensitive to luminance than to color. For this reason, a high-frequency luminance signal representing the luminance of each pixel and a low-frequency color signal by the above-described interpolation processing and smoothing processing are generated from the color imaging signal obtained by the single-chip CCD, and the luminance signal and color A method of reconstructing a color image signal using a signal has been proposed (Japanese Patent Application Laid-Open No. 10-200906,
No. 9-65075). According to this method, more information is given to a luminance component having high sensitivity in human visual characteristics, so that a color image signal capable of reproducing an image having an apparently high resolution can be obtained.

[0004]

However, in the method described in Japanese Patent Application Laid-Open No. H10-200906, a luminance signal is estimated from a low-frequency color signal generated from an imaging signal obtained by a single CCD. Because
The high-frequency luminance signal obtained by this method is blurred near the edges included in the image, and as a result, there is a problem that the resolution of the image is reduced.

In the method described in Japanese Patent Application Laid-Open No. 9-65075, a high-frequency luminance signal at each pixel position is obtained by an interpolation operation from signal values at each pixel position and pixel positions around the pixel position. Therefore, the resolution does not decrease so much as compared with the method described in the above-mentioned JP-A-10-200906. However, since the image signals obtained by the single-chip CCD do not include all the color signals (eg, RGB) at each pixel position, streak-like artifacts occur near the boundary of the color edge, and granularity occurs near the edge. There is a problem that artifacts occur.

That is, if there is a hue that a pixel position originally has, an interpolation operation is performed using a signal value that is complementary to or close to the hue to obtain a signal value at that pixel position. Is more likely to occur. For example, when the hue is R, an interpolation operation is performed using the G signal value to obtain a signal value at that pixel position. When the signal value of the pixel position is originally small enough to be regarded as noise, the signal value is reduced by the interpolation operation. Is amplified, and granular artifacts occur at the edge boundaries.

The present invention has been made in view of the above circumstances, and it is possible to obtain a high-frequency signal without generating an artifact near an edge with respect to an imaging signal obtained by an imaging device such as a single CCD. It is an object of the present invention to provide a computer-readable recording medium on which a program for causing a computer to execute a signal generation method and apparatus and a signal generation method are recorded.

[0008]

According to the present invention, there is provided a signal generation method, comprising the steps of: obtaining a plurality of types of photoelectric conversion elements having different spectral sensitivities on a single surface based on a color imaging signal obtained from the imaging device; A signal generation method for generating a high-frequency color signal including high-frequency color information at each pixel position corresponding to a photoelectric conversion element, the method comprising: interpolating based on a signal value at each pixel position and a pixel position around each pixel position. In the signal generation method of performing the operation to generate the high-frequency color signal at each of the pixel positions, based on the color imaging signal, generate a low-frequency color signal representing low-frequency color information at each of the pixel positions, A low-frequency color difference signal representing low-frequency color difference information at each of the pixel positions is generated based on the low-frequency color signal, and each of the image signals is generated based on the low-frequency color difference signal. Generating hue / saturation information at a position, selecting a pixel position to be subjected to the interpolation operation based on the hue / saturation information, performing an interpolation operation based on a signal value of the selected pixel position; It is characterized by generating a color signal.

Here, "an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface"
Means an image pickup device such as the above-mentioned single-plate CCD. In addition, each photoelectric conversion element is R (red), G
Not only (green) and B (blue) but also C (cyan), M (magenta), Y (yellow), or CMYG obtained by adding G (green) to CMY may have spectral sensitivity. The arrangement of these photoelectric conversion elements is not limited to a specific one.

"Including color information of high frequency" means that color information in all frequency bands from low frequency to high frequency is included.

The "low-frequency color signal representing low-frequency color information" is based on, for example, a linear interpolation operation or a spline interpolation operation based on a signal value of a pixel position at which a low-frequency color signal is to be obtained and pixel values of surrounding pixel positions. Means a low-frequency color signal at that pixel position. For this reason, the resolution of the image obtained by the low-frequency color signal is smaller than the resolution of the image obtained by all the photoelectric conversion elements, and as a result,
The low frequency color signal represents low frequency color information.

Further, "hue / saturation information" refers to information representing the magnitude of change in hue and / or saturation at each pixel position, and the hue and / or saturation values themselves.

"Selecting a pixel position for performing an interpolation operation based on hue / saturation information" means that an artifact is not generated near an edge in a high-frequency color signal.
This means that a pixel position used for performing an interpolation operation is selected from a pixel position at which a high-frequency color signal is calculated and pixel positions around the pixel position. More specifically, a pixel position corresponding to the hue or saturation change direction or a pixel position corresponding to the hue or saturation value itself is selected.

In the signal generation method according to the present invention, it is preferable that the hue / saturation information is a hue and / or saturation value at each of the pixel positions. Alternatively, the change amount is preferably a change in saturation.

[0015] The signal generating apparatus according to the present invention corresponds to each of the photoelectric conversion elements based on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A signal generation device that generates a high-frequency color signal including high-frequency color information at each pixel position, performing an interpolation operation based on the signal values at each of the pixel positions and pixel positions around the respective pixel positions. In the signal generation device that generates the high-frequency color signal at a pixel position, based on the color imaging signal, a low-frequency color signal generation unit that generates a low-frequency color signal representing low-frequency color information at each pixel position, Low-frequency color difference signal generation means for generating a low-frequency color difference signal representing low-frequency color difference information at each of the pixel positions based on the low-frequency color signal; A hue / saturation information generation unit that generates hue / saturation information at each of the pixel positions based on a frequency / color difference signal, and a selection unit that selects a pixel position where the interpolation calculation is performed based on the hue / saturation information, And a high-frequency color signal generating means for generating the high-frequency color signal by performing an interpolation operation based on the signal value at the selected pixel position.

In the signal generating apparatus according to the present invention, the hue / saturation information generating means is means for generating the hue / saturation information as hue and / or saturation values at each of the pixel positions. It is preferable that the unit generates the hue / saturation information as a change in hue and / or saturation at each pixel position.

The signal generation method according to the present invention may be recorded on a computer-readable recording medium and provided as a program for causing a computer to execute the signal generation method.

[0018]

According to the present invention, a low-frequency color signal at each pixel position is generated from a color imaging signal obtained by an imaging device such as a single-chip CCD, and a low-frequency color signal is generated based on the low-frequency color signal. A color difference signal is generated, and from this low frequency color difference signal, hue saturation information at each pixel position,
That is, values of hue and saturation change, hue, and saturation values at each pixel position are generated. Then, a pixel position to be used for performing an interpolation operation is selected based on the hue / saturation information, and an interpolation operation is performed based on a signal value of the selected pixel position to generate a high-frequency color signal. .
As described above, according to the present invention, since the high-frequency color signal is generated based on the hue saturation information, for example, when a high-frequency color signal is generated at a certain pixel position, the pixel is determined based on the hue saturation information. The signal value can be obtained without using a signal value that is in a complementary color relationship with or close to the hue of the position, thereby preventing the occurrence of an artifact at an edge boundary.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram illustrating a configuration of a signal generation device according to an embodiment of the present invention. FIG.
As shown in the signal generation device according to the embodiment of the present invention,
From the imaging signal S0 composed of the color signals R0, G0, B0 obtained in each photoelectric conversion element constituting the single CCD 1,
A high-frequency signal S1 composed of high-frequency color signals R1, G1, and B1 at each pixel position is generated. Low-frequency color signal generating means 2 for generating signals RL, GL, BL;
Based on GL and BL, low-frequency luminance and color difference signals YL and C
Low frequency luminance / color difference signal generating means 3 for generating rL and CbL
And the hue H based on the low-frequency color difference signals CrL and CbL.
hue / saturation information generating means 4 for generating hue / saturation information comprising hue and saturation Sat;
The change amount calculating means 5 for calculating the change amount V of t, and the hue / saturation information generated by the hue / saturation information generating means 4 and the change amount V calculated by the change amount calculating means 5, each pixel position And the pixel position around it,
Color signals R1 and G including high-frequency color information at each pixel position are obtained by performing an interpolation operation based on the signal value of the selected pixel position.
1 and B1.

The signal generating device according to the present invention is provided in an image processing device provided in an image pickup device for reading an image from a digital camera or a film. This image processing apparatus regenerates a color image signal by generating a high-frequency color signal from an imaging signal S0 obtained by a single-chip CCD 1. In order to generate a high-frequency color signal in the image processing apparatus, The signal generation device according to the present embodiment is used.

FIG. 2 is a diagram showing the arrangement of the photoelectric conversion elements of the single-plate CCD 1. FIG. 2A shows an arrangement in which R, G, and B channel photoelectric conversion elements having spectral sensitivities are alternately arranged in R, G, and B, and FIG. The line in which the lines are alternately arranged and the line in which the G and B channels are alternately arranged are alternately arranged in the horizontal direction, and the G channel which affects the luminance is twice as large as the R and G channels. FIG. 2C shows C,
C, M, and Y channel photoelectric conversion elements having spectral sensitivity in M and Y are alternately arranged. FIG. 2D shows a photoelectric conversion element in which a G channel is added to the C, M, and Y channels. They are arranged alternately. Where RG
Since the chromaticity of each primary color is defined for B and CMY, they can be mutually converted. Therefore, the signal generation device according to the present invention may generate the high-frequency luminance signal YH from the imaging signal S0 obtained from any arrangement of the single-chip CCDs 1. In the present embodiment, the signal generation device shown in FIG. Description will be made assuming that processing is performed on an image pickup signal S0 obtained in the single-plate CCD 1 having the arrangement of the photoelectric conversion elements shown in FIG.

The low-frequency color signal generating means 2 includes a single CCD 1
The low-frequency color signals RL, GL, and BL representing low-frequency color information at each pixel position are generated from the image pickup signal S0 obtained from the image signal S0. First, generation of the low-frequency color signal RL will be described. FIG. 3 shows a state in which only the R channel element is extracted from the array of photoelectric conversion elements shown in FIG. Since there is no R signal at the position of the element indicated by X in FIG. 3 (hereinafter referred to as pixel position X), the R signal at the pixel position X is converted into a single CCD 1 Are obtained by performing an interpolation operation in this order in the vertical direction and the horizontal direction. As this interpolation operation, in addition to linear interpolation, a higher-order interpolation operation such as a B-spline interpolation operation that emphasizes smoothness and a Cubic spline interpolation operation that emphasizes sharpness can be applied.

Here, the Cubic spline interpolation calculation and the B spline interpolation calculation will be described. The imaging signal S0 used in the present embodiment is a sampling point (pixel) X _k−2 , X _k−1 , X _k , X _{k + 1} , X _k arranged in one direction sampled at a regular interval. the corresponding signal value in the _{k-2 ... (S k-} 2, S k-1, S k, S k + 1 S k + 2 ...) for those having. Cubic spline interpolating operation, the interpolation data in the original sampling points (pixels) X _k ~X _k + interpolation point provided between ₁ X _p Cubic spline interpolation equations of third order which represents the interpolated data Y '(1) Y _k-1 ,
Interpolation coefficients c respectively corresponding to Y _k , Y _{k + 1} , and Y _{k + 2
k-1} , _ck , _{ck + 1} , _{ck + 2} are obtained by the following calculations.

Y ′ = c _k−1 Y _k−1 + c _k Y _k + c _{k + 1} Y _{k + 1} + c _{k + 2} Y _{k + 2} (1) c _k−1 = (− t ³ + 2t ² −t) ) / 2 _ck = (3t ³ -5t ² +2) / 2 _{ck + 1} = (-3t ³ + 4t ² + t) / 2 _{ck + 2} = (t ³ −t ² ) / 2 (where t ( 0 ≦ t ≦ 1) means that the grid interval is 1 and the pixel X
_The position of the interpolation point _{Xp in} the pixel _{Xk + 1} direction with respect to _k is shown. In the B-spline interpolation operation, the interpolation data Y _k in the cubic B-spline interpolation expression (2) representing the interpolation data Y ′ of the interpolation point X _p provided between the original sampling points X _{k to} X _{k + 1 is} calculated. Interpolation coefficients b _k−1 , b _k , b _{k + 1} , and b _{k + 2} corresponding to ₋₁ , Y _k , Y _{k + 1} , and Y _{k + 2} , respectively, are obtained by the following calculations. .

Y ′ = b _k−1 Y _k−1 + b _k Y _k + b _{k + 1} Y _{k + 1} + b _{k + 2} Y _{k + 2} (2) b _k−1 = (− t ³ + 3t ² −3t + 1) _{) / 6 b k = (3t} 3 -6t 2 +4) / 6 b k + 1 = (- 3t 3 + 3t 2 + 3t + 1) / 6 b k + 2 = t 3/6 ( where, t (0 ≦ t ≦ 1 ) Indicates that the grid interval is 1 and the pixel X
_The position of the interpolation point _{Xp in} the pixel _{Xk + 1} direction with respect to _k is shown. In the present embodiment, it is preferable to perform a B-spline interpolation operation from the viewpoint of smoothing the color signal. In addition, the color signal RL at the pixel position X is obtained by simply performing a filtering process using a low-pass filter without performing the interpolation operation. May be obtained. As a low-pass filter, when the Nyquist frequency when sampling is performed by all the photoelectric conversion elements of the single-chip CCD 1 is fs, the original color signal R0 of the single-chip CCD 1 having the arrangement of the photoelectric conversion elements shown in FIG. One of all signals in the vertical and horizontal directions
Since only / 2 exists, it is preferable to cut high-frequency components of 1/2 fs or more. Note that the characteristics of the low-pass filter are preferably changed according to the arrangement of the photoelectric conversion elements in the single-chip CCD 1.

Here, it is preferable to perform the interpolation operation as described below in order to reduce the operation time. FIG. 4 is a diagram for explaining this preferred interpolation operation. Here, the calculation of the color signal RL will be described. In FIG. 4, the pixel position of ○ is the pixel position where the color signal R0 originally exists, □ is the pixel position where the color signal RL calculated by the interpolation operation in the X direction exists, and × is the pixel position where the color signal R0 is present in the Y direction. Indicates the pixel position where the calculated color signal RL exists, and indicates the color signal RL at the pixel position x from the color signal RL at the 4 × 4 pixel position near the interpolation point for calculating the color signal RL.
Is what you want. First, for the four lines near the interpolation point for calculating the color signal RL at the pixel position x to be determined in the X direction (here, the lines are lines in the X direction, and lines 1 to 4 respectively), the color at the pixel position □ The signal RL is obtained, and signals for four lines are stored in the line buffer. In FIG. 4, the pixel position at which the color signal RL is calculated is indicated by ■.

Next, the color signal RL at the pixel position x is calculated based on the calculated color signal RL at the pixel position 分 の for the four lines. Then, to calculate the color signal RL at the pixel position □ of the next line (line 5),
The color signal RL of line 1 is erased from the line buffer, the color signal RL of line 5 is stored in the line buffer, and the color signal RL of the pixel position □ is calculated in the same manner as described above. The color signal RL at the pixel position x is calculated. Hereinafter, by repeating this process for each line, the color signals RL at all the pixel positions □ and × can be calculated to obtain the color signals RL at all the pixel positions.

The color signal BL is also subjected to the above-described interpolation calculation or filtering by a low-pass filter so that the pixel position X at which the color signal B0 does not exist is determined.
Is obtained.

Since the signal value of the color signal G0 can be obtained at more pixel positions than the color signal R0 and the color signal B0, the frequency characteristic of the color signal GL is expressed by the color signal R0.
In order to match with L and the color signal BL, the pixel is halved.
Then, the color signal GL at the pixel position X where the color signal G0 does not exist is obtained by performing the above-described interpolation operation or the filtering process using the low-pass filter after the thinning.

The low frequency luminance / color difference signal generating means 3 derives the following equation (3) from the low frequency color signals RL, GL, BL.
The low frequency luminance signal YL and the low frequency color difference signal Cr
L and CbL are calculated.

YL = 0.3RL + 0.6GL + 0.1BL CrL = 0.7RL-0.6GL-0.1BL (3) CbL = -0.3RL-0.6GL + 0.9BL Note that the low-frequency luminance signal YL is a high-frequency color signal. The low frequency color difference signals CrL and CbL are input to the signal generation means 6 and input to the hue / saturation information generation means 4.

The hue / saturation information generating means 4 outputs the low-frequency color-difference signal C generated by the low-frequency luminance / color-difference signal generating means 3.
Based on rL and CbL, the following equations (4) and (5)
Generates the hue Hue and the saturation Sat.

Hue = tan ⁻¹ (CbL / CrL) (4) Sat = √ (CrL ² + CbL ² ) (5) Here, R represented by 8 bits (R = 255, G =
0, B = 0), G (R = 0, G = 255, B = 0), B
(R = 0, G = 0, B = 255) and C (R = 0, G
= 255, B = 255), M (R = 255, G = 0, B
= 255) and Y (R = 255, G = 255, B = 0) are shown in FIG. 5 for the relationship between the hue Hue and the saturation Sat. In FIG. 5, the emission direction from the origin is the saturation S
at, the angle θ with respect to the CrL axis represents the hue Hue, and the numbers attached to the respective symbols of R, G, B, C, M, and Y are as follows:
(Hue, saturation) calculated by the above equations (4) and (5).

Based on the hue Hue and the saturation Sat generated by the hue / saturation information generating means 4, the change amount calculating means 5 calculates the hue Hue and the saturation S at each pixel position.
A change amount V as an added value of the change amounts of at is obtained. The change amount V can be obtained by performing a filtering process using a Robinson edge detection filter on the hue Hue and the saturation Sat in a range of 3 × 3 pixels centered on each pixel position (image analysis). Handbook, Mikio Takagi, Hirohisa Shimoda, University of Tokyo Press, 1991, January 17, PP554). FIG. 6 shows this Robinson edge detection filter. The filter shown in FIG. 6A is a filter for calculating the edge strength in the direction of arrow A shown in FIG. 7, and the filter shown in FIG.
The direction shown in FIG. 6C is indicated by an arrow C in FIG.
The filter shown in (d) is a filter for calculating the edge strength in the direction of arrow D. Specifically, regarding the hue Hue, as shown in FIG. 8, the hue Hue at each pixel position is shown.
Is Hueij (i, j = −1 to 1), the change amount V1 in the direction of arrow A can be obtained by the following equation (6).

V1 = | Hue _-1-1 + 2Hue _0-1 + Hue _1-1 -Hue _-11 -2Hue ₀₁ -Hue ₁₁ | +1 (6) Here, 1 is added to the change amount V1. This is because the value is not set to 0. Then, the amounts of change V2, V3, and V4 in the directions of arrows B, C, and D can be obtained in the same manner. Also,
The change amount of the saturation Sat can be obtained in the same manner as the change amount of the hue Hue.

Although the amounts of change V1 to V4 are obtained by the Robinson edge detection filter here, they may be obtained by a differential filter or a difference filter.

Thus, the hue H at each pixel position
When the change amounts V1 to V4 of the ue and the saturation Sat are obtained, the average value of the change amounts V1 to V4 is obtained, and the average value of the change amounts of the hue Hue and the saturation Sat is added to obtain the change amount V. The change amount V is the hue Hue
And saturation Sat, or may be calculated by weighting and adding them.

The high-frequency color signal generator 6 generates the hue Hue and the saturation Sat generated by the hue / saturation information generator 4.
Of the high-frequency color signals R1, G1, B based on the
1 is generated. First, the change amount V
The calculation of the color signals R1, G1, and B1 based only on will be described.

First, the case where the changes in hue Hue and saturation Sat are small (case 1) will be described. In addition,
“Small change” means that the change amount V is, for example, less than 40. When there is little change in hue Hue and saturation Sat, it can be said that there is little change in color around the pixel position. In this case, as shown in the following equation (7), the high frequency at the pixel position is high. The relationship is established that the difference between the color signal and the low-frequency color signal is substantially equal. On the other hand, as shown in FIG. 9, when the change in the saturation Sat is small even when the hue Hue changes (case 2),
Since the RGB color signals at the pixel position are substantially equal and the periphery of the pixel position can be said to be gray, the relationship of Expression (7) is similarly established. Note that this relationship holds when each color signal is represented by the logarithm of the amount of light, and when the color signal is represented by the amount of light, the relationship is expressed by Expression (7 '). However, in the present embodiment, each color signal is described as being represented by the logarithm of the amount of light. Equations (7) and (7 ')
In the above, Y1 is a high-frequency luminance signal (including a low-frequency component).

Y1-YL = R1-RL = G1-GL = B1-BL (7) YL: RL: GL: BL = Y1: R1: G1: B1 (7 ') Therefore, in the low-frequency color signal generating means 2, Since the low-frequency color signals RL, GL, and BL are calculated, if the color signal (for example, R0) originally existing at the pixel position is used as the high-frequency color signal R1 using the relationship of Expression (7), other high-frequency signals are obtained. The color signals (G1, B1) can be calculated.

[0041] Specifically, when subjected to reference numbers in the pixel position as shown in FIG. 10, in the pixel position R ₄ R0 =
Is R1, the low-frequency luminance signal YL (R _4), the high-frequency luminance signal Y1 (R _4), the color signals R0 originally included in the pixel position R ₄
(R ₄ ) and the low-frequency color signal RL (R ₄ ) satisfy the following equation (8).

YL (R ₄ ) −RL (R ₄ ) = Y 1 (R ₄ ) −R 0 (R ₄ ) (8) Therefore, the luminance signal YH (R ₄ ) representing only the high-frequency component at the pixel position R ₄ is become _{YH (R 4) = Y1 (} R 4) -YL (R 4) = R0 (R 4) -RL (R 4) (9). Therefore, the color signal GL of the low-frequency to high-frequency luminance signal YH (R ₄₎ at the pixel position R _4, by adding to the BL, the high frequency of the color signal at the pixel position _{R 4 G1 (R 4),} B1 (R ₄ ) is given by the following equations (10) and (1)
It can be obtained as shown in 1).

[0043] _{G1 (R 4) = GL (} R 4) + R0 (R 4) -RL (R 4) (10) B1 (R 4) = BL (R 4) + R0 (R 4) -RL (R 4) (11) Here, as shown in FIGS. 11A to 11C, calculation of a high-frequency color signal in a portion where the absolute values of the RGB signals are different and only the difference between the edges indicated by arrows in the diagram is substantially equal explain. First, as shown in FIG. 12, when the interpolation operation is performed only with the RGB signals around the pixel position where the high-frequency color signal is calculated, a false color is generated at the edge portion as shown in FIG. On the other hand, if it is assumed that the relationship shown in Expression (7) holds, as shown in FIGS. 11D to 11F, there is no difference in edge shape depending on the calculated high-frequency color signal. As shown in FIG. 13B, no false color is generated. In FIGS. 11E and 11F, R0 (or R1) -RL is added to the low-frequency color signals GL and BL to calculate the high-frequency color signals G1 and B1 at the pixel position of R. Is what you are doing.

When the value of the saturation Sat is small,
Since the respective RGB color signals are substantially equal, Y1 (R ₄ ) = R
0 (R ₄ ), and based on this relationship, the following equation (1)
2), (13) by high-frequency color signal G1 (R _4), B
1 (R ₄ ) may be calculated.

[0045] _{G1 (R 4) = GL (} R 4) + R0 (R 4) -YL (R 4) (12) B1 (R 4) = BL (R 4) + R0 (R 4) -YL (R 4) (13) The R signal and G signal at the B pixel position and the R signal and B signal at the G pixel position can be calculated in the same manner as described above.

Next, when the hue Hue is constant and the saturation Sat changes (case 3), the high-frequency color signals R1, G1, B1 are calculated as follows. FIG. 14 shows the hue H
FIG. 7 is a diagram for explaining calculation of a high-frequency color signal when ue is constant and saturation Sat changes. Here, a case will be described in which the hue Hue is constant at R and the saturation exists in the hatched portion in FIG. In this case, high-frequency color signals R1, G1, and B1 are calculated by weighting and adding a signal calculated on the assumption that R has high saturation and a signal calculated on the assumption that R has low saturation. You. For example, assuming that a high-frequency color signal calculated on the assumption that R has high saturation is Rh1, and a high-frequency color signal calculated on the assumption that R has low saturation is R11, the color signal R1 is represented by the following equation ( 14). Here, high saturation means that the saturation Sat is 20 or more, and low saturation means that the saturation Sat is less than 10.

R1 = α · Rh1 + (1−α) · R11 (14) where α is a weighting coefficient determined according to the saturation Sat. Note that the signal R11 has a saturation even if the hue Hue described above changes. What is necessary is just to use the high frequency color signal R1 calculated when Sat is small. The signal Rh1 is calculated by the same processing as the processing based on the values of the hue Hue and the saturation Sat itself, which will be described later.

On the other hand, when the change in both the hue Hue and the saturation Sat is large (case 4), the hue Hu is also changed.
A high-frequency color signal is calculated by the same processing as that based on the values of e and the saturation Sat itself.

Next, calculation of a high-frequency color signal based on the values of the hue Hue and the saturation Sat itself will be described. First, the calculation of the color signal according to the value of the hue Hue (case 5) will be described. The calculation of the color signal in accordance with the value of the hue Hue is based on the calculation of the signal Rh1 in the equation (14) in the case where the saturation Sat is high and the change in both the hue Hue and the saturation Sat ( Also applies to case 4). First, when it is determined that the hue Hue is R, the luminance signal at that pixel position is R
Is dominant, the relationship shown in the following equation (15) is established.

[0050] Y1-YL = R1-RL ( 15) wherein, when the pixel position B ₀ in FIG. 10 pixel positions to calculate a high frequency color signals R1, G1, B1, from the relation of equation (15), Y1 (B ₀ ) −YL (B ₀ ) = R 1 (B ₀ ) −RL (B ₀ ) (16) The color signal R1 (B ₀ ) in the equation (16) is calculated based on the color signal R0 at the pixel positions R ₀ , R ₁ , R ₃ , R ₄ near the pixel position B ₀ . Specifically, it may be the average value of the color signals R0 at the four pixel positions R ₀ , R ₁ , R ₃ , and R ₄ , and may be the pixel value in the direction where the change in the hue Hue and / or the saturation Sat is the least. The calculation may be performed using only the signal value. Here, the change of hue Hue and / or saturation Sat is the smallest direction assumed to be the direction of the pixel position _{R 3, R1 (B 0)} = BL (B 0) + R0 (R 3) -RL (B 0 ) (17) enables calculates color signals R1 (B ₀₎ at the pixel position B _0. Here, in the equation (17), R1 (B ₀ ) =
May R0 (R _3). Note that the R signal and the G signal at the B pixel position and the R signal at the G pixel position,
The B signal can be similarly calculated. If the hue Hue is determined to be G or B, a high-frequency color signal is obtained in the same manner as described above based on the relationship of Y1-YL = G1-GL (18) Y1-YL = B1-BL (19) R1,
G1 and B1 can be calculated.

On the other hand, when it is determined that the hue Hue is cyan (C), since the G signal and the B signal are dominant in the luminance signal at the pixel position, the following equation (20) is used. The relationship is established.

[0052] Y1-YL = G1-GL = B1-BL (20) Here, if the pixel position to calculate the high frequency color signals R1, G1, B1 and the pixel position B ₀ in FIG. 10, formula (20) the following equation from the relationship (21), the color signals R1 (B ₀₎ of the RF at the pixel position B ₀ by (22), G1 (B ₀₎
Can be requested.

[0053] _{R1 (B 0) = RL (} B 0) + B0 (B 0) -BL (B 0) (21) G1 (B 0) = GL (B 0) + B0 (B 0) -BL (B 0) (22) on the other hand, the high frequency of the color signal at the pixel position G ₅ R1
(G ₅ ) and B1 (G ₅ ) can be obtained by the following equations (23) and (24).

[0054] _{R1 (G 5) = RL (} G 5) + G0 (G 5) -GL (G 5) (23) B1 (G 5) = BL (G 5) + G0 (G 5) -GL (G 5) (24) in the high-frequency color signal at the pixel position R ₄ G1, B1
Is determined in the same manner as when the hue Hue is determined to be R as described above. Here, if the change of hue Hue and / or saturation Sat is the least direction pixel position G _{6 direction, G1 (R 4) = GL} (R 4) + G0 (G 6) -GL (R 4 _{) = G0 (G 6) (} 25) makes it possible to calculate the color signals G1 at a pixel position R _4, when the change in the hue hue and / or saturation Sat is the least direction is the direction of the pixel position B ₃ the can calculate the _{B1 (R 4) = BL (} R 4) + B0 (B 3) -BL (R 4) = B0 (B 3) color signals B1 at the pixel position R ₄ by (26).

When the hue Hue is determined to be magenta or yellow, the following equations (27) and (2)
By using the relationship shown in 8), the high-frequency color signals R1, G1, and B1 at each pixel position can be calculated in the same manner as described above.

Y1-YL = R1-RL = B1-BL (27) Y1-YL = R1-RL = G1-GL (28) Here, when a signal value at a certain pixel position is obtained, the pixel position is originally determined. When there is a hue Hue having the hue Hue, the high-frequency color signal R
If 1, G1 and B1 are determined, artifacts are likely to occur. For example, when the signal value of the pixel position is obtained by using the signal value of G while the hue Hue is R, the signal value of G is originally small enough to be regarded as noise. Amplified and grain artifacts occur at the edge boundaries. For this reason, the hue Hu at the pixel position where the signal is calculated as in the present embodiment.
In accordance with the value of e, high-frequency color signals R1, G1, and B are used without using color signals having a complementary color relationship with the hue Hue.
By calculating 1, it is possible to suppress artifacts occurring at the edge boundaries.

If the hue Hue is an intermediate hue between a certain hue and a certain hue, for example, if the hue Hue is an intermediate hue between R and magenta, the color signal calculated on the premise that the hue Hue is R And the high-frequency color signals R1, G1, and B1 may be calculated by weighting and adding the color signals calculated on the assumption that the hue Hue is magenta. For example, considering the R signal, the hue Hue is R
Rr is the high-frequency color signal calculated on the assumption that
1. Assuming that a high-frequency color signal calculated on the assumption that the hue Hue is magenta is Rm1, the color signal R1 is calculated by the following equation (29).

R1 = β · Rr1 + (1−β) · Rm1 (29) where β is a weight coefficient according to the hue Hue. Next, for the calculation of the color signal according to the saturation Sat (case 7), When the degree Sat is small, the color signals R1, G1 and B1 are calculated in the same manner as in the case where the saturation Sat is small even though the hue Hue is changed, and the saturation Sat is set to a predetermined value (for example, 20). Is also large, the saturation Sa
Regardless of t, the color signals R1, G1, and B1 are calculated by the processing according to the hue Hue described above.

Next, calculation of a color signal according to the hue Hue, the saturation Sat and the variation V (case 8) will be described. In this method, high-frequency color signals R1, G1, and B1 are calculated by performing weighted addition according to the values of the hue Hue and the saturation Sat. Note that as a specific value, the hue H
A description will be given using ue = 100 °, saturation Sat = 15, and change amount V = 45.

Here, since the hue Hue = 100 ° means a hue intermediate between R and C, it is determined that the hue Hue is R in the processing according to the hue Hue described above, and High-frequency color signals are obtained for both cases where the hue Hue is determined to be C, weighted and added according to the value of the hue Hue, and a high-frequency color signal (here, SHh) corresponding to the hue Hue is obtained. ) Is calculated. Here, the high-frequency color signals obtained when the hue Hue is determined to be R and C are determined to be SH.
Assuming that (R) and SH (C), the high-frequency color signal SHh can be calculated using the weighting coefficient a by the following equation (30).

SHh = (1−a) · SH (R) + a · SH (C) (30) Here, assuming that the weighting coefficient a is as shown in FIG. 15, SHh = (1− (100−97) ) / (156-97)) * SH (R) + (100-97) / (156-97) * SH (C) (31) The high-frequency color signal SHh can be calculated.

Next, the high-frequency color signal SHh thus calculated is weighted and added according to the variation V. Here, the high-frequency color signal calculated according to the change amount V is SH
Assuming that c, the high-frequency color signal SHhc can be calculated using the weighting coefficient b by the following equation (32).

SHhc = (1−b) · SHh + b · SHc (32) Here, assuming that the weighting factor b is as shown in FIG. 16, SHhc = (1− (45−40) / (60−40)) ) * SHh + (45-40) / (60-40) * SHc (33) The high-frequency color signal SHhc can be calculated. The high-frequency color signals calculated according to the change amount V are the high-frequency color signals R1, G1, and B1 calculated in cases 1 to 4 described above.

Further, the high frequency color signal SHhc calculated in this way is weighted and added according to the saturation Sat.
Here, assuming that a high-frequency color signal calculated according to the saturation Sat is SHs, a weight coefficient c is calculated by the following equation (34).
Can be used to calculate the high-frequency color signal SHhcs.

SHhcs = (1−c) · SHhc + c · SHs (34) Here, assuming that the weighting coefficient c is as shown in FIG. 17, SHhcs = (1− (15−10) / (20−10) ) * SHhc + (15-10) / (20-10) * SHs (35) The high frequency color signal SHhcs can be calculated.

Next, the operation of the present embodiment will be described. FIG. 18 is a flowchart showing the operation of the present embodiment. First, photographing is performed by the single-plate CCD 1 to obtain an imaging signal S0 including the color signals R0, G0, and B0 (step S1). Next, the low-frequency color signal generation means 2 generates low-frequency color signals RL, GL, and BL (step S2). On the other hand, in the low-frequency luminance / color difference signal generating means 3, the low-frequency luminance signal YL and the color difference signals CrL, C
bL is generated (step S3). Then, the low-frequency color difference signals CrL and CbL are input to the hue / saturation information generating means 4, where hue / saturation information including the hue Hue and the saturation Sat is generated (step S4). On the other hand, hue H
ue and the saturation Sat are input to the change amount calculating means 5,
Here, the change amount V of the hue Hue and the saturation Sat is calculated (step S5).

When the hue Hue, the saturation Sat, and the variation V are obtained in this manner, the high-frequency color signal generation means 6
In the above, as described above, the hue Hue and the saturation Sa
High-frequency color signals R1, G1, B1 at each pixel position are generated according to the value of t or the amount of change V,
Thereby, a high frequency signal S1 is obtained (step S6),
The process ends.

As described above, according to the present embodiment, the hue H
Since the high-frequency color signals R1, G1, and B1 are generated based on the values of ue and / or the saturation Sat, or the hue / saturation information including the variation V thereof, for example, the high-frequency signal S1 at a certain pixel position Can be calculated based on the hue / saturation information without using a signal value that is complementary to or close to the hue at the pixel position. This can be prevented from occurring.

In the above-described embodiment, the low-frequency color signals RL, GL, and BL are calculated by the method shown in FIG. 4. However, the present invention is not limited to this. The color signals RL, GL, BL can be calculated.

Also, in the above embodiment, the weight coefficients a to c are set to Hue, V, S as shown in FIGS.
Although the function changes linearly according to the value of at, the function is not limited to this, and may be a function that changes three-dimensionally according to the values of Hue, V, and Sat.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating a configuration of a signal generation device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an arrangement of photoelectric conversion elements.

FIG. 3 is a diagram showing a pixel position of an R signal.

FIG. 4 is a diagram for explaining an interpolation operation for calculating a low-frequency color signal;

FIG. 5 is a diagram showing a relationship between hue and saturation.

FIG. 6 is a diagram showing a Robinson edge detection filter;

FIG. 7 is a diagram showing a direction in which edge strength is obtained.

FIG. 8 is a diagram for explaining calculation of edge strength;

FIG. 9 is a diagram illustrating a region having low saturation in the relationship between hue and saturation.

FIG. 10 is a diagram showing an array of pixel positions with reference numerals attached thereto;

FIG. 11 is a diagram for explaining calculation of a high-frequency color signal;

FIG. 12 is a diagram for explaining an interpolation operation;

FIG. 13 is a diagram showing an interpolation calculation result of an edge portion;

FIG. 14 is a diagram for explaining a case where saturation changes in the relationship between hue and saturation.

FIG. 15 is a diagram showing a weighting coefficient a;

FIG. 16 is a diagram showing a weight coefficient b;

FIG. 17 is a diagram showing a weighting coefficient c;

FIG. 18 is a flowchart showing the operation of the present embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Single-chip CCD 2 Low frequency color signal generation means 3 Low frequency luminance color difference signal generation means 4 Hue saturation information generation means 5 Change amount calculation means 6 High frequency color signal generation means

Claims (9)

[Claims] 1. A high-frequency signal at each pixel position corresponding to each photoelectric conversion element, based on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A signal generation method for generating a high-frequency color signal including color information, wherein the high-frequency color signal at each pixel position is interpolated based on a signal value at each pixel position and a pixel position around each pixel position. In the signal generation method of generating a signal, a low-frequency color signal representing low-frequency color information at each of the pixel positions is generated based on the color imaging signal, and each of the pixel positions is generated based on the low-frequency color signal. Generating a low-frequency color difference signal representing low-frequency color difference information, and generating hue saturation information at each of the pixel positions based on the low-frequency color difference signal; A signal generation method comprising: selecting a pixel position on which the interpolation operation is performed based on saturation information; performing an interpolation operation based on a signal value of the selected pixel position to generate the high-frequency color signal. Method. 2. The signal generation method according to claim 1, wherein the hue / saturation information is a hue and / or saturation value at each of the pixel positions. 3. The signal generation method according to claim 1, wherein the hue / saturation information is a change in hue and / or saturation at each of the pixel positions. 4. A high-frequency signal at each pixel position corresponding to each photoelectric conversion element, based on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A signal generation device for generating a high-frequency color signal including color information, wherein the high-frequency color signal at each pixel position is interpolated based on a signal value at each of the pixel positions and pixel positions around the respective pixel positions. A signal generation device that generates a signal, based on the color imaging signal, a low-frequency color signal generation unit that generates a low-frequency color signal representing low-frequency color information at each of the pixel positions; A low-frequency color-difference signal generating unit that generates a low-frequency color-difference signal representing low-frequency color-difference information at each of the pixel positions; Hue / saturation information generating means for generating hue / saturation information at each of the pixel positions; selecting means for selecting a pixel position for performing the interpolation operation based on the hue / saturation information; A signal generating apparatus comprising: a high-frequency color signal generating unit that performs an interpolation operation based on a signal value to generate the high-frequency color signal. 5. The hue / saturation information generating unit according to claim 4, wherein the hue / saturation information generating unit generates the hue / saturation information as a hue and / or saturation value at each of the pixel positions. Signal generator. 6. The device according to claim 4, wherein the hue / saturation information generating unit generates the hue / saturation information as a change in hue and / or saturation at each of the pixel positions. 5. The signal generating device according to 5. 7. A high-frequency signal at each pixel position corresponding to each photoelectric conversion element, based on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A signal generation method for generating a high-frequency color signal including color information, wherein the high-frequency color signal at each pixel position is interpolated based on a signal value at each pixel position and a pixel position around each pixel position. In a computer-readable recording medium recording a program for causing a computer to execute a signal generation method for generating a signal, the program represents low-frequency color information at each of the pixel positions based on the color imaging signal. A step of generating a low-frequency color signal; and a low-frequency color difference information at each of the pixel positions based on the low-frequency color signal. Generating a low-frequency color-difference signal representing the following, generating hue-saturation information at each pixel position based on the low-frequency-color-difference signal, and performing the interpolation operation based on the hue-saturation information A computer-readable recording medium comprising: a step of selecting a pixel position; and a step of performing an interpolation operation based on a signal value of the selected pixel position to generate the high-frequency color signal. 8. The step of generating the hue / saturation information includes: converting the hue / saturation information into a hue and / or a hue at each of the pixel positions.
8. The computer-readable recording medium according to claim 7, wherein said recording medium is generated as a saturation value. 9. The method of generating hue / saturation information, comprising: converting the hue / saturation information to hue and / or
9. The computer-readable recording medium according to claim 7, wherein the computer-readable recording medium is generated as a change in saturation.