JP2010183246A - Color camera - Google Patents

Abstract

A cyclic noise reduction circuit in a single-plate color camera requires a frame memory and has a large hardware configuration. In addition, a noise signal noise reduction circuit, a color difference signal noise reduction circuit, and two noise reduction circuits are required, and the circuit scale is large and expensive. An object of the present invention is to provide a color camera that has a smaller hardware configuration than the prior art and is less expensive than the prior art.
In the color camera of the present invention, noise reduction processing is performed at a RAW signal stage before color separation, so that the noise reduction processing is realized by only one system processing, thereby reducing the circuit configuration to ½. In addition, the high-luminance correction process is performed before the noise reduction process to eliminate the color balance shift in the moving subject.
[Selection] Figure 11

Description

  The present invention relates to a color camera, and more particularly to a noise removal circuit for a video signal acquired by an image sensor in a single-plate color camera.

FIG. 1 is a block diagram showing a configuration example of a cyclic noise reduction circuit in a conventional single-plate color camera. As a circuit for removing a noise component of a video signal, a cyclic noise reduction circuit as shown in FIG. 1 is effective and has been conventionally used.
In the noise reduction circuit of FIG. 1, the video signal acquired by the image sensor of the single-plate color camera is input to one input terminal of the multiplier 11 as RAW signal data. The multiplier 11 multiplies the coefficient (1-K) input from the other input terminal by the input RAW signal and outputs the product to one input terminal of the adder 12. The adder 12 adds the signal input from the multiplier 11 to the signal input from the other input terminal, outputs the signal to the frame memory 14, and outputs the output signal of the noise reduction circuit as a video signal after noise reduction. To do.
The frame memory 14 delays the input video signal by one frame period, and then outputs it to one input terminal of the multiplier 13. The multiplier 13 multiplies the video signal delayed by one frame period by a coefficient K input from the other input terminal, and outputs the product to the other input terminal of the adder 12.
The coefficient K setting unit 16 outputs the set coefficient K to the other input terminal of the multiplier 13 and also outputs the coefficient K to the coefficient (1-K) generation unit 15. The coefficient (1-K) generation unit 15 generates a coefficient (1-K) based on the input coefficient K and outputs it to the other input terminal of the multiplier 11.
That is, the noise reduction circuit as shown in FIG. 1 is configured to perform cumulative addition using a recursive filter, and only noise that is a random component is canceled, and noise reduction is realized by effectively extracting signal components. Yes.
Note that K is a natural number of 0 <K <1.

In addition, as a general single-plate color camera, as shown in FIG. 2, the signal (RAW signal) output from the image sensor (sensor) is color-separated and corrected into a luminance signal and a color signal. Perform noise reduction on each. FIG. 2 is a block diagram showing components until a luminance signal and a color signal are separated from an image sensor in a conventional single-plate color camera.
2, an image sensor (sensor) 21 outputs a subject image formed through an optical system (not shown) to the color separation filter unit 22 as a video signal of a RAW signal. The color separation filter unit 22 generates a luminance signal and a color signal from the input RAW signal, outputs the luminance signal to the luminance signal noise reduction unit 23, and outputs the color signal to the color signal noise reduction unit 24.
Each of the luminance signal noise reduction unit 23 and the color signal noise reduction unit 24 removes noise from the input signal, for example, with a cyclic noise reduction circuit as described with reference to FIG.

The signal output from the image sensor has the configuration shown in FIG. FIG. 3 shows the case of a primary color type image pickup device, but a case of a complementary color type image pickup device is also conceivable. Here, for the sake of simplicity, description will be made with a primary color type image sensor.
In a color camera such as a single-plate color camera, the imaging device is a solid-state imaging device in which a plurality of types of color filters having different transmittances are grounded in a predetermined arrangement on the surface of each pixel. That is, the image sensor is a solid-state image sensor provided with a single-plate color filter. This color filter is provided with a different type of color filter for each adjacent pixel. Therefore, the video signal output from the image sensor is a so-called RAW image signal in which each pixel has only single color information.
As a single plate type color filter installed in such an image sensor, for example, a Bayer filter including an R (red) filter, a G (green) filter, and a B (blue) filter is configured. For example, this Bayer filter includes a line in which G filters and B filters are alternately arranged in the horizontal direction and a line in which R filters and G filters are alternately arranged in the horizontal direction so that they are alternately arranged in the vertical direction. . That is, the lines in which the G filter and the R filter are alternately arranged in the vertical direction and the lines in which the B filter and the G filter are alternately arranged in the vertical direction are alternately arranged in the horizontal direction. With such an image sensor having a Bayer filter, a video signal (RAW signal) that becomes an RGB signal is output as an array of colors as shown in FIG. , Only a part of pixels of the image sensor is reflected and displayed.)

First, the relationship between the RAW signal and the color separation circuit will be briefly described.
In each pixel, there is only a color signal of 1CH. Generally, in color separation correction, color components for 3CH are extracted by interpolation from surrounding signal information. For example, in the G-phase pixel in FIG. 3, interpolation is performed from surrounding R-phase signals to generate a G-phase R signal. Similarly, interpolation is performed from surrounding B-phase signals to generate G-phase B signals.
Further, in the R-phase pixel, interpolation is performed from surrounding G-phase signals to generate R-phase G signals. Similarly, an R-phase B signal is generated from peripheral B-phase signals.
Similarly, the B-phase pixel generates a 3CH signal.
From the obtained R, G, and B 3CH signals, matrix calculation is performed and converted into a luminance signal Y, a color difference signal RY, and a color difference signal BY according to the following equations. R, G, and B in this equation are the luminance values of the signals for each CH.

Luminance Y = 0.299R + 0.587G + 0.114B (1)
Color difference R−Y = 0.700R−0.590G−0.110B (2)
Color difference BY = −0.300R−0.590G + 0.890B Expression (3)

Before the conversion, the color balance is adjusted for each of the R, G, and B signals after color separation interpolation.
There are various factors in which the color balance is lost, but the biggest factor is the difference in sensitivity characteristics between CHs for each color in the image sensor.
In an image sensor of a single-plate type color camera, R, G, and B color components are extracted as R signals, G signals, and B signal CH signals from R, G, and B color filters provided for each pixel. It has a configuration. These R signals, G signals, and B signals have different imaging sensitivities because the color filters are different. The same applies to the case where a complementary color type image sensor is used.
In the case of an RGB primary color type image pickup device, the image pickup sensitivity of the G signal is generally high, and the image pickup sensitivity of the R signal and the B signal is low. In some cases, a sensitivity difference of twice or more occurs. This sensitivity difference causes the color balance to be lost. In order to correct this, conventionally, white balance control is required. White balance control refers to the correction of color misregistration between RGB in the R, G, and B signals after a white subject is projected and color-separated and interpolated (after the circuit shown in FIG. 2). In addition, the color balance between RGB is corrected.

FIG. 4 is a block diagram showing a conventional example of the configuration of the white balance control circuit. The R signal, the G signal, and the B signal are respectively input to the multipliers 41R, 41G, and 41B for each CH, and are output from the R-GAIN correction unit 44R, the G-GAIN correction unit 44G, and the B-GAIN correction unit 44B, respectively. The corrected data is multiplied, and the respective products are output and output to the R-CH integrator 42R, the G-CH integrator 42G, and the B-CH integrator 42B.
The R-CH integrator 42R, the G-CH integrator 42G, and the B-CH integrator 42B each output an integration value to the CPU 43, and the CPU 43 outputs a correction signal based on the input integration value to the R-GAIN correction unit. 44R, G-GAIN correction unit 44G, and B-GAIN correction unit 44B. The R-GAIN correction unit 44R, the G-GAIN correction unit 44G, and the B-GAIN correction unit 44 output correction data corresponding to the input correction signal to the multipliers 41R, 41G, and 41B, respectively. With these correction data, the input R signal, G signal, and B signal are clipped and output by multipliers 41R, 41G, and 41B, respectively.

Based on the above configuration, the problem of noise reduction at the RAW signal stage will be further described.
In general, as described above, the imaging sensitivity between each color CH is high for the G signal and low for the R signal and the B signal. As shown in FIG. 6, in the color camera, when white light is incident, as the light amount increases, the output of the image sensor first overflows and collapses the charge of the G signal. In this case, the R signal and the B signal are not yet overflowing because the sensitivity is low. FIG. 6 shows the input / output characteristics of each color signal. FIG. 6 is a diagram illustrating the input level on the horizontal axis and the output level on the vertical axis. Similarly, FIGS. 7 to 10 and FIGS. 12 and 13 shown in the prior art and embodiments of the present invention show input / output characteristics with the horizontal axis representing the input level and the vertical axis representing the output level. FIG.

FIG. 7 shows characteristics after the color balance between RGB is corrected by the white balance control circuit shown in FIG. In FIG. 7, the horizontal axis in FIG. 4 represents the R signal, the G signal, and the B signal, and the vertical axis represents the respective output.
In FIG. 7, the signal level is clipped at 100% for the G signal, whereas the R signal and the B signal are extended at a high luminance portion where the signal is 100% or more. For this reason, the color balance is lost and the color becomes magenta.
However, since a signal of 100% or more is clipped to 100% at the camera output stage, it becomes 100% as in the case of R, G, and B, and no color shift occurs. Note that the clip is a function for setting all level signals equal to or higher than Lc to Lc, where the clip value is Lc.
The characteristics in that case are shown in FIG. As shown in FIG. 8, the G signal is clipped at 100% or more (Lc = 100%) and becomes white after output.

JP 2005-175864 A Japanese Unexamined Patent Publication No. 07-087511

As can be seen from the above description of the conventional example, the cyclic noise reduction circuit in the single-plate color camera requires a frame memory and has a large hardware configuration. In addition, a noise signal noise reduction circuit, a color difference signal noise reduction circuit, and two noise reduction circuits are required, and the circuit scale is large and expensive.
Japanese Patent Application Laid-Open No. H10-228561 describes that a cyclic noise removal filter is used for RAW signal data in order to reduce the amount of data to be calculated. The control of the white balance processing is also executed with the RAW signal data. However, the purpose is to simplify noise removal in moving image shooting at low illuminance, and no consideration is given to countermeasure technology at high luminance.
Patent Document 2 describes a technique for providing a threshold value for each color component in order to prevent coloring at high luminance. However, although white balance control is processed in the RAW signal, there is no description regarding noise reduction processing.
In view of the above problems, an object of the present invention is to provide a color camera that has a small hardware configuration and is cheaper than the conventional one.

In order to solve the above problems, the present invention reduces the circuit configuration by half by implementing noise reduction processing with only one system processing by performing noise reduction processing at the RAW signal stage before color separation. did. Further, the high-luminance correction process is performed before the noise reduction process, thereby eliminating the color balance shift in the moving subject.
That is, the present invention provides a high-intensity correction unit that corrects a sensitivity difference of a color signal with respect to a RAW signal obtained by an image sensor, and a noise that performs noise removal processing on the signal that has been corrected for the sensitivity difference of the color signal. A color camera comprising a removal circuit and means for performing color separation processing on the noise-removed signal.

  According to the present invention, it is possible to provide a noise reduction circuit that has a small hardware configuration and is less expensive than the conventional one. Further, by inserting a high brightness correction circuit before the noise reduction circuit, it is possible to provide a color camera that does not cause a color balance shift even for a moving subject.

It is a block diagram which shows the structural example of the conventional cyclic | annular noise reduction circuit. It is a block diagram which shows the component until it isolate | separates a luminance signal and a color signal from the image pick-up element in the conventional single plate type color camera. It is a figure for demonstrating the output signal in the case of a primary color type image sensor. It is a block diagram which shows an example of a structure of the conventional white balance control circuit. It is a block diagram which shows the structure of the color camera of one Example of this invention. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a block diagram which shows the structure of the color camera of one Example of this invention. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a figure which shows an input / output characteristic by taking an input level on a horizontal axis and taking an output level on a vertical axis. It is a block diagram which shows the structure of the white balance control circuit of one Example of this invention.

The color camera of the present invention, for example, in a color camera such as a single-plate color camera, performs cyclic noise removal processing at a RAW signal stage before color separation interpolation is performed on a signal obtained by an image sensor (sensor). A noise reduction circuit is provided.
Further preferably, it further includes means for correcting the sensitivity difference of the color signal in the previous stage of the noise reduction circuit that performs the cyclic noise removal process, and for the signal after the sensitivity difference correction process, a signal having a specific value or more is detected. A process of replacing with a constant value is performed, and after eliminating the color collapse of the high luminance signal, a cyclic noise removal process is performed.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of each drawing, the same reference numerals are given to components having the same function, including the drawings describing the prior art, and description thereof will be omitted as much as possible to avoid duplication.

An embodiment of the color camera of the present invention will be described with reference to FIG.
FIG. 5 is a block diagram showing components until the luminance signal and the color signal are separated from the image sensor in the single-plate color camera of the present invention. FIG. 5 is a block diagram showing the configuration of a color camera according to an embodiment of the present invention when the white balance control circuit described in FIG. 4 is configured to process at the RAW signal stage.
In FIG. 5, the RAW signal output from the image sensor (sensor) is subjected to noise reduction and then subjected to color separation correction. That is, the image sensor (sensor) 21 outputs a subject image formed through an optical system (not shown) to the noise reduction unit 53 as a video signal of a RAW signal. The noise reduction unit 53 removes noise from the input RAW signal and outputs it to the color separation filter unit 52. The color separation filter unit 52 generates and outputs a luminance signal and a color signal from the input noise-removed RAW signal.
The noise reduction unit 53 removes noise from the input signal, for example, using a cyclic noise reduction circuit as described with reference to FIG.

  According to the embodiment of FIG. 5, by performing noise reduction processing at the RAW signal stage before color separation, the circuit configuration can be reduced to ½ by realizing noise reduction processing with only one system processing. .

However, when noise reduction is performed with the block configuration shown in FIG. 1 in the RAW signal stage, there is no problem with still images. However, when there is a motion between frames in a moving image, a coefficient is applied and cumulative addition is performed, so that the input video is multiplied by a gain.
For example, when a step (step) image that changes from level = 0 to level = L is included in the input image, the noise reduction output image level NR is expressed by the following equation.
NR = L × (1−K) × (1 + K ₁ + K ₂ + K ₃ +... + K _n ) (3)
It becomes.
Here, n is the number of frames.
When the coefficient K is 0.75 (K = 0.75),
The output level NR is 0.25, 0.4375, 0.578,... And the number of frames n until the output level NR reaches 95% of the level L is 10 (n = 10).
That is, if the video moves even for a moment, it takes about 10 frames to settle down as a video after noise reduction even if the motion stops. Is displayed.

In addition, when strong light is incident and a signal in which the G signal is crushed moves at the output stage of the image sensor, color separation processing and white balance processing are performed, as shown in FIG. A phenomenon (white crushing) different from the clip processing shown occurs.
In other words, at the output stage of the image sensor, the image is the same as in FIG. 6, but the image at the moment when there is a motion is multiplied by a coefficient K with an n-order polynomial when the noise reduction process is executed.
For example, if the coefficient K is 0.75, the NR is 0.25 (25%) at the level L in the first frame (n = 1), and the NR at the level L is 0.4375 in the next frame (n = 2). (About 44%). Therefore, the input / output characteristics are as shown in FIG. When this is subjected to white balance processing, the characteristics shown in FIG. 10 are obtained, and a magenta color is generated in an area of 100% or less. That is, when a subject with high brightness moves, white crushing occurs according to the input / output characteristics shown in FIG. That is, during the noise reduction process, the input / output characteristics shown in FIG. 10 are followed, so that the color balance is lost at high luminance, and the white display portion appears magenta. 9 and 10 show the input / output characteristics in the case of the second frame (n = 2) from the moving image.
Eliminate this problem, improve color reproducibility, ensure that the color balance is not lost even at high brightness, and the white display area is displayed in its original white color even when a bright subject moves. Is the next issue.

In order to solve the following problem, in another embodiment of the present invention, as shown in FIG. 11, a high luminance correction circuit is inserted before the RAW signal noise reduction circuit.
The color balance of a color camera according to another embodiment of the present invention will be described with reference to the block configurations of FIGS. FIG. 11 is a block diagram showing the configuration of the color camera of one embodiment of the present invention. FIG. 14 is a block diagram showing the configuration of a white balance control circuit according to an embodiment of the present invention.

11 is obtained by inserting a high-intensity correction circuit 110 shown in FIG. 14 between the image sensor (sensor) 21 and the noise reduction unit 53 in addition to the configuration of FIG. At the output stage of the image sensor 21, the input / output characteristics are the same as those in FIG.
That is, in FIG. 11, the high luminance correction circuit 110 in FIG. 14 corrects the sensitivity difference among the R, G, and B CHs for the RAW signal input from the image sensor 21 first, and then the noise reduction unit 53. Output to. The noise reduction unit 53 removes noise from the input RAW signal and outputs it to the color separation filter unit 52 as in FIG. The color separation filter unit 52 generates and outputs a luminance signal and a color signal from the input noise-removed RAW signal.

Details of the high luminance correction circuit 110 of FIG. 14 will be described. The RAW signal data output from the image sensor 21 is input to the multiplier 161. The multiplier 161 multiplies the input RAW signal data by the signal input from the selector 166, and the product together with the clipping circuit 167 for each corresponding color signal, the R-CH integrator 162R and the Gr-CH integrator 162Gr. , Gb-CH integrator 162Gb or B-CH integrator 162B.
For example, the RAW signal data is input to the multiplier 161 for each pixel from the left to the right of the horizontal line, and the following processing is performed (see FIG. 3).
When the pixel of which the horizontal line color component is G, B, G, B, G, B,... Is processed, the multiplier 161 first outputs the video signal of the G signal component in each pixel. Data is output to the Gb-CH integrator 162Gb, and video signal data of the B signal component is output to the B-CH integrator 162B. Next, when the pixels whose color components of the horizontal line are R, G, R, G, R, G,... Are processed, the video signal data of the R signal component is R- The signal is output to the CH integrator 162R, and the video signal data of the G signal component is output to the Gr-CH integrator 162Gr.

The control unit (CNT) 165 outputs a predetermined control signal to the R-CH integrator 162R, the Gr-CH integrator 162Gr, the Gb-CH integrator 162Gb, the B-CH integrator 162B, and the selector 166. The integrators 162R, 162Gr, 162Gb, and 162B select and control data writing and reading timings and correction data output by the selector 166.
For example, when the imaging device is a CCD, a clock signal is acquired from a timing generator (not shown) for driving the CCD, counted by a counter (not shown), and when a predetermined count value is reached, a predetermined value is obtained. Outputs timing signals and selection control signals.

  The R-CH integrator 162R, the Gr-CH integrator 162Gr, the Gb-CH integrator 162Gb, and the B-CH integrator 162B, respectively, write a signal, A value obtained by integrating within a time range or a predetermined sample amount is read and output to the CPU 163.

The CPU 163 outputs a correction signal to the R-GAIN correction unit 164R, the Gr-GAIN correction unit 164Gr, the Gb-GAIN correction unit 164Gb, and the B-GAIN correction unit 164B based on the input integral values.
Preferably, the CPU 163 controls the control unit 165 via a control line (not shown) to change the timing cycle of the control unit 165 and the selection control cycle.

The R-GAIN correction unit 164R, the Gr-GAIN correction unit 164Gr, the Gb-GAIN correction unit 164Gb, and the B-GAIN correction unit 164B each calculate correction data corresponding to the input correction signal, and calculate the correction data Are output to the selector 166, respectively. The selector 166 selects each of the input correction data according to the timing of the timing signal input from the control unit 165 and outputs the selected correction data to the multiplier 161.
The clip circuit 167 applies a predetermined clip to the input signal and outputs it.
The following processing has been described with reference to FIG.

As in the RAW signal-output signal characteristic input to the multiplier 161 shown in FIG. 7, the signal level is clipped at 100% for the G signal, whereas the R signal and the B signal are 100% or more. It extends in the high brightness area. For this reason, the color balance is lost, and in the case of high luminance, the color balance is lost, and the white display portion appears magenta and becomes magenta. However, when this is clipped by the clipping circuit 167, the input / output characteristics shown in FIG. 12 can be obtained.
Therefore, when there is a movement in the subsequent stage through the output of the noise reduction circuit 110, the input / output characteristics shown in FIG. 13 are obtained, the color balance is not lost, and no color shift occurs.

As described above, according to the invention of FIG. 11, it is possible to provide a noise reduction circuit that has a small hardware configuration and is less expensive than the conventional one. Further, by inserting a high brightness correction circuit before the noise reduction circuit, it is possible to provide a color camera that does not cause a color balance shift even for a moving subject.
In other words, the color camera of the present invention corrects the sensitivity difference between the color signal, the image sensor that captures the subject and outputs the RAW signal, the high-intensity correction unit that corrects the sensitivity difference between the color signals of the RAW signal. A noise removal circuit for performing noise removal processing on the processed signal, and means for performing color separation processing on the noise-removed signal.

  In the above embodiment, the cyclic noise reduction circuit and the white balance control circuit of the television camera have been described with the hardware configuration. However, it goes without saying that all or part of the circuit including these may be software configured by a DSP or FPGA.

  In the above embodiment, the primary color type image sensor has been described. However, instead of the primary color type image sensor, a complementary color type image sensor may be used.

  11: Multiplier, 12: Adder, 13: Multiplier, 14: Frame memory, 15: Coefficient (1-K) generation unit, 16: Coefficient K setting unit, 21: Image sensor, 22: Color separation filter unit, 23: luminance signal noise reduction unit, 24: color signal noise reduction unit, 41R, 41G, 41B: multiplier, 42R: R-CH integrator, 42G: G-CH integrator, 42B: B-CH integrator, 43 : CPU, 44R: R-GAIN correction unit, 44G: G-GAIN correction unit, 44B: B-GAIN correction unit, 52: Color separation filter unit, 53: Noise reduction unit, 110: High luminance correction circuit, 161: Multiplication 162R: R-CH integrator, 162Gr: Gr-CH integrator, 162Gb: Gb-CH integrator, 162B: B-CH integrator, 16 : CPU, 164R: R-GAIN correction unit, 164Gr: Gr-GAIN correction unit, 164Gb: Gb-GAIN correction unit, 164B: B-GAIN correction unit, 165: control unit, 166: Selector, 167: clip circuit.

Claims (1)

  In a color camera, an image pickup device that picks up an image of a subject and outputs a RAW signal, a high-intensity correction unit that corrects a sensitivity difference between the color signals of the RAW signal, and a noise that is generated by correcting the sensitivity difference between the color signals A color camera comprising: a noise removal circuit that performs a removal process; and a unit that performs a color separation process on the noise-removed signal.

Images