JP2007158964A - Image processing apparatus and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a flicker component in an image picked up by an XY address scan type solid-state imaging device, using an image processing apparatus of small circuit scale. <P>SOLUTION: An integrated value due to an integration processing unit 211 is averaged for each field by a filter processing unit 213, a differential value of an integrated value in an adjacent field due to a differential computing unit 214 is normalized by a normalization processing unit 215 using an output value of the filter processing unit 213, and a spectrum of the normalized differential value is extracted by a DFT processing unit 220. Furthermore, the filter processing unit 213 is comprised of a recursive filter, and a value that becomes greater as a filter coefficient (k) for the filter processing unit 213 as an amount of camera shake detection due to a camera shake detection unit is greater, is set under control of a system control. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that processes an image signal, and an imaging apparatus having this function, and in particular, an image processing apparatus and an imaging apparatus suitable for processing an image signal captured by an XY address scanning type solid-state imaging device. About.

  When shooting a subject with a video camera under the illumination of a blinking light source such as a fluorescent lamp that is turned on by a commercial AC power supply, the captured image is taken due to the difference between the luminance change (light quantity change) frequency of the light source and the vertical synchronization frequency of the camera. It is known that a change in brightness over time, so-called fluorescent lamp flicker, occurs on the top. In particular, when an XY address scanning type imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) type image sensor is used, the exposure timing for each horizontal line is different, and therefore flicker on the captured image is periodic in the vertical direction. It is observed as a striped pattern due to variations in brightness level or hue.

  As a method for removing such flicker components from the captured image signal, a correction method based on the relationship between the shutter speed and the flicker level (referred to as a shutter correction method) and a flicker waveform are detected. A method of applying the inverse waveform to the image signal as a correction gain (referred to as a gain correction method) is known. Of these, the flicker reduction method using the gain correction method detects the flicker frequency spectrum by analyzing the change in the signal level of the image signal, and corrects the signal level of the image signal based on the amplitude value of the spectrum. There was a method (for example, refer patent document 1).

In the flicker reduction method using the gain correction method, the integral value of the image signal is averaged, the integral value for each region is divided by the averaged value, and the spectrum is calculated using the divided value. Thus, it is known that flicker components can be extracted with high accuracy even when the subject moves. Further, as an integration value averaging process, a technique using, for example, a cyclic filter or an FIR (Finite Impulse Response) filter is known (see, for example, Patent Document 2).
JP 2004-222228 A (paragraph numbers [0072] to [0111], FIG. 4) JP 2001-1111887 (paragraph numbers [0055] to [0064], FIG. 8)

  By the way, when the FIR filter is used for averaging the integration values of the image signals described above, it is easy to obtain a desired frequency characteristic, but the memory capacity for temporarily storing the image signals is large. For example, the circuit scale is relatively large. On the other hand, when the recursive filter is used, it is difficult to obtain a desired frequency characteristic as compared with the case of the FIR filter. In particular, when a large change occurs in the subject due to camera shake or the like, the flicker component cannot be sufficiently suppressed from the image signal by averaging, and the flicker component extraction accuracy is lowered. However, when a cyclic filter is used, there is an advantage that the circuit scale can be reduced as compared with the case of the FIR filter.

  The present invention has been made in view of these points, and provides an image processing apparatus with a small circuit scale that can accurately detect a flicker component of an image captured by an XY address scanning type solid-state imaging device. Objective.

  Another object of the present invention is to provide an imaging device including a detection device with a small circuit scale that can accurately detect a flicker component of an image captured by an XY address scanning type solid-state imaging device.

  In the present invention, in order to solve the above problems, in an image processing apparatus for processing an image signal, an integration means for integrating the image signal in units of one horizontal synchronization period or more, and an integration value by the integration means is set to one field or 1 A filter processing means comprising a cyclic filter that averages every frame, and an integration value by the integration means or a difference value between integration values in adjacent fields or frames is normalized by using an output value of the filter processing means. Generated from the normalization means, the frequency analysis means for extracting the spectrum of the integral value or the difference value after normalization by the normalization means, and the spectrum extracted by the frequency analysis means on the screen under fluorescent lamp illumination. Flicker estimation means for estimating flicker components, camera shake detection means for detecting camera shake during imaging, and camera shake detection Depending on the vibration detection amount by stages, the image processing apparatus is provided, characterized in that it comprises a filter control means for changing a set value of the filter coefficient for the filtering unit.

  In such an image processing apparatus, the integration value by the integration means is averaged for each field or frame by the filter processing means, and the integration value by the integration means or the difference value of the integration values in adjacent fields or frames is filtered. By normalizing by the normalizing means using the output value of the processing means and extracting the spectrum of the normalized integral value or difference value by the frequency analyzing means, the flicker component can be detected with high accuracy even from a non-uniform image. The Further, by configuring the filter processing means with a recursive filter, the memory capacity for holding a signal necessary for the filter processing is reduced, and the processing load is reduced. Then, the filter control means changes the filter coefficient for the filter processing means in accordance with the amount of camera shake detected by the camera shake detection means, thereby making it difficult for false detection of flicker components when the screen motion is large.

  According to the image processing apparatus of the present invention, by configuring the filter processing means for averaging the integral values with a cyclic filter, it is possible to reduce the memory capacity for holding a signal necessary for the filter processing, and to reduce the processing load Can be reduced. Further, by changing the filter coefficient for the filter processing means in accordance with the amount of camera shake detection, it is difficult for false detection of flicker components when the screen motion is large. Therefore, an image processing apparatus with high flicker component detection accuracy and a small circuit scale can be realized.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<System configuration>
FIG. 1 is a block diagram showing a main configuration of an imaging apparatus according to an embodiment of the present invention.

  1 includes an optical block 11, a driver 11a, a CMOS image sensor (hereinafter abbreviated as a CMOS sensor) 12, a timing generator (TG) 12a, an analog front end (AFE) circuit 13, and a camera processing circuit 14. A system controller 15 and a camera shake detector 16.

  The optical block 11 includes a lens for condensing light from the subject on the CMOS sensor 12, a drive mechanism for moving the lens to perform focusing and zooming, a shutter mechanism, an iris mechanism, and the like. The driver 11 a controls driving of each mechanism in the optical block 11 based on a control signal from the system controller 15.

  In the CMOS sensor 12, a plurality of pixels including a photodiode (photogate), a transfer gate (shutter transistor), a switching transistor (address transistor), an amplification transistor, a reset transistor (reset gate), etc. are two-dimensionally formed on a CMOS substrate. And a vertical scanning circuit, a horizontal scanning circuit, an image signal output circuit, and the like are formed. The CMOS sensor 12 is driven based on a timing signal output from the TG 12a, and converts incident light from the subject into an electrical signal. The TG 12 a outputs a timing signal under the control of the system controller 15.

In this embodiment, the analog image signal obtained from the CMOS sensor 12 is a primary color signal of each color of RGB (Red, Green, Blue), but may be a complementary color system color signal, for example.
The AFE circuit 13 is configured, for example, as one IC (Integrated Circuit), and improves the S / N (Signal / Noise) ratio with respect to the image signal output from the CMOS sensor 12 by CDS (Correlated Double Sampling) processing. The sample is held so as to be maintained, the gain is controlled by AGC (Auto Gain Control) processing, A / D (Analog / Digital) conversion is performed, and a digital image signal is output. Note that the circuit for performing the CDS process may be formed on the same substrate as the CMOS sensor 12.

  The camera processing circuit 14 is configured as, for example, one IC, and performs various signal corrections such as white balance adjustment, AF (Auto Focus), and AE (Auto Exposure), which will be described later, on the image signal from the AFE circuit 13. Perform signal processing or execute part of the processing. In the present embodiment, in particular, a flicker reduction unit 20 is provided that reduces the flicker signal component generated on the screen during imaging under a fluorescent lamp from the image signal.

  The system controller 15 is a microcontroller composed of, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and by executing a program stored in the ROM or the like, Each part of this imaging apparatus is controlled in an integrated manner. Further, it communicates with the camera processing circuit 14 to execute a part of the calculation for the various camera signal processing. Further, in the present embodiment, as will be described later, a filter coefficient k for filtering the image signal is set in the flicker reducing unit 20 of the camera processing circuit 14.

  The camera shake detection unit 16 detects the amount of camera shake applied to the imaging apparatus and outputs the detection result to the system controller 15. The camera shake detection unit 16 is preferably configured by another detection mechanism separated from the captured image signal system, such as an angular velocity sensor such as a gyro sensor or an acceleration sensor.

  In this image pickup apparatus, the signals received and photoelectrically converted by the CMOS sensor 12 are sequentially supplied to the AFE circuit 13, subjected to CDS processing and AGC processing, and then converted into digital signals. The camera processing circuit 14 performs image quality correction processing on the digital image signal supplied from the AFE circuit 13, and finally converts the digital image signal into a luminance signal (Y) and a color difference signal (RY, BY) and outputs the result.

  The image data output from the camera processing circuit 14 is supplied to a graphic I / F (interface) (not shown) and converted into an image signal for monitor display, whereby a camera-through image is displayed on the monitor. When the system controller 15 is instructed to record an image by a user input operation or the like, the image data from the camera processing circuit 14 is supplied to an encoder (not shown) and subjected to a predetermined compression encoding process. It is recorded on a recording medium (not shown). At the time of recording a still image, image data for one frame is supplied from the camera processing circuit 14 to the encoder, and at the time of recording a moving image, the processed image data is continuously supplied to the encoder. .

<Flicker detection method>
Next, a flicker detection method will be described. First, FIG. 2 is a diagram for explaining flicker.

  Flicker occurs when shooting is performed under a blinking light source such as a fluorescent lamp, and when the image is captured by an XY address scanning type imaging device such as a CMOS sensor, a period in the vertical direction as shown in an image P in FIG. It is observed as a change in luminance level and hue. The image P shows a state in which flicker appears as a bright and dark stripe pattern when the subject is uniform.

  Here, for example, a fluorescent lamp with a commercial AC power supply with a frequency of 50 Hz has a blinking frequency of 100 Hz. Therefore, in a video signal of the NTSC (National Television Standards Committee) system with a field frequency of 60 Hz, the number of lines per field is M. When this occurs, one period L of the striped pattern is (M × 60/100) lines. In addition, such periodic fluctuation in one field occurs 100/60 = 1.66 period. That is, such a periodic variation is repeated every three fields.

FIG. 3 is a block diagram illustrating an internal configuration of the flicker detection unit.
As shown in FIG. 3, the flicker reducing unit 20 detects the image signal, normalizes the detected value and outputs the normalized integrated value calculating unit 210, and DFT processing for performing DFT processing on the normalized detection value Unit 220, a flicker generation unit 230 that estimates a flicker component from the result of spectrum analysis by DFT, and a calculation unit 240 that executes a calculation for removing the estimated flicker component from the image signal. Further, the normalized integral value calculation unit 210 includes an integration processing unit 211, an integration value delay unit 212, a filter processing unit 213, a difference calculation unit 214, and a normalization processing unit 215.

  In the imaging apparatus according to the present embodiment, the processing by the block shown in FIG. 3 is executed for each of the luminance signal and color difference signal constituting the video signal. Alternatively, it may be executed for at least the luminance signal, and may be executed for the color difference signal and each color signal as necessary. Further, the luminance signal may be executed at the stage of the color signal before being synthesized with the luminance signal, and the process at this stage of the color signal may be performed at any stage of the color signal by the primary color or the color signal by the complementary color. May be executed. When these color signals are executed, processing by the block shown in FIG. 3 is executed for each color signal.

Hereinafter, the flicker detection method will be described in detail with reference to FIG.
In general, the flicker component is proportional to the signal intensity of the subject. Therefore, when an input image signal (RGB primary color signal or luminance signal before flicker reduction) in an arbitrary field n and an arbitrary pixel (x, y) for a general subject is In ′ (x, y), In ′ (x, y) x, y) is expressed by the following equation (1) as a sum of a signal component not including a flicker component and a flicker component proportional to the signal component.
In ′ (x, y) = [1 + Γn (y)] × In (x, y) (1)
Here, In (x, y) is a signal component, Γn (y) × In (x, y) is a flicker component, and Γn (y) is a flicker coefficient. One horizontal period is sufficiently shorter than the light emission period (1/100 second) of the fluorescent lamp, and the flicker coefficient can be regarded as constant in the same line of the same field, so the flicker coefficient is represented by Γn (y).

  In order to generalize Γn (y), it is described in a form expanded to a Fourier series as shown in equation (2). Thus, the flicker coefficient can be expressed in a format that covers all the light emission characteristics and afterglow characteristics that differ depending on the type of fluorescent lamp.

  In the equation (2), λ0 is the wavelength of the flicker in the screen shown in FIG. 2, and when the number of read lines per field is M, it corresponds to L (= M / λ0) lines. ω0 is a normalized angular frequency normalized by λ0.

  γm is the amplitude of each flicker component (m = 1, 2, 3,...). Φm, n indicates the initial phase of each next flicker component, and is determined by the light emission period (1/100 second) of the fluorescent lamp and the exposure timing. However, Φm, n is determined by the vertical synchronization frequency and the fluorescent lamp frequency as shown in Equation (3), and the difference in Φm, n from the immediately preceding field is expressed by Equation (4).

<Calculation and storage of integral value>
In the flicker reducing unit 20 shown in FIG. 3, first, the input image signal In ′ (x, y) is represented by the equation (5) in the integration processing unit 211 in order to reduce the influence of the pattern for flicker detection. In this way, integration is performed over one line in the horizontal direction of the screen, and an integrated value Fn (y) is calculated. Note that α _n (y) in equation (5) is an integral value over one line of the signal component In (x, y) as represented by equation (6).

  The calculated integral value Fn (y) is temporarily stored in the integral value delay unit 212 for flicker detection in subsequent fields. The integral value delay unit 212 is configured to always output an integral value one field before by storing the inputted integral value for one vertical synchronization period.

<Filter processing and difference calculation>
Here, if the subject is uniform, the integral value α _n (y) of the signal component In (x, y) becomes a constant value, and therefore, from the input image signal In ′ (x, y), α _n (y ) × Γn (y), it is easy to extract the flicker component.

However, in a general subject, the integral value α _n (y) also includes a component of m × ω0, so that a luminance component and a color component as flicker components and a signal component of the subject itself are accurately separated. Therefore, it is difficult to extract only flicker components purely. Furthermore, since the component of the second term (α _n (y) × Γn (y)) is very small with respect to the first term (α _n (y)) in the third row of the equation (5), the flicker component Are almost buried in the signal component.

On the other hand, in this embodiment, in order to remove the influence of α _n (y) from the integrated value Fn (y), the average value AVE [Fn of integrated values averaged for each field by the filter processing unit 213. (Y)] is used to extract the flicker component. The filter processing unit 213 is configured by a recursive filter. When the transfer function of the filter is H (z) and the filter coefficient is k, the average value AVE [Fn (y)] of the integral values is calculated by the following equations (7) and (8). In addition, Formula (7) represents the average value of integral values converted by z.

  By averaging the integration values using a recursive filter, it is no longer necessary to hold integration values for a plurality of fields for filter processing as in the case of using an FIR filter, and the memory capacity for that purpose is reduced. be able to. Further, the processing load is reduced as compared with the FIR filter. Therefore, the average value AVE [Fn (z)] in which the flicker component is canceled can be obtained by a circuit having a small circuit scale and reduced manufacturing cost and power consumption.

  However, the above explanation is about the case where the average value of the integral values output from the cyclic filter is calculated on the assumption that the approximation of the above equation (9) is established. The approximation of (9) does not hold. For this reason, when the movement of the subject is large, the filter coefficient k of the recursive filter is increased. Thereby, the possibility of erroneous detection is reduced. Conversely, when the movement of the subject is small, the flicker component is more strongly suppressed by reducing the filter coefficient k. Such control of the filter coefficient k will be described later with reference to FIGS.

  In the present embodiment, the difference calculation unit 214 further integrates the integration value Fn (y) of the field from the integration processing unit 211 and the integration value Fn_1 (y) of the previous field from the integration value delay unit 212. And a difference value Fn (y) −Fn_1 (y) represented by the following equation (10) is calculated. This equation (10) is also premised on the approximation of the above equation (9).

<Normalization of difference value>
In the flicker reduction unit 20 of FIG. 3, the normalization processing unit 215 further calculates the difference value Fn (y) −Fn — 1 (y) from the difference calculation unit 214 as the average value AVE for each field from the filter processing unit 213. Normalization is performed by dividing by [Fn (y)], and a normalized difference value gn (y) is calculated.

  The normalized difference value gn (y) is expanded as shown in Expression (11) by the above product (8), Expression (10), and the trigonometric product formula. It is represented by (12). In addition, | Am | and θm in the equation (12) are expressed by the equations (13) and (14), respectively.

  The difference value Fn (y) −Fn_1 (y) from the difference calculation unit 214 remains affected by the signal intensity of the subject, and therefore the level of luminance change and color change due to flicker varies depending on the region. On the other hand, as described above, normalization by the normalization processing unit 215 makes it possible to adjust the luminance change and color change due to flicker to the same level over the entire region.

<Estimation of flicker components by spectral extraction>
| Am | and θm represented by the above equations (13) and (14) are the amplitude and initial phase of each spectrum of the normalized difference value gn (y), respectively. If the difference value gn (y) after normalization is Fourier-transformed to detect the amplitude | Am | and the initial phase θm of each spectrum, the following equation (2) is obtained by the following equations (15) and (16): The amplitude γm and the initial phase Φm, n of each next flicker component shown in FIG.

  Therefore, in the flicker reduction unit 20 of FIG. 3, the DFT processing unit 220 corresponds to data corresponding to one wavelength of flicker (for L lines) of the normalized difference value gn (y) from the normalization processing unit 215. Is subjected to discrete Fourier transform.

  If the DFT operation is DFT [gn (y)] and the DFT result of order m is Gn (m), the DFT operation is expressed by the following equation (17). However, W in Formula (17) is represented by Formula (18).

  Moreover, the relationship between Formula (13) and (14) and Formula (17) is represented by following Formula (19) and (20) by the definition of DFT.

  Therefore, from the equations (15), (16), (19), and (20), the amplitude γm and the initial phase Φm, n of each next flicker component can be obtained by the following equations (21) and (22).

  The DFT processing unit 220 first extracts a spectrum by the DFT calculation defined by Expression (17), and then calculates the amplitude γm and the initial phase Φm of each next flicker component by the calculations of Expressions (21) and (22). , N is estimated. Here, the data length of the DFT calculation is set to one wavelength (L line) of the flicker waveform because a discrete spectrum group that is an integral multiple of ω0 can be directly obtained.

  In general, fast Fourier transform (FFT) is used as Fourier transform in digital signal processing. However, since FFT requires that the data length be a power of 2, in the present embodiment, frequency analysis is performed by DFT, and data processing is simplified accordingly. However, FFT can also be used by processing input / output data.

  Under actual fluorescent lamp illumination, flicker components can be sufficiently approximated even if the order m is limited to several orders, so that it is not necessary to output all data in the DFT calculation. Therefore, in this application for detecting flicker, there is no disadvantage in terms of calculation efficiency compared with FFT.

  Next, the flicker generation unit 230 uses the estimated value of the amplitude γm and the initial phase Φm, n by the DFT processing unit 220 to execute the calculation process of the above equation (2), and correctly reflects the flicker component. The flicker coefficient Γn (y) is calculated. Even in the calculation processing of equation (2), under actual fluorescent lamp illumination, the total order is not infinite, but limited to a predetermined order, for example, the second order, and the higher order processing is omitted. However, the flicker component can be approximated sufficiently in practice.

<Calculation for flicker reduction>
The above equation (1) can be modified as the following equation (23). Based on this equation (23), the calculation unit 240 adds “1” to the flicker coefficient Γn (y) from the flicker generation unit 230 and then divides the image signal by this added value. Thereby, the flicker component contained in the input image signal In ′ (x, y) is removed with high accuracy.
In (x, y) = In ′ (x, y) / [1 + Γn (y)] (23)
According to the above flicker detection / reduction processing, the flicker component is completely buried in the signal component with the integrated value Fn (y), even in a region such as a black background portion where the flicker component is minute or a portion with low illuminance, By calculating the difference value Fn (y) -Fn_1 (y) and normalizing it with the average value AVE [Fn (y)], the flicker component can be detected with high accuracy.

  Further, in calculating the flicker coefficient Γn, since the order can be limited to several orders, flicker detection can be performed with high accuracy by a relatively simple process. Note that estimating the flicker component from the spectrum up to an appropriate order approximates the normalized difference value gn (y) without completely reproducing it. Even if a discontinuous part occurs in the difference value gn (y) after conversion, the flicker component of that part can be estimated with high accuracy.

  In the above process, the difference value Fn (y) −Fn — 1 (y) is normalized by the average value AVE [Fn (y)], thereby effectively ensuring a finite calculation accuracy. However, for example, when the required calculation accuracy can be satisfied, the integral value Fn (y) may be directly normalized by the average value AVE [Fn (y)].

<Coefficient setting for flicker processing section>
By the way, the above-described equation (7) defines a transfer function of the filter processing unit 213 which is a recursive filter. In the filter processing unit 213, the smaller the filter coefficient k is set, the greater the low-pass filter effect in the time direction, and the greater the effect of suppressing the flicker component in the normalized signal. Conversely, as the filter coefficient k is set larger, the low-pass filter effect is reduced and the flicker component suppression effect is also reduced. However, the larger the filter coefficient k is, the easier it is to follow the temporal change of the input image signal.

  Therefore, in the present embodiment, the motion detection unit 16 detects the screen motion, and the filter coefficient k set for the filter processing unit 213 is changed according to the detection result, so that the screen motion is matched. To optimize the operation of the recursive filter.

FIG. 4 is a flowchart showing the flow of filter coefficient setting control.
The system controller 15 sets the filter coefficient k to the filter processing unit 213 by executing the process of FIG. First, the system controller 15 acquires a detected value of the amount of camera shake from the camera shake detection unit 16 (step S1). Next, a filter coefficient k corresponding to the acquired camera shake amount is acquired by referring to a coefficient setting table stored in advance such as a ROM (not shown) indicating the correspondence between the camera shake amount and the filter coefficient k (step S2). And the acquired filter coefficient k is set with respect to the filter process part 213 (step S3). By such processing, the frequency characteristic at the time of averaging processing by the filter processing unit 213 changes according to the detected amount of camera shake.

FIG. 5 is an example of a graph showing the relationship between the camera shake amount set in the coefficient setting table and the filter coefficient.
As shown in FIG. 5, in the coefficient setting table, the filter coefficient k gradually increases as the camera shake amount is increased from the threshold value thrA to the threshold value thrB (however, thrA <thrB) by the camera shake detection unit 16. Set to do. As a result, when the amount of camera shake is small and the movement of the subject is small, the flicker component is strongly suppressed by the averaging process. In addition, as the movement of the subject increases due to camera shake, the suppression effect of the flicker component gradually decreases, but it becomes easier to follow the movement of the screen. Accuracy is improved. That is, while using a recursive filter for averaging processing, flicker detection accuracy can be improved overall regardless of the amount of camera shake. In addition, by continuously changing the filter coefficient k according to the amount of camera shake, it is possible to reduce a sense of discomfort given to the user when viewing the screen due to the influence of flicker, and as a result, it is possible to improve the image quality. .

  In the example of FIG. 5, when the amount of camera shake is equal to or less than the threshold value thrA and equal to or greater than the threshold value thrB, the filter coefficient k is set to a constant value to maintain an appropriate flicker suppression effect, While reducing the sense of discomfort at the time, the coefficient setting control is facilitated.

  There is generally a method for detecting the movement of the screen from the captured image signal itself. However, with such a technique, the possibility of erroneous detection increases when a flicker component is included, and the accuracy of flicker detection cannot always be improved when used in the coefficient setting control described above. For this reason, it is desirable that the movement of the screen is detected by a detection mechanism different from the captured image signal system, such as an angular velocity sensor or an acceleration sensor.

  As described above, according to the imaging apparatus according to the present embodiment, the cyclic filter is used for the averaging process at the time of flicker detection, so that the calculation is performed as compared with the case where the FIR filter is used. The capacity of the memory for storing the image signal used for the recording can be reduced, and the processing load can be reduced. Therefore, the circuit scale of the flicker reduction unit 20 can be suppressed, and the manufacturing cost can be reduced. The filter coefficient k is changed in accordance with the amount of camera shake detection, so that the flicker component is not erroneously detected due to the movement of the screen, the flicker component is reduced accurately, and the sense of incongruity is small. A captured image can be obtained.

  In the above-described embodiment, the processing in the case of using an interlace scanning type imaging device has been described. However, even in the case of the progressive scanning method, if the processing is performed for each frame instead of the field, the basic In particular, the same processing as described above can be applied.

  In the above-described embodiment, the case where a CMOS image sensor is used as the image sensor has been described. However, when another XY address scanning image sensor such as a MOS image sensor other than the CMOS image sensor is used. The present invention is applicable. Furthermore, the present invention can also be applied to various imaging devices using XY address scanning type imaging devices, and devices such as mobile phones and PDAs (Personal Digital Assistants) having such an imaging function. . Further, the present invention can also be applied to an image processing apparatus that processes an image signal from a small camera such as a videophone or game software connected to a PC (personal computer) or the like.

  Further, the above processing functions can be realized by a computer. In this case, a program describing the processing contents of the functions that the apparatus should have (functions corresponding to the above flicker reduction unit, filter coefficient setting function by the system controller, etc.) is provided. And the said processing function is implement | achieved on a computer by running the program with a computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical disk, and a semiconductor memory.

  In order to distribute the program, for example, portable recording media such as an optical disk and a semiconductor memory on which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

It is a block diagram which shows the principal part structure of the imaging device which concerns on embodiment of this invention. It is a figure for demonstrating flicker. It is a block diagram which shows the internal structure of a flicker detection part. It is a flowchart which shows the flow of the filter coefficient setting control. It is an example of the graph which shows the relationship between the camera-shake amount set to a coefficient setting table, and a filter coefficient.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Optical block, 11a ... Driver, 12 ... CMOS type image sensor (CMOS sensor), 12a ... Timing generator (TG), 13 ... Analog front end (AFE) circuit, 14 ... Camera processing circuit, DESCRIPTION OF SYMBOLS 15 ... System controller, 16 ... Camera shake detection part, 20 ... Flicker reduction part, 210 ... Normalization integration value calculation part, 211 ... Integration processing part, 212 ... Integration value delay part, 213 ... Filter processing , 214... Difference calculation unit, 215... Normalization processing unit, 220... DFT processing unit, 230.

Claims (9)

In an image processing apparatus that processes an image signal,
Integrating means for integrating the image signal in units of one horizontal synchronization period or more;
Filter processing means comprising a recursive filter for averaging the integration value by the integration means for each field or frame;
Normalization means for normalizing the integration value by the integration means or the difference value of the integration values in adjacent fields or frames using the output value of the filter processing means;
A frequency analysis means for extracting a spectrum of an integral value or a difference value after normalization by the normalization means;
Flicker estimation means for estimating a flicker component generated on a screen under fluorescent lamp illumination from the spectrum extracted by the frequency analysis means;
Camera shake detection means for detecting camera shake during imaging;
Filter control means for changing a set value of a filter coefficient for the filter processing means in accordance with a camera shake detection amount by the camera shake detection means;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the filter control unit sets a larger value as the filter coefficient as the camera shake detection amount is larger.   The filter control means further holds the filter coefficients at individual constant values when the camera shake detection amount is equal to or smaller than a predetermined lower limit value and when equal to or greater than a predetermined upper limit value. Item 3. The image processing apparatus according to Item 2.   The image processing apparatus according to claim 1, wherein the camera shake detection unit includes an angular velocity sensor or an acceleration sensor.   2. The image processing apparatus according to claim 1, wherein the normalizing unit divides an integrated value obtained by the integrating unit or a difference value between integrated values in adjacent fields or frames by an output value of the filter processing unit. .   2. The image processing apparatus according to claim 1, further comprising a correction unit that corrects the image signal so as to cancel the flicker component estimated by the flicker estimation unit. In an imaging apparatus that captures an image using an XY address scanning type solid-state imaging device,
Integrating means for integrating the image signal obtained by imaging in units of one horizontal synchronization period or more;
Filter processing means comprising a recursive filter for averaging the integration value by the integration means for each field or frame;
Normalization means for normalizing the integration value by the integration means or the difference value of the integration values in adjacent fields or frames using the output value of the filter processing means;
A frequency analysis means for extracting a spectrum of an integral value or a difference value after normalization by the normalization means;
Flicker estimation means for estimating a flicker component generated on a screen under fluorescent lamp illumination from the spectrum extracted by the frequency analysis means;
Camera shake detection means for detecting camera shake during imaging;
Filter control means for changing a set value of a filter coefficient for the filter processing means in accordance with a camera shake detection amount by the camera shake detection means;
An imaging device comprising: In an image processing method for detecting flicker generated on a captured image under fluorescent lamp illumination,
A step of detecting a camera shake during imaging,
The filter control means changing the set value of the filter coefficient for the filter processing means comprising a recursive filter according to the amount of camera shake detected by the camera shake detection means;
An integrating means for integrating an image signal obtained by imaging in units of one horizontal synchronization period or more;
The filter processing means averaging the integration value by the integration means for each field or frame;
Normalizing means normalizing the integrated value by the integrating means or the difference value of the integrated values in adjacent fields or frames using the output value of the filter processing means;
A step of extracting a spectrum of an integral value or a difference value after normalization by the normalization means;
Flicker estimation means estimating a flicker component generated on the captured image from the spectrum extracted by the frequency analysis means;
An image processing method comprising: In an image processing program for causing a computer to execute processing for detecting flicker generated on a captured image under fluorescent lamp illumination,
Integration means for integrating the image signal obtained by imaging in units of one horizontal synchronization period or more;
A filter processing means comprising a recursive filter for averaging the integration value by the integration means for each field or frame;
Normalization means for normalizing the integration value by the integration means or the difference value of the integration values in adjacent fields or frames using the output value of the filter processing means;
Frequency analysis means for extracting the spectrum of the integral value or difference value after normalization by the normalization means,
Flicker estimation means for estimating a flicker component generated on the captured image from the spectrum extracted by the frequency analysis means;
Camera shake detection means for detecting camera shake during imaging,
Filter control means for changing a set value of a filter coefficient for the filter processing means in accordance with a camera shake detection amount by the camera shake detection means;
An image processing program for causing the computer to function.

Images