CN101668106B - Image processing device, imaging device, and image processing method - Google Patents

Abstract

An image processing device, an imaging device, and an image processing method are provided that perform synchronization processing to generate image data having multiple colors per pixel based on image data obtained by an imaging device having an imaging element equipped with multiple color filters. A predetermined granular pattern is added to the image data after the synchronization processing.

Description

Image processing device, imaging device, and image processing method

technical field

The present invention relates to an image processing device and an image processing method for adding a granular pattern similar to that obtained in a silver halide photograph to image data obtained by a digital camera or the like.

Background technique

A noise component mainly generated by an imaging device is superimposed on a digital image obtained by a digital camera or the like, and this noise component is one cause of degradation in image quality. Generally, noise components superimposed on an image are reduced by various methods in an effort to improve image quality.

Also in silver salt photographs, grainy noise mainly caused by silver salt grains occurs, which is one of the causes of image quality degradation as in digital images. Conversely, graininess due to grainy noise also exists as an expression of silver salt photographs. Facts of certain value.

In this context, in order to obtain the visual effect of silver salt grains that give digital images the same value as silver salt photographs, various methods of superimposing a grainy pattern similar to the graininess of silver salt grains on digital images have been proposed. technology. For example, in Patent Document 1 (US5641596B1), random numbers are generated in units of pixels, frequency filtering processing and resizing are performed, and graininess of expected graininess is superimposed on the image. In addition, in Patent Document 2 (JP11-85955A), a grainy pattern is calculated by subtracting a smoothed image from an exposure image obtained from a uniformly exposed color film.

However, the aforementioned Patent Documents 1 and 2 refer to the calculation of graininess or grainy patterns superimposed on image data, but do not describe an image processing device and an image processing method necessary for generating a desired graininess as a final image.

Contents of the invention

According to a first aspect of the present invention, there is provided an image processing device including: a simultaneous processing unit for generating image data having a plurality of color components per pixel based on image data obtained by an imaging device, the The imaging device includes an imaging element on which a plurality of color filters are arranged, and a granular pattern adding unit that adds a predetermined granular pattern to the image data generated by the synchronization processing unit.

According to a second aspect of the present invention, there is provided an image processing apparatus including: a granular pattern adding unit for adding a predetermined pattern to image data subjected to predetermined image processing including at least a first gradation conversion process. a granular pattern; and a second gradation conversion unit that performs a second gradation conversion process different from the first gradation conversion process on the image data to which the predetermined granular pattern is added.

According to a third aspect of the present invention, there is provided an image processing method, the image processing method comprising: a first step of generating image data having multiple color components for each pixel based on the image data obtained by the imaging device, the imaging device It has an imaging element on which a plurality of color filters are arranged; and in a second step, adding a predetermined granular pattern to the generated image data.

According to a fourth aspect of the present invention, there is provided an imaging device comprising: an imaging unit that captures a subject to obtain first image data; an image processing unit that performs image processing on the first image data to generate 2nd image data; a size setting unit that sets a recording size for recording; a reduced image data generation unit that reduces the size of the second image data based on the recording size set by the size setting unit and the size of the second image data The second image data generates third image data; a granular pattern adding mode setting unit that sets a granular pattern adding mode for adding a predetermined granular pattern to the second or third image data; and a granular pattern adding unit When the granular pattern addition mode is set by the granular pattern addition mode setting unit, the recording size set by the size setting unit is a size generated by reducing the second image data , the granular pattern adding unit adds the predetermined granular pattern to the third image data, and when the recording size set by the size setting unit is a size generated without reducing the second image data The granular pattern adding unit adds the predetermined granular pattern to the second image data.

According to a fifth aspect of the present invention, there is provided an imaging device comprising: an imaging unit that captures a subject to obtain first image data; an image processing unit that performs image processing on the first image data to generate 2nd image data; a size setting unit that sets a recording size for recording; a reduced image data generation unit that uses a recording size based on the recording size set by the size setting unit and the size of the second image data A predetermined reduction ratio for reducing the second image data to generate third image data; a granular pattern enlarging unit for enlarging the granular pattern attached to the image data at an enlargement ratio corresponding to the predetermined reduction ratio; the granular pattern an adding mode setting unit that sets a granular pattern adding mode for adding a predetermined granular pattern to the image data; and a granular pattern adding unit that adds a granular pattern to the image data. When the granular pattern addition mode is set by the granular pattern addition mode setting unit, when the recording size set by the size setting unit is a size generated by reducing the second image data After adding the enlarged granular pattern to the second image data by the granular pattern adding unit, the reduced image data generating unit reduces and adds the enlarged image at the predetermined reduction ratio. The second image data of the granular pattern.

According to a sixth aspect of the present invention, there is provided an image processing device that adds a predetermined grain pattern to input image data. This image processing device includes: a first granular pattern adding unit that adds the predetermined granular pattern to the input image data; and a reduced image data generating unit that generates a reduced image having fewer pixels than the input image data. data; and a second granular pattern adding unit for adding the predetermined granular pattern to the reduced image data.

According to a seventh aspect of the present invention, there is provided an image processing device that adds a predetermined grain pattern to input image data. The image processing apparatus includes: a reduced image data generating unit that generates reduced image data having fewer pixels than the input image data at a predetermined reduction ratio; and a granular pattern enlarging unit that generates the reduced image data at a predetermined reduction ratio a magnification factor enlarging the predetermined granular pattern; and a granular pattern adding section which adds the enlarged granular pattern to the input image data, the reduced image data generation section reducing the added granular pattern at the predetermined magnification rate Image data of the magnified granular pattern was obtained.

According to an eighth aspect of the present invention, there is provided an image processing method for adding a predetermined grain pattern to image data. The image processing method includes: a step of generating reduced image data having fewer pixels than the image data; and a step of adding the predetermined grain pattern to the reduced image data.

According to a ninth aspect of the present invention, there is provided an image processing method for adding a predetermined grain pattern to image data. The image processing method includes: a step of enlarging the granular pattern at a magnification rate corresponding to a predetermined reduction rate; adding the enlarged granular pattern to the image data; The step of adding the image data of the enlarged granular pattern.

According to the present invention, the processing of adding a granular pattern to the image data after the synchronization processing is performed, so the following problem caused by the synchronization processing after the addition of the granular pattern can be prevented, that is, the unique fineness of the silver salt particles. The graininess of the image disappears, or the grainy pattern is misidentified as the structure of the image, resulting in a geometric pattern that is not originally in the image. Therefore, the graininess of silver halide photographs can be appropriately reproduced.

Description of drawings

FIG. 1 is a block diagram showing the configuration of a digital camera including an image processing device according to a first embodiment.

FIG. 2 is a block diagram showing a specific configuration of an image processing circuit.

3A is a flowchart showing the early part of the processing flow when shooting with a digital camera including the image processing device according to the first embodiment.

3B is a flowchart showing the latter part of the flow of processing when shooting with a digital camera including the image processing device according to the first embodiment.

FIG. 4 is a diagram showing examples of various gains, imaging sizes, and image sizes in a normal mode.

FIG. 5 is a diagram showing examples of various gains, imaging sizes, and image sizes in the silver salt style mode.

FIG. 6 is a diagram showing examples of various gains, camera sizes, and image sizes when editing JPEG compressed data.

FIG. 7 is a diagram showing an example of gradation conversion characteristics of gradation conversion processing performed before synthesizing a granular pattern.

FIG. 8 is a diagram showing an example of gradation conversion characteristics when gradation conversion processing of a Y signal is performed.

FIG. 9 is a diagram showing an example of gradation conversion characteristics when gradation conversion processing of a C signal is performed.

FIG. 10 is a flowchart showing the flow of processing at the time of editing RAW data.

FIG. 11 is a flowchart showing the flow of processing at the time of editing JPEG compressed data.

12 is a block diagram showing the configuration of a digital camera to which the image processing device of the second embodiment is applied.

13 is a flowchart showing the flow of main processing performed by a digital camera to which the image processing device of the second embodiment is applied.

FIG. 14A is a flowchart showing specific processing contents of a still image shooting/recording operation performed in step S1302 of the flowchart shown in FIG. 13 .

FIG. 14B is a flowchart showing a flow of processing succeeding the processing of the flowchart shown in FIG. 14A .

FIG. 15 is a flowchart showing specific processing contents of live view display performed in step S1305 of the flowchart shown in FIG. 13 .

FIG. 16 is a flowchart showing the content of processing performed after step S1413 in FIG. 14A in the still image shooting/recording operation processing performed by the image processing apparatus according to the third embodiment.

17 is a flowchart showing specific processing contents of live view display processing performed in the image processing device according to the third embodiment.

Detailed ways

(first embodiment)

FIG. 1 is a block diagram showing the configuration of a digital camera including an image processing device according to a first embodiment. This digital camera has an imaging unit 1, an A/D conversion unit 2, a microcomputer 3 (hereinafter referred to as microcomputer 3), a RAM 4, a ROM 5, an image processing circuit 6, an operation unit 7, a display unit 8, and a memory interface 9 (hereinafter referred to as I/F9), and the recording medium 10.

The imaging unit 1 is composed of a single-plate color imaging device (hereinafter simply referred to as the imaging device), an imaging optical system, and their drive units. The imaging device has a plurality of color filters arranged in front of photodiodes constituting each pixel. The color filters are arranged in a Bayer arrangement, for example. The Bayer array has rows in which R pixels and G (Gr) pixels are alternately arranged in the horizontal direction, rows in which G (Gb) pixels and B pixels are alternately arranged, and these two rows are alternately arranged in the vertical direction. The imaging element receives light collected by a lens not shown by a photodiode constituting a pixel, and performs photoelectric conversion, thereby outputting the amount of light as an amount of charge to the A/D conversion unit 2 . In addition, the imaging element may be a CMOS type element or a CCD type element. In addition, the color filters may be configured in an arrangement other than the Bayer arrangement, or may be configured in colors other than R, G, and B.

The A/D conversion unit 2 converts the electrical signal output from the imaging unit 1 into a digital image signal (hereinafter referred to as image data).

The microcomputer 3 is a control unit that performs overall control of the digital camera. For example, the microcomputer 3 performs focus control of the imaging optical system inside the imaging unit 1, exposure control of the imaging element, recording control when recording image data on the recording medium 10, display control when displaying image data on the display unit 8, and the like.

The RAM 4 is a storage unit for temporarily storing various data such as image data obtained by the A/D conversion unit 2 and image data processed by the image processing circuit 6 described later. The ROM 5 stores various parameters necessary for the operation of the digital camera, data of a granular pattern similar to that of silver salt grains, and the like. The data of the granular pattern is obtained by a known method, and is separated into a Y (brightness) component and a C (color) component and stored in the ROM 5 . In addition, the ROM 5 also stores various programs executed by the microcomputer 3 . The microcomputer 3 reads parameters required for various programs from the ROM 5 and executes various processes according to the programs stored in the ROM 5 .

The image processing circuit 6 performs various image processing on the image data read from the RAM 4 . Details of the image processing performed by the image processing circuit 6 will be described later. Image data subjected to image processing in the image processing circuit 6 is recorded on the recording medium 10 through the I/F 9 . The recording medium 10 is, for example, a recording medium composed of a memory card that can be inserted and removed from the main body of the digital camera, but is not limited thereto.

The operation unit 7 is an operation member such as a power button, a release button, and various input keys. When the user operates any one of the operating members of the operating unit 7, the microcomputer 3 executes various programs corresponding to the user's operation. The power button is an operation member for instructing on/off the power of the digital camera. When the power button is pressed, the microcomputer 3 turns on or off the power of the digital camera. The release button is configured to have two switches, a first release switch and a second release switch. When the release button is half-pressed and the first release switch is turned on, the microcomputer 3 executes photography preparation programs such as AE processing and AF processing. Then, when the release button is fully pressed and the second release switch is turned on, the microcomputer 3 executes the imaging program to perform imaging. The operation unit 7 can set an image recording mode and a shooting mode.

The bus 11 is a transmission path for transmitting various data generated inside the digital camera to various parts in the digital camera. Bus 11 is connected to imaging unit 1 , A/D conversion unit 2 , microcomputer 3 , RAM 4 , ROM 5 , image processing circuit 6 , operation unit 7 , display unit 8 , and I/F unit 9 .

FIG. 2 is a block diagram showing a specific configuration of the image processing circuit 6 . The image processing circuit 6 has a first noise reduction unit (first NR unit in the figure) 21, a white balance correction unit (WB unit in the figure) 22, a synchronization processing unit 23, a color conversion unit 24, a gradation conversion unit 25, YC conversion section 26, edge extraction section 27, edge emphasis section 28, Y gradation conversion section 29, Y 2nd noise reduction section (Y 2nd NR section in the figure) 30, C gradation conversion section 31, C 2nd noise A reduction section (C 2nd NR section in the figure) 32 , a resizing section 33 , a JPEG compression section 34 , a JPEG decompression section 35 , addition sections 36 and 37 , and multiplication sections 38 and 39 .

The first noise reduction unit 21 performs noise reduction processing on the image data converted by the A/D conversion unit 2 and stored in the RAM 4 based on a noise reduction gain NR1 described later. The noise reduction processing refers to processing for correcting pixel defects of an imaging device and processing for reducing random noise generated during imaging. Here, at least one of processing for correcting pixel defects of the imaging device and processing for reducing random noise generated during imaging may be performed.

The white balance correcting unit 22 performs processing of correcting the white balance of the noise-reduced image data.

The synchronization processing unit 23 performs processing for simultaneously converting image data based on the Bayer array into image data composed of R, G, and B color information for each pixel. The synchronized image data is subjected to predetermined color conversion processing in the color conversion unit 24 , and then to gradation conversion processing (first gradation conversion processing) in the gradation conversion unit 25 .

The YC conversion section 26 converts the image data after the gradation conversion process into a Y (brightness) signal and a C (color) signal. The converted Y signal is output to the adder 36 , and the C signal is output to the adder 37 .

The edge extraction unit 27 performs processing for extracting an edge from the image data subjected to noise reduction processing by the first noise reduction unit 21 . The edge enhancement unit 28 performs an edge enhancement process of multiplying the edge data extracted by the edge extraction unit 27 by a predetermined gain.

The multiplier 38 multiplies the Y component data of the grain pattern stored in the ROM 5 by a gain GRN_Y described later, and outputs the result to the adder 36 . The adder 36 adds the edge data input from the edge enhancement unit 28 and the Y component data of the granular pattern input from the multiplier 38 to the Y signal. The Y signal is input from the YC converter 26 at the time of photography and editing of RAW data, and is input from the RAM 4 at the time of editing of JPEG compressed data. The Y signal output from the adder 36 is subjected to a gradation conversion process (second gradation conversion process) described later in the Y gradation conversion unit 29 , and then is subjected to noise reduction processing in the Y second noise reduction unit 30 .

The multiplier 39 multiplies the C component data of the granular pattern stored in the ROM 5 by a gain GRN_C described later, and outputs the result to the adder 37 . The adder 37 adds the C component data of the granular pattern input from the multiplier 39 to the C signal. The C signal is input from the YC converter 26 at the time of photography and editing of RAW data, and is input from the RAM 4 at the time of editing of JPEG compressed data. The C signal output from the adder 37 is subjected to a gradation conversion process (second gradation conversion process) described later in the C gradation conversion unit 31 , and then is subjected to noise reduction processing in the C second noise reduction unit 32 .

The resizing unit 33 resizes the Y signal and the C signal after the noise reduction processing according to the image size at the time of recording. The JPEG compression unit 34 performs JPEG compression on the resized Y signal and C signal. JPEG-compressed data is recorded on the recording medium 10 through I/F9. The JPEG decompression unit 35 reads out the JPEG compressed data recorded on the recording medium 10, and performs decompression processing to return the data to the state before compression.

＝Flow of processing during photography＝

3A and 3B are flowcharts showing the flow of processing when shooting with a digital camera including the image processing device according to the first embodiment. After the user turns on the release button, the process of step S10 starts.

Known photometry processing and ranging processing are performed in step S10, and then known exposure control and focus control are performed in step S20. In step S30 , imaging processing is executed by the imaging unit 1 . In step S40 , after the imaging process is executed by the imaging unit 1 , the image data obtained by performing the A/D conversion process in the A/D conversion unit 2 is temporarily stored in the RAM 4 .

In step S50, it is determined whether the recording format of the image is the JPEG format or the RAW format. The image recording format can be set by the user operating the operation unit 7 before shooting. When it is determined that the recording format of the image is JPEG format, the process proceeds to step S60, and when it is determined that the recording format of the image is RAW format, the process proceeds to step S250 of FIG. 3B.

In step S60, it is determined whether the recording mode of the image is the silver salt style mode or the normal mode. The silver salt style mode is a mode in which a grainy pattern similar to the graininess of silver salt grains is superimposed on image data in order to obtain the same visual effect as a silver salt photograph. The image recording mode can be set by the user operating the operation unit 7 before shooting. When it is determined that the image recording mode is the normal mode, the process proceeds to step S70.

In step S70, the noise reduction gain NR1 in the normal mode is calculated. FIG. 4 is a diagram showing examples of various gains, imaging sizes, and image sizes in a normal mode. The various gains include: gain NR1 used when the first noise reduction unit 21 performs noise reduction processing; gain GRN_Y multiplied by the Y component of the granular pattern; gain GRN_C multiplied by the C component of the granular pattern; Gain NR2_Y used when the noise reduction unit 30 performs noise reduction processing; and gain NR2_C used when the second noise reduction unit 32 performs noise reduction processing.

Various gains are determined according to the ISO sensitivity set during shooting. For example, when the ISO sensitivity is 800, the gain NR1 is 4, the gain NR2_Y is 2, and the gain NR2_C is 2. However, in the normal mode, since the grain pattern is not superimposed on the image data, the columns of the gains GRN_Y and GRN_C are left blank. In addition, the gains GRN_Y and GRN_C may be respectively set to 0 to perform processing of superimposing the grain pattern.

The imaging noise corresponding to each ISO sensitivity is based on the imaging noise when the ISO sensitivity is 100, and the smaller the value, the greater the noise. That is, the greater the ISO sensitivity, the greater the imaging noise. Similarly, the image noise is based on the image noise when the ISO sensitivity is 100, and the smaller the value, the greater the noise. Therefore, the greater the ISO sensitivity, the greater the image noise.

Since the data table for specifying various gains corresponding to the respective ISO sensitivities shown in FIG. 4 is stored in the ROM 5 , the data table is read out to calculate the gain NR1 corresponding to the ISO sensitivity during shooting. In addition, the larger the value of the gain NR1 is, the larger the noise reduction effect during the noise reduction processing is.

On the other hand, when it is determined in step S60 that the image recording mode is the silver salt style mode, the process proceeds to step S80. In step S80, the noise reduction gain NR1 in the silver salt style mode is calculated. FIG. 5 is a diagram showing examples of various gains, imaging sizes, and image sizes in the silver salt style mode. In FIG. 5 , the observation methods of various gains, imaging sizes, and image sizes are the same as those in FIG. 4 .

In the silver salt style mode, regardless of the ISO sensitivity, in order to keep the graininess on the final image data constant, the intensity of the synthesized grainy pattern is controlled according to the amplification of the imaging noise corresponding to the ISO sensitivity (gain GRN_Y, GRN_C) Noise reduction strength (NR2_Y, NR2_C) after compositing with grainy pattern. Here, the higher the ISO sensitivity is, the smaller the gains GRN_Y and GRN_C are, and the more the strength of the grainy pattern synthesized on the image data is weakened. As shown in FIG. 5 , the image noise of the image data after adding the grain pattern is constant regardless of the ISO sensitivity.

In addition, in order to maintain the graininess of the grainy pattern superimposed on the image data, the gains NR2_Y and NR2_C are set weaker than those in the normal mode. In addition, regarding the strength of the grain pattern, the chromatic aberration (C) is relatively weakened compared with the luminance (Y), so that unnecessary color noise does not occur.

Since a data table for specifying various gains corresponding to ISO sensitivity shown in FIG. 5 is stored in ROM 5 , the data table is read out to calculate gain NR1 corresponding to ISO sensitivity during shooting.

In step S90, the first noise reduction unit 21 reads the image data stored in the RAM 4, and performs noise reduction processing using the noise reduction gain NR1 calculated in step S70 or step S80.

In step S100 , in the white balance correcting unit 22 , a process of correcting the white balance is performed on the image data after the noise reduction process. In step S110 , synchronization processing is performed in the synchronization processing unit 23 , and in step S120 , color conversion processing is performed in the color conversion unit 24 .

In step S130 , the gradation conversion process is performed in the gradation conversion unit 25 . Here, in consideration of the gamma characteristic of the display unit 8 , a gradation conversion process of expanding dark areas and compressing bright areas is performed. FIG. 7 is a diagram showing an example of gradation conversion characteristics when the gradation conversion process is performed in step S130. After the grayscale conversion process is performed, go to step S140 in FIG. 3B .

In step S140, in the YC conversion section 26, the image data after the gradation conversion process is converted into a Y (brightness) signal and a C (color) signal.

In step S150, it is determined whether the recording mode of the image is the silver salt style mode or the normal mode. This determination is the same as the determination in step S60. When it is determined that the image recording mode is the normal mode, the process proceeds to step S210, and when it is determined that the image recording mode is the silver salt style mode, the process proceeds to step S160.

In step S160, the gains GRN_Y and GRN_C in the silver salt style mode are calculated. When calculating the gains GRN_Y and GRN_C, the above-mentioned data table shown in FIG. 5 is read from the ROM 5 to calculate the gains GRN_Y and GRN_C corresponding to the ISO sensitivity at the time of shooting.

In step S170, the multiplier 38 reads the Y component data of the granular pattern from the ROM 5, and multiplies the read Y component data of the granular pattern by the gain GRN_Y calculated in step S160. Then, in the multiplication unit 39 , the C component data of the granular pattern are read from the ROM 5 , and the read C component data of the granular pattern are multiplied by the gain GRN_C calculated in step S160 .

In step S180 , the addition unit 36 adds the edge data input from the edge emphasis unit 28 and the Y component data of the grain pattern input from the multiplication unit 38 to the Y signal converted by the YC conversion unit 26 . Then, in the adder 37 , the C component data of the granular pattern input from the multiplier 39 is added to the C signal converted by the YC converter 26 .

In step S190 , in the Y gradation conversion unit 29 , the gradation conversion table for the Y signal is read from the ROM 5 , and at the same time, the C gradation conversion unit 31 reads the gradation conversion table for the C signal from the ROM 5 .

FIG. 8 is a diagram showing an example of gradation conversion characteristics when gradation conversion processing of a Y signal is performed. The gradation conversion characteristic is a characteristic of emphasizing the contrast of image data, that is, a characteristic of compressing dark parts and bright parts and stretching the intermediate style. Furthermore, FIG. 9 is a diagram showing an example of the gradation conversion characteristics when the gradation conversion processing of the C signal is performed. A gradation conversion table of the gradation conversion characteristics shown in FIG. 8 and a gradation conversion table of the gradation conversion characteristics shown in FIG. 9 are stored in the ROM 5 . After the grayscale conversion table is read out from the ROM5, the process goes to step S200.

In step S200, in the Y gradation converting section 29, the Y signal input from the adding section 36 is subjected to gradation conversion processing using the gradation conversion table for the Y signal read in step S190. In the gradation conversion unit 31 , the gradation conversion process is performed on the C signal input from the addition unit 37 using the gradation conversion table for the C signal read in step S190 .

In step S210, in the Y second noise reduction unit 30, noise reduction processing is performed on the Y signal after the gradation conversion process, and at the same time, in the C second noise reduction unit 32, the C signal after the gradation conversion process Perform noise reduction processing. In the noise reduction processing, gains NR2_Y, NR2_C corresponding to the normal mode or the silver salt style mode are used. For example, in the determination of step S150, when it is determined that it is the normal mode, the data table shown in FIG. 4 is read from the ROM 5, and the gains NR2_Y and NR2_C corresponding to the ISO sensitivity during shooting are obtained. Then, in the judgment of step S150, if it is judged to be the silver salt style mode, the data table shown in FIG. 5 is read from the ROM 5, and the gains NR2_Y and NR2_C corresponding to the ISO sensitivity during shooting are obtained. The Y second noise reduction unit 30 performs noise reduction processing on the Y signal using the gain NR2_Y, and the C second noise reduction unit 32 performs noise reduction processing on the C signal using the gain NR2_C.

In step S220, in the resizing unit 33, the Y signal and the C signal after the noise reduction process are resized according to the image size at the time of recording. In step S230 , the JPEG compression unit 34 performs JPEG compression on the resized Y signal and C signal. In step S240, the JPEG-compressed Y signal and C signal are stored in RAM4.

In step S250, shooting information such as the recording mode and exposure conditions of the image is generated as file header information. In step S260, the generated file header information is added to the data compressed by JPEG and temporarily stored in the RAM 4, and recorded in the recording medium 10 through the I/F9.

＝Processing flow when editing RAW data＝

In the flow chart showing the processing flow at the time of shooting shown in FIG. 3A, when it is determined in step S50 that the image recording format is the RAW format, the image processing after step S60 is not performed, but is recorded as RAW data in the recording format. Medium 10. Hereinafter, the processing when reading the RAW data recorded on the recording medium 10 and performing image processing will be described.

FIG. 10 is a flowchart showing the flow of processing at the time of editing RAW data. In the flowchart of FIG. 10 , steps that perform the same processing as those performed in the flowchart of FIG. 3A are assigned the same step numbers and detailed explanations are omitted. In addition, since the processing after step S130 in FIG. 10 is the same as the processing in the flowchart shown in FIG. 3B , illustration and description thereof will be omitted.

In step S1000, image data in RAW format is read from the recording medium 10 through the I/F9. In step S1010, shooting information is read out from the file header information attached to the image data. In step S40, the read image data and imaging information are stored in RAM4.

The processing after step S60 is the same as the processing in the flowchart shown in FIGS. 3A and 3B .

= Processing when editing JPEG compressed data =

In the flow chart showing the processing flow at the time of photographing shown in FIGS. 3A and 3B , when it is determined that the recording mode of the image is the normal mode by the determination of step S60 and step S150, the image data is not superimposed with silver salt particles. A grainy pattern with a similar graininess. The following describes the process of reading image data recorded in the normal mode at the time of photography, superimposing a grainy pattern similar to that of silver salt grains, and recording it on the recording medium 10 .

FIG. 11 is a flowchart showing the flow of processing at the time of editing JPEG compressed data. In the flowchart of FIG. 11 , steps that perform the same processing as those performed in the flowcharts of FIGS. 3A , 3B, and 10 are given the same step numbers and detailed explanations are omitted.

In step S1100, the JPEG-compressed image data is read from the recording medium 10 through the I/F9.

In step S1110, the JPEG decompression unit 35 performs decompression processing for restoring the image data read in step S1100 to the state before JPEG compression. The image data decompressed and the imaging information read in step S1010 are stored in RAM 4 in step S40.

In step S1120 , the image data stored in RAM 4 is displayed on display unit 8 . Therefore, the user can confirm the image on which the granular pattern is superimposed.

In step S160A, gains GRN_Y and GRN_C in the silver salt style mode at the time of editing JPEG compressed data are calculated. FIG. 6 is a diagram showing an example of a data table used in processing when editing JPEG compressed data. In FIG. 6 , the observation methods of various gains, imaging sizes, and image sizes are the same as those in FIG. 4 and FIG. 5 .

When editing JPEG compressed data, in order to obtain the same image quality as that obtained during shooting processing in silver salt style mode, control the strength of the grain pattern (GRN_Y) in consideration of ISO sensitivity and gain NR1 when shooting in normal mode , GRN_C) and the combined noise reduction strength (NR2_Y, NR2_C).

Since a data table for specifying various gains corresponding to ISO sensitivity shown in FIG. 6 is stored in ROM 5 , this data table is read out to calculate gains GRN_Y and GRN_C corresponding to ISO sensitivity during shooting.

In step S210A, in the Y second noise reduction unit 30, noise reduction processing is performed on the Y signal after the gradation conversion process, and at the same time, in the C second noise reduction unit 32, the C signal after the gradation conversion process is processed. Perform noise reduction processing. Here, the data table shown in FIG. 6 is read from the ROM 5, and the gains NR2_Y and NR2_C corresponding to the ISO sensitivity at the time of shooting are obtained. The Y second noise reduction unit 30 performs noise reduction processing on the Y signal using the obtained gain NR2_Y, and the C second noise reduction unit 32 performs noise reduction processing on the C signal using the obtained gain NR2_C.

As described above, according to the image processing device of the first embodiment, the processing of adding a predetermined grain pattern is performed after the synchronization processing is performed on the image data. Therefore, it is possible to prevent the following problems caused by the synchronizing process after adding the granular pattern, that is, the unique fine graininess of the silver salt particles disappears, or the granular pattern is misidentified as the structure of the image, resulting in an image that is not originally geometric patterns, so the graininess of silver halide photographs can be appropriately reproduced.

In addition, according to the image processing device of the first embodiment, the first gradation conversion process is performed on the image data subjected to the synchronization process, and a predetermined grain pattern is added to the image data after the gradation conversion process. Therefore, it is possible to prevent the grainy pattern from being emphasized and the grainy feeling from being destroyed in dark areas due to the gradation conversion process after adding the grainy pattern.

In addition, according to the image processing device of the first embodiment, the second gradation conversion processing different from the first gradation conversion processing is performed on the image data to which the granular pattern is added, so that the original gradation characteristics of the image and the corresponding gradation characteristics can be simultaneously realized. Control of the graininess of the gray scale. Since this second gradation conversion process is a gradation conversion process that emphasizes the contrast of the image data, it is possible to suppress the graininess of the brightest and darkest parts of the image, emphasize the graininess of the intermediate style, and express the uniqueness of the silver halide photograph. Unique graininess.

Furthermore, according to the image processing device of the first embodiment, the edge used in the edge enhancement process is extracted from the image data before the predetermined grain pattern is added. If the edge is extracted from the image data after the granular pattern is added, the part that is not the edge may be mistakenly extracted as the edge, but by extracting the edge from the image data before the granular pattern is added, the edge can be prevented. Error extraction.

Furthermore, according to the image processing device of the first embodiment, since the intensity of the grain pattern added to the image data is adjusted, a desired grain pattern can be added. In particular, since the strength of the granular pattern is adjusted according to the imaging sensitivity at the time of imaging, it is possible to add an appropriate granular pattern in consideration of the noise component generated according to the imaging sensitivity.

In addition, according to the image processing device of the first embodiment, predetermined noise reduction processing is performed on the image data obtained by the imaging device, synchronization processing is performed on the image data after the noise reduction processing, and a predetermined granular pattern is added. The device has an imaging element provided with a plurality of color filters. Therefore, the graininess of silver halide photographs can be appropriately reproduced.

Furthermore, according to the image processing device of the first embodiment, the degree of noise reduction processing and the intensity of the predetermined grain pattern added to the image data are controlled according to the imaging sensitivity when imaging with the imaging device. noise components to attach a suitable grainy pattern. In particular, when the imaging sensitivity at the time of imaging is different, the degree of noise reduction processing and the strength of the predetermined grain pattern are controlled so that the graininess of the predetermined grain pattern added to the image data is substantially the same, so that high resolution can be maintained. Again, a grainy feel with good stability can be obtained regardless of photographic sensitivity.

(second embodiment)

12 is a block diagram showing the configuration of a digital camera to which the image processing device of the second embodiment is applied. The digital camera shown in FIG. 12 is composed of a camera body 100 and an interchangeable lens 200 .

The interchangeable lens 200 has a lens 1010 , a flash memory 1011 , a microcomputer 1012 , a driver 1013 , and an aperture 1014 . The interchangeable lens 200 is connected to the camera body 100 via I/F999 and can communicate.

The camera body 100 has a mechanical shutter 101, an imaging element 102, an analog processing unit 103, an analog/digital conversion unit 104 (hereinafter referred to as an A/D conversion unit 104), a bus 105, an SDRAM 106, an image processing unit 107, an AE processing unit 108, AF processing section 109, JPEG processing section 110, memory interface 111 (hereinafter referred to as memory I/F 111), recording medium 112, LCD driver 113, LCD 114, microcomputer 115, operation section 116, flash memory 117, enlargement/reduction section 118, and Image addition unit 119 .

The lens 1010 focuses the optical image of the subject on the imaging element 102 . The lens 1010 can be a single-focus lens or a zoom lens.

Microcomputer 1012 is connected to I/F 999 , flash memory 1011 , and drive 1013 , reads/writes information stored in flash memory 1011 , and controls drive 1013 . Microcomputer 1012 can also communicate with microcomputer 115 through I/F999, send various information to microcomputer 115, and receive information such as aperture value from microcomputer 115.

The driver 1013 receives instructions from the microcomputer 1012 to drive the lens 1010 to change the focal distance and focus position, and to drive the aperture 1014 at the same time. The aperture 1014 is provided near the lens 1010 and adjusts the amount of light on the subject.

The mechanical shutter 101 is driven by an instruction from the microcomputer 115 to control the timing for exposing the subject by the imaging element 102 .

The imaging element 102 is an imaging element in which a color filter in a Bayer arrangement is arranged in front of a photodiode constituting each pixel. The Bayer array has rows in which R pixels and G (Gr) pixels are alternately arranged in the horizontal direction, rows in which G (Gb) pixels and B pixels are alternately arranged, and these two rows are alternately arranged in the vertical direction. The imaging element 102 receives the light condensed by the lens 1010 by photodiodes constituting the pixels, performs photoelectric conversion, and outputs the amount of light to the analog processing unit 103 as an amount of charge. In addition, the imaging element 102 may be a CMOS type element or a CCD type element.

The analog processing unit 103 performs waveform shaping after reducing reset noise and the like on the electric signal (analog image signal) read from the imaging element 102 , and further performs gain enhancement so as to achieve target brightness. The A/D conversion unit 104 converts the analog image signal output from the analog processing unit 103 into a digital image signal (hereinafter referred to as image data).

The bus 105 is a transmission path for transmitting various data generated inside the digital camera to various parts in the digital camera. Bus 105 and A/D conversion section 104, SDRAM 106, image processing section 107, AE processing section 108, AF processing section 109, JPEG processing section 110, memory I/F 111, LCD driver 113, microcomputer 115, enlargement/reduction section 118, and The image addition unit 119 is connected.

Image data output from the A/D conversion unit 104 is temporarily stored in the SDRAM 106 via the bus 105 . SDRAM 106 is a storage unit for temporarily storing image data obtained by A/D conversion unit 104, image data processed by image processing unit 107, JPEG processing unit 110, enlargement/reduction unit 118, and image addition unit 119, etc. kinds of data.

The image processing section 107 includes an optical black (optical black) subtraction section 1071 (hereinafter referred to as OB subtraction section), a white balance correction section 1072 (hereinafter referred to as WB correction section 1072), a synchronization processing section 1073, a gamma/color reproduction processing Section 1074 , color matrix calculation section 1075 , edge enhancement processing section 1076 , and noise reduction processing section 1077 (hereinafter referred to as NR processing section 1077 ) perform various image processing on image data read from SDRAM 106 .

The OB subtraction unit 1071 performs optical black subtraction processing (hereinafter referred to as OB subtraction processing) on the image data. The OB subtraction process is a process of subtracting an optical black value (hereinafter referred to as an OB value) caused by a dark current of the imaging element 102 or the like from the pixel value of each pixel constituting image data.

The WB correction unit 1072 multiplies the image data by a white balance gain corresponding to the white balance mode to correct the white balance. The white balance mode can be set by the user according to light sources such as sunny days, cloudy days, electric lights, and fluorescent lights.

The synchronization processing unit 1073 performs processing to simultaneously convert image data based on the Bayer array into image data composed of R, G, and B color information for each pixel. The gamma/color reproduction processing section 1074 performs gamma correction processing and color reproduction processing of changing the color tone of an image.

The color matrix calculation unit 1075 performs linear conversion of multiplying the image data by the color matrix, and corrects the color of the image data. The edge enhancement processing unit 1076 extracts an edge from the image data, multiplies the extracted edge data by a predetermined gain, and adds it to the image data, thereby performing processing for emphasizing the edge of the image data. The NR processing unit 1077 performs noise reduction processing by processing using a filter for reducing high frequencies, coring processing, and the like.

The image processing unit 107 selects each of the sections 1071 to 1077 provided therein as necessary, and performs each process. The image data processed by the image processing unit 107 is stored in the SDRAM 106 .

The AE processing unit 108 calculates subject brightness from image data. The data used to calculate the brightness of the subject can also be the output of a dedicated light metering sensor. The AF processing unit 109 extracts a signal of a high-frequency component from image data, and acquires a focus evaluation value through AF (Auto Focus: automatic focus) integration processing.

Enlargement/reduction unit 118 performs processing for reducing or enlarging the size of image data stored in SDRAM 106 to a specified size. When reducing image data, the enlargement/reduction unit 118 uses a low-pass filter to reduce the number of pixels of the image after reducing high-frequency components so that foldback distortion does not occur.

The image addition unit 119 adds image data after image processing to image data of a granular pattern similar to the graininess of silver salt grains, and outputs image data in which the granular pattern is superimposed. Image data on which the grain pattern is superimposed is stored in SDRAM 106 . In addition, the image data after image processing before being added to the image data of the grain pattern includes not only image data not subjected to reduction processing in the enlarging/reducing unit 118 but also image data subjected to reduction processing.

When recording image data, the JPEG processing unit 110 reads the image data from the SDRAM 106 , compresses the read image data according to the JPEG compression method, and temporarily stores the compressed JPEG image data in the SDRAM 106 . The microcomputer 115 adds a JPEG header required to constitute a JPEG file to the JPEG image data stored in the SDRAM 106 to generate a JPEG file, and records the generated JPEG file in the recording medium 112 through the memory I/F 111 . The recording medium 112 is, for example, a recording medium composed of a memory card that can be inserted and removed from the camera body 100 , but is not limited thereto.

LCD driver 113 causes LCD 114 to display images. The display of images includes REC view display for short-term display of image data captured just now, playback display of JPEG files recorded in the recording medium 112, and dynamic image display such as live view display. When reproducing the JPEG file recorded on the recording medium 112 , the JPEG processing unit 110 reads out the JPEG file recorded on the recording medium 112 , performs decompression processing, and temporarily stores the decompressed image data in the SDRAM 106 . LCD driver 113 reads the decompressed image data from SDRAM 106 , converts the read image data into video signals, and outputs them to LCD 114 to display images.

The microcomputer 115 functioning as a control unit collectively controls various programs of the digital camera body 100 . An operation unit 116 and a flash memory 117 are connected to the microcomputer 115 .

The operation unit 116 is an operation member such as a power button, a release button, and various input keys. When the user operates any one of the operation members of the operation unit 116, the microcomputer 115 executes various programs corresponding to the user's operation. The power button is an operation member for instructing on/off the power of the digital camera. When the power button is pressed, the microcomputer 115 turns on or off the power of the digital camera. The release button is configured to have two switches, a first release switch and a second release switch. When the release button is half-pressed and the first release switch is turned on, the microcomputer 115 executes an imaging preparation program such as AE processing and AF processing. Then, when the release button is fully pressed and the second release switch is turned on, the microcomputer 115 executes the imaging program to perform imaging.

The flash memory 117 stores various parameters necessary for the operation of the digital camera such as white balance gain and low-pass filter coefficient corresponding to the white balance mode, image data of a grainy pattern similar to that of silver salt grains, and data for a specific digital camera. The serial number of the camera, etc. Furthermore, the flash memory 117 also stores various programs executed by the microcomputer 115 . Microcomputer 115 reads parameters required for various programs from flash memory 117 in accordance with programs stored in flash memory 117 and executes various processes.

13 is a flowchart showing the flow of main processing performed by a digital camera to which the image processing device of the second embodiment is applied. When the user presses the power button and the digital camera is turned on, the microcomputer 115 starts the process of step S1301.

In step S1301, it is determined whether or not photography is being performed, that is, whether or not the release button has been pressed. When it is determined that the release button is pressed, proceed to step S1302, and perform still image shooting/recording actions. The still image shooting/recording operation will be described later using the flowcharts shown in FIGS. 14A and 14B .

When it is determined in step S1301 that the release button has not been pressed, the process proceeds to step S1303. In step S1303, it is determined whether there is an instruction to change the recording size of image data. The user can change (designate) the recording size of the image data by operating the buttons of the operation unit 116 . The recording size can be selected from, for example, pixel sizes of 4032×3024 (full size), 2560×1920, 1280×960, and 640×480. When it is determined that the user has instructed to change the recording size, the process proceeds to step S1304.

In step S1304, change/setting to a recording size corresponding to the user's instruction is performed.

On the other hand, when it is determined in step S1303 that there is no instruction to change the recording size, the process proceeds to step S1305. Live view display is performed in step S1305. The specific processing of the live view display will be described later using the flowchart shown in FIG. 15 .

In step S1306, it is determined whether the power of the digital camera is turned off. When it is determined that the power supply has not been turned off, the process returns to step S1301. On the other hand, when it is determined that the user has pressed the power button and the power has been turned off, the processing of the flowchart ends.

In addition, in the above-mentioned flowchart of FIG. 13 , the case where the live view display is performed when the release button is not pressed after the power is turned on, and there is no instruction to change the recording size of the image data, may be The live view display is performed when the user instructs the live view display.

14A and 14B are flowcharts showing specific processing contents of still image shooting/recording operations performed in step S1302 of the flowchart shown in FIG. 13 . The processing from step S1401 to step S1403 is processing performed using the first release switch, and the processing after step S1404 is processing performed using the second release switch.

In step S1401 , an in-focus evaluation value is calculated in the AF processing unit 109 . The microcomputer 115 issues a command to the driver 1013 to drive the lens 1010 based on the focus evaluation value.

In step S1402 , subject brightness is calculated in the AE processing unit 108 . In step S1403 , the aperture and shutter speed are calculated based on the brightness of the subject and referring to the aperture value and shutter speed determination table stored in the flash memory 117 .

In step S1404, it is determined whether the user presses the release switch fully and whether the second release switch is turned on. When it is determined that the second release switch is not turned on, the process proceeds to step S1405. In step S1405, it is determined whether the release button is restored. When it is determined that the release button has been restored from the half-pressed state, the photographing/recording operation ends, and when it is determined that the release button has not been restored, the process returns to step S1404.

On the other hand, when it is determined in step S1404 that the second release switch is turned on, the process proceeds to step S1406. Photography is performed in step S1406. Regarding photography, it is the same method as used in the past. The driver 1013 drives the aperture 1014 to a set aperture value according to instructions from the microcomputer 1012 . Then, based on the calculated shutter speed, the mechanical shutter 101 is controlled to perform shooting, and image data (first image data) is obtained.

In step S1407 , the OB subtraction unit 1071 performs OB subtraction processing of subtracting the OB value obtained at the time of imaging from the image data obtained by imaging.

In step S1408 , the WB correction unit 1072 multiplies the image data subjected to the OB subtraction process by the white balance gain corresponding to the white balance mode to correct the white balance. In addition, the white balance mode can be set every time a photograph is taken by the user operating an input key included in the operation unit 116 . The microcomputer 115 sets the white balance mode in accordance with the user's operation on the operation unit 116 . Furthermore, when the digital camera has an automatic white balance function for automatically adjusting the white balance, the microcomputer 115 automatically sets a white balance mode corresponding to the light source at the time of shooting.

In step S1409 , the synchronization processing is performed by the synchronization processing unit 1073 on the image data subjected to the white balance correction processing. In step S1410, the color matrix calculation unit 1075 performs a color matrix calculation of multiplying the synchronized image data by a color matrix corresponding to the white balance mode.

In step S1411 , the gamma/color reproduction processing unit 1074 performs gamma correction processing and color reproduction processing for changing the color tone of the image on the image data subjected to the color matrix calculation.

In step S1412 , edge emphasis processing is performed by the edge emphasis processing unit 1076 on the image data subjected to gamma correction processing and color reproduction processing.

In step S1413 , the NR processing unit 1077 performs noise reduction processing on the image data subjected to the edge enhancement processing. The noise reduction process is a coring process based on a coring parameter, or a process using a filter that reduces high frequencies according to a noise reduction parameter (hereinafter referred to as NR parameter).

After performing the processing of step S1413, the process goes to step S1414 of FIG. 14B. In step S1414, the image data after the noise reduction processing (second image data) is read out from the SDRAM 106 by the enlargement/reduction unit 118, and the data size of the read image data (full size) is reduced to the REC finder display size. processing. The REC finder display size is, for example, a pixel size of 640×480.

In step S1415, it is determined whether or not the grain pattern adding mode of image data for adding a grain pattern to image data has been set. The user can perform the setting of the granular pattern adding mode in advance according to the operation of the operation unit 116 . When it is determined that the granular pattern adding mode is not set, the process proceeds to step S1417, and when it is determined that the granular pattern adding mode is set, the process proceeds to step S1416.

In step S1416, the image adding unit 119 performs processing of adding the image data of the grain pattern stored in the flash memory 117 to the image data reduced in step S1414. Then, the gamma/color reproduction processing unit 1074 performs gradation conversion processing for emphasizing the contrast of the image data on the added image data. This gradation conversion process suppresses the graininess of the brightest and darkest parts of the image while emphasizing the graininess of the intermediate style, expressing the graininess unique to silver halide photographs.

In step S1417, REC finder display for displaying image data on LCD 114 for a very short time is performed. The image data for REC viewfinder display is image data with a grain pattern added when the process of step S1416 is performed, and image data without a grain pattern added when the process of step S1416 is not performed.

In step S1418, it is determined whether or not the set recording size is a reduced size. Here, if the set recording size is the full size, it is determined that it is not a reduced size, and when other recording sizes are set, it is determined that it is a reduced size. If it is determined that the set recording size is not a reduced size, the process proceeds to step S1420, and if it is determined to be a reduced size, the process proceeds to step S1419.

In step S1419, the enlarging/reducing section 118 performs the process of reading out the full-size image data stored in the SDRAM 106 after the noise reduction process in step S1413, and reducing the data size of the image data to the recording size. . The record size is the data size changed in step S1304 of the flowchart shown in FIG. 13 .

In step S1420, it is determined whether the granular pattern adding mode is set. When it is determined that the granular pattern adding mode is not set, the process goes to step S1422, and when it is determined that the granular pattern adding mode is set, the process goes to step S1421.

In step S1421, the image adding unit 119 performs processing of adding the image data of the grain pattern stored in the flash memory 117 to the image data (third image data) reduced to the recording size in step S1419. Then, the added image data is subjected to gradation conversion processing for emphasizing the contrast of the image data by the gamma/color reproduction processing unit 1074 . This gradation conversion process suppresses the graininess of the brightest and darkest parts of the image while emphasizing the graininess of the intermediate style, expressing the graininess unique to silver halide photographs.

In addition, when it is determined in step S1418 that the set recording size is not a reduced size and the process proceeds to step S1421, image data of a grainy pattern is added to the image data subjected to noise reduction processing in step S1413. In this case, the image data of the grain pattern added to the image data not reduced and the image data of the grain pattern added to the image data after reduction are the same data.

In step S1422 , JPEG compression is performed on the image data in the JPEG processing unit 110 . The image data subject to JPEG compression is image data to which a grainy pattern has been added when the process of step S1421 has been performed, and image data to which a grainy pattern has not been added when the process of step S1421 has not been performed. In step S1423, photography information such as the recording mode and exposure conditions of the image is generated as file header information, the generated file header information is added to the image data compressed by JPEG, and recorded on the recording medium through the memory I/F 111 112 in.

FIG. 15 is a flowchart showing specific processing contents of live view display performed in step S1305 of the flowchart shown in FIG. 13 .

In step S1501 , subject brightness is calculated in the AE processing unit 108 . In step S1502 , the aperture and shutter speed are calculated based on the brightness of the subject and referring to the aperture value and shutter speed determination table stored in the flash memory 117 .

Photography is performed in step S1503. This photography is photography for live view display, that is, photography using an electronic shutter. The imaging for live view display is the same as the conventionally used method, so detailed description is omitted here.

In step S1504 , the OB subtraction unit 1071 performs OB subtraction processing of subtracting the OB value obtained at the time of imaging from the image data obtained by imaging. In step S1505 , the WB correcting unit 1072 multiplies the image data subjected to the OB subtraction process by the white balance gain corresponding to the white balance mode to correct the white balance.

In step S1506 , the synchronization processing is performed on the image data subjected to the white balance correction processing by the synchronization processing unit 1073 . In step S1507, the color matrix calculation unit 1075 performs a color matrix calculation of multiplying the synchronized image data by a color matrix corresponding to the white balance mode.

In step S1508, the gamma/color reproduction processing unit 1074 performs gamma correction processing and color reproduction processing for changing the color tone of the image on the image data subjected to the color matrix calculation. In step S1509 , edge emphasis processing is performed on the image data subjected to gamma correction processing and color reproduction processing by the edge emphasis processing unit 1076 .

In step S1510 , the NR processing unit 1077 performs noise reduction processing on the image data subjected to the edge enhancement processing. The noise reduction process is a coring process based on a coring parameter, or a process using a filter that reduces high frequencies according to a noise reduction parameter (hereinafter referred to as NR parameter).

In step S1511, the enlarging/reducing unit 118 performs the process of reading out the image data stored in the SDRAM 106 after performing the noise reduction processing in step S1510, and reducing the data size of the image data to the live view display image size. . The live view display image size is, for example, a pixel size of 640×480.

In step S1512, it is determined whether the grain pattern adding mode is set. When it is determined that the granular pattern adding mode is not set, the process goes to step S1514, and when it is determined that the granular pattern adding mode is set, the process goes to step S1513.

In step S1513, the image adding unit 119 performs a process of adding the image data of the grain pattern stored in the flash memory 117 to the image data reduced in step S1511. Then, the added image data is subjected to gradation conversion processing for emphasizing the contrast of the image data by the gamma/color reproduction processing unit 1074 . This gradation conversion process suppresses the graininess of the brightest and darkest parts of the image while emphasizing the graininess of the intermediate style, expressing the graininess unique to silver halide photographs.

In step S1514 , the image data is displayed on LCD 114 by LCD driver 113 . Here, when the process of step S1513 is performed, the image data with the grain pattern added is displayed, and when the process of step S1513 is not performed, the image data without the grain pattern is displayed. By repeating the processing from step S1501 to step S1514, live view display can be continuously performed.

As described above, according to the image processing device of the second embodiment, there is provided an image processing device that adds a predetermined granular pattern to image data, and when generating reduced image data with a relatively small number of pixels, adds a predetermined granular pattern to the reduced image data, Therefore, it is possible to obtain reduced image data on which a grainy pattern having a sufficiently grainy feeling is superimposed. That is, when image data with a grain pattern added is reduced to generate reduced image data as in the prior art, the grain pattern may be destroyed during reduction, but according to the image processing device of the second embodiment, there is no grain pattern being destroyed. destroy that situation.

(third embodiment)

In the image processing device according to the second embodiment, image data is reduced and then a predetermined grainy pattern is added, thereby generating reduced image data that maintains a grainy feel. In the image processing device according to the third embodiment, the granular pattern is enlarged at a magnification rate corresponding to the reduction rate at the time of image reduction, the image data before reduction and the enlarged granular pattern are added, and the image data is reduced, thereby Reduced image data to which a grainy pattern is added is generated.

The main processing flow performed by the digital camera to which the image processing device of the third embodiment is applied is the same as the processing flow shown in FIG. 13 . However, the still image shooting/recording processing performed in step S1302 is different from the live view display processing performed in step S1305, and these processing contents will be described below.

First, still image shooting/recording processing performed in the image processing device of the third embodiment will be described. In the still image shooting/recording processing performed by the image processing device of the third embodiment, the processing from step S1401 to step S1413 shown in FIG. 14A is the same as that of the second embodiment, and the processing performed after step S1413 different. Therefore, the processing performed after step S1413 will be described below.

FIG. 16 is a flowchart showing the content of processing performed after step S1413 in FIG. 14A in the still image shooting/recording operation processing performed by the image processing apparatus according to the third embodiment. In step S1601, which proceeds after the processing of step S1413, it is determined whether or not the grain pattern adding mode is set. When it is determined that the granular pattern adding mode is not set, the process goes to step S1604, and when it is determined that the granular pattern adding mode is set, the process goes to step S1602.

In step S1602, first, the reciprocal of the reduction ratio when the captured full-size image data is reduced to the record-size image data is set as the enlargement ratio. Then, the image data of the granular pattern stored in the flash memory 117 is magnified at a set magnification factor to generate enlarged granular pattern image data. For example, when the reduction ratio is 1/2, the enlargement ratio is set to 2 times, and the image data of the grain pattern stored in the flash memory 117 is enlarged by 2 times. This processing may be performed by the microcomputer 115, or may be performed by an enlarged granular pattern data generating unit provided separately.

In step S1603, the enlargement/reduction unit 118 performs a process of reading out the full-size image data stored in the SDRAM 106 after performing the noise reduction process in step S1413, and comparing them with the enlarged granular image data generated in step S1602. The pattern image data is added. Then, the added image data is subjected to gradation conversion processing for emphasizing the contrast of the image data by the gamma/color reproduction processing unit 1074 . This gradation conversion process suppresses the graininess of the brightest and darkest parts of the image while emphasizing the graininess of the intermediate style, expressing the graininess unique to silver halide photographs.

In step S1604, the enlarging/reducing section 118 performs the process of reading the image data added to the enlarged grain pattern image data from the SDRAM 106, and reducing the data size of the read image data (full size) to REC viewfinder display size. Wherein, when the processing of steps S1602 and S1603 is not performed, the image data to which the enlarged granular pattern image data is not added is read from SDRAM 106, and the data size of the read image data is reduced to the REC finder display size.

In step S1605, a REC finder display is performed for displaying the image data reduced in step S1604 on LCD 114 for a very short time.

In step S1606, it is determined whether the set recording size is a reduced size. If it is determined that the set recording size is not a reduced size, the process goes to step S1608, and if it is determined to be a reduced size, the process goes to step S1607.

In step S1607, the enlargement/reduction unit 118 performs a process of reading out the full-size image data stored in the SDRAM 106 and reducing the data size to a recording size. The image data to be reduced is image data added to the enlarged granular pattern image data when the processing of step S1603 is performed, and image data not added to the enlarged granular pattern image data when the processing of step S1603 is not performed.

In step S1608 , JPEG compression is performed on the image data in the JPEG processing unit 110 . The image data subject to JPEG compression is image data that has been reduced when the process of step S1607 is performed, and image data that has not been reduced when the process of step S1607 has not been performed. In step S1609, photographing information such as the recording mode and exposure conditions of the image is generated as file header information, the generated file header information is added to the image data compressed by JPEG, and recorded on the recording medium through the memory I/F 111 112 in.

Next, live view display processing performed in the image processing device according to the third embodiment will be described.

17 is a flowchart showing specific processing contents of live view display processing performed in the image processing device according to the third embodiment. Steps that perform the same processing as those in the flowchart shown in FIG. 15 are denoted by the same reference numerals, and detailed description thereof will be omitted.

The processing from step S1501 to step S1510 is the same as that of the flowchart shown in FIG. 15 . In step S1701 succeeding step S1510, it is determined whether or not the granular pattern adding mode is set. When it is determined that the granular pattern adding mode is not set, the process goes to step S1704, and when it is determined that the granular pattern adding mode is set, the process goes to step S1702.

In step S1702, the reciprocal of the reduction ratio when reducing the full-size image data obtained by photography to the image data of the recording size is set as the enlargement ratio, and the granular pattern stored in the flash memory 117 is converted into The image data of the image is enlarged, thereby generating enlarged grain pattern image data. This processing may be performed by the microcomputer 115, or may be performed by an enlarged granular pattern data generating unit provided separately.

In step S1703, the enlargement/reduction unit 118 performs a process of reading out the full-size image data stored in the SDRAM 106 after the noise reduction process in step S1510, and comparing them with the enlarged granular image data generated in step S1702. The pattern image data is added. Then, the added image data is subjected to gradation conversion processing for emphasizing the contrast of the image data by the gamma/color reproduction processing unit 1074 . This gradation conversion process suppresses the graininess of the brightest and darkest parts of the image while emphasizing the graininess of the intermediate style, expressing the graininess unique to silver halide photographs.

In step S1704, the enlarging/reducing unit 118 performs processing of reading the image data added to the enlarged grain pattern image data from the SDRAM 106, and reducing the data size of the read image data into a live view display image. size. Wherein, when the processing of steps S1702 and S1703 is not performed, the image data to which the enlarged granular pattern image data is not added is read from SDRAM 106, and the data size of the read image data is reduced to the live view display image size.

In step S1514 , the image data reduced in step S1704 is displayed on LCD 114 . By repeating the processing from step S1501 to step S1514, live view display can be continuously performed.

As described above, according to the image processing apparatus of the third embodiment, there is provided an image processing apparatus that generates reduced image data having fewer pixels than the image data at a predetermined reduction ratio, and enlarges the grain pattern at an enlargement ratio corresponding to the predetermined reduction ratio. , after adding the enlarged grainy pattern to the image data, it is reduced at a predetermined reduction rate. Therefore, it is possible to obtain reduced image data on which a grainy pattern having a sufficiently grainy feeling is superimposed.

In addition, in the second embodiment, when performing still image shooting/recording processing, as shown in FIG. As shown, only one time is sufficient, so the processing time can be shortened. In addition, in step S1603, the granular pattern image data enlarged by the reciprocal value of the reduction ratio when the image data is reduced to the recording size, that is, the magnification ratio is added, and the REC viewfinder display is performed. When the REC viewfinder display size is different from the recording size, the enlarged granular pattern image data will not become the data size corresponding to the REC viewfinder display size, but since the REC viewfinder display is only performed for a very short time, it is more likely to cause problems when displaying Small.

In addition, in the description of each of the above-mentioned embodiments, an example in which an image processing device is applied to a digital camera has been cited and described, but it may be configured such that a computer executes a program for realizing the processing described in the embodiments. In this case, the computer has a main storage device such as a CPU and a RAM, and a computer-readable storage medium storing a program for realizing all or part of the processing described in each embodiment. Here, this program is called an image processing program. Further, the CPU reads out the image processing program stored in the storage medium, and executes information processing/calculation processing, thereby realizing the same processing as that of the above-mentioned image processing device.

Wherein, the computer-readable storage medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like. Furthermore, the above-mentioned program may be distributed to computers via a communication line, and the computer that receives the distribution may execute the program.

The present invention is not limited to the first to third embodiments described above, and various modifications and applications are possible within a range not departing from the gist of the present invention. For example, an example of a grainy pattern similar to the graininess of silver salt grains is given as a predetermined grainy pattern to be added to image data, but the grainy pattern is not limited to this grainy pattern.

In the first embodiment, the gains GRN_Y and GRN_C for determining the intensity of the grain pattern added to the image data are obtained from the ISO sensitivity at the time of shooting, but they may be set to any value by the user according to the operation of the operation unit 7. value. According to this method, a granular pattern of intensity desired by the user can be added to image data. In addition, the gains GRN_Y and GRN_C for determining the intensity of the grain pattern may be set according to the degree of noise on the image.

In the first embodiment, the case where JPEG compression is performed on image data is described, but the compression method of image data is not limited to the JPEG format.

In the third embodiment, when the REC finder display is performed, the granular pattern enlarged by the reciprocal of the reduction ratio when the image data is reduced to the recording size, that is, the magnification is added. However, the granular pattern may be enlarged and added to the image data at a magnification ratio that is the reciprocal of the reduction ratio when the image data is reduced to the REC viewfinder display size, and then reduced to the REC viewfinder display size at the reduction ratio. Zoom out and display the REC viewfinder.

Japanese Patent Application No. 2008-226060 filed on September 3, 2008, Japanese Patent Application No. 2008-226074 filed on September 3, 2008, and Japanese Patent Application No. 2009 filed on April 10, 2009 The content of No. 2009-95626 is incorporated in this application by reference.

Claims (14)

1. An image processing device, the image processing device has: a simultaneous processing section that generates image data having a plurality of color components per pixel based on image data obtained by an image pickup device having an image pickup element provided with a plurality of color filters; a first gradation conversion unit that performs a first gradation conversion process on the image data generated by the synchronization processing unit, and a granular pattern adding unit for adding a predetermined granular pattern to the image data subjected to the first gradation conversion process by the first gradation conversion unit, The second gradation conversion unit performs a second gradation conversion process different from the first gradation conversion process on the image data to which the predetermined granular pattern is added. 2. The image processing device according to claim 1, wherein the second gradation conversion unit performs gradation conversion processing for emphasizing the contrast of image data as the second gradation conversion processing. 3. The image processing device according to claim 1 , further comprising an edge extraction unit for extracting an edge used in edge emphasis processing from image data before the predetermined granular pattern is added. the edge of. 4. The image processing device according to any one of claims 1 to 3, further comprising an intensity adjustment unit that adjusts the intensity of the predetermined grain pattern added to the image data. strength. 5 . The image processing device according to claim 4 , wherein the intensity adjustment unit adjusts the intensity of the predetermined granular pattern according to an imaging sensitivity when the imaging device is used for imaging. 6 . 6. The image processing device according to claim 4, The image processing device further includes a setting unit for setting the intensity of the predetermined granular pattern added to the image data, The intensity adjustment unit adjusts the intensity of the predetermined granular pattern according to a user's operation on the setting unit. 7. The image processing device according to claim 1, The image processing device further includes a noise reduction processing unit that performs predetermined noise reduction processing on image data obtained by an image pickup device having an image pickup element provided with a plurality of color filters, The synchronization processing unit generates image data having a plurality of color components per pixel, based on the image data subjected to the predetermined noise reduction processing. 8. The image processing device according to claim 7, wherein the noise reduction processing unit performs at least one of processing for correcting pixel defects of the imaging device and processing for reducing random noise generated when taking pictures with the imaging device. processing as the predetermined noise reduction processing. 9. The image processing device according to claim 7 or 8, further comprising a control unit configured to control the degree of noise reduction processing, and the intensity of the predetermined granular pattern added to the image data generated by the synchronization processing unit. 10. The image processing device according to claim 9, wherein the control unit controls the degree of the noise reduction processing and the intensity of the predetermined granular pattern so that when the photographing sensitivity of the photographing by the imaging device is different. The graininess of the predetermined grain pattern added to the image data is made the same. 11. An image processing device, the image processing device having: a granular pattern adding unit for adding a predetermined granular pattern to image data subjected to predetermined image processing including at least a first gradation conversion process; and The gradation conversion unit performs a second gradation conversion process different from the first gradation conversion process on the image data to which the predetermined granular pattern is added. 12. The image processing device according to claim 11, wherein the gradation conversion unit performs gradation conversion processing for emphasizing contrast of image data as the second gradation conversion processing. 13. An image processing method, the image processing method comprising: In a first step, image data having multiple color components per pixel is generated based on image data obtained by an imaging device having an imaging element provided with a plurality of color filters; The second step is to implement the first grayscale conversion process on the image data having multiple color components for each pixel, In a third step, adding a predetermined grain pattern to the image data subjected to the first gradation conversion process; and In a fourth step, a second gradation conversion process different from the first gradation conversion process is performed on the image data to which the predetermined grain pattern is added. 14. The image processing method according to claim 13, The image processing method further includes the step of performing predetermined noise reduction processing on image data obtained by an image pickup device having an image pickup element provided with a plurality of color filters, In the first step, image data having a plurality of color components per pixel is generated based on the image data subjected to the predetermined noise reduction processing.

Images