WO2006033235A1

WO2006033235A1 - Image processing method, image processing device, imaging device, and image processing program

Info

Publication number: WO2006033235A1
Application number: PCT/JP2005/016383
Authority: WO
Inventors: Hiroaki Takano; Tsukasa Ito; Takeshi Nakajima; Kenji Kuwae; Takeshi Saito; Misae Tasaki; Daisuke Sato
Original assignee: Konica Minolta Photo Imaging, Inc.
Priority date: 2004-09-22
Filing date: 2005-09-07
Publication date: 2006-03-30
Also published as: JP2006092133A

Abstract

An image adjustment unit divides acquired image data into areas consisting of at least a combination of a predetermined lightness and hue or a combination of a distance of the acquired image data from an external brim of the screen and the lightness. For each of the areas divided, an occupancy indicating the ratio with respect to the entire acquired image data is calculated, and a deviation amount indicating the deviation of the gradation distribution of the acquired image data is calculated. According to the calculated occupancy and deviation amount, an index to specify the imaging condition is calculated. According to the calculated index and a judgment map separated into areas in advance according to the accuracy of the imaging condition, the imaging condition of the acquired image data is identified. According to the identified imaging condition, a method for gradation adjustment of the acquired image data is decided. By using the decided gradation adjustment method, the acquired image data is subjected to the gradation conversion.

Description

Specification

Image processing method, image processing apparatus, imaging apparatus, and image processing program

The present invention relates to an image processing method, an image processing device, an imaging device, and an image processing program.

Background art

Conventionally, brightness correction of film scan images and digital camera images has been performed by correcting the average brightness of the entire image to a value desired by the user. In normal shooting, the light source conditions such as forward light, backlight, and strobe change variously, resulting in a large area with a large luminance deviation in the image. In addition to correcting the average luminance, discriminant analysis, Additional correction using values calculated by regression analysis was necessary. However, the discriminant regression analysis method has a problem that it is difficult to discriminate shooting conditions (shooting scenes) because the parameters for calculating the stroboscopic scene and the backlight scene force are very similar.

[0003] Patent Document 1 discloses a method for calculating an additional correction value in place of the discriminant regression analysis method. The method described in Patent Document 1 deletes the high luminance region and the low luminance region from the luminance histogram indicating the cumulative number of pixels of luminance (frequency number), and further uses the frequency frequency limit to limit the luminance. An average value is calculated, and a difference between the average value and the reference luminance is obtained as a correction value.

[0004] Patent Document 2 describes a method of determining a light source state at the time of photographing in order to compensate for the extraction accuracy of a face region. In the method described in Patent Document 2, first, a face candidate area is extracted, and the average brightness of the extracted face candidate area is calculated with respect to the entire image. (Shooting close-up flash)) and adjust the tolerance of the judgment criteria for the face area. In Patent Document 2, as a method for extracting a face candidate region, a method using a two-dimensional histogram of hue and saturation described in JP-A-6-67320, JP-A-8-122944, JP-A-8-184925 The pattern matching and pattern search methods described in Japanese Patent Laid-Open No. 9138471 and Japanese Patent Laid-Open No. 9138471 are used for bow I.

[0005] Patent Document 2 discloses a method for removing a background region other than a face as disclosed in JP-A-8-122944. And the method described in Japanese Patent Application Laid-Open No. 8-184925 and the method of discriminating using the ratio of the linear portion, the line object property, the contact ratio with the outer edge of the screen, the density contrast, the pattern of the density change, and the periodicity. Has been. A method that uses a one-dimensional histogram of density is described to determine shooting conditions. This method is based on an empirical rule that the face area is dark and the background area is bright in the case of backlighting, and that the face area is bright and the background area is dark in the case of close-up flash photography.

Patent Document 1: JP 2002-247393 A

Patent Document 2: JP 2000-148980 A

[0006] However, the technique described in Patent Document 1 reduces the influence of a region having a large luminance deviation in a backlight scene and a strobe scene, but in a shooting scene in which a person is a main subject, There was a problem that the brightness of was inappropriate. In addition, the technique described in Patent Document 2 can achieve the effect of compensating for the identification of the face area in the case of typical backlight or close-up flash photography, but if it does not apply to the typical composition, There was a problem that the effect could not be obtained.

[0007] Also, most of image data taken with a digital camera is underexposure due to underexposure, even though there are many images taken with inappropriate exposure conditions. When discriminating shooting conditions derived from light source conditions such as backlighting or close-up flash photography, there is a problem that the under image is mistaken for close-up flash photography and the over image is mistaken for a backlit scene, leading to density correction in the exact opposite direction. there were.

[0008] As described above, in order to realize quantitative correction based on a person's face area, improvement of the imaging condition discrimination capability is an important issue. Furthermore, measures against misjudgment of shooting conditions, such as evaluating the reliability of the judgment result of shooting conditions, are desired.

Disclosure of the invention

[0009] An object of the present invention is to determine a gradation adjustment method based on an index that quantitatively represents shooting conditions (light source conditions and exposure conditions) of shot image data and a discrimination map that evaluates the reliability of the index. This is to improve the lightness reproducibility of the subject.

[0010] In order to solve the above-mentioned problem, the form described in Item 1 is that the captured image data includes at least a predetermined combination of brightness and hue and a distance from the outer edge of the screen of the captured image data. An occupancy ratio calculating step of dividing the area into any one of the combinations of brightness and calculating an occupancy ratio indicating a ratio of the entire photographic image data for each of the divided areas; and a gradation of the photographic image data A bias amount calculating step for calculating a bias amount indicating a distribution bias; and

To specify the shooting condition by multiplying the occupation ratio of each area calculated in the occupancy ratio calculation step and the deviation amount calculated in the deviation amount calculation step by a coefficient set in advance according to the shooting condition. An index calculation step for calculating the index of the image, and a determination for determining the shooting condition of the captured image data based on the calculated index and a determination map divided in advance according to the accuracy of the shooting condition! Process,

An adjustment method determining step for determining a gradation adjustment method for the captured image data in accordance with the determined shooting conditions;

A gradation conversion step of performing gradation conversion processing on the captured image data using the determined gradation adjustment method.

Brief Description of Drawings

FIG. 1 is a perspective view showing an external configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an internal configuration of the image processing apparatus according to the present embodiment.

3 is a block diagram showing a main part configuration of the image processing unit in FIG.

圆 4] A diagram showing an internal configuration (a) of the scene determination unit, an internal configuration (b) of the ratio calculation unit, and an internal configuration (c) of the image processing condition calculation unit.

FIG. 5 is a flowchart showing a flow of processing executed in an image adjustment processing unit.

FIG. 6 is a flowchart showing index calculation processing executed in the scene determination unit.

FIG. 7 is a flowchart showing a first occupancy ratio calculation process for calculating a first occupancy ratio for each brightness and hue region.

FIG. 8 is a diagram showing an example of a program for converting RGB power into the HSV color system.

FIG. 9 is a diagram showing the brightness (V) —hue (H) plane and the region rl and region r2 on the V—H plane.

FIG. 10 is a diagram showing the lightness (V) —hue (H) plane, and regions r3 and r4 on the V—H plane.

FIG. 11 is a diagram showing a curve representing a first coefficient for multiplying the first occupancy for calculating index 1; FIG. 12 is a diagram showing a curve representing a second coefficient for multiplying the first occupancy for calculating index 2;

FIG. 13 is a flowchart showing a second occupancy ratio calculation process for calculating a second occupancy ratio based on the composition of captured image data.

FIG. 14 is a diagram showing areas nl to n4 determined according to the distance from the outer edge of the screen of captured image data.

FIG. 15 is a diagram showing, by region (n1 to n4), curves representing a third coefficient for multiplying the second occupancy ratio for calculating index 3;

FIG. 16 is a flowchart showing a bias amount calculation process executed in a bias calculator.

FIG. 17 is a flowchart showing tone adjustment condition determination processing executed in the image processing condition calculation unit.

[FIG. 18] Plot of indices 4-6 calculated according to shooting conditions.

FIG. 19 is a diagram showing a discrimination map for discriminating shooting conditions.

[FIG. 20] A diagram showing the relationship between an index for specifying shooting conditions, parameters A to C, and gradation adjustment methods A to C.

FIG. 21 is a diagram showing a gradation conversion curve corresponding to each gradation adjustment method.

FIG. 22 is a diagram showing a luminance frequency distribution (histogram) (a), a normalized histogram (b), and a block-divided histogram (c).

FIG. 23 is a diagram ((a) and (b)) for explaining the deletion of the low luminance region and the high luminance region of the luminance histogram power, and a diagram for explaining the limitation of the luminance frequency ((c) and ( d)).

FIG. 24 is a diagram showing a gradation conversion curve representing gradation adjustment conditions when the photographing condition is backlight or under.

FIG. 25 is a block diagram showing a configuration of a digital camera to which the imaging apparatus of the present invention is applied.

BEST MODE FOR CARRYING OUT THE INVENTION

[0012] Preferred embodiments according to the present invention will be described below.

[0013] In the form described in Item 2, in the image processing method described in Item 1, in the occupancy ratio calculating step, the captured image data is divided into regions that are combinations of predetermined brightness and hue, and For each divided area, an occupancy ratio indicating a ratio of the entire captured image data is set. calculate.

[0014] The form described in Item 3 is the image processing method described in Item 2,

And an adjustment amount determination step for determining a gradation adjustment amount for the captured image data based on the calculated index. In the gradation conversion step, the level determined by the adjustment method determination step is determined. Using the tone adjustment method, the photographed image data is subjected to gradation conversion processing of the gradation adjustment amount determined in the adjustment amount determination step.

[0015] In the form described in Item 4, in the image processing method described in Item 1, in the occupation rate calculating step, the captured image data is a combination of a distance from the outer edge of the screen of the captured image data and brightness. Is divided into predetermined areas, and for each of the divided areas, an occupancy ratio indicating a ratio of the entire captured image data is calculated.

[0016] The form described in Item 5 is the image processing method described in Item 4,

And an adjustment amount determination step for determining a gradation adjustment amount for the captured image data based on the calculated index. In the gradation conversion step, the level determined by the adjustment method determination step is determined. Using the tone adjustment method, tone conversion processing of the tone adjustment amount determined in the adjustment amount determination step is performed on the captured image data.

[0017] In the form described in Item 6, in the image processing method described in Item 1, in the occupancy rate calculating step, the captured image data is divided into regions corresponding to combinations of predetermined brightness and hue, For each divided area, a first occupancy ratio indicating the ratio of the entire captured image data is calculated, and the captured image data is combined with the distance from the outer edge of the captured image data and the brightness. Dividing into predetermined areas, and for each of the divided areas, calculate a second occupancy ratio indicating the ratio of the entire captured image data,

In the index calculation step, the first occupancy rate and the second occupancy rate calculated in the occupancy rate calculation step and the deviation amount calculated in the deviation amount calculation step are preset according to the shooting conditions. The index for specifying the shooting condition is calculated by multiplying the obtained coefficient.

[0018] The form described in Item 7 is the image processing method described in Item 6,

The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index, and in the gradation conversion step, the adjustment method Using the gradation adjustment method determined in the determination step, the photographed image data is subjected to gradation conversion processing of the gradation adjustment amount determined in the adjustment amount determination step.

[0019] In the form described in Item 8, in the image processing method described in Item 4 or 5, the number of accumulated pixels is calculated for each distance and brightness from the outer edge of the screen of the captured image data. A occupancy ratio calculating step, wherein the occupancy ratio is calculated based on the generated two-dimensional histogram.

[0020] In the form described in Item 9, in the image processing method described in Item 6 or 7, the cumulative number of pixels is calculated for each distance and brightness from the outer edge of the screen of the captured image data. Including a step of creating a dimension histogram, wherein the second occupancy rate is calculated based on the created two-dimensional histogram in the occupancy rate calculation step.

[0021] In the form described in Item 10, in the image processing method described in Item 2 or 3, a two-dimensional histogram is created by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data. In the occupation rate calculation step, the occupation rate is calculated based on the created two-dimensional histogram.

[0022] In the form described in Item 11, in the image processing method described in Item 6 or 7, a two-dimensional histogram is calculated by calculating the cumulative number of pixels for each predetermined hue and brightness of the photographed image data. Including a step of creating a ram, and in the occupation rate calculating step, the first occupation rate is calculated based on the created two-dimensional histogram.

[0023] In the image processing method according to any one of Items 1, 2, 3, 6, 7, 9 to: L1, the form described in Item 12 includes a predetermined high brightness in the index calculation step. The coefficients with different signs are used for the skin color hue area of the color tone and the hue area other than the high brightness skin color hue area.

[0024] In the image processing method according to any one of Items 1, 2, 3, 6, 7, and 9 to 12, the form described in Item 13 is an intermediate lightness of a flesh color hue region in the index calculation step. Different code coefficients are used for the region and the lightness region other than the intermediate lightness region.

[0025] In the form described in Item 14, in the image processing method described in Item 12, the brightness area of the hue area other than the high brightness skin color hue area is a predetermined high brightness area.

[0026] In the form described in Item 15, in the image processing method described in Item 13, the brightness area other than the intermediate brightness area is a brightness area in the flesh-color hue area. [0027] In the form described in Item 16, in the image processing method described in Item 12 or 14, the skin color hue region of high brightness includes a region in the range of 170 to 224 in terms of brightness value of the HSV color system. It is.

[0028] In the form described in Item 17, in the image processing method described in Item 13 or 15, the intermediate brightness region includes a region in the range of 85 to 169 in terms of the brightness value of the HSV color system.

[0029] The form described in Item 18 is the image processing method according to any one of Items 12, 14, and 16, wherein a hue region other than the high brightness skin color hue region includes a blue hue region, At least one of the green hue regions is included.

[0030] In the form described in Item 19, in the image processing method described in any one of Items 13, 15, and 17, the lightness region other than the intermediate lightness region is a shadow region.

[0031] The form described in Item 20 is the image processing method described in Item 18, wherein the hue value of the blue hue region is in the range of 161 to 250 in terms of the hue value of the HSV color system. The hue value in the hue area is in the range of 40 to 160 in the HSV color system.

In the form described in Item 21, in the image processing method described in Item 19, the brightness value of the shadow area is in the range of 26 to 84 as the brightness value of the HSV color system.

[0033] The form described in Item 22 and Item 23 is the image processing method described in any one of Items 12 to 21, wherein the hue value of the flesh color hue region is a hue value of the HSV color system. ~ 39 and 3

It is in the range of 30-359.

[0034] The form described in Item 24 and Item 25 is the image processing method described in any one of Items 12 to 23, in which the skin color hue region is expressed by a predetermined conditional expression based on lightness and saturation. It is divided into one area.

[0035] In the form described in Item 26, in the image processing method described in any one of Items 1 to 25, the deviation amount includes a brightness deviation amount of the photographed image data, and a screen of the photographed image data. It contains at least one of the average brightness value in the center and the brightness difference value calculated under different conditions.

Here, the brightness deviation amount of the photographed image data is, for example, the standard deviation or variance of the brightness of the photographed image data. In addition, the average brightness value of the captured image data at the center of the screen is, for example, the average luminance value at the center of the screen, or the skin color area at the center of the screen. The average luminance value of. Further, the brightness difference value calculated under different conditions is, for example, a difference value between the maximum brightness value and the minimum brightness value of the captured image data.

[0037] The form according to Item 27 is the image processing method according to any one of Items 1 to 26, wherein the photographing condition indicates a light source condition and an exposure condition at the time of photographing,

The light source condition includes a flash photographing state, and the exposure condition includes an under photographing state derived from a shutter speed and an aperture value at the time of photographing.

[0038] In the form described in Item 28, the photographed image data is at least in a region including any combination of a predetermined brightness and hue and a combination of the distance from the outer edge of the screen and the brightness of the photographed image data. An occupancy ratio calculating unit that divides and calculates an occupancy ratio indicating a ratio of the entire captured image data for each of the divided areas;

A deviation amount calculation unit that calculates a deviation amount indicating a deviation in gradation distribution of the photographed image data, an occupancy rate of each area calculated by the occupancy rate calculation unit, and a deviation amount calculated by the deviation amount calculation unit An index calculation unit that calculates an index for specifying the shooting condition by multiplying a preset coefficient according to the shooting condition;

A discriminator for discriminating the shooting condition of the shot image data based on the calculated index and a discrimination map pre-divided into regions according to the accuracy of the shooting condition;

An adjustment method determining unit that determines a gradation adjustment method for the captured image data according to the determined shooting condition;

An image processing apparatus including a gradation conversion unit that performs gradation conversion processing on the captured image data using the determined gradation adjustment method.

[0039] In the form described in Item 29, in the image processing device described in Item 28, the occupancy ratio calculation unit divides the captured image data into regions that are combinations of predetermined brightness and hue, and For each divided area, an occupancy ratio indicating the ratio of the entire captured image data is calculated.

[0040] The form described in Item 30 is the image processing apparatus described in Item 29,

Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index is provided, and in the gradation conversion unit, the gradation adjustment method determined by the adjustment method determination unit The adjustment is performed on the captured image data using A gradation conversion process of the gradation adjustment amount determined by the amount determination unit is performed.

[0041] In the form described in Item 31, in the image processing device described in Item 28, the occupancy ratio calculation unit includes a combination of the distance from the outer edge of the screen of the captured image data and the brightness. The area is divided into predetermined areas, and for each of the divided areas, an occupation ratio indicating a ratio of the entire captured image data is calculated.

[0042] The form described in Item 32 is the image processing device described in Item 31,

And an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion unit, gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit is performed on the captured image data using the gradation adjustment method determined by the adjustment method determination unit. Apply.

[0043] In the form described in Item 33, in the image processing device described in Item 28, the occupancy ratio calculation unit divides the captured image data into regions that are combinations of predetermined brightness and hue, and For each divided area, a first occupancy ratio indicating the ratio of the entire captured image data is calculated, and the captured image data is combined with the distance from the outer edge of the captured image data and the light intensity combination power. A second occupancy ratio indicating a ratio of the entire captured image data is calculated for each of the divided areas, and the index calculation unit calculates the occupancy ratio calculation unit. An index for identifying the shooting condition by multiplying the first occupancy ratio and the second occupancy ratio by the coefficient set in advance according to the shooting condition to the deviation amount calculated by the deviation amount calculation unit. Is calculated.

The form described in Item 34 is the image processing apparatus described in Item 33,

Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index is provided, and in the gradation conversion unit, the gradation adjustment method determined by the adjustment method determination unit Is used to perform gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the captured image data.

In the form described in Item 35, in the image processing device described in Item 31 or 32, a two-dimensional histogram is obtained by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data. A histogram creation unit to create, and the occupancy rate calculation unit, The occupation ratio is calculated based on the created two-dimensional histogram.

[0046] In the form described in Item 36, in the image processing device described in Item 33 or 34, a two-dimensional histogram is obtained by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data. A histogram creation unit is provided, and the occupation rate calculation unit calculates the second occupation rate based on the created two-dimensional histogram.

[0047] In the form described in Item 37, in the image processing device described in Item 29 or 30, a two-dimensional histogram is created by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data. A histogram creation unit is provided, and the occupation rate calculation unit calculates the occupation rate based on the created two-dimensional histogram.

[0048] In the form described in Item 38, in the image processing device described in Item 33 or 34, a two-dimensional histogram is created by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data. A histogram creation unit is provided, and the occupation rate calculation unit calculates the first occupation rate based on the created two-dimensional histogram.

[0049] In the image processing device described in any one of the forms described in Item 39, Item 28, 29, 30, 33, 34, 36 to 38, the index calculation unit has a predetermined high brightness. Different code coefficients are used for the flesh-color hue area and the hue area other than the high-brightness flesh-color hue area.

[0050] In the image processing apparatus according to any one of Items 28, 29, 30, 33, 34, and 36 to 39, the index calculation unit includes an intermediate part of a flesh-color hue region. Different code coefficients are used for the lightness region and the lightness region other than the intermediate lightness region.

[0051] In the form described in Item 41, in the image processing device described in Item 39, the lightness region of the hue region other than the high-brightness skin color hue region is a predetermined high-lightness region.

[0052] In the form described in Item 42, in the image processing device described in Item 40, the lightness area other than the intermediate lightness area is a lightness area in the flesh-color hue area.

[0053] In the form described in Item 43, in the image processing device described in Item 39 or 41, the high-brightness skin color hue region includes a region in the range of 170 to 224 in terms of the brightness value of the HSV color system. It is.

[0054] In the form described in Item 44, in the image processing device described in Item 40 or 42, the intermediate brightness region includes a region in the range of 85 to 169 in terms of the brightness value of the HSV color system. [0055] In the form described in Item 45, in the image processing device described in any one of Items 39, 41, and 43, the hue region other than the high brightness skin color hue region includes a blue hue region, At least one of the green hue regions is included.

[0056] In the form described in Item 46, in the image processing device described in any one of Items 40, 42, and 44, the lightness region other than the intermediate lightness region is a shadow region.

[0057] In the form described in Item 47, in the image processing device described in Item 45, the hue value of the blue hue region is in the range of 161 to 250 in terms of the hue value of the HSV color system. The hue value of the green hue region is in the range of 40 to 160 as the hue value of the HSV color system.

[0058] In the form described in Item 48, in the image processing device described in Item 46, the brightness value of the shadow area is in the range of 26 to 84 as the brightness value of the HSV color system.

[0059] In the image processing device according to any one of Items 39 to 48, the hue value of the flesh color hue region is 0 to 39 in the HSV color system. And 33

It is in the range of 0-359.

[0060] In the image processing device according to any one of Items 39 to 50, the skin color hue region is divided into two regions according to a predetermined conditional expression based on lightness and saturation. It is divided into.

[0061] In the form described in Item 53, in the image processing device described in any one of Items 28 to 52, the deviation amount includes a deviation amount of brightness of photographed image data, and a screen of the photographed image data. It contains at least one of the average brightness value in the center and the brightness difference value calculated under different conditions.

[0062] In the form described in Item 54, in the image processing apparatus described in any one of Items 28 to 53, the photographing condition indicates a light source condition and an exposure condition at the time of photographing. A robot shooting state is included, and the exposure condition includes an under shooting state derived from the shutter speed and aperture value at the time of shooting.

[0063] In the form described in Item 55, an image capturing unit that captures captured image data by capturing a subject, and the captured image data include at least a combination of predetermined brightness and hue and a screen of the captured image data. Is divided into areas consisting of any combination of distance and brightness from the outer edge, and the ratio of the divided area to the entire captured image data is indicated for each of the divided areas. An occupancy rate calculation unit for calculating an occupancy rate;

A gradation conversion unit that performs gradation conversion processing on the photographed image data using the determined gradation adjustment method.

[0064] In the form described in Item 56, in the imaging device described in Item 55, the occupation ratio calculation unit divides the photographed image data into regions having a combination of predetermined brightness and hue, and the divided image data is divided. For each area, an occupancy ratio indicating the ratio of the entire captured image data is calculated.

[0065] The form according to Item 57 is the imaging apparatus according to Item 56,

Furthermore, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index is provided, and in the gradation conversion, the gradation adjustment determined by the adjustment method determination unit Using the method, the photographed image data is subjected to gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit.

[0066] In the form described in Item 58, in the imaging device described in Item 55, the occupancy ratio calculation unit includes a combination of a distance from the outer edge of the screen of the captured image data and brightness. The area is divided into predetermined areas, and for each of the divided areas, an occupation ratio indicating a ratio of the entire captured image data is calculated.

[0067] The configuration according to Item 59, in the imaging device according to Item 58,

[0068] In the form described in Item 60, in the imaging device described in Item 55, the occupation ratio calculation unit divides the photographed image data into regions having a combination of predetermined brightness and hue, and the divided image data is divided. For each region, a first occupancy ratio indicating the ratio of the entire captured image data is calculated, and the captured image data is determined to be a predetermined combination power of distance and brightness from the outer edge of the captured image data screen. A second occupancy ratio indicating a ratio of the entire captured image data is calculated for each of the divided areas, and the index calculation unit calculates the first occupancy ratio calculated by the occupancy ratio calculation unit. The index for identifying the shooting condition is calculated by multiplying the occupancy ratio and the second occupancy ratio by the deviation amount calculated by the deviation amount calculation unit by a coefficient set in advance according to the shooting condition. To do.

[0069] The configuration according to Item 61, in the imaging device according to Item 60,

[0070] In the form described in Item 62, in the imaging device described in Item 58 or 59, a two-dimensional histogram is calculated by calculating a cumulative pixel count for each distance and brightness from the outer edge of the screen of the captured image data. A histogram creation unit is provided, and the occupancy rate calculation unit calculates the occupancy rate based on the created two-dimensional histogram.

[0071] In the configuration described in Item 63, in the imaging device described in Item 60 or 61, a two-dimensional histogram is obtained by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data. It has a histogram creation part to create,

The occupation ratio calculation unit calculates the second occupation ratio based on the created two-dimensional histogram.

[0072] In the form described in Item 64, in the imaging device described in Item 56 or 57, the two-dimensional histogram is calculated by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data. A histogram creation unit is provided, and the occupancy rate calculation unit calculates the occupancy rate based on the created 2D histogram. [0073] In the configuration described in Item 65, the two-dimensional histogram is calculated by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data in the imaging device described in Item 60 or 61. A histogram creation unit is provided, and the occupancy rate calculation unit calculates the first occupancy rate based on the created 2D histogram.

[0074] In the imaging device according to any one of the forms described in Item 66, Item 55, 56, 57, 60, 61, 63 to 65, the index calculation unit includes a skin color having a predetermined high brightness. Different sign coefficients are used for the hue area and the hue area other than the high brightness skin color hue area.

[0075] In the imaging device according to any one of the forms described in Item 67, Item 55, 56, 57, 60, 61, 63 to 66, the index calculation unit includes the intermediate lightness of the flesh color hue region. Different sign coefficients are used for the region and the lightness region other than the intermediate lightness region.

In the form described in Item 68, in the imaging device described in Item 66, the brightness region of the hue region other than the high-brightness skin color hue region is a predetermined high brightness region.

[0077] In the form described in Item 69, in the imaging device described in Item 67, the brightness area other than the intermediate brightness area is a brightness area in the flesh-color hue area.

[0078] In the form described in Item 70, in the imaging device described in Item 66 or 68, the skin color hue region of high brightness includes a region in the range of 170 to 224 in terms of brightness value of the HSV color system. .

[0079] In the form described in Item 71, in the imaging device described in Item 67 or 69, the intermediate brightness region includes a region in the range of 85 to 169 in terms of the brightness value of the HSV color system.

[0080] In the imaging device according to any one of Items 66, 68, and 70, the form according to Item 72 includes a blue hue region and a green hue region in the hue region other than the high brightness skin color hue region. At least one of them.

[0081] In the form described in Item 73, in the imaging device described in any one of Items 67, 69, and 71, the lightness region other than the intermediate lightness region is a shadow region.

[0082] In the form described in Item 74, in the imaging device described in Item 72, the hue value of the blue hue region is in a range of 161 to 250 in terms of the hue value of the HSV color system, The hue value in the green hue region is in the range of 40 to 160 in the HSV color system.

In the form described in Item 75, in the imaging device described in Item 73, the brightness value of the shadow area is in the range of 26 to 84 as the brightness value of the HSV color system. [0084] In the imaging device according to any one of Items 66 to 75, the hue value of the flesh color hue region is a hue value of the HSV color system in the form described in Item 76 and 77. It is in the range of 39 and 330-359.

[0085] In the imaging device according to any one of Items 66 to 77, the form described in Item 78 and 79 includes two skin color hue regions according to a predetermined conditional expression based on brightness and saturation. Divided into regions.

[0086] In the configuration described in Item 80, in the imaging device described in any one of Items 55 to 79, the deviation amount includes a brightness deviation amount of captured image data, a screen of the captured image data. It includes at least one of the average brightness value in the center and the brightness difference value calculated under different conditions.

[0087] In the form described in Item 81, in the imaging device described in any one of Items 55 to 80, the imaging condition indicates a light source condition and an exposure condition at the time of imaging, and the light source condition includes a strobe. The exposure condition includes an under shooting state derived from the shutter speed and aperture value at the time of shooting.

[0088] The form according to Item 82 is provided in a computer for executing image processing.

The photographed image data is divided into at least a predetermined combination of brightness and hue and a combination of the distance and brightness from the outer edge of the screen of the photographed image data, and for each of the divided areas, An occupancy ratio calculating step for calculating an occupancy ratio indicating the ratio of the entire captured image data;

A deviation amount calculating step of calculating a deviation amount indicating a deviation in gradation distribution of the captured image data;

An adjustment method determining step for determining a gradation adjustment method for the captured image data in accordance with the determined shooting conditions; A gradation conversion step of performing gradation conversion processing on the photographed image data using the determined gradation adjustment method.

[0089] In the form described in Item 83, in the image processing program described in Item 82, in the occupancy ratio calculating step, the captured image data is divided into regions that are combinations of predetermined brightness and hue. Then, for each of the divided areas, an occupancy ratio indicating a ratio of the entire captured image data is calculated.

[0090] The form described in Item 84 is the image processing program described in Item 83,

And an adjustment amount determining step for determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination step is performed on the captured image data using the gradation adjustment method determined by the adjustment method determination step. Apply.

[0091] In the form described in Item 85, in the image processing program described in Item 82, in the occupation rate calculation step, the captured image data is combined with the distance from the outer edge of the screen of the captured image data and the brightness. The area is divided into predetermined areas, and for each of the divided areas, an occupation ratio indicating a ratio of the entire captured image data is calculated.

[0092] The form described in Item 86 is the image processing program described in Item 85,

The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index, and the gradation adjustment method determined by the adjustment method determination step in the gradation conversion. Then, gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination step is performed on the captured image data.

[0093] In the form described in Item 87, in the image processing program described in Item 82, in the occupancy ratio calculation, the captured image data is divided into regions that are combinations of predetermined brightness and hue. For each of the divided areas, a first occupancy ratio indicating a ratio of the entire captured image data is calculated, and the captured image data is combined with the distance between the outer edge force of the captured image data and the lightness combination power. And a second occupancy ratio indicating the ratio of the entire captured image data is calculated for each of the divided areas. In the indicator calculation step, the second occupancy ratio is calculated by the occupancy ratio calculation step. First occupancy and second occupancy An index for specifying the shooting condition is calculated by multiplying the ratio and the deviation amount calculated in the bias amount calculation step by a coefficient set in advance according to the shooting condition.

[0094] The form described in Item 88 is the image processing program described in Item 87,

The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index, and the gradation adjustment method determined by the adjustment method determination step in the gradation conversion step Is used to perform gradation conversion processing of the gradation adjustment amount determined in the adjustment amount determination step.

[0095] In the form described in Item 89, in the image processing program described in Item 85 or 86, the cumulative number of pixels is calculated for each distance and brightness from the outer edge of the screen of the captured image data. A step of creating a histogram, wherein the occupation rate is calculated based on the two-dimensional histogram created in the occupation rate calculation step.

[0096] In the form described in Item 90, in the image processing program described in Item 87 or 88, the cumulative number of pixels is calculated for each distance and brightness from the outer edge of the screen of the captured image data. A step of creating a histogram, wherein the second occupancy ratio is calculated based on the two-dimensional histogram created in the occupancy ratio calculation step.

[0097] In the form described in Item 91, in the image processing program described in Item 83 or 84, a step of creating a two-dimensional histogram by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data And the occupancy ratio calculating step calculates the occupancy ratio based on the created two-dimensional histogram.

[0098] In the form described in Item 92, the image processing program described in Item 87 or 88 is a histogram for creating a two-dimensional histogram by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data. A first step of calculating the first occupancy rate based on the generated two-dimensional histogram.

[0099] In the image processing program according to any one of the forms described in Item 93, Item 82, 83, 84, 87, 88, 90 to 92, in the index calculation step, a predetermined high brightness value is obtained. A coefficient with a different sign is used for the flesh-color hue area and the hue area other than the high-brightness flesh-color hue area.

[0100] The configuration described in item 94, described in any one of the items of items 82, 83, 84, 87, 88, 90 to 93 In the image processing program, in the index calculation step, coefficients with different signs are used in the intermediate brightness area of the flesh color hue area and the brightness areas other than the intermediate brightness area.

[0101] In the form described in Item 95, in the image processing program described in Item 93, the brightness region of the hue region other than the high-brightness skin color hue region is a predetermined high brightness region.

[0102] In the form described in Item 96, in the image processing program described in Item 94, the brightness region other than the intermediate brightness region is a brightness region in the flesh-color hue region.

[0103] In the form described in Item 97, in the image processing program described in Item 93 or 95, the high-brightness skin color hue region includes a region in the range of 170 to 224 in terms of the brightness value of the HSV color system. It is.

[0104] In the form described in Item 98, in the image processing program described in Item 94 or 96, the intermediate brightness region includes a region in the range of 85 to 169 in terms of the brightness value of the HSV color system.

[0105] In the image processing program according to any one of Items 93, 95, and 97, the form described in Item 99 includes a blue hue region, a green hue in a hue region other than the high brightness skin color hue region. At least one of the regions is included.

[0106] In the form described in Item 100, in the image processing program according to any one of Items 94, 96, and 98, the brightness area other than the intermediate brightness area is a shadow area.

[0107] In the form described in Item 101, in the image processing program described in Item 99, the hue value of the blue hue region is in the range of 161 to 250 as the hue value of the HSV color system, and the green color The hue value in the hue region is in the range of 40 to 160 in the HSV color system.

[0108] In the form described in Item 102, in the image processing program described in Item 100, the brightness value of the shadow area is in the range of 26 to 84 as the brightness value of the HSV color system.

[0109] In the image processing program according to any one of Items 93 to 102, the form described in Item 103 and 104 is the hue value of the HSV color system in the image processing program according to any one of Items 93 to 102.

9 and within the range of 330-359.

[0110] In the image processing program according to any one of Items 93 to 104, the form described in Items 105 to 106 is based on a predetermined conditional expression based on lightness and saturation in the skin color hue region.

Divided into two areas. [0111] In the form described in Item 107, in the image processing program described in any one of Items 82 to 106, the deviation amount includes the brightness deviation amount of the photographed image data, and the photographed image data. At least one of the average brightness value at the center of the screen and the brightness difference value calculated under different conditions is included.

[0112] In the form described in Item 108, in the image processing program described in any one of Items 82 to 107, the photographing condition indicates a light source condition and an exposure condition at the time of photographing, and the light source condition includes A flash shooting state is included, and the exposure condition includes an under shooting state derived from the shutter speed and aperture value at the time of shooting.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, the configuration of the present embodiment will be described.

FIG. 1 is a perspective view showing an external configuration of the image processing apparatus 1 according to the embodiment of the present invention. As shown in FIG. 1, the image processing apparatus 1 is provided with a magazine loading section 3 for loading a photosensitive material on one side surface of a housing 2. Inside the housing 2 are provided an exposure processing unit 4 for exposing the photosensitive material and a print creating unit 5 for developing and drying the exposed photosensitive material to create a print. On the other side of the casing 2, a tray 6 for discharging the prints produced by the print creation unit 5 is provided.

In addition, a CRT (Cathode Ray Tube) 8 serving as a display device, a film scanner unit 9 that is a device for reading a transparent document, a reflective document input device 10, and an operation unit 11 are provided at the top of the housing 2. Yes. The CRT8 power print is composed of a display means that displays the image of the image information to be created on the screen. Further, the housing 2 is provided with an image reading unit 14 that can read image information recorded on various digital recording media, and an image writing unit 15 that can write (output) image signals to various digital recording media. ing. In addition, a control unit 7 that centrally controls each of these units is provided inside the housing 2.

[0117] The image reading unit 14 includes a PC card adapter 14a and a floppy (registered trademark) disk adapter 14b, and a PC card 13a and a floppy (registered trademark) disk 13b can be inserted therein. . For example, the PC card 13a has a memory in which a plurality of frame image data captured by a digital camera is recorded. For example, a plurality of frame image data captured by a digital camera is recorded on the floppy (registered trademark) disk 13b. PC card 1 Recording media on which frame image data is recorded in addition to 3a and floppy disk 13b include, for example, multimedia card (registered trademark), memory stick (registered trademark), MD data, CD-ROM, etc. .

[0118] The image writing unit 15 includes a floppy (registered trademark) disk adapter 15a, an MO adapter 15b, and an optical disk adapter 15c. The floppy disk (registered trademark) disks 16a and M016b and the optical disk 16c, respectively. Can be inserted. Examples of the optical disk 16c include CD-R, DVD-R, and the like.

In FIG. 1, the operation unit 11, the CRT 8, the film scanner unit 9, the reflective document input device 10, and the image reading unit 14 have a structure in which the casing 2 is integrally provided. Either one or more powers may be provided as separate bodies!

[0120] In the image processing apparatus 1 shown in FIG. 1, the force print creation method exemplified by the photosensitive material exposed to light and developed to create a print is not limited to this. A method such as a kuget method, an electrophotographic method, a heat sensitive method, or a sublimation method may be used. <Main components of image processing device 1>

FIG. 2 shows a main part configuration of the image processing apparatus 1. As shown in FIG. 2, the image processing apparatus 1 includes a control unit 7, an exposure processing unit 4, a print generation unit 5, a film scanner unit 9, a reflection original input device 10, an image reading unit 14, a communication means (input) 32, The image writing unit 15, data storage unit 71, template storage unit 72, operation unit 11, CRT 8, and communication unit (output) 33 are configured.

[0121] The control unit 7 is constituted by a microcomputer, and is stored in a storage unit (not shown) such as a ROM (Read Only Memory).

Unit) (operation not shown) controls the operation of each part of the image processing apparatus 1

The control unit 7 includes an image processing unit 70 according to the image processing apparatus of the present invention. Based on an input signal (command information) from the operation unit 11, the control unit 7 receives from the film scanner unit 9 and the reflection original input device 10. The scanned image signal, the image signal read from the image reading unit 14, and the image signal input from the external device via the communication means 32 are subjected to image processing to obtain an image for exposure. Image information is formed and output to the exposure processing unit 4. Further, the image processing unit 70 performs a conversion process corresponding to the output form on the image signal subjected to the image processing, and outputs it. As output destinations of the image processing unit 70, there are a CRT 8, an image writing unit 15, a communication means (output) 33, and the like.

The exposure processing unit 4 performs image exposure on the photosensitive material and outputs the photosensitive material to the print creating unit 5. The print creating unit 5 develops the exposed photosensitive material and dries it to create prints Pl, P2, and P3. Print P1 is a service size, high-definition size, panorama size, etc. print P2 is an A4 size print, and print P3 is a business card size print.

[0124] The film scanner unit 9 reads a frame image recorded on a transparent original such as a developed negative film N or a reversal film imaged by an analog camera, and obtains a digital image signal of the frame image. The reflective original input device 10 reads an image on the print P (photo print, document, various printed materials) by a flat bed scanner, and obtains a digital image signal.

The image reading unit 14 reads frame image information recorded on the PC card 13 a and the floppy (registered trademark) disk 13 b and transfers the frame image information to the control unit 7. The image reading unit 14 includes, as image transfer means 30, a PC card adapter 14a, a floppy (registered trademark) disk adapter 14b, and the like. The image reading unit 14 reads frame image information recorded on the PC card 13a inserted into the PC card adapter 14a or the floppy disk 13b inserted into the floppy disk adapter 14b. And transfer to the control unit 7. For example, a PC card reader or a PC card slot is used as the PC card adapter 14a.

[0126] The communication means (input) 32 receives an image signal representing a captured image and a print command signal from another computer in the facility where the image processing apparatus 1 is installed or a distant computer via the Internet or the like. To do.

The image writing unit 15 includes a floppy (registered trademark) disk adapter 15a, an MO adapter 15b, and an optical disk adapter 15c as the image transport unit 31. In accordance with a write signal input from the control unit 7, the image writing unit 15 is connected to the floppy disk 16a inserted in the floppy disk adapter 15a and the MO adapter 15b. The MO 16b inserted, the optical disc 16c inserted into the optical disc adapter 15c, and the image signal generated by the image processing method of the present invention are written.

[0128] The data storage means 71 stores and sequentially stores image information and order information corresponding to the image information (information on how many sheets of image power are to be created, information on the print size, etc.).

[0129] The template storage means 72 stores the background image, illustration image, etc., which are sample image data corresponding to the sample identification information Dl, D2, and D3, and data of at least one template that sets the synthesis region. RU A predetermined template is selected from a plurality of templates that are set by the operator's operation and stored in advance in the template storage means 72, and the frame image information is synthesized by the selected template and designated sample identification information Dl, D2, The sample image data selected based on D3 is combined with the image data based on the order and the Z or character data to create a print based on the specified sample. The synthesis using this template is performed by the well-known Chromaki method.

[0130] The sample identification information Dl, D2, and D3 for designating the print sample is configured to be input from the operation unit 211. However, these sample identification information is stored in the print sample or order. Since it is recorded on the sheet, it can be read by reading means such as OCR. Or it can also input by an operator's keyboard operation.

[0131] In this way, sample image data is recorded corresponding to sample identification information D1 that specifies a print sample, sample identification information D1 that specifies a print sample is input, and this sample identification information that is input Select sample image data based on D1, and combine the selected sample image data with the image data and Z or character data based on the order to create prints based on the specified samples. Users can actually order samples for printing and can meet the diverse requirements of a wide range of users.

[0132] Also, the first sample identification information D2 designating the first sample and the image data of the first sample are stored, and the second sample identification information D3 designating the second sample and the first sample identification data D3 are stored. The image data of two samples is stored, the sample image data selected based on the designated first and second sample identification information D2, D3, the image data based on the order, and the Z or character data To create a print based on the specified sample. Therefore, it is possible to synthesize a wider variety of images, and to create prints that meet a wide variety of user requirements.

[0133] The operation unit 11 includes information input means 12. The information input means 12 is composed of, for example, a touch panel and outputs a pressing signal from the information input means 12 to the control unit 7 as an input signal. The operation unit 11 may be configured with a keyboard, a mouse, and the like. The CRT 8 displays image information and the like according to the display control signal input from the control unit 7.

[0134] The communication means (output) 33 sends an image signal representing a photographed image after the image processing of the present invention and order information attached thereto to other links in the facility where the image processing apparatus 1 is installed. The computer transmits to a distant computer via the Internet or the like.

As shown in FIG. 2, the image processing apparatus 1 includes an image input unit that captures image information obtained by dividing and metering images of various digital media and image originals, an image processing unit, and a processed image. Image output means for displaying images, printing output, writing to image recording media, and order information attached to image data for remote computers via the communication line via another communication line or computer Means for transmitting. <Internal configuration of image processor 70>

FIG. 3 shows the internal configuration of the image processing unit 70. As shown in FIG. 3, the image processing unit 70 includes an image adjustment processing unit 701, a film scan data processing unit 702, a reflection original scan data processing unit 703, an image data format decoding processing unit 704, a template processing unit 705, and a CRT specific process. A unit 706, a printer specific processing unit A707, a printer specific processing unit B708, and an image data format creation processing unit 709 are configured.

[0136] The film scan data processing unit 702 performs a calibration operation specific to the film scanner unit 9, negative / positive reversal (in the case of a negative document), dust scratch removal, contrast adjustment, and the like on the image data input from the film scanner unit 9. It performs processing such as granular noise removal and sharpening enhancement, and outputs the processed image data to the image adjustment processing unit 701. In addition, the film size, negative / positive type, information on the main subject optically or magnetically recorded on the film, information on the shooting conditions (for example, information content described in APS), etc. are also output to the image adjustment processing unit 701. . [0137] The reflection document scan data processing unit 703 performs a calibration operation specific to the reflection document input device 10, negative / positive reversal (in the case of a negative document), dust flaw removal, and contrast adjustment for the image data input from the reflection document input device 10. Then, processing such as noise removal and sharpening enhancement is performed, and the processed image data is output to the image adjustment processing unit 701.

[0138] The image data format decoding processing unit 704 applies a compression code to the image data input from the image transfer means 30 and Z or the communication means (input) 32 according to the data format of the image data as necessary. Processing such as restoration and conversion of the color data expression method is performed, the data is converted into a data format suitable for computation in the image processing unit 70, and output to the image adjustment processing unit 701. In addition, when the size of the output image is specified from any of the operation unit 11, the communication means (input) 32, and the image transfer means 30, the image data format decoding processing unit 704 detects the specified information. And output to the image adjustment processing unit 701. Information about the size of the output image specified by the image transfer means 30 is embedded in the header information and tag information of the image data acquired by the image transfer means 30.

The image adjustment processing unit 701 is based on a command from the operation unit 11 or the control unit 7, and includes a film scanner unit 9, a reflective document input device 10, an image transfer unit 30, a communication unit (input) 32, and a template. The image data received from the image processing unit 705 is subjected to image processing described later (see FIGS. 6, 7, 13, and 17) for image formation optimized for viewing on the output medium. Digital image data is generated and output to the CRT specific processing unit 706, the printer specific processing unit A707, the printer specific processing unit B708, the image data format creation processing unit 709, and the data storage unit 71.

[0140] In the optimization process, for example, assuming that the image is displayed on a CRT display monitor compliant with the sRGB standard, the process is performed so that the optimum color reproduction is obtained within the color gamut of the sRGB standard. If output to silver salt photographic paper is assumed, processing is performed to obtain the optimum color reproduction within the color gamut of silver salt photographic paper. In addition to the compression of the color gamut, gradation processing from 16 bits to 8 bits, reduction of the number of output pixels, and processing for handling output characteristics (LUT) of the output device are also included. It goes without saying that noise compression, sharpening, gray balance adjustment, saturation adjustment, or gradation compression processing such as masking and baking are performed.

[0141] The image adjustment processing unit 701 determines the shooting condition of the shot image data as shown in FIG. A scene discriminating unit 710 that determines tone adjustment conditions (tone adjustment method, tone adjustment amount) and a tone conversion unit 711 that performs tone conversion processing according to the determined tone adjustment conditions. The

In the present embodiment, the photographing conditions are classified into light source conditions and exposure conditions.

[0143] The light source condition is derived from the light source at the time of photographing, the positional relationship between the main subject (mainly a person) and the photographer. In the broader sense, it also includes the type of light source (sunlight, strobe light, tandasten lighting and fluorescent lamps). Backlit scenes occur when the sun is located behind the main subject. A strobe (close-up) scene occurs when the main subject is strongly irradiated with strobe light. Both scenes have the same brightness (light / dark ratio), and the relationship between the brightness of the foreground and background of the main subject is merely reversed.

On the other hand, the exposure conditions are derived from the settings of the camera shutter speed, aperture value, etc., and underexposure is under, proper exposure is normal, and overexposure is over. In a broad sense, so-called “white jump” and “shadow collapse” are also included. Under all light source conditions, under or over exposure conditions can be used. Especially in DSC (digital still camera) with a narrow dynamic range, even if the automatic exposure adjustment function is used, due to the setting conditions aimed at suppressing overexposure, the frequency of underexposed exposure conditions is high. High,.

FIG. 4 (a) shows the internal configuration of the scene discrimination unit 710. As shown in FIG. 4A, the scene discriminating unit 710 includes a ratio calculating unit 712, a bias calculating unit 722, an index calculating unit 713, and an image processing condition calculating unit 714. As shown in FIG. 4B, the ratio calculation unit 712 includes a color system conversion unit 715, a histogram creation unit 716, and an occupation rate calculation unit 717.

The color system conversion unit 715 converts the RGB (Red, Green, Blue) value of the captured image data into the HSV color system. The HSV color system is a representation of image data with three elements: Hue, Saturation, and Value (Value or Brightness), and is based on the color system proposed by Munsell. Was devised.

In the claims and the present embodiment, “brightness” means “brightness” that is generally used unless otherwise noted. In the following description, the power to use V (0 to 255) of the HSV color system as “brightness” is the unit system that expresses the brightness of any other color system. It may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated. The photographed image data in the present embodiment is assumed to be image data having a person as a main subject.

[0148] The histogram creation unit 716 creates a two-dimensional histogram by dividing the captured image data into regions composed of a predetermined combination of hue and brightness, and calculating the cumulative number of pixels for each of the divided regions. In addition, the histogram creation unit 716 divides the captured image data into predetermined regions having a combination power of distance and brightness from the outer edge of the screen of the captured image data, and calculates the cumulative number of pixels for each of the divided regions. To create a two-dimensional histogram. Note that the captured image data is divided into regions that have the combined power of distance, brightness, and hue from the outer edge of the screen of the captured image data, and a three-dimensional histogram is created by calculating the cumulative number of pixels for each divided region. You may make it do. In the following, a method of creating a two-dimensional histogram will be adopted.

[0149] The occupancy calculation unit 717 indicates the ratio of the cumulative number of pixels calculated by the histogram creation unit 716 to the total number of pixels (the entire captured image data) for each region divided by the combination of brightness and hue. Calculate the first occupancy (see Table 1). The occupancy calculation unit 717 also calculates the total number of pixels calculated by the histogram creation unit 716 for each area divided by the combination of the distance from the outer edge of the screen of the captured image data and the brightness (the captured image data). Calculate the second occupancy ratio (see Table 4) indicating the ratio of the total occupancy.

[0150] The bias calculation unit 722 calculates a bias amount indicating the bias of the gradation distribution of the captured image data.

Here, the deviation amount is a standard deviation of luminance values of photographed image data, a luminance difference value, a skin color average luminance value at the center of the screen, an average luminance value at the center of the screen, and a skin color luminance distribution value. The processing for calculating these deviation amounts will be described in detail later with reference to FIG.

[0151] The index calculation unit 713 sets the first coefficient set in advance (for example, by discriminant analysis) to the first occupancy rate calculated for each area in the occupancy rate calculation unit 717 according to the imaging conditions. By multiplying (see Table 2) and taking the sum, index 1 for specifying the shooting conditions is calculated. Index 1 indicates the characteristics of flash photography such as indoor photography, close-up photography, and high brightness of the face color, and is used to separate the image that should be identified as a flash from other shooting conditions. [0152] When calculating the index 1, the index calculation unit 713 uses coefficients of different signs for a predetermined high-lightness skin color hue region and a hue region other than the high-lightness skin color hue region. Here, the skin color hue region of a predetermined high lightness includes a region of 170 to 224 in the lightness value of the HSV color system. Further, the hue area other than the predetermined high brightness skin color hue area includes at least one of the high brightness areas of the blue hue area (hue values 161 to 250) and the green hue area (hue values 40 to 160).

In addition, the index calculation unit 713 sets the first occupancy calculated for each area in the occupancy ratio calculation unit 717 to the second occupancy previously set according to the imaging conditions (for example, by discriminant analysis). By multiplying the coefficient (see Table 3) and taking the sum, index 2 for specifying the shooting conditions is calculated. Indicator 2 is a composite indication of characteristics during backlighting such as outdoor shooting, sky blue high brightness, and low facial color, and is used to separate the image that should be identified as backlight from other shooting conditions. .

[0154] When calculating the index 2, the index calculation unit 713 uses different codes for the intermediate brightness area of the flesh color hue area (hue values 0 to 39, 330 to 359) and the brightness areas other than the intermediate brightness area. The coefficient of is used. The intermediate brightness area of the flesh tone hue area includes areas with brightness values of 85 to 169. The brightness area other than the intermediate brightness area includes, for example, a shadow area (brightness value 26-84).

[0155] Furthermore, the index calculation unit 713 sets the second occupancy calculated for each area in the occupancy calculation unit 717 to the third occupancy set in advance (for example, by discriminant analysis) according to the imaging conditions. By multiplying the coefficient (see Table 5) and taking the sum, index 3 for specifying the shooting conditions is calculated. Indicator 3 shows the difference in contrast between the center and outside of the screen of the captured image data between backlight and strobe, and quantitatively shows only the image that should be identified as backlight or strobe. When calculating the index 3, the index calculation unit 713 uses different values of coefficients depending on the distance from the outer edge of the screen of the captured image data.

[0156] In addition, the index calculation unit 713 is set in advance (for example, by discriminant analysis) to the index index 3 and the average luminance value of the skin color region in the center of the screen of the captured image data according to the shooting conditions. Index 4 is calculated by multiplying the coefficient and taking the sum. In addition, the index calculation unit 713 calculates the average luminance value of the skin color area in the index 2, index 3, and center of the screen. Each index 5 is calculated by multiplying a coefficient set in advance (for example, by discriminant analysis) according to the photographing condition and taking the sum. In addition, the index calculation unit 713 multiplies the deviation amount calculated by the bias calculation unit 722 by a fourth coefficient (see Table 6) set in advance (for example, by discriminant analysis) according to the shooting conditions. Then, index 6 is calculated by taking the sum. A specific calculation method of the indices 1 to 6 in the index calculation unit 713 will be described in detail in the operation description of the present embodiment described later.

FIG. 4C shows the internal configuration of the image processing condition calculation unit 714. As shown in FIG. 4C, the image processing condition calculation unit 714 includes a scene determination unit 718, a gradation adjustment method determination unit 719, a gradation adjustment parameter calculation unit 720, and a gradation adjustment amount calculation unit 721. The

[0158] The scene discriminating unit 718 is divided into regions in advance according to the values of the index 4, the index 5 and the index 6 calculated by the index calculating unit 713 and the accuracy of the shooting conditions, and discriminates the reliability of the index Based on the map (see FIG. 19), the shooting condition of the shot image data is determined.

The gradation adjustment method determination unit 719 determines a gradation adjustment method for the captured image data according to the imaging conditions determined by the scene determination unit 718. For example, when the shooting condition is direct light or strobe light, as shown in Fig. 21 (a), the method of correcting the translation (offset) of the pixel value of the input captured image data (tone adjustment method A) is Applied. When the shooting condition is backlit or under, as shown in FIG. 21 (b), a method (tone adjustment method B) for gamma correction of the pixel value of the input captured image data is applied. If the shooting condition is between backlight and direct light (low accuracy region (1)), or between strobe and under light (low accuracy region (2)), as shown in Fig. 21 (c). Gamma correction and translation (offset) correction (tone adjustment method C) are applied to the pixel values of the input captured image data.

[0160] The tone adjustment parameter calculation unit 720 calculates parameters (key correction values, etc.) necessary for tone adjustment based on the values of the index 4, the index 5, and the index 6 calculated by the index calculation unit 713. Put out.

The gradation adjustment amount calculation unit 721 calculates (determines) the gradation adjustment amount for the captured image data based on the gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit 720. Specifically, the gradation adjustment amount calculation unit 721 is determined by the gradation adjustment method determination unit 719. A gradation conversion curve corresponding to the gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit 720 is selected from a plurality of gradation conversion curves set in advance corresponding to the determined gradation adjustment method. . Note that the tone conversion curve (tone adjustment amount) is calculated based on the tone adjustment parameter calculated by the tone adjustment parameter calculation unit 720.

[0162] The method for determining the shooting condition in the scene determination unit 718 and the method for calculating the gradation adjustment parameter in the gradation adjustment parameter calculation unit 720 will be described in detail in the operation description of the present embodiment described later.

In FIG. 3, the tone conversion unit 711 performs tone conversion of the captured image data according to the tone conversion curve determined by the tone adjustment amount calculation unit 721.

[0164] Based on a command from the image adjustment processing unit 701, the template processing unit 705 reads predetermined image data (template) from the template storage unit 72 and synthesizes the image data to be processed and the template. The template processing is performed, and the image data after the template processing is output to the image adjustment processing unit 701.

[0165] The CRT specific processing unit 706 performs processing such as changing the number of pixels and color matching on the image data input from the image adjustment processing unit 701 as necessary, and displays information such as control information that needs to be displayed. The combined display image data is output to CRT8.

[0166] The printer-specific processing unit A707 performs printer-specific calibration processing, color matching, pixel number change processing, and the like as necessary, and outputs processed image data to the exposure processing unit 4.

[0167] When an external printer 51 such as a large-format ink jet printer can be connected to the image processing apparatus 1 of the present invention, a printer specific processing unit B708 is provided for each printer apparatus to be connected. The printer-specific processing unit B708 performs printer-specific calibration processing, color matching, pixel number change, and the like, and outputs processed image data to the external printer 51.

[0168] The image data format creation processing unit 709 converts the image data input from the image adjustment processing unit 701 to various general-purpose image formats represented by JPEG, TIFF, Exif, and the like as necessary. The processed image data is output to the image transport unit 31 and the communication means (output) 33. Note that the film scan data processing unit 702, the reflection original scan data processing unit 703, the image data format decoding processing unit 704, the image adjustment processing unit 701, the CRT specific processing unit 706, and the printer specific processing unit shown in FIG. The categories A707, printer specific processing unit B708, and image data format creation processing unit 709 are provided to help understand the functions of the image processing unit 70, and are not necessarily realized as physically independent devices. For example, it may be realized as a type of software processing by a single CPU.

Next, the operation in the present embodiment will be described.

First, the flow of processing executed by the image adjustment processing unit 701 will be described with reference to the flowchart of FIG.

[0172] First, the size of the captured image data is reduced (step T1). As a method for reducing the size of the captured image data, a known method (for example, a bilinear method, a bicubic method, a two-arrest naver method, or the like) can be used. The reduction ratio is not particularly limited, but is preferably about 1Z2 to LZ10 of the original image from the viewpoint of processing speed and the accuracy of determining the photographing condition.

[0173] Next, DSC white balance adjustment correction processing is performed on the reduced captured image data (step T2), and the shooting conditions are specified based on the corrected captured image data. An index calculation process for calculating the indices (index 1 to 6) is performed (step Τ 3). The index calculation process of step IV3 will be described in detail later with reference to FIG.

[0174] Next, based on the index calculated in Step ステップ 3 and the discrimination map, the shooting conditions of the shot image data are determined, and the tone adjustment conditions (tone adjustment method, tone adjustment for the shot image data) are determined. Gradation adjustment condition determination processing for determining (amount) is performed (step Τ4). The gradation adjustment condition determination processing in step IV4 will be described in detail later with reference to FIG.

Next, gradation conversion processing is performed on the original photographed image data in accordance with the gradation adjustment condition determined in step Τ4 (step Τ5). Then, the sharpness adjustment processing is performed on the captured image data after the gradation conversion processing (step ステップ 6). In step Τ6, it is preferable to adjust the processing amount according to the shooting conditions and output print size.

[0176] In the next step, a process for removing the noise enhanced by the gradation adjustment and the enhancement of sharpness is performed (step Τ7). Next, the type of media that outputs the captured image data Color correction processing is performed to convert the color space according to the type (step T8), and the captured image data after image processing is output to the designated media.

Next, with reference to the flowchart in FIG. 6, the index calculation process (step Τ3 in FIG. 5) executed in the scene determination unit 710 will be described. In the following index calculation processing, “photographed image data” is image data reduced in step T1 in FIG.

[0178] First, in the ratio calculation unit 712, the captured image data is divided into predetermined image areas, and an occupation ratio indicating the ratio of each divided area to the entire captured image data (first occupation ratio, second occupation ratio). ) Is calculated (step S1). Details of the occupation rate calculation process will be described later with reference to FIGS.

[0179] Next, in the bias calculation unit 722, a bias amount calculation process for calculating a bias amount indicating a bias in the gradation distribution of the photographed image data is performed (step S2). The bias amount calculation processing in step S2 will be described in detail later with reference to FIG.

[0180] Next, an index for specifying the light source condition is calculated based on the occupation ratio calculated by the ratio calculation unit 712 and the coefficient set in advance according to the imaging condition (step S3). Also, an index for specifying the exposure condition is calculated based on the occupation ratio calculated in the ratio calculation unit 712 and a coefficient set in advance according to the shooting condition (step S4), and this index calculation process is performed. finish. The method for calculating the indices in steps S3 and S4 will be described in detail later.

Next, the first occupancy ratio calculation process executed in the ratio calculation unit 712 will be described in detail with reference to the flowchart of FIG.

[0182] First, the RGB values of the photographed image data are converted into the HSV color system (step S10). Figure 8 shows an example of a conversion program (HSV conversion program) that obtains hue values, saturation values, and brightness values by converting to the RGB power HSV color system, using program code (c language). In the HSV conversion program shown in Fig. 8, the digital image data values that are input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, The degree value is OutS, the lightness value is OutV, and the unit is defined as 0 to 255.

[0183] Next, the shot image data is divided into areas having a predetermined combination of brightness and hue. Then, a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step Sll). Hereinafter, the area division of the captured image data will be described in detail.

[0184] Lightness (V) i, Lightness value power ^) to 25 (vl), 26-50 (ν2), 51 to 84 (ν3), 85 to 169 (ν4), 170 to 199 (v5), 200 It is divided into seven areas of ~ 224 (v6) and 225 ~ 255 (v7). Hue (H) is a flesh color range (HI and H2) with a hue value of 0 to 39, 330 to 359, a green hue range (H3) with a hue value of 0 to 160, and a blue hue range with a hue value of 161 to 250. It is divided into four areas: (H4) and red hue area (H5). Note that the red hue area (H5) is not used in the following calculations because of the fact that it contributes little to the determination of the shooting conditions. The flesh-color hue area is further divided into a flesh-color area (HI) and other areas (H2). Hereinafter, out of the flesh color hue area (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following formula (1) is defined as the flesh color area (HI), and the formula (1) is not satisfied! /, Let the region be (H2).

[0185] 10 <Saturation (S) <175,

Hue '(H) = Hue (H) + 60 (when 0≤Hue (H) <300),

Hue, (H) = Hue (H) — 300 (when 300 ≤ Hue (H) 360 360),

Luminance (Y) = InRX0.30 + InGXO.59 + InBXO. 11 (A), Hue '(H) Z Luminance (Y) <3.0Χ (Saturation (S) Z255) +0.7 (1)

Therefore, the number of divided areas of the captured image data is 4X7 = 28. It is also possible to use V and brightness (V) in equation (1).

[0186] When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S12). The occupation rate calculation process ends. Assuming that Rij is the first occupancy calculated in the divided area, which is the combined power of the lightness area vi and the hue area Hj, the first occupancy ratio in each divided area is expressed as shown in Table 1.

[0187] [Table 1] [First occupancy]

[0188] Next, a method of calculating the index 1 and the index 2 will be described.

[0189] Table 2 shows, for each divided area, the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting, that is, the brightness state of the face area at the time of strobe shooting. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the photographing conditions.

[0190] [Table 2]

[First factor

[0191] Figure 9 shows the brightness (v) —hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the area (rl) distributed in the high brightness skin color hue area in Fig. 9, and the other hue is blue. Hue region (r2) force A negative (one) coefficient is used for the first occupancy calculated. Figure 11 shows a curve (coefficient curve) where the first coefficient in the flesh-color area (HI) and the first coefficient in the other areas (green hue area (H3)) change continuously over the entire brightness. ). According to Table 2 and Fig. 11, in the high lightness (V = 170 to 224) region, the sign of the first coefficient in the skin color region (HI) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (one).

[0192] When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Equation (2). [0193] [Equation 1]

Hk region sum = t ^{Rikx Cik} (2)

[0194] Therefore, the sum of the HI 'regions is expressed by the following equations (2-1) to (2-4).

HI area sum: R11X (-44.0) + R21X (-16.0) + (omitted)

... + R71X (-11.3) (2-1)

Sum of H2 region = R12X0.0 + R22X8.6 + (omitted)

... + R72X (-11.1) (2-2)

Sum of H3 regions = R13XO.0 + R23X (-6.3) + (omitted)

... + R73X (-10.0) (2-3)

Sum of H4 region = R14X0.0 + R24X (-1.8) + (omitted)

... + R74X (-14.6) (2-4)

Index 1 is defined as equation (3) using the sum of the H1 to H4 regions shown in equations (2-1) to (2-4).

[0195] Indicator 1 = sum of H1 regions + sum of H2 regions + sum of H3 regions + sum of H4 regions +4.424 (3)

Table 3 shows the second coefficient required for each divided area to calculate the index 2 that quantitatively shows the accuracy of backlighting, that is, the brightness state of the face area during backlighting. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy ratio Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

[0196] [Table 3]

[Second factor]

FIG. 10 shows the brightness (V) —hue (H) plane. According to Table 3, a negative (one) coefficient is used for the occupancy calculated from the area (r4) distributed in the intermediate lightness of the flesh-colored hue area in Fig. 10. A positive (+) coefficient is used for the occupation ratio calculated from the low brightness (shadow) area (r3) of the flesh tone hue area. Fig. 12 shows the second coefficient in the flesh color region (HI) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and Fig. 12, the sign of the second coefficient of the intermediate lightness area of the flesh tone hue area with a lightness value of 85 to 169 (v4) is negative (-), and the lightness value is 26 to 84 (v2, The sign of the second coefficient in the low lightness (shadow) region of v3) is positive (+), and it is obvious that the sign of the coefficient in both regions is different.

[0198] If the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in equation (4).

[0199] [Equation 2]

Hk region sum = £ Riit X Dik (4)

[0200] Therefore, the sum of the H1 to H4 regions is expressed by the following equations (41) to (44).

HI area sum = R11X (--27.0) + R21X4.5 + (omitted)

... + R71X (-24.0) (4-1)

Sum of H2 region = R12X0.0 + R22X4.7 + (omitted)

... + R72X (-8.5) (4-2)

Sum of H3 regions = R13XO.0 + R23XO.0 + (omitted)

... + R73XO.0 (4-3)

Sum of H4 region = R14X0.0 + R24X (-5.1) + (omitted)

... + R74X7.2 (4-4)

Index 2 is defined as equation (5) using the sum of the H1 to H4 regions shown in equations (41) to (44).

[0201] Indicator 2 = H1 region sum + H2 region sum + H3 region sum + H4 region sum +1.554 (5)

Since the index 1 and the index 2 are calculated based on the brightness and hue distribution amount of the captured image data, the index 1 and the index 2 are effective in determining the imaging condition when the captured image data is a color image.

Next, the second occupancy ratio calculation process executed in the ratio calculation unit 712 to calculate the index 3 will be described in detail with reference to the flowchart of FIG. [0203] First, the RGB values of the photographed image data are converted into the HSV color system (step S20). Next, the captured image data is divided into areas where the combined power of the distance from the outer edge of the captured image screen and the brightness is determined, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided area. (Step S21). Hereinafter, the area division of the captured image data will be described in detail.

[0204] Figs. 14 (a) to 14 (d) show four regions nl to n4 divided according to the distance from the outer edge of the screen of the captured image data. The area nl shown in FIG. 14 (a) is the outer frame, the area n2 shown in FIG. 14 (b) is the area inside the outer frame, and the area n3 shown in FIG. 14 (c) is the area n2. Further, an inner area, an area n4 shown in FIG. 14 (d) is an area at the center of the captured image screen. In addition, the lightness is divided into seven regions from vl to v7 as described above. Therefore, when the captured image data is divided into areas that are a combination of the distance from the outer edge of the captured image screen and the brightness, the number of divided areas is 4 × 7 = 28.

[0205] When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S22). The occupation rate calculation process ends. Assuming that Qij is the second occupancy calculated in the divided area, which is the combination of the lightness area vi and the screen area nj, the second occupancy ratio in each divided area is shown in Table 4.

[0206] [Table 4]

[Second occupancy

Next, a method for calculating the index 3 will be described.

[0208] Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the photographing conditions.

[0209] [Table 5]

[0210] FIG. 15 shows the third coefficient in the screen areas nl to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

[0211] If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in equation (6).

[0212] [Equation 3] (6)

[0213] Therefore, the sum of the nl to n4 regions is expressed by the following equations (6-1) to (6-4).

Sum of nl region: Q11X40. 1 + Q21X37.0 + (omitted)

... + Q71X22.0 (6-1)

Sum of n2 regions = Q12X (-14.8) + Q22X (— 10.5) + (omitted)

... + Q72X0.0 (6-2)

n3 area B = Q13X24.6 + Q23X12.1 + (omitted)

... + Q73X10. 1 (6-3)

n4 area sum = Q14X1.5 + Q24X (-32.9) + (omitted)

... + Q74X (1 52.2) (6-4)

Index 3 is defined as equation (7) using the sum of the N1 to H4 regions shown in equations (6-1) to (6-4).

[0214] Indicator 3 = sum of nl regions + sum of n2 regions + sum of n3 regions + sum of n4 regions 12.6201

(7)

Index 3 is calculated based on the compositional characteristics (distance of the outer edge force of the screen of the captured image data) based on the brightness distribution position of the captured image data. Therefore, the shooting conditions for the monochrome image as well as the color image are determined. It is also effective for discrimination. Next, with reference to the flowchart in FIG. 16, the bias amount calculation process (step S 2 in FIG. 6) executed in the bias calculation unit 722 will be described.

[0216] First, the luminance Y (brightness) of each pixel is calculated from the RGB (Red, Green, Blue) values of the captured image data using Equation (A), and the standard deviation (xl) of the luminance is calculated. (Step S23). The standard deviation (xl) of luminance is expressed as shown in Equation (8).

[0217] [Numeric 4] ( ₈ )

( ⁸ )

[0218] In equation (8), the pixel luminance value is the luminance of each pixel of the captured image data, and the average luminance value is the average value of the luminance of the captured image data. The total number of pixels is the number of pixels of the entire captured image data.

Next, as shown in Expression (9), a luminance difference value (x2) is calculated (step S24).

[0220] Luminance difference value (x2) = (maximum luminance value, average luminance value) Z255 (9)

In equation (9), the maximum luminance value is the maximum luminance value of the captured image data.

[0221] Next, the average luminance value (x3) of the flesh color region in the center of the screen of the photographed image data is calculated (step S25), and further, the average luminance value (x4) in the center of the screen is calculated (step S25). S26). Here, the center of the screen is, for example, an area composed of an area n3 and an area n4 in FIG.

[0222] Next, the flesh color luminance distribution value (x5) is calculated (step S27), and this deviation amount calculation processing ends. When the maximum brightness value of the skin color area of the captured image data is Yskin—max, the minimum brightness value of the skin color area is Yskin—min, and the average brightness value of the skin color area is Yskin—ave, the skin color brightness distribution value (x5) is It is expressed as 10).

[0223] x5 = (Yskin― max― Yskin― min) / 2 ― Yskin― ave (10)

Let x6 be the average luminance value of the skin color area in the center of the screen of the captured image data. Here, the central portion of the screen is, for example, a region composed of region n2, region n3, and region n4 in FIG. At this time, index 4 is defined as in equation (11) using index indexes 3 and x6, and index 5 is defined as in equation (12) using index 2, index 3, and x6.

[0224] Indicator 4 = 0. 46 X indicator 1 + 0. 61 X indicator 3 + 0. 01 χ χ6— 0. 79 (11) Index 5 = 0. 58X Index 2 + 0.18X Index 3 + (— 0.03) Χχ6 + 3. 34 (12) Here, the weighting coefficient multiplied by each index in Equation (11) and Equation (12) is It is set in advance according to the shooting conditions.

[0225] The index 6 is obtained by multiplying the deviation amounts (xl) to (x5) calculated in the deviation amount calculation processing by a fourth coefficient set in advance according to the imaging conditions. Table 6 shows the fourth coefficient, which is a weighting coefficient by which each deviation is multiplied.

[0226] [Table 6]

4th coefficient

[0227] The index 6 is expressed as in Expression (13).

[0228] Indicator 6 = xlX0.02 + x2Xl. 13 + x3XO.06 + x4X (-0.01) + x5XO.03— 6.49 (13)

This index 6 has luminance histogram distribution information that is based only on the compositional characteristics of the captured image data screen, and is particularly effective in distinguishing between a flash photography scene and an under photography scene.

Next, the gradation adjustment condition determination process (step T4 in FIG. 5) executed in the image processing condition calculation unit 714 will be described with reference to the flowchart in FIG.

[0230] First, the shooting condition of the shot image data is determined based on the index (indexes 4 to 6) calculated by the index calculation unit 713 and the determination map divided in advance according to the shooting conditions. (Step S30). Hereinafter, a method for determining the photographing condition will be described.

[0231] Fig. 18 (a) shows 60 images taken under each of the following conditions: forward light, backlight, and strobe, and index 4 and index 5 were calculated for a total of 180 digital image data. The values of index 4 and index 5 are plotted. Figure 18 (b) shows the results of plotting the values of index 4 and index 6 for images where 60 images were taken under each strobe and under shooting conditions, and index 4 was greater than 0.5.

[0232] The discriminant map is used to evaluate the reliability of the index. As shown in FIG. It consists of strobe and under basic areas, a low accuracy area (1) between backlight and direct light, and a low accuracy area (2) between strobe and under. Note that there are other low-accuracy regions such as a low-accuracy region between backlight and strobe on the discrimination map, but they are omitted in this embodiment.

[0233] Table 7 shows a plot of each index value shown in Fig. 18 and the shooting conditions discriminated based on the discrimination map of Fig. 19.

[0234] [Table 7]

As described above, the light source condition can be quantitatively determined based on the values of the index 4 and the index 5, and the exposure condition can be quantitatively determined based on the values of the index 4 and the index 6. In addition, the low accuracy region (1) between the forward light and the backlight can be distinguished from the values of the indicators 4 and 5, and the low accuracy region (2) between the strobe and the under can be determined from the values of the indicators 4 and 6. Can be determined.

[0236] When the shooting conditions are determined, a gradation adjustment method for the shot image data is determined in accordance with the determined shooting conditions (step S31). As shown in Fig. 20, when the shooting condition is normal light or strobe light, the gradation adjustment method A (Fig. 21 (a)) is selected, and when the shooting condition is backlight or under, the gradation adjustment method B (Fig. 21 ( b)) is selected. Also, if the shooting condition is between backlight and direct light or between strobe and under (that is, a low-accuracy region on the discrimination map), tone adjustment method C (Fig. 21 (c)) is selected. The

[0237] As described above, when the shooting condition is direct light, since the correction amount is relatively small, it is possible to apply the gradation adjustment method A in which the pixel value of the shot image data is corrected in parallel (offset). A viewpoint power capable of suppressing gamma fluctuation is also preferable. Also, when the shooting conditions are backlit or under, the amount of correction is relatively large, so applying gradation adjustment method A significantly increases the gradation where there is no image data, resulting in black turbidity or whiteness. Will lead to a decrease in the brightness of The Therefore, when the shooting condition is backlight or under, it is preferable to apply the gradation adjustment method B in which the pixel value of the shot image data is gamma corrected. Also, in the case of shooting conditions in the low accuracy area on the discrimination map, the gradation adjustment method for one of the adjacent shooting conditions is A or B in any low accuracy area, so both gradation adjustment methods are mixed. It is preferable to apply the gradation adjustment method C described above. By setting the low-accuracy region in this way, the processing result can be smoothly transferred even when different gradation adjustment methods are used. In addition, it is possible to reduce variations in density between multiple photo prints taken of the same subject. Note that the tone conversion curve shown in FIG. 21 (b) is convex upward, but may be convex downward. In addition, the tone conversion curve shown in FIG. 21 (c) is convex downward, but may be convex upward.

[0238] When the gradation adjustment method is determined, parameters necessary for gradation adjustment are calculated based on the index calculated by the index calculation unit 713 (step S32). Hereinafter, the calculation method of the gradation adjustment parameter calculated in step S32 will be described. In the following, it is assumed that the 8-bit captured image data has been converted to 16-bit in advance, and the unit of the captured image data value is 16-bit.

[0239] The following parameters P1 to P10 are calculated as parameters necessary for gradation adjustment (gradation adjustment parameters).

[0240] P1: Average brightness of the entire shooting screen

P2: Block division average brightness

P3: Average brightness of skin tone area (HI)

P4: Brightness correction value 1 = P1 -P2

P5: Reproduction target correction value = Brightness reproduction target value (30360) -P4

P6: Offset value 1 = P5— P1

P7: Key correction value

P7 ': Key correction value 2

P8: Brightness correction value 2

P9: Offset value 2 = P5— P8— P1

P 10: Offset value 3 Here, the calculation method of the meter P2 will be described with reference to FIG. 22 and FIG.

[0241] First, a CDF (cumulative density function) is created in order to normalize captured image data. Next, the maximum and minimum values of the CDF force obtained are determined. The maximum and minimum values are obtained for each RGB. Here, the maximum and minimum values for each of RGB obtained, respectively, Rmax, R mmゝGmax, GmmゝBmax, Bmm and to ₀

[0242] Next, normalized image data for any pixel (Rx, Gx, Bx) of the captured image data is calculated. Rx normalized data in R plane is R, Gx in G plane

point

If the normalized data of G is G and the normalized data of Bx in the B plane is B, then

point point

The normalized data R 1, G 2, and B are respectively expressed as Expression (14) to Expression (16).

point point point

R = {(Rx-Rmin) / (Rmax-Rmin)} X 65535 (14)

point

G = {(Gx-Gmin) / (Gmax-Gmin)} X 65535 (15)

point

B = {(Bx-Bmin) / (Bmax-Bmin)} X 65535 (16)

point

Next, the luminance N of the pixel (Rx, Gx, Bx) is calculated by Expression (17).

point

[0243] N = (B + G + R) / 3 (17)

point point point point

Figure 22 (a) shows the frequency distribution (histogram) of the brightness of RGB pixels before normalization. In FIG. 22 (a), the horizontal axis represents luminance, and the vertical axis represents pixel frequency. This histogram is created for each RGB. When the luminance histogram is created, regularity is applied to the captured image data for each plane according to equations (14) to (16). Fig. 22 (b) shows a histogram of brightness calculated by equation (17). Since the captured image data is normally entered at 65535, each pixel takes an arbitrary value between the maximum value of 65535 and the minimum value power.

When the luminance histogram shown in FIG. 22 (b) is divided into blocks divided by a predetermined range, a frequency distribution as shown in FIG. 22 (c) is obtained. In Fig. 22 (c), the horizontal axis is the block number (luminance) and the vertical axis is the frequency.

[0245] Next, a process of deleting the no, illite, and shadow areas from the luminance histogram shown in Fig. 22 (c) is performed. This is because the average brightness is extremely high in white walls and snow scenes, and the average brightness is very low in dark scenes, so highlights and shadow areas can adversely affect average brightness control. by. Therefore, by limiting the highlight and shadow areas of the luminance histogram shown in Fig. 22 (c), the influence of both areas is reduced. Let If the high-brightness region (highlight region) and the low-brightness region (shadow region) are deleted from the luminance histogram shown in Fig. 23 (a) (or Fig. 22 (c)), the result is as shown in Fig. 23 (b).

[0246] Next, as shown in Fig. 23 (c), an area having a frequency greater than a predetermined threshold is deleted from the luminance histogram. This is because if there is a part with an extremely high frequency, the data in this part has a strong influence on the average brightness of the entire photographed image, so that erroneous correction is likely to occur. Therefore, as shown in Fig. 23 (c), the number of pixels above the threshold is limited in the luminance histogram. Figure 23 (d) shows the luminance histogram after the pixel number limiting process.

[0247] Each block number of the luminance histogram (Fig. 23 (d)) obtained by deleting the high luminance region and the low luminance region from the normalized luminance histogram and further limiting the cumulative number of pixels, The parameter P2 is the average luminance value calculated based on each frequency.

[0248] The norm P1 is an average value of the luminance of the entire captured image data, and the parameter P3 is an average value of the luminance of the skin color region (HI) in the captured image data. The key correction value for parameter P7, the key correction value 2 for parameter P7 ', and the luminance correction value 2 for parameter P8 are defined as shown in equations (18), (19), and (20), respectively.

[0249] P7 (Key correction value) = [P3— ((Indicator 5Z6) X 18000) + 22000)] Z24. 78 (18)

P7 '(key correction value 2) = [P3— ((index 6Z6) X 10000 + 30000)] Z24. 78 (19)

P8 (Luminance correction value 2) = (Indicator 4Z6) X 17500 (20)

The offset value 3 of the meter P10 is a gradation adjustment parameter in the case of shooting conditions corresponding to the low accuracy region (1) or (2) on the discrimination map. The calculation method of the meter P10 will be described below.

[0250] First, among the indexes in the corresponding low-accuracy region, a reference index is determined. For example, in the low accuracy region (1), the index 5 is determined as the reference index, and in the low accuracy region (2), the index 6 is determined as the reference index. Then, by normalizing the value of the reference index in the range of 0 to 1, the reference index is converted into a normalized index. Normalized indicator Is defined as in equation (21).

[0251] Normalized index = (Standard index Minimum index value) Z (Maximum index value Minimum index value) (21) In equation (21), the maximum index value and minimum index value are within the corresponding low accuracy range. The maximum and minimum values of the reference index.

[0252] The correction amounts at the boundary between the corresponding low accuracy region and the two regions adjacent to the low accuracy region are ex and β, respectively. The correction amounts α and j8 are fixed values calculated in advance using the reproduction target value defined at the boundary of each region on the discrimination map. The nomometer P10 is expressed as in equation (22) using the normalized index of equation (21) and the correction amounts a and β.

[0253] PIO = (j8— α) X normalization index + α (22)

In the present embodiment, the correlation between the normal index and the correction amount is a linear relationship, but it may be a curve relationship in which the correction amount shifts more gradually.

[0254] When the tone adjustment parameter is calculated, the tone adjustment amount for the captured image data is determined based on the calculated tone adjustment parameter (step S33), and the tone adjustment condition determination process is performed. finish. In step S33, specifically, from the plurality of gradation conversion curves set in advance corresponding to the gradation adjustment method determined in step S31, it corresponds to the gradation adjustment parameter calculated in step S32. The gradation conversion curve to be selected is selected (determined). Note that the gradation conversion curve (gradation adjustment amount) may be calculated based on the gradation adjustment parameter calculated in step S32. When the gradation conversion curve is determined, the photographed image data is gradation converted according to the determined gradation conversion curve.

[0255] A method for determining a gradation conversion curve for each shooting condition will be described below.

When the shooting condition is direct light, offset correction (parallel shift of 8-bit value) that matches parameter P1 with P5 is performed by the following equation (23).

[0256] RGB value of output image = RGB value of input image + P6 (23)

Therefore, when the shooting condition is normal light, a gradation conversion curve corresponding to Equation (23) is selected from the plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (23). <In the case of backlight>

When the photographing condition is backlight, a gradation conversion curve corresponding to the parameter P7 (key correction value) shown in Expression (18) is selected from the plurality of gradation conversion curves shown in FIG. 21 (b). A specific example of the gradation conversion curve in Fig. 21 (b) is shown in Fig. 24. The correspondence between the value of parameter P7 and the selected gradation conversion curve is shown below.

[0257] -50 <P7 <+ 50 → L3

+ 50≤P7 <+ 150 → L4

+ 150≤P7 + + 250 → L5

— 150 <P7≤— If 50 → L2

-250 <P7≤-150 → L1

When the photographing condition is backlight, it is preferable to perform the dodging process together with the gradation conversion process. In this case, it is desirable to adjust the degree of the dodging process according to the index 5 indicating the backlight intensity.

When the shooting condition is under, as in the case of backlighting, it corresponds to the parameter P7 ′ (key correction value 2) shown in Equation (19) from among the multiple tone transformation curves shown in FIG. 21 (b). A tone conversion curve is selected. The correspondence between the value of parameter P7 'and the gradation conversion curve selected in Fig. 24 is shown below.

[0258] 50 <? 7, <+50 → 3

+ 50≤P7 '<+ 150 → L4

+ 150≤P7 '<+ 250 → L5

— If 150 <P7, ≤— 50 → L2

— 250 P P7, ≤— 150 → L1

Note that when the shooting condition is under, dodging is not performed as shown in the case of backlight.

When the shooting condition is a strobe, offset correction (parallel shift of 8-bit value) is performed using equation (24). [0259] RGB value of output image = RGB value of input image + P9 (24)

Therefore, when the shooting condition is a strobe, a gradation conversion curve corresponding to Expression (24) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, calculate (determine) the gradation conversion curve based on Equation (24).

For low-accuracy area shooting conditions, offset correction (parallel shift of 8-bit value)

).

[0260] RGB value of output image = RGB value of input image + P10 (25)

Therefore, in the case of the low accuracy region, the gradation conversion curve corresponding to the equation (25) is selected from the plurality of gradation conversion curves shown in FIG. Alternatively, calculate (determine) the gradation conversion curve based on Equation (25).

[0261] In the present embodiment, when the gradation conversion process is actually performed on the captured image data, the above-described gradation adjustment conditions are changed from 16 bits to 8 bits.

[0262] As described above, according to the image processing apparatus 1 of the present embodiment, an index that quantitatively indicates the shooting condition of the captured image data is calculated, and the index is calculated according to the calculated index and the accuracy of the shooting condition. Based on the discrimination map divided in advance, the shooting conditions are determined, the gradation adjustment method for the captured image data is determined according to the determination result, and the gradation adjustment amount (gradation conversion) of the captured image data is determined. By determining (curve), it is possible to appropriately correct the brightness of the subject. In particular, by determining the shooting condition based on the determination map, the reliability of the determination result is increased, and appropriate gradation conversion according to the shooting condition can be performed.

[0263] In addition to index 1 that quantitatively indicates the accuracy for flash photography and index 2 that quantitatively indicates the accuracy for backlighting, index 3 is also used to derive the compositional elemental power of the captured image data. By using this to determine the shooting conditions, the determination accuracy of the shooting conditions can be improved. In addition, by using the compositional element of the captured image data and the index 6 that also calculates the distribution information power of the histogram, it is possible to distinguish between a strobe shooting scene and an under shooting scene, further improving the shooting condition discrimination accuracy. be able to.

The image processing method shown in the above embodiment is also applied to an imaging apparatus such as a digital camera. Is possible. FIG. 25 shows a configuration of a digital camera 200 to which the imaging apparatus of the present invention is applied. Digital camera 200, Fig. 25 [As shown], CPU201, optical system 202, image sensor 咅 203, AF calculation 咅 204, WB calculation 咅 205, AE calculation 咅 206, lens control 咅 207, image processing unit 208 , Display unit 209, recording data creation unit 210, recording medium 211, scene mode setting key 212, color space setting key 213, release button 214, and other operation keys 215 c

The CPU 201 comprehensively controls the operation of the digital camera 200. The optical system 202 is a zoom lens, and forms a subject image on a charge-coupled device (CCD) image sensor in the imaging sensor unit 203. The imaging sensor unit 203 photoelectrically converts an optical image by a CCD image sensor, converts it into a digital signal (AZD conversion), and outputs it. The image data output from the imaging sensor unit 203 is input to the AF calculation unit 204, the WB calculation unit 205, the AE calculation unit 206, and the image processing unit 208.

[0265] The AF calculation unit 204 calculates and outputs the distances of the AF areas provided at nine places in the screen. The determination of the distance is performed by determining the contrast of the image, and the CPU 201 selects a value at the closest distance among them and sets it as the subject distance. The WB calculation unit 205 calculates and outputs a white balance evaluation value of the image. The white balance evaluation value is a gain value required to match the RGB output value of a neutral subject under the light source at the time of shooting, and is calculated as the ratio of RZG and BZG with reference to the G channel. The calculated evaluation value is input to the image processing unit 208, and the white balance of the image is adjusted. The AE calculation unit 206 calculates and outputs an appropriate exposure value for the image data, and the CPU 201 calculates an aperture value and a shutter speed value so that the calculated appropriate exposure value matches the current exposure value. The aperture value is output to the lens control unit 207, and the corresponding aperture diameter is set. The shutter speed value is output to the image sensor unit 203, and the corresponding CCD integration time is set.

[0266] The image processing unit 208 performs processing such as white balance processing, CCD filter array interpolation processing, color conversion, primary gradation conversion, and sharpness correction on the captured image data, and then performs the above-described implementation. Similar to the form, an index (index 1 to 6) for specifying the shooting condition is calculated, the shooting condition is determined based on the calculated index, and the gradation conversion process determined based on the determination result is performed. By doing so, it is converted into a preferable image. Its PEG compression And so on. The JPEG-compressed image data is output to the display unit 209 and the recording data creation unit 210.

[0267] The display unit 209 displays the captured image data on the liquid crystal display and various information according to instructions from the CPU 201. The recording data creation unit 210 formats JPEG-compressed image data and various captured image data input from the CPU 201 into an Exif (Exchangeable Image File Format) file, and records the data on the recording medium 211. In the recording media 211, there is a part called maker note as a space where each manufacturer can write free information, and it is also possible to record the judgment result of shooting conditions and index 4, index 5 and index 6. Oh ,.

[0268] In the digital camera 200, the shooting scene mode can be switched by a user setting. That is, three modes can be selected as a shooting scene mode: a normal mode, a portrait mode, and a landscape mode scene. When the user operates the scene mode setting key 212 and the subject is a person, the portrait mode and the landscape mode are selected. In case of, switch to landscape mode to perform primary gradation conversion suitable for the subject. In addition, the digital camera 200 records the selected shooting scene mode information by adding it to the maker note portion of the image data file. The digital camera 200 also records the position information of the AF area selected as the subject in the image file in the same manner.

[0269] Note that in the digital camera 200, the user can set the output color space using the color space setting key 213. As the output color space, sRGB (IEC61966-2-i; ^ RAW can be selected. When sRGB is selected, the image processing in this embodiment is executed. When Raw is selected, The image processing of this embodiment is not performed, and the image is output in a color space unique to CCD.

[0270] As described above, according to the digital camera 200 to which the imaging apparatus of the present invention is applied, as in the above-described image processing apparatus 1, an index that quantitatively indicates the shooting condition of the shot image data is calculated, and The shooting conditions are determined based on the calculated index, the gradation adjustment method for the captured image data is determined according to the determination result, and the gradation adjustment amount (gradation conversion curve) of the captured image data is determined. Accordingly, it is possible to appropriately correct the brightness of the subject. In this way, appropriate gradation conversion processing according to the shooting conditions is performed inside the digital camera 200. Thus, even when the digital camera 200 and the printer are directly connected without using a personal computer, a preferable image can be output.

[0271] Note that the description in this embodiment can be changed as appropriate without departing from the spirit of the present invention.

[0272] For example, a face image may be detected from the photographed image data, the photographing condition may be determined based on the detected face image, and the gradation adjustment condition may be determined. In addition, Exif information may be used to determine the shooting conditions. By using Exif information, it is possible to further improve the accuracy of determining the shooting conditions.

Industrial applicability

[0273] According to the present invention, an index that quantitatively indicates the shooting condition of the captured image data is calculated, the shooting condition is determined based on the calculated index, and the captured image data is determined according to the determination result. By determining the gradation adjustment method, it is possible to appropriately correct the brightness of the subject. In particular, by determining the shooting conditions based on a discrimination map that has been divided into regions according to the accuracy of the shooting conditions, the reliability of the determination results can be increased, and appropriate gradation conversion according to the shooting conditions can be performed. It becomes possible.

[0274] Further, by calculating an index that quantitatively indicates the shooting condition of the captured image data, and determining the gradation adjustment amount of the captured image data based on the calculated index, the brightness of the subject is appropriately corrected. It becomes possible to do.

Claims

The scope of the claims

[1] The photographed image data is divided into at least a region having a combination of a predetermined brightness and hue and a combination of a distance from the outer edge of the screen and the brightness of the photographed image data. An occupancy ratio calculating step for calculating an occupancy ratio indicating the ratio of the entire captured image data;

A gradation conversion step of performing gradation conversion processing on the photographed image data using the determined gradation adjustment method.

[2] In the occupancy rate calculating step, the captured image data is divided into regions each having a predetermined combination of brightness and hue, and an occupancy rate indicating a ratio of the entire captured image data for each of the divided regions. 2. The image processing method according to claim 1, wherein the value is calculated.

[3] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, the gradation adjustment amount determined in the adjustment amount determination step is applied to the photographed image data using the gradation adjustment method determined in the adjustment method determination step. 3. The image processing method according to claim 2, wherein gradation conversion processing is performed.

[4] In the occupancy rate calculating step, the captured image data is divided into predetermined areas that are combinations of distance and brightness from the outer edge of the screen of the captured image data, and each divided area is divided. The image processing method according to claim 1, wherein an occupancy ratio indicating a ratio of the entire captured image data is calculated.

[5] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, the gradation adjustment amount determined in the adjustment amount determination step is applied to the photographed image data using the gradation adjustment method determined in the adjustment method determination step. 5. The image processing method according to claim 4, wherein gradation conversion processing is performed.

[6] In the occupancy ratio calculating step, the photographed image data is divided into regions each having a predetermined combination of brightness and hue, and a first ratio indicating the ratio of the divided regions to the entire photographed image data is obtained. The captured image data is divided into predetermined areas having a combination power of distance and brightness from the outer edge of the screen of the captured image data, and the captured image data is divided for each of the divided areas. Calculate a second occupancy ratio that represents the percentage of the entire image data,

In the index calculation step, the first occupancy rate and the second occupancy rate calculated in the occupancy rate calculation step and the deviation amount calculated in the deviation amount calculation step are preset according to the shooting conditions. The index for specifying the shooting conditions is calculated by multiplying

The image processing method according to claim 1.

[7] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, the gradation adjustment amount determined in the adjustment amount determination step is applied to the photographed image data using the gradation adjustment method determined in the adjustment method determination step. 7. The image processing method according to claim 6, wherein gradation conversion processing is performed.

[8] including a step of creating a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data,

5. The image processing method according to claim 4, wherein the occupation ratio is calculated based on the created two-dimensional histogram based on the occupation ratio calculation step.

[9] including a step of creating a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data,

Based on the created two-dimensional histogram, the occupation rate calculating step The image processing method according to claim 6, wherein the second occupancy is calculated.

[10] By calculating the cumulative number of pixels for each predetermined hue and brightness of the photographed image data

Including the step of creating a two-dimensional histogram,

3. The image processing method according to claim 2, wherein the occupation ratio is calculated based on the created two-dimensional histogram after the occupation ratio calculation step.

[11] By calculating the cumulative number of pixels for each predetermined hue and brightness of the photographed image data

Including the step of creating a two-dimensional histogram,

7. The image processing method according to claim 6, wherein the first occupation ratio is calculated based on the created two-dimensional histogram based on the occupation ratio calculation step.

12. The index calculation step according to claim 1, wherein coefficients having different signs are used in a predetermined high-lightness skin color hue region and a hue region other than the high-lightness skin color hue region. Image processing method.

13. The image processing method according to claim 1, wherein in the index calculation step, coefficients with different signs are used in an intermediate brightness area of the flesh color hue area and a brightness area other than the intermediate brightness area.

14. The image processing method according to claim 12, wherein the brightness area of the hue area other than the high brightness skin color hue area is a predetermined high brightness area.

15. The image processing method according to claim 13, wherein the brightness area other than the intermediate brightness area is a brightness area in a flesh-color hue area.

16. The image processing method according to claim 12, wherein the high-brightness skin color hue region includes a region having a brightness value of 170 to 224 in the HSV color system.

17. The image processing method according to claim 13, wherein the intermediate brightness area includes an area having a brightness value in the HSV color system of 85 to 169.

18. The image processing method according to claim 12, wherein the hue region other than the high brightness skin color hue region includes at least one of a blue hue region and a green hue region.

19. The image processing method according to claim 13, wherein the brightness area other than the intermediate brightness area is a shadow area.

[20] The hue value of the blue hue region is in the range of 161 to 250 in the HSV color system. The image processing method according to claim 18, wherein a hue value of the green hue region is in a range of 40 to 160 in terms of a hue value of the HSV color system.

[21] The image processing method according to item 19, wherein the lightness value of the shadow area is a lightness value of the HSV color system within a range of 26 to 84.

22. The image processing method according to claim 12, wherein a hue value of the flesh color hue region is in a range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

23. The image processing method according to claim 13, wherein a hue value of the flesh color hue region is in a range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

24. The image processing method according to claim 12, wherein the skin color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

25. The image processing method according to claim 13, wherein the skin color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

[26] The deviation amount includes at least one of a deviation amount of brightness of the photographed image data, an average value of brightness at the center of the screen of the photographed image data, and a difference value of brightness calculated under different conditions. The image processing method according to claim 1, wherein one is included.

[27] The shooting condition indicates a light source condition and an exposure condition at the time of shooting,

The image processing method according to claim 1, wherein the light source condition includes a strobe shooting state, and the exposure condition includes an under shooting state derived from a shutter speed and an aperture value at the time of shooting.

[28] The photographed image data is divided into areas each including at least a combination of a predetermined brightness and hue and a combination of the distance from the outer edge of the screen and the brightness of the photographed image data. An occupancy ratio calculation unit for calculating an occupancy ratio indicating a ratio of the entire captured image data;

Discrimination map pre-divided into regions according to the calculated index and the accuracy of the shooting conditions And a determination unit that determines a shooting condition of the captured image data, and an adjustment method determination unit that determines a gradation adjustment method for the captured image data according to the determined shooting condition. When,

An image processing apparatus comprising: a gradation conversion unit that performs gradation conversion processing on the photographed image data using the determined gradation adjustment method.

[29] The occupancy ratio calculation unit divides the captured image data into regions having a combination of predetermined brightness and hue, and an occupancy ratio indicating a ratio of the entire captured image data for each of the divided regions. 29. The image processing device according to claim 28, which calculates

[30] Furthermore, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

The gradation conversion unit performs gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the captured image data, using the gradation adjustment method determined by the adjustment method determination unit. 30. The image processing device according to claim 29, wherein the image processing device is applied.

[31] The occupancy ratio calculation unit divides the photographed image data into predetermined regions having a combination of the distance of the outer edge force and the brightness of the screen of the photographed image data, and the photographed image data for each of the divided regions. 29. The image processing device according to claim 28, wherein an occupancy ratio indicating a ratio of the entire data is calculated.

[32] Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

The gradation conversion unit performs gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the captured image data, using the gradation adjustment method determined by the adjustment method determination unit. 32. The image processing device according to claim 31 to be applied.

[33] The occupancy rate calculating step section divides the captured image data into regions each having a predetermined combination of brightness and hue, and indicates the ratio of the divided region to the entire captured image data. In addition to calculating the occupation ratio of 1, the captured image data is divided into predetermined areas having a combination power of the distance from the outer edge of the screen of the captured image data and the brightness, and the captured image data is divided for each of the divided areas. Calculate a second occupancy that represents the percentage of the total data, In the index calculation unit, the first occupancy rate and the second occupancy rate calculated by the occupancy rate calculation unit and the deviation amount calculated by the deviation amount calculation unit are set in advance according to shooting conditions. 29. The image processing device according to claim 28, wherein an index for specifying the photographing condition is calculated by multiplying by a coefficient.

[34] Furthermore, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

The gradation conversion unit performs gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the captured image data, using the gradation adjustment method determined by the adjustment method determination unit. 34. The image processing device according to claim 33.

[35] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data,

32. The image processing device according to claim 31, wherein the occupancy ratio calculation unit calculates the occupancy ratio based on the created two-dimensional histogram.

[36] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data,

34. The image processing device according to claim 33, wherein the occupancy ratio calculation unit calculates the second occupancy ratio based on the created two-dimensional histogram.

[37] A histogram creation unit that creates a two-dimensional histogram by calculating a cumulative number of pixels for each predetermined hue and brightness of the captured image data,

30. The image processing device according to claim 29, wherein the occupancy ratio calculation unit calculates the occupancy ratio based on the created two-dimensional histogram.

[38] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data,

34. The image processing device according to claim 33, wherein the occupancy ratio calculation unit calculates the first occupancy ratio based on the created two-dimensional histogram.

[39] The index calculation unit according to claim 28, wherein coefficients of different signs are used in a predetermined high-lightness skin color hue region and a hue region other than the high-lightness skin color hue region. Image processing device.

40. The image processing device according to claim 28, wherein the index calculation unit uses different code coefficients for an intermediate brightness area of the flesh hue hue area and a brightness area other than the intermediate brightness area.

41. The image processing device according to claim 39, wherein a brightness area of a hue area other than the high brightness skin color hue area is a predetermined high brightness area.

42. The image processing apparatus according to claim 40, wherein the brightness area other than the intermediate brightness area is a brightness area within a flesh-color hue area.

[43] The image processing device according to [39], wherein the high-brightness skin color hue region includes a region having a brightness value of 170 to 224 in the HSV color system.

44. The image processing apparatus according to claim 40, wherein the intermediate brightness area includes an area having a brightness value in the HSV color system of 85 to 169.

45. The image processing device according to claim 39, wherein the hue area other than the high brightness skin color hue area includes at least one of a blue hue area and a green hue area.

46. The image processing device according to claim 40, wherein the brightness area other than the intermediate brightness area is a shadow area.

[47] The hue value of the blue hue region is in the range of 161 to 250 as the hue value of the HSV color system, and the hue value of the green hue region is from 40 to 160 as the hue value of the HSV color system. 46. The image processing device according to claim 45, which is within the range of.

[48] The image processing device according to [46], wherein the brightness value of the shadow area is a brightness value of the HSV color system in the range of 26 to 84.

49. The image processing apparatus according to claim 39, wherein a hue value of the flesh color hue region is in a range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

50. The image processing apparatus according to claim 40, wherein a hue value of the flesh color hue region is in a range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

51. The image processing device according to claim 39, wherein the skin color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

52. The image processing device according to claim 40, wherein the flesh-color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

[53] The deviation amount includes at least one of a brightness deviation amount of the photographed image data, an average brightness value at the center of the photographed image data screen, and a brightness difference value calculated under different conditions. 29. The image processing device according to claim 28, wherein one is included.

[54] The shooting condition indicates a light source condition and an exposure condition at the time of shooting,

29. The image processing apparatus according to claim 28, wherein the light source condition includes a flash photographing state, and the exposure condition includes an under photographing state derived from a shutter speed and an aperture value at the time of photographing.

[55] an imaging unit that captures captured image data by capturing a subject;

The photographed image data is divided into at least areas of a combination of a predetermined brightness and hue and a combination of the distance from the outer edge of the screen of the photographed image data and the brightness, and for each of the divided areas, An occupancy ratio calculation unit that calculates an occupancy ratio indicating the ratio of the entire captured image data

An imaging apparatus comprising: a gradation conversion unit that performs gradation conversion processing on the captured image data using the determined gradation adjustment method.

[56] The occupancy ratio calculation unit divides the photographed image data into regions composed of a combination of predetermined brightness and hue, and determines a ratio occupied in the entire photographed image data for each of the divided regions. 56. The imaging device according to claim 55, which calculates the occupancy indicated.

[57] Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion unit, the gradation adjustment method determined by the adjustment method determination unit 57. The imaging apparatus according to claim 56, wherein a gradation conversion process of a gradation adjustment amount determined by the adjustment amount determination unit is performed on the photographed image data using.

[58] In the occupancy rate calculation unit, the captured image data is divided into predetermined regions that are combinations of distance and brightness from the outer edge of the screen of the captured image data, and for each of the divided regions, 56. The imaging device according to claim 55, wherein an occupancy ratio indicating a ratio of the entire captured image data is calculated.

[59] Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

The gradation conversion unit performs gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the captured image data, using the gradation adjustment method determined by the adjustment method determination unit. 59. An imaging device according to claim 58.

[60] The occupancy ratio calculating step divides the photographed image data into regions each having a predetermined combination of brightness and hue, and indicates the proportion of the entire photographed image data for each of the divided regions. The first occupancy ratio is calculated, and the captured image data is divided into a predetermined area having a combination power of the distance from the outer edge of the screen of the captured image data and the brightness, and the captured image data is divided into each of the divided areas. Calculate a second occupancy that represents the percentage of the total image data,

In the index calculation unit, the first occupancy rate and the second occupancy rate calculated by the occupancy rate calculation unit, and the deviation amount calculated by the deviation amount calculation unit are set in advance according to shooting conditions. 56. The imaging apparatus according to claim 55, wherein an index for specifying the imaging condition is calculated by multiplying by a coefficient.

[61] Further, an adjustment amount determination unit that determines a gradation adjustment amount for the captured image data based on the calculated index,

The gradation conversion unit performs gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination unit on the photographed image data using the gradation adjustment method determined by the adjustment method determination unit. The imaging device according to claim 60, wherein said imaging device is applied.

[62] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data, 59. The imaging apparatus according to claim 58, wherein the occupancy ratio calculation unit calculates the occupancy ratio based on the created two-dimensional histogram.

[63] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the captured image data,

61. The imaging apparatus according to claim 60, wherein the occupancy ratio calculation unit calculates the second occupancy ratio based on the created two-dimensional histogram.

[64] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data,

57. The imaging apparatus according to claim 56, wherein the occupancy rate calculation unit calculates the occupancy rate based on the created two-dimensional histogram.

[65] A histogram creation unit that creates a two-dimensional histogram by calculating the cumulative number of pixels for each predetermined hue and brightness of the captured image data,

61. The imaging apparatus according to claim 60, wherein the occupancy ratio calculation unit calculates the first occupancy ratio based on the created two-dimensional histogram.

[66] The index calculation unit according to claim 55, wherein coefficients having different signs are used in a predetermined high-lightness skin color hue region and a hue region other than the high-lightness skin color hue region. Imaging device.

[67] The imaging device according to [55], wherein the index calculation unit uses coefficients of different signs for an intermediate brightness region of a flesh-color hue region and a brightness region other than the intermediate brightness region.

68. The imaging device according to claim 66, wherein a brightness area of a hue area other than the high brightness skin color hue area is a predetermined high brightness area.

[69] The imaging device according to [67], wherein the lightness region other than the intermediate lightness region is a lightness region in a flesh-color hue region.

[70] The imaging device according to [66], wherein the skin color hue region of high brightness includes a region having a brightness value of 170 to 224 in the HSV color system.

71. The imaging apparatus according to claim 67, wherein the intermediate brightness area includes an area having a brightness value in the HSV color system of 85 to 169.

72. The imaging apparatus according to claim 66, wherein the hue area other than the high brightness skin color hue area includes at least one of a blue hue area and a green hue area.

[73] The imaging device according to [67], wherein the brightness area other than the intermediate brightness area is a shadow area.

[74] The hue value of the blue hue region is in the range of 161 to 250 as the hue value of the HSV color system, and the hue value of the green hue region is from 40 to 160 as the hue value of the HSV color system. 75. The imaging device according to claim 72, which is within the range of.

[75] The imaging device according to item 73, wherein the brightness value of the shadow region is a brightness value of the HSV color system in a range of 26 to 84.

[76] The imaging device according to item 66, wherein the hue value of the flesh color hue region is in the range of 0 to 39 and 330 to 359 in the hue value of the HSV color system.

[77] The imaging device according to [67], wherein the hue value of the flesh color hue region is in the range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

78. The imaging device according to claim 66, wherein the skin color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

79. The imaging apparatus according to claim 67, wherein the skin color hue area is divided into two areas by a predetermined conditional expression based on brightness and saturation.

[80] The deviation amount includes at least one of a brightness deviation amount of the photographed image data, an average brightness value of the photographed image data at the center of the screen, and a brightness difference value calculated under different conditions. 56. The imaging device according to claim 55, wherein one is included.

[81] The shooting conditions indicate a light source condition and an exposure condition at the time of shooting,

56. The imaging apparatus according to claim 55, wherein the light source condition includes a strobe shooting state, and the exposure condition includes an under shooting state derived from a shutter speed and an aperture value at the time of shooting.

[82] In the computer for image processing,

The photographed image data is divided into at least areas of a combination of a predetermined brightness and hue and a combination of the distance from the outer edge of the screen of the photographed image data and the brightness, and for each of the divided areas, Indicates the percentage of the entire shot image data An occupancy ratio calculating step for calculating an occupancy ratio;

[83] In the occupation ratio calculation step, the captured image data is divided into areas each having a predetermined combination of brightness and hue, and an occupation indicating the ratio of the divided areas to the entire captured image data is divided. 83. An image processing program according to claim 82, wherein the rate is calculated.

[84] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, gradation conversion processing of the gradation adjustment amount determined by the adjustment amount determination step is performed on the captured image data using the gradation adjustment method determined by the adjustment method determination step. 84. An image processing program according to claim 83.

[85] In the occupancy ratio calculating step, the captured image data is divided into predetermined regions having a combination power of distance and brightness from the outer edge of the screen of the captured image data, and the captured image data is divided into the divided regions. 83. The image processing program according to claim 82, wherein an occupancy ratio indicating a ratio of the entire data is calculated.

[86] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, the gradation adjustment method determined by the adjustment method determination step 86. The image processing program according to claim 85, wherein a gradation conversion process of the gradation adjustment amount determined by the adjustment amount determination step is performed on the photographed image data.

[87] In the occupancy ratio calculating step, the photographed image data is divided into regions each having a predetermined combination of brightness and hue, and a first ratio indicating a ratio of the divided regions to the entire photographed image data is obtained. The captured image data is divided into predetermined areas having a combination power of distance and brightness from the outer edge of the screen of the captured image data, and the captured image data is divided for each of the divided areas. Calculate a second occupancy ratio that represents the percentage of the entire image data,

In the index calculation unit, the first occupancy ratio and the second occupancy ratio calculated in the occupancy ratio calculation step, and the deviation amount calculated in the deviation amount calculation step are set in advance according to the shooting conditions. 83. The image processing program according to claim 82, wherein an index for specifying the photographing condition is calculated by multiplying by a coefficient.

[88] The method further includes an adjustment amount determination step of determining a gradation adjustment amount for the captured image data based on the calculated index,

In the gradation conversion step, the gradation adjustment amount determined by the adjustment amount determination step is determined for the captured image data using the gradation adjustment method determined by the adjustment method determination step. 90. An image processing program according to claim 87, which performs a tone conversion process.

[89] including a step of creating a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the photographed image data,

86. The image processing program according to claim 85, wherein the occupancy rate is calculated based on the created two-dimensional histogram based on the occupancy rate calculation step.

[90] including a step of creating a two-dimensional histogram by calculating the cumulative number of pixels for each distance and brightness from the outer edge of the screen of the photographed image data,

90. The image processing program according to claim 87, wherein the second occupation ratio is calculated based on the created two-dimensional histogram based on the occupation ratio calculation step.

[91] By calculating the cumulative number of pixels for each predetermined hue and brightness of the photographed image data

Including the step of creating a two-dimensional histogram, 84. The image processing program according to claim 83, wherein, in the occupancy rate calculating step, the occupancy rate is calculated based on the created two-dimensional histogram.

[92] By calculating the cumulative number of pixels for each predetermined hue and brightness of the photographed image data

Including the step of creating a two-dimensional histogram,

90. The image processing program according to claim 87, wherein the first occupation ratio is calculated based on the created two-dimensional histogram based on the occupation ratio calculation step.

93. The index calculation step according to claim 82, wherein coefficients having different signs are used in a predetermined high-lightness skin color hue region and a hue region other than the high-lightness skin color hue region. Image processing program.

[94] The image processing program according to item 82, wherein, in the index calculation step, coefficients having different signs are used in an intermediate brightness area of the flesh hue hue area and a brightness area other than the intermediate brightness area.

95. The image processing program according to claim 93, wherein the brightness area of the hue area other than the high brightness skin color hue area is a predetermined high brightness area.

96. The image processing program according to claim 94, wherein the brightness area other than the intermediate brightness area is a brightness area in a flesh-color hue area.

[97] The image processing program according to [93], wherein the high-brightness skin color hue region includes a region having a brightness value of 170 to 224 in the HSV color system.

98. The image processing program according to claim 94, wherein the intermediate brightness area includes an area having a brightness value in the HSV color system of 85 to 169.

99. The image processing program according to claim 93, wherein the hue area other than the high brightness skin color hue area includes at least one of a blue hue area and a green hue area.

[100] The image processing program according to item 94, wherein the brightness area other than the intermediate brightness area is a shadow area.

[101] The hue value of the blue hue region is in the range of 161 to 250 as the hue value of the HSV color system, and the hue value of the green hue region is from 40 to 160 as the hue value of the HSV color system. The image processing program according to claim 99, which is within the range of.

[102] The brightness value of the shadow area is a value of 26 to 84 in the HSV color system. 100. The image processing program according to item 100.

[103] The image processing program according to item 93, wherein the hue value of the flesh color hue region is in the range of 0 to 39 and 330 to 359 as the hue value of the HSV color system.

[104] The image processing program according to item 94, wherein a hue value of the flesh color hue region is in a range of 0 to 39 and 330 to 359 as a hue value of the HSV color system.

105. The image processing program according to claim 93, wherein the skin color hue area is divided into two areas by a predetermined conditional expression based on brightness and saturation.

106. The image processing program according to claim 94, wherein the skin color hue region is divided into two regions by a predetermined conditional expression based on brightness and saturation.

[107] The deviation amount includes at least one of the brightness deviation amount of the photographed image data, the average brightness value at the center of the screen of the photographed image data, and the brightness difference value calculated under different conditions. 83. An image processing program according to claim 82, wherein one is included.

[108] The shooting conditions indicate a light source condition and an exposure condition at the time of shooting,

83. The image processing program according to claim 82, wherein the light source condition includes a strobe shooting state, and the exposure condition includes a shutter speed at the time of shooting and an under shooting state derived from an aperture value.