JP2006180185A - Color adjustment method, color adjustment apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique determining a target color in response to the attribute of an object for color adjustment and a scene represented by a color image. <P>SOLUTION: This color adjustment apparatus is provided with: a scene feature quantity calculation section 11 for analyzing the color image and calculating a scene feature quantity for digitizing the attribute of the color adjustment object and a scene represented by the color image; a target color definition generating section 21 for generating a target color definition for cross-referencing the scene feature quantity with the target color; a target color calculation section 12 for calculating the target color on the basis of the calculated scene feature quantity and the generated target color definition; an adjustment object color calculation section 31 for calculating the color of the adjustment object; and a color adjustment section 32 for adjusting the adjustment target color to the target color. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a color adjustment method for adjusting a specific target color of a color image.

  Color adjustment may be performed on a natural image taken by a digital camera or the like. Color adjustment is to correct the color balance, brightness, etc. of the entire image, or to correct a specific area or a specific color to be adjusted. Of these, typical examples of partial color adjustments such as the latter include a color that is stored by many people as an image of a specific target color (hereinafter referred to as `` memory color ''). Adjustment) For example, a person's skin may become an unhealthy color such as ocher in a natural image, but in such a case, an adjustment may be made to make the skin color look healthy. Also, even if the sky is clear, it is normal that the natural image is blue, which is close to light blue. However, as a printed image, many people imagine it as the color of the sky when it is clear. Since it looks better when it is a bright blue color, it is often corrected to such a color.

For this type of color adjustment, several methods have been proposed as conventional techniques.
First, there is a conventional technique for performing color adjustment so that gradation inversion does not occur and color continuity is not impaired (see, for example, Patent Document 1). In this Patent Document 1, by moving a point without gradation reversal from the target color to the target color in the adjustment target color space including the target color to be adjusted and the target color to be adjusted. Allows specific color adjustments specified. Since the movement of the point is performed only in the adjustment target color space, it does not affect other colors.
There is also a conventional technique for performing color adjustment so that the result of color adjustment is not far from the original image (see, for example, Patent Document 2). In this Patent Document 2, the given target color is not used as it is, but the given target color is converted into the optimum target color so that it becomes a natural color with respect to the original image, and this optimum target color is converted. Is the goal of color adjustment. Specifically, the distance between the color of the region to be adjusted extracted from the original image and the target color is calculated, and control is performed so that the optimum target color is obtained by a suppression function for the distance.

Japanese Unexamined Patent Publication No. 2004-112694 (pages 7-12, FIGS. 1-6) JP 2004-254303 A (pages 7-9, FIG. 3-6)

By the way, as a target color that is a target of color adjustment, it is not necessary to set a fixed color according to an object. The target color may change depending on the attribute of the object to be color-adjusted and the scene represented by the image. For example, the skin color of a person may vary depending on attributes such as gender and race, and may vary depending on scenes such as a scene around the person.
However, the techniques described in Patent Documents 1 and 2 only control a target color given in advance, and there is a problem that the target color itself cannot be appropriately determined according to the scene.

  The present invention has been made to solve the above technical problem, and its purpose is to achieve a target color according to the attribute of the object to be color-adjusted and the scene represented by the color image. Is to be able to determine.

For this purpose, in the present invention, the target color is determined according to the scene or the like represented by the color image. That is, the color adjustment method of the present invention is a method for adjusting the color of a color image, the step of acquiring a feature amount relating to the content of the color image, and a target for color adjustment in the color image based on the acquired feature amount. Determining a target color.
Here, the target color can be determined by the following method.
The first method is to determine a target color as a color adjustment target in a color image based on a reference target color and an acquired feature amount.
Second, based on the definition information that defines the correspondence between the feature amount related to the content of the color image and the target color that should be the target of color adjustment in the color image from which the feature amount was acquired, and the acquired feature amount, This is a method of determining a target color that is a target of color adjustment in a color image.

The present invention can also be understood as an apparatus that determines a target color according to a scene or the like represented by a color image. In this case, the color adjustment device of the present invention is a device that adjusts the color of a color image, a feature amount acquisition unit that acquires a feature amount related to the content of the color image, and a feature amount acquired by the feature amount acquisition unit. And a target color determining unit that determines a target color that is a target of color adjustment in the color image.
Here, the target color can be determined by the following method.
First, the target color determination unit determines a target color that is a target of color adjustment in a color image based on the reference target color and the feature amount acquired by the feature amount acquisition unit. .
Second, the target color determination unit defines definition information that defines the correspondence between the feature amount related to the content of the color image and the target color that should be the target of color adjustment in the color image from which the feature amount is acquired, and the feature amount acquisition. This is a method of determining a target color as a color adjustment target in a color image based on the feature amount acquired by the unit.

On the other hand, the present invention can also be understood as a program for realizing a predetermined function in a computer. In that case, the program of the present invention has a function of acquiring a feature amount related to the contents of the color image in a computer and a function of determining a target color as a target of color adjustment in the color image based on the acquired feature amount. It is for realizing.
Here, the target color can be determined by the following method.
The first method is to determine a target color as a color adjustment target in a color image based on a reference target color and an acquired feature amount.
Second, based on the definition information that defines the correspondence between the feature amount related to the content of the color image and the target color that should be the target of color adjustment in the color image from which the feature amount was acquired, and the acquired feature amount, This is a method of determining a target color that is a target of color adjustment in a color image.

  According to the present invention, the target color can be determined according to the attribute of the target object to be color adjusted and the scene represented by the color image.

  The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below in detail with reference to the accompanying drawings. In the present embodiment, the term “scene” is used as a concept that includes not only the scene around the color adjustment target but also the attributes of the target.

(First embodiment)
FIG. 1 is a diagram showing a functional configuration of the color adjustment apparatus according to the first embodiment of the present invention.
As shown in the figure, the color adjustment apparatus according to the present embodiment includes a scene feature amount calculation unit 11 and a target color calculation unit 12.
The scene feature amount calculation unit 11 has a function of analyzing a scene related to a color image and calculating a scene feature amount obtained by quantifying the scene. It is also a part that can be done. In addition, the target color calculation unit 12 is a part having a function of calculating a target color according to the degree of the scene feature amount, and can be grasped as a target color determination unit if not limited to calculation. It is.
Each of these functions can be realized only by hardware, but can also be realized by a combination of hardware and software. In the latter case, each function is realized by a CPU (not shown) of the color adjustment apparatus reading and executing a program stored in a memory (not shown), for example.

Next, the operation of the color adjustment apparatus having such a functional configuration will be described in detail.
First, the operation of the scene feature quantity calculation unit 11 will be described.
The scene feature amount calculation unit 11 first receives an input of a color image.
FIG. 2 is a diagram showing an example of such a color image. Although various objects can be considered for color adjustment, in this embodiment, the color of the person's skin is adjusted. Accordingly, it is assumed that the color image to be input includes a person as shown in FIG.
In the present embodiment, it is assumed that there is only one person in the image in order to simplify the description. For example, an image analysis unit may be provided in the preceding stage of the scene feature amount calculation unit 11 to determine whether or not only one person is included in the input color image by a known method. In this case, only when the color image includes one person, the color image is input to the scene feature amount calculation unit 11.

The scene feature amount calculation unit 11 inputs such a color image and analyzes the scene. The scene analysis may be performed for each partial region by a known method such as a combination of region division and a neural network. For example, in FIG. 2, the analysis is performed by extracting partial areas such as clouds, sky, mountains, human faces, and chairs.
When the scene analysis result is digitized, a scene feature amount as shown in FIG. 3A is obtained. Here, “sex”, “race”, and “scene” are selected as the feature types.

Here, a method for calculating a scene feature amount for gender, race, and scene will be briefly described.
First, gender can be determined by, for example, the face color and hairstyle. That is, by extracting the connected components of the pixels in the color range of the target object (face, hair, etc.) to be detected and excluding those in which the color gradient in the connected component exceeds a predetermined threshold Identify the area of the object. Then, for example, the determination is made such that a woman is biased when the face color is biased toward a lighter color and a male is biased toward a darker color. Alternatively, it is possible to make a determination such that a woman has a long hair and a man has a short hair. Further, gender may be determined in consideration of the face color, hairstyle and other factors.

Also, the race can be discriminated to some extent by using the color of the face as a clue, like gender.
On the other hand, in the example of FIG. 2, it is assumed that the scene is likely to be an image taken outdoors due to the presence of clouds, sky, and mountains. For detection of clouds, sky, mountains, etc., the connected components of the pixels in the color range of the object to be detected (clouds, sky, mountains, etc.) are extracted, and the gradient of the color in the connected components among them is predetermined. It can be extracted by excluding those exceeding the threshold value.

  Note that the scene feature amount is not necessarily represented by a binary value of “0” or “1”. In the case of gender, “0” is set when the probability of being 100% is male, and “1” is set when the probability of being 100% is female, and there is a possibility of being male or female. The degree may be expressed by a numerical value between “0” and “1”. A scale indicating the degree of sex, race, and scene in such a case is shown in FIG. In each of “gender degree”, “race degree”, and “scene degree” in FIG. 3B, scene feature values for gender, race, and scene exist between “0” and “1”. become. In FIG. 2, since a Japanese woman is photographed outdoors, the gender degree shows a value close to that of a woman, the racial degree shows a value close to that of a Japanese person, and the scene degree becomes a value close to the outdoors.

If there are three types of features as shown in FIG. 3, the scene feature amount can be represented by a three-dimensional vector. However, the scene feature amount may be represented by a dimension corresponding to the number of feature types, and when the feature type is one, it may be a scalar instead of a vector.
Furthermore, the types of features are not limited to “gender”, “race”, and “scene”, and any feature can be used as long as the original image means. In FIG. 3, the scene indicates whether it is indoor or outdoor, but a fine scene such as whether it is a night view or an image at the sea may be adopted.

Next, the operation of the target color calculation unit 12 will be described.
FIG. 4 is a flowchart showing an example of the operation flow of the target color calculation unit 12.
As illustrated in FIG. 3, when scene feature amounts are obtained for a plurality of feature types, first, the target color calculation unit 12 prompts the user to select a feature type used for determining the target color. (Step 101). When the feature type is selected by the user, the scene feature amount corresponding to the feature type is acquired from the scene feature amounts calculated by the scene feature amount calculation unit 11 (step 102). For example, if “gender” is selected as the feature type, the gender degree “0.95” is acquired from the scene feature amount of FIG.

On the other hand, the target color calculation unit 12 sets the first target color to be set when the scene feature amount is “0” and the second target color to be set when the scene feature amount is “1”. Holding. Based on the information to be held and the scene feature value acquired in step 102, a ratio of adopting the first target color and the second target color is obtained (step 103).
Then, a target color value is calculated according to the obtained ratio (step 104).

For example, if “gender” is selected as the feature type, the target color calculation unit 12 calculates a target color value based on information as shown in FIG.
FIG. 5A is a diagram showing information defining a target color for men and a target color for women for a person's skin. Here, the chromaticity of these target colors is set on the a ^* b ^* plane of the L ^* a ^* b ^* space.
FIG. 5B is a diagram showing the relationship between the gender degree value acquired in step 102 and the female target color adoption rate indicating how close the target color is adopted to the female target color. is there. If the gender degree is “0”, the female target color adoption rate is “0”, and the male target color is adopted. Conversely, if the gender degree is “1”, the female target color adoption rate is “1”, and the female target color is adopted. Further, if the gender degree is “0.95” as shown in FIG. 3A, the target color adoption rate for women is “Y” as shown in FIG. 5B. In this case, in FIG. 5A, the line segment connecting the point representing the male target color and the point representing the female target color corresponds to a point (indicated by a black circle) obtained by internally dividing Y: 1−Y. The color is the target color that should be adopted.

Finally, the target color calculation unit 12 outputs the calculated target color value to a storage device (not shown) (step 105).
Thus, the operation of the first embodiment is completed.

In this embodiment, since the degree of gender is focused, the male target color and the female target color are used as reference target colors. However, if attention is paid to other types of features, a target color may be set as appropriate according to the type. In a particularly easy-to-understand example, settings such as setting the skin color close to wheat as the target color if the degree considered to have been taken in the sea are high, and setting the light skin color as the target color if the degree considered to be taken in the sea are low It is.
In the present embodiment, the target color calculation unit 12 holds the information of FIG. 5, but the information may be stored in a storage device (not shown) and read from there by the target color calculation unit 12. .

(Second embodiment)
FIG. 6 is a diagram showing a functional configuration of the color adjustment apparatus according to the second embodiment of the present invention.
As shown in the figure, the color adjustment apparatus according to the present embodiment includes a scene feature amount calculation unit 11, a target color calculation unit 12, and a target color definition generation unit 21.
As in the first embodiment, the scene feature quantity calculation unit 11 is a part having a function of analyzing a scene related to a color image and calculating a scene feature quantity obtained by quantifying the scene, and is limited to being obtained by calculation. Otherwise, it is a part that can be grasped as a feature amount acquisition unit. Further, the target color calculation unit 12 has a function of calculating a target color using a target color definition described later, and can be grasped as a target color determination unit if not limited to calculation. It is. The target color definition generation unit 21 has a function of generating a target color definition in which a scene and a target color are associated with each other and storing the target color definition in a storage device (not shown).
Each of these functions can be realized only by hardware, but can also be realized by a combination of hardware and software. In the latter case, each function is realized by a CPU (not shown) of the color adjustment apparatus reading and executing a program stored in a memory (not shown), for example.

Here, an example of the target color definition generated by the target color definition generation unit 21 will be described.
FIG. 7 is an example of a neural network that learns a target color for a scene feature amount. Here, the target color is given in the L ^* a ^* b ^* color space for n scene feature values. For example, the neural network connection coefficient after learning can be used as the target color definition.
FIG. 8 is an example of a look-up table that stores the correspondence between scene feature amounts and target colors. Again, the target colors are given in the L ^* a ^* b ^* color space for n scene feature values.
7 and 8, no specific example is shown, but the scene feature amount may include gender, race, scene, etc., as in FIG. 3, or other scene feature amounts may be included. It may be adopted.

Next, the operation of the color adjustment apparatus having such a functional configuration will be described in detail. The operation of the scene feature amount calculation unit 11 is the same as that of the first embodiment, and thus the description thereof will be omitted. Only the operation of the target color calculation unit 12 will be described.
FIG. 9 is a flowchart showing an example of the operation flow of the target color calculation unit 12.
First, the target color calculation unit 12 acquires the scene feature value calculated by the scene feature value calculation unit 11 (step 201). If there are n scene feature amounts, all of them are acquired.
Next, the target color calculation unit 12 reads the target color definition generated by the target color definition generation unit 21 from a storage device (not shown) (step 202).
A corresponding target color is acquired based on the n combinations of scene feature values acquired in step 201 (step 203). If the neural network of FIG. 7 is used, a target color can be obtained by giving a combination of n scene feature amounts to the neural network. If the lookup table of FIG. 8 is used, a target color can be obtained by searching the lookup table using a combination of n scene feature amounts as a key.
Thus, the operation of the second embodiment is completed.

In the example of FIG. 8, the scene feature amount is binary of “0” or “1”, and the target color is set for these n combinations. However, in the first embodiment, as shown in FIG. 3A, the scene feature amount is not necessarily represented by a binary value of “0” or “1”, and an intermediate value thereof is used. It can also be taken.
Hereinafter, how to obtain the target color in this case will be briefly described.
For example, it is assumed that the scene feature quantity i is expressed as F _i and (F ₁ , F ₂ ,..., F _n ) is acquired as a combination of scene feature quantities. In this case, first, the target color calculation unit 12 obtains a hypercube including a point F corresponding to the combination of the scene feature amounts in the n-dimensional space. Then, target colors corresponding to 2 ⁿ lattice points of the hypercube are obtained, and a target color corresponding to F is calculated by linear interpolation.

(Third embodiment)
FIG. 10 is a diagram showing a functional configuration of the color adjustment apparatus according to the third embodiment of the present invention.
As shown in the figure, the color adjustment apparatus according to the present embodiment includes a scene feature amount calculation unit 11, a target color calculation unit 12, a target color definition generation unit 21, an adjustment target color calculation unit 31, and a color adjustment unit 32. With.
Since the scene feature amount calculation unit 11, the target color calculation unit 12, and the target color definition generation unit 21 are the same as those described in the first embodiment or the second embodiment, description thereof will be omitted. On the other hand, the adjustment target color calculation unit 31 is a part having a function of calculating a color to be adjusted. For example, when adjusting the skin color of a person, a skin color relating to the person such as a face region may be specified using a known technique. The color adjustment unit 32 calculates the color toward the target color calculated in the first embodiment or the second embodiment and stored in a storage device (not shown) with respect to the adjustment target color thus determined. Make adjustments. In this case, for example, partial color adjustment without gradation inversion can be performed using the method described in Patent Document 1 (Japanese Patent Laid-Open No. 2002-112694).
Each of these functions can be realized only by hardware, but can also be realized by a combination of hardware and software. In the latter case, each function is realized by a CPU (not shown) of the color adjustment apparatus reading and executing a program stored in a memory (not shown), for example.
This is the end of the description of the third embodiment.

In the first to third embodiments described above, for the sake of simplicity, a color image showing a single person is input. However, a color image showing a plurality of persons is input. It is good. In that case, which person's skin color is to be adjusted may be determined based on a user instruction or based on system judgment.
In addition to the skin color of a person, it is possible to adjust the colors of various objects such as the sky color and grass color. However, it is conceivable that the type of feature for calculating the scene feature amount differs between the case of adjusting the color of human skin, the case of adjusting the sky color, and the case of adjusting the color of grass. In that case, prior to the operation of the scene feature amount calculation unit 11 described in the first embodiment or the second embodiment, a color adjustment target is determined by a user instruction or by system judgment, You may perform the process which specifies the kind of feature-value suitable for the object.

  Further, in the present embodiment, the scene feature amount is analyzed by analyzing the image. However, the scene feature amount may be calculated by acquiring a feature related to the content of the image from the accompanying information added to the image data at the time of photographing with a digital camera or the like and digitizing the feature. As the accompanying information in this case, information added to the main image in an Exif (Exchangeable Image File Format) file can be employed. Exif is an image file standard for a digital camera proposed by Fuji Photo Film Co., Ltd. and standardized by JEIDA (Japan Electronics Industry Promotion Association).

  In the present embodiment, hardware for performing image processing is not particularly limited. However, for example, it may be realized by a computer such as a personal computer, or may be realized by an image processing unit (IPS) of the image forming apparatus. ) May be realized. In the former case, the function of the image processing apparatus of the present invention may be incorporated into an application program that runs on a computer, and the printer driver that receives the image data to be printed may have the color of the present invention. Adjustment processing may be performed. In the latter case, scanned image data or image data received via a communication line is received by the IPS of the image forming apparatus, color adjustment processing is performed, and image formation is performed based on the image data after the color adjustment processing. You just have to do it.

  As described above, in the first to third embodiments of the present invention, by defining or generating information for deriving the target color from the feature amount of the scene represented by the entire image, such feature amount is obtained. A suitable target color can be determined.

It is the block diagram which showed the function structure of the 1st Embodiment of this invention. It is the figure which showed an example of the color image input in the 1st thru | or 3rd Embodiment of this invention. It is a figure for demonstrating the scene feature-value calculated in the 1st thru | or 3rd Embodiment of this invention. It is the flowchart which showed operation | movement of the target color calculation part in the 1st Embodiment of this invention. It is the figure which showed an example of the information hold | maintained at the target color calculation part in the 1st Embodiment of this invention. It is the block diagram which showed the function structure of the 2nd Embodiment of this invention. It is the figure which showed the 1st example of the target color definition which the target color calculation part in the 2nd Embodiment of this invention refers. It is the figure which showed the 2nd example of the target color definition which the target color calculation part in the 2nd Embodiment of this invention refers. It is the flowchart which showed operation | movement of the target color calculation part in the 2nd Embodiment of this invention. It is the block diagram which showed the function structure of the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Scene feature-value calculation part, 12 ... Target color calculation part, 21 ... Target color definition production | generation part, 31 ... Adjustment object color calculation part, 32 ... Color adjustment part

Claims (12)

In a color adjustment method for adjusting the color of a color image,
Obtaining a feature amount relating to the content of the color image;
Determining a target color that is a target of color adjustment in the color image based on the acquired feature amount.   The color adjustment according to claim 1, wherein in the determining step, a target color that is a target of color adjustment in the color image is determined based on a target color that is a reference and the acquired feature amount. Method.   In the determining step, definition information defining a correspondence between a feature amount related to the content of the color image and a target color to be a color adjustment target in the color image from which the feature amount is acquired, and the acquired feature amount The color adjustment method according to claim 1, further comprising: determining a target color as a color adjustment target in the color image based on the above.   The definition information is a neural network in which a correspondence between a feature amount related to the content of the color image and a target color to be a color adjustment target in the color image from which the feature amount is acquired is learned. Item 4. The color adjustment method according to Item 3.   The definition information is a look-up table storing a correspondence between a feature amount related to the content of the color image and a target color to be a color adjustment target in the color image from which the feature amount is acquired. The color adjustment method according to claim 3.   The color adjustment method according to claim 1, wherein the feature amount is a numerical value of a scene represented by the color image. In a color adjustment device that adjusts the color of a color image,
A feature amount acquisition unit for acquiring a feature amount related to the content of the color image;
A color adjustment apparatus comprising: a target color determination unit that determines a target color that is a target of color adjustment in the color image based on the feature amount acquired by the feature amount acquisition unit.   The target color determination unit determines a target color to be a color adjustment target in the color image based on a reference target color and the feature amount acquired by the feature amount acquisition unit. The color adjusting device according to claim 7.   The target color determination unit defines definition information defining a correspondence between a feature amount related to the content of the color image and a target color to be a color adjustment target in the color image from which the feature amount is acquired, and the feature amount acquisition The color adjustment device according to claim 7, wherein a target color that is a target of color adjustment in the color image is determined based on the feature amount acquired by the unit.   The color adjustment apparatus according to claim 7, wherein the feature amount is a numerical value of a scene represented by the color image. On the computer,
A function for acquiring feature values related to the contents of a color image;
A program for realizing a function of determining a target color that is a target of color adjustment in the color image based on the acquired feature amount.   12. The program according to claim 11, wherein the feature amount is a numerical value of a scene represented by the color image.

Images