JP2017037622A - Image processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase a possibility of achieving both improvement in the visibility and suppression in the occurrence of noise or whiteout in comparison to a case where an image processing device is configured to decide the holding degree regardless of the emphasis degree when emphasizing a reflectance component of an original image by using the emphasis degree of the reflectance component of the original image and the holding degree of a dark part or bright part of the original image.SOLUTION: An image processing device 10 includes: a reflectance estimation unit 12 which generates a reflectance image using a reflectance component of an original image as a pixel value from the original image; a bright/dark part holding parameter decision unit 13 which decides the holding degree of at least one of a dark part and a bright part of the original image by using the original image and emphasis degree of the reflectance component of the original image; and an image reproduction unit 17 which generates an emphasis image with the reflectance component of the original image emphasized by using the reflectance image, emphasis degree and holding degree.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and a program.

  An illumination light component estimation unit that calculates an estimated illumination light component of the original image, an estimated illumination light component correction unit that outputs a corrected estimated illumination light component obtained by correcting the estimated illumination light component, and an estimation in which the luminance component of the original image is corrected A Retinex processing unit that corrects the luminance component by dividing by the illumination light component, and a gain adjustment unit that adjusts the gain of the corrected luminance component, and the estimated illumination light component correction unit includes 0 to a reference value The estimated illumination light component of the value is converted into the maximum value of the corrected estimated illumination light component, and the estimated illumination light component having a value larger than the reference value is a value having a larger value of the corrected estimated illumination light component than the estimated illumination light component value. An image processing apparatus using such a characteristic that the rate of increase of the corrected estimated illumination light component decreases as the value of the estimated illumination light component increases and the value of the estimated illumination light component increases is known (for example, Patent Document 1). reference).

JP 2012-108898 A

  Here, when the configuration in which the estimated illumination light component having a value from 0 to the reference value is converted to the maximum value of the corrected estimated illumination light component and the Retinex processing is performed is adopted, the dark portion is left as the original image. Therefore, even if the generation of noise is suppressed, the visibility is not improved. Further, the estimated illumination light component having a value larger than the reference value is set so that the value of the corrected estimated illumination light component is larger than the value of the estimated illumination light component, and as the value of the estimated illumination light component increases, If a configuration is adopted in which the rate of increase in the corrected estimated illumination light component is reduced and the Retinex process is used, the effect of Retinex is weakened as the bright part becomes brighter. Even if the occurrence of is suppressed, the visibility is not improved as expected.

  In order to improve the visibility and suppress the occurrence of noise or whiteout, the reflectance component of the original image is used by using the degree of enhancement of the reflectance component of the original image and the degree of retention of the dark or bright part of the original image. Can be emphasized. At that time, it is conceivable to adopt a configuration that determines the retention level regardless of the enhancement level. However, since the degree of occurrence of noise and whiteout differs depending on the degree of enhancement of the reflectance component of the original image, when adopting a configuration in which the holding degree is determined regardless of the degree of enhancement, the visibility and noise or whiteout are determined. There is a high possibility that it is not possible to achieve both the suppression of the occurrence of water.

  In order to improve visibility and suppress the occurrence of pseudo contour, the reflectance of the original image is determined by using the degree of enhancement of the reflectance component of the original image and the degree of retention of the specific color that generates the pseudo contour of the original image. It is possible to emphasize the ingredients. At that time, it is conceivable to adopt a configuration that determines the retention level regardless of the enhancement level. However, since the degree of occurrence of pseudo contour varies depending on the degree of enhancement of the reflectance component of the original image, when adopting a configuration in which the retention degree is determined regardless of the degree of enhancement, visibility is improved and generation of pseudo contour is suppressed. There is a high probability that

  It is an object of the present invention to maintain an original image with a reflectance component regardless of the degree of enhancement using the degree of enhancement of the reflectance component of the original image and the degree of retention of the dark or bright portion of the original image. Compared to the case where the degree is determined, there is an increase in the possibility of achieving both improvement in visibility and suppression of occurrence of noise or whiteout.

  Another object of the present invention is to enhance the reflectance component of the original image using the degree of enhancement of the reflectance component of the original image and the degree of retention of the specific color that generates the pseudo contour of the original image. The object is to increase the possibility that both the improvement of visibility and the suppression of the occurrence of pseudo contours can be achieved compared to the case where the degree of retention of the specific color is determined regardless of the degree of enhancement.

According to the first aspect of the present invention, there is provided a reflectance image generating unit that generates a reflectance image having a reflectance component of the original image as a pixel value from the original image, the original image, and the reflectance component of the original image. Using the degree of emphasis and the determination means for determining the degree of retention of at least one of the dark part and the bright part of the original image, the reflectance image, the degree of emphasis, and the degree of retention, An image processing apparatus comprising: an enhanced image generating unit configured to generate an enhanced image in which the reflectance component of the original image is enhanced.
The invention according to claim 2 is characterized in that the determining means determines a brightness / darkness holding degree which is a holding degree of the dark part and the bright part of the original image as the holding degree. An image processing apparatus.
According to a third aspect of the present invention, the determination means is configured such that the darker pixel of the original image has a higher degree of holding the bright and dark part, and the brighter pixel of the original image has a higher degree of holding of the bright and dark part. The image processing apparatus according to claim 2, wherein the degree of bright and dark area retention is determined.
The invention according to claim 4 is characterized in that the determining means determines the brightness / darkness portion holding degree so that the higher the degree of emphasis, the higher the brightness / darkness portion holding degree. This is an image processing apparatus.
According to a fifth aspect of the present invention, the determination means includes, as the holding degree, a dark part holding degree that is a holding degree of a dark part of the original image and a bright part holding degree that is a holding degree of a bright part of the original image. The image processing apparatus according to claim 1, wherein:
According to a sixth aspect of the present invention, the determining unit determines the dark portion retention degree so that the darker pixel of the original image has a higher dark portion retention degree, and the brighter the original image, the brighter the pixel. The image processing apparatus according to claim 5, wherein the bright part holding degree is determined so that the part holding degree is high.
According to a seventh aspect of the present invention, the determination means determines the dark part holding degree so that the dark part holding degree is higher as the emphasis degree is higher, and the bright part holding is higher as the emphasis degree is higher. The image processing apparatus according to claim 5, wherein the degree of bright portion retention is determined so that the degree becomes higher.
The image processing apparatus according to claim 1, further comprising setting means for setting the degree of emphasis previously associated with an operation performed on the screen by the operator. It is.
The invention according to claim 9 uses a reflectance image generating means for generating a reflectance image having the reflectance component of the original image as a pixel value from the original image, and the degree of enhancement of the reflectance component of the original image. A first determining means for determining a first holding degree that is a holding degree of a portion of a specific color predetermined as a color for generating a pseudo contour of the original image, the reflectance image, and the enhancement degree. And an enhanced image generating means for generating an enhanced image in which the reflectance component of the original image is enhanced using the first holding degree.
The invention according to claim 10 further includes an extracting unit that extracts the specific color portion from the original image, and the enhanced image generating unit holds the first color portion of the original image in the first holding state. 10. The image processing according to claim 9, wherein the enhanced image is generated using a degree, and the portion other than the specific color portion of the original image is not enhanced using the first holding degree. Device.
The invention according to claim 11 is characterized in that the first determining means determines the first holding degree so that the higher the degree of emphasis is, the higher the first holding degree is. An image processing apparatus according to claim 9.
According to a twelfth aspect of the present invention, a second holding degree that is a holding degree of at least one of a dark part and a bright part of the original image is determined using the original image and the enhancement degree. The image processing apparatus according to claim 9, further comprising: determining means, wherein the emphasized image generating means further generates the emphasized image using the second holding degree.
According to a thirteenth aspect of the present invention, there is provided a computer having a function of generating a reflectance image having a reflectance component of the original image as a pixel value from the original image, Using the enhancement degree, the function for determining the retention degree of at least one of the dark part and the bright part of the original image, the reflectance image, the enhancement degree, and the retention degree are used. And a function for generating an enhanced image in which the reflectance component of the image is enhanced.
According to the fourteenth aspect of the present invention, the computer uses a function of generating a reflectance image having the pixel value of the reflectance component of the original image from the original image, and the degree of enhancement of the reflectance component of the original image. , A function for determining a first holding degree that is a holding degree of a specific color portion predetermined as a color for generating a pseudo contour of the original image, the reflectance image, the enhancement degree, and the first And a function for generating an enhanced image in which the reflectance component of the original image is enhanced.

According to the first aspect of the present invention, when the reflectance component of the original image is enhanced using the enhancement degree of the reflectance component of the original image and the degree of retention of the dark portion or the bright portion of the original image, Regardless of the configuration in which the degree of holding the dark part or the bright part is determined regardless of the possibility, it is possible to improve both the visibility and the suppression of the occurrence of noise or whiteout.
According to the second aspect of the present invention, when the reflectance component of the original image is emphasized using the enhancement degree of the reflectance component of the original image and the retention levels of the dark part and the bright part of the original image, Regardless of the configuration in which the degree of holding the dark part and the bright part is determined regardless of the possibility, it is possible to improve both visibility and suppress the occurrence of noise and whiteout.
According to the invention of claim 3, the reflectance component of the original image is enhanced using the enhancement degree of the reflectance component of the original image, the retention degree of the dark part of the original image, and the retention degree of the bright part of the original image. As compared with the case where it is configured, it becomes easier to suppress the occurrence of noise and whiteout.
According to the fourth aspect of the present invention, compared to the case where the high degree of retention of the dark part and the bright part of the original image is determined irrespective of the high degree of enhancement of the reflectance component of the original image. The possibility of suppressing the occurrence of noticeable noise and whiteout when the degree of emphasis is increased is increased.
According to the invention of claim 5, the reflectance component of the original image is enhanced using the enhancement degree of the reflectance component of the original image, the retention degree of the dark part of the original image, and the retention degree of the bright part of the original image. Compared to the case where the darkness retention level and the brightness retention level are determined regardless of the enhancement level, it is possible to achieve both improvement in visibility and suppression of occurrence of noise and overexposure. The possibility increases.
According to the invention of claim 6, when the reflectance component of the original image is enhanced using the enhancement degree of the reflectance component of the original image and the retention degree of the bright part and the dark part of the original image. In comparison, the accuracy of suppressing the occurrence of noise and whiteout becomes higher.
According to the seventh aspect of the present invention, the high degree of holding of the dark part of the original image and the high degree of holding of the bright part of the original image are determined regardless of the high degree of enhancement of the reflectance component of the original image. Compared to the case where it is configured, there is a higher possibility of suppressing the occurrence of noticeable noise and whiteout when the degree of emphasis is increased.
According to the invention of claim 8, it is possible to suppress the occurrence of noise and whiteout while ensuring the visibility set by the operator.
According to the invention of claim 9, when enhancing the reflectance component of the original image by using the enhancement degree of the reflectance component of the original image and the holding degree of the specific color that generates the pseudo contour of the original image, Compared to a case where the degree of holding a specific color is determined regardless of the degree of enhancement, there is a higher possibility that both improved visibility and suppression of the occurrence of pseudo contours can be achieved.
According to the invention of claim 10, the reflectance component is emphasized over the entire original image by using the enhancement degree of the reflectance component of the original image and the retention degree of the specific color according to the distance from the specific color. Compared with the case where it comprises, the load of the process which balances the improvement of visibility and suppression of generation | occurrence | production of a pseudo contour is reduced.
According to the eleventh aspect of the present invention, when the degree of holding the specific color that generates the pseudo contour of the original image is determined regardless of the degree of enhancement of the reflectance component of the original image. In comparison, the possibility of suppressing the occurrence of a pseudo contour that is noticeable when the degree of emphasis is increased is increased.
According to the invention of claim 12, when the reflectance component of the original image is emphasized by further using the degree of retention of the dark part or the bright part of the original image, the degree of retention of the dark part or the bright part is determined regardless of the degree of enhancement. Compared with the case where it is configured to do so, there is a high possibility that the suppression of the occurrence of noise or whiteout can be further achieved.
According to the invention of claim 13, when enhancing the reflectance component of the original image using the enhancement degree of the reflectance component of the original image and the degree of retention of the dark part or bright part of the original image, Regardless of the configuration in which the degree of holding the dark part or the bright part is determined regardless of the possibility, it is possible to improve both the visibility and the suppression of the occurrence of noise or whiteout.
According to the invention of claim 14, when enhancing the reflectance component of the original image by using the enhancement degree of the reflectance component of the original image and the holding degree of the specific color that generates the pseudo contour of the original image, Compared to a case where the degree of holding a specific color is determined regardless of the degree of enhancement, there is a higher possibility that both improved visibility and suppression of the occurrence of pseudo contours can be achieved.

1 is a block diagram illustrating a functional configuration example of an image processing apparatus according to a first embodiment of the present invention. It is the figure which showed the mode of the production | generation of the multilayer image by the multilayer image generation part in case an original image is an RGB image. (A)-(c) is the figure which showed that the frequency of the image of each layer of a multilayer image changes with the value of (sigma). It is the figure which showed the mode of the production | generation of the multilayer image by the multilayer image generation part in case an original image is a brightness | luminance chromaticity image. It is the graph which showed the example of the relationship between the brightness-and-darkness part retention parameter determined in the 1st Embodiment of this invention, and the pixel value of an original image. It is the graph which showed the example of the change with respect to the pixel value of the original image of the brightness-and-darkness part retention parameter when a visibility parameter is large. It is the block diagram which showed the function structural example of the image processing apparatus in the 2nd Embodiment of this invention. It is the graph which showed the example of the relationship between the dark part holding parameter determined in the 2nd Embodiment of this invention, and the pixel value of an original image. It is the graph which showed the example of the relationship between the bright part holding parameter determined in the 2nd Embodiment of this invention, and the pixel value of an original image. It is the graph which showed the example of the relationship between the product of a dark part holding parameter and a bright part holding parameter, and the pixel value of an original image. It is the graph which showed the example of the relationship between the bright part holding parameter and the pixel value of an original image when it is desired to suppress whiteout. 6 is a graph showing an example of a relationship between a product of a dark part retention parameter and a bright part retention parameter and a pixel value of an original image when whiteout is to be suppressed. It is the graph which showed the example of the change with respect to the pixel value of the original image of the dark part holding parameter in case a visibility parameter is large. It is the graph which showed the example of the change with respect to the pixel value of the original image of the bright part holding parameter in case a visibility parameter is large. It is the block diagram which showed the function structural example of the image processing apparatus in the 3rd Embodiment of this invention. It is the figure which showed the example of the method of determining the intensity | strength of each parameter according to user interaction. It is the figure which showed the example of the method of determining the intensity | strength of each parameter according to user interaction. 5 is a flowchart showing an operation example of the image processing apparatus according to the first to third embodiments of the present invention. It is the block diagram which showed the function structural example of the image processing apparatus in the 4th Embodiment of this invention. It is the graph which showed the example of the relationship between the skin part retention parameter determined in the 4th Embodiment of this invention, and a visibility parameter. It is the block diagram which showed the function structural example of the image processing apparatus in the 5th Embodiment of this invention. It is the graph which showed the example of the relationship between the final reproduction parameter about the skin area | region determined by the 5th Embodiment of this invention, and other than a skin area | region, and the pixel value of an original image. The graph which illustrated how the relationship between the final reproduction parameter about the skin area | region determined in the 5th Embodiment of this invention and the pixel value of an original image changes according to the change of a visibility parameter. It is. It is the flowchart which showed the operation example of the image processing apparatus in the 4th and 5th embodiment of this invention. It is the block diagram which showed the hardware structural example of the image processing apparatus in embodiment of this invention.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

[Background of the embodiment]
The work of creating a document by handling images is generally performed while watching a monitor using a personal computer (PC). In such work, an increasing number of users use ICT (Information and Communication Technology) devices such as tablets that are rapidly spreading in recent years.

  In general, in an office environment such as office work or DTP (DeskTop Publishing) work, it is rarely affected by changes in ambient light. On the other hand, an ICT device that can be carried comfortably has the advantage of being able to work regardless of location, but also has the disadvantage of being greatly influenced by the carried destination, such as changes in ambient light.

  In addition to the work for creating a document as described above, the work for handling images includes a work for storing images taken by a user with a camera-equipped tablet in each device. Scenes where users show images to each other and explain the situation with images are also often seen.

  As described above, as a feature of the monitor environment in recent years, unlike the conventional monitor environment, it is easy to use and has various usage places. In this recent monitor environment, the “viewability” is more important than color matching because the use method and the use environment are different from the conventional one.

  “Visibility” is a characteristic of whether or not a visual target is clearly visible. There are basic methods in the image processing field represented by gamma correction, histogram equalization, dynamic range compression, and the like as methods for improving the visibility of an image.

  In gamma correction, a curve that enlivens a dark part or a target region is generated and applied to pixel values to brighten the dark part. In the histogram equalization, a curve that eliminates the bias of the histogram of the image is generated and applied to the pixel value, thereby reproducing the histogram to be smoothed. In dynamic range compression, low luminance and high luminance are expressed without reducing contrast by changing the correction amount according to the peripheral luminance of the image.

  Some methods for improving visibility using visual characteristics use the Retinex principle. Retinex is a basic principle that enhances visibility by emphasizing the reflectance component based on the idea that humans perceive a scene by reflectance.

  In Retinex processing, it is necessary to estimate the illumination component in order to estimate the reflectance component. At present, a smoothed image that has been smoothed is generally used as an illumination component. As a method using this smoothed image, SSR (Single-Scale Retinex) using a single smoothed image, MSR (Multi-Scale Retinex) using a plurality of smoothed images, and IMSR (Integrated Multi-Scale Retinex) are used. ) In SSR, since a single smoothed image is used, if the smoothing degree is small, noise may be generated in the dark area, and if the smoothing degree is large, an unnatural band-like defect ( Halo effect) may occur. On the other hand, in MSR and IMSR, it is said that the generation of noise and halo effect is reduced because the smoothed image is multilayered.

  However, if the enhancement degree (reproduction parameter) of the reflectance component is increased, noise or whiteout may occur depending on the image even in MSR or IMSR.

  Therefore, in the present embodiment, the holding parameter of the dark part or the bright part that suppresses noise or whiteout is controlled according to the visibility parameter and the brightness information of the image. That is, even if the pixel value is the same, the final emphasis level is changed by changing the retention parameter of the dark part or the bright part according to the visibility parameter.

  In addition, due to human physical characteristics, it is easy to create shadows on the boundary between the face outline and neck, the bent limbs, and the like. If there is such a shadow, a false contour may be generated by the Retinex process. The pseudo contour is more conspicuous as the visibility parameter is higher, and gives an unnatural impression. For example, the pseudo contour of the skin portion becomes more noticeable as the visibility parameter is increased to improve the visibility.

  Therefore, in the present embodiment, the skin part holding parameter for suppressing the pseudo contour is controlled according to the level of visibility to be reproduced. That is, even if the pixel value is the same, the final emphasis level is changed by changing the skin holding parameter according to the visibility parameter.

[First Embodiment]
FIG. 1 is a block diagram illustrating a functional configuration example of an image processing apparatus 10 according to the first embodiment of the present invention. As illustrated, the image processing apparatus 10 according to the present embodiment includes an illumination estimation unit 11, a reflectance estimation unit 12, a bright / dark part retention parameter determination unit 13, and an image reproduction unit 17.

The illumination estimation unit 11 assumes that the pixel value for the position (x, y) of the original image is I (x, y) and the illumination component is I _L (x, y), and I _L based on I (x, y). (X, y) is estimated (hereinafter, this estimated illumination component image is referred to as an “illumination estimated image”). As one characteristic of visual recognition, it is known that the perception amount of one point (one pixel in the image) of light entering the retina affects the average of the peripheral luminance of the one point. The ambient luminance corresponds to the estimated illumination component, and the illumination component is estimated by, for example, performing a convolution represented by a moving average method or a Gaussian function.

  Here, x and y represent the position of a certain pixel, k represents a coefficient to be normalized so that the result becomes 1 when integration is performed for pixels of the filter size of image processing, and σ represents a smoothing degree (scale ). The above function is an example, and any filter may be used as long as the result is a smoothed image. For example, although there is a bilateral filter known as a smoothing filter that performs edge preservation with a filter that is a function obtained by transforming Equation 1, it may be used.

  According to the Retinex principle, it is known that the human visual characteristic estimates illumination light from the periphery of a region of interest. Therefore, the smoothed image represents the estimated illumination light. However, since the suitable scale differs depending on the scene, for example, it is desirable to estimate the illumination light by taking a weighted sum of N layer images from scale 1 to scale N as follows.

Here, I _L (x, y) represents the pixel value of the estimated illumination image, G _n (x, y) represents Equation 1 for the scale n, and I (x, y) represents the pixel value of the luminance image. , W _n represents a weight for the scale n, and a symbol in which “×” is surrounded by “◯” represents convolution. Incidentally, W _n may be as simple as 1 / N, may be varied depending on the layer.

  FIG. 2 shows how the multilayer image generation unit 110 in the illumination estimation unit 11 generates a multilayer image. FIG. 2 shows an example in which N-layer images from scale 1 to scale N are generated as multilayer images when the original image is composed of three RGB planes. Here, since the layers of scale 1, scale 2,..., Scale N are layers obtained by changing σ of Formula 1, the degree of smoothing is different. As shown in FIGS. 3A to 3C, when σ is changed, the frequency of the image changes. Specifically, when σ is small as shown in (a), the frequency becomes high, when σ is large as shown in (c), the frequency becomes low, and when σ is medium as shown in (b), the frequency becomes high. Also moderate.

  As described above, the multilayer image may be an N layer of three planes of RGB, but may be as shown in FIG. FIG. 4 shows an example in which an RGB image is converted in advance into a luminance chromaticity image such as a YCbCr image. In this example, only one Y plane representing luminance is converted into a multilayer image. . This is because the illumination component may be estimated using only the luminance. Here, the YCbCr color space is used as the color space represented by the luminance chromaticity, but an L * a * b * color space, an HSV color space (where HS is converted into chromaticity coordinates), and the like are used. May be. L * may be used as the luminance image when the L * a * b * color space is used, and V may be used as the luminance image when the HSV color space is used. Further, only an image of one scale layer used in the SSR may be used as the illumination estimation image.

  In the present embodiment, an illumination estimation image is used as an example of an illumination image having the illumination component of the original image as a pixel value, and an illumination estimation unit 11 is provided as an example of an illumination image generation unit that generates an illumination image. Yes.

  The reflectance estimation unit 12 estimates the reflectance of the original image by obtaining a ratio of the pixel value of the original image to the pixel value of the illumination estimated image. Specifically, an image representing the reflectance (hereinafter referred to as “reflectance estimation image”) is obtained as follows.

Here, I (x, y) represents the pixel value of the original image, I _L (x, y) represents the pixel value of the illumination estimated image, and I _R (x, y) represents the pixel value of the reflectance estimated image. Represents. When the illumination estimation image is calculated for each of the three RGB planes, Equation 3 is also interpreted as I _R (x, y) corresponding to the three RGB planes. Further, when only the luminance component (Y of YCbCr, L of L * a * b *, V of HSV) is used, Expression 3 may be used for one plane.

  In this embodiment, a reflectance estimation image is used as an example of a reflectance image in which the reflectance component of the original image is a pixel value, and the reflectance is used as an example of a reflectance image generation unit that generates a reflectance image. An estimation unit 12 is provided.

  The bright / dark part holding parameter determination unit 13 determines the bright / dark part holding parameter according to the visibility parameter and the luminance information of the original image. Details will be described later.

  The image reproduction unit 17 performs processing for enhancing the reflectance component based on the reflectance estimation image generated by the reflectance estimation unit 12 and the original image. For example, a reproduction image in which the reflectance component is emphasized is generated by the following reproduction formula.

Here, I (x, y) represents the pixel value of the original image, I _R (x, y) represents the pixel value of the reflectance estimated image, and I ^ (x, y) represents the pixel value of the reproduced image. Represent. Α is a visibility parameter for emphasizing visibility, and β is a light / dark part retention parameter determined by the light / dark part retention parameter determination unit 13. α is assumed to change according to user instructions and image characteristics. The user instruction includes, for example, selection of a finger movement amount and a mode on the screen. I ^ (x, y) becomes the reflectance component itself when α = 1 and β = 1, and becomes the pixel value of the original image when α = 0 or β = 0. In the present specification, a hat symbol is added immediately above a character in a mathematical expression, but is added after a character in a sentence.

  The final reproduction parameter is a parameter obtained by multiplying α and β. The reproduction parameter has a characteristic that, when the visibility parameter α is increased, the intensity of the dark / dark portion retention parameter β increases as the dark pixel or the bright pixel increases. By doing so, dark part noise or bright part whiteout is suppressed.

  Although several Retinex reproduction formulas have been proposed, any reproduction formula may be used as long as the weight of the reflectance component is changed. For example, the following formula may be used.

  Here, α and β are parameters representing the gain of the reflectance, and γ is a parameter representing the intercept of the reproduction formula. FIG. 1 shows the case where the image reproduction unit 17 generates a reproduction image using the original image. However, when the mathematical formula 5 is used, the image reproduction unit 17 generates the reproduction image without using the original image. become. Further, α corresponds to the visibility parameter in FIG. 1, β corresponds to the bright / dark part retention parameter in FIG. 1, and γ is a parameter not shown in FIG.

  In the present embodiment, a reproduced image is used as an example of an enhanced image in which the reflectance component of the original image is enhanced, and an image reproduction unit 17 is provided as an example of an enhanced image generation unit that generates an enhanced image. .

  Next, the bright / dark part retention parameter determination unit 13 will be described in detail using Equation 4 as an example. The bright / dark part holding parameter β determined by the bright / dark part holding parameter determination unit 13 is a parameter responsible for suppressing dark part noise and bright part whiteout. Basically, as shown in FIG. 5, the darker or brighter the pixel, the smaller the bright / dark portion retention parameter β. Then, since the weight of the original image becomes high, the dark portion and the bright portion of the original image are maintained. Therefore, dark part noise and bright part whiteout generated by Retinex processing are suppressed. On the other hand, the bright / dark area retention parameter β is increased at intermediate pixel values. Then, since the weight of the Retinex reproduction image becomes high, it is considered that the visibility is improved.

  For example, a function of the following formula can be considered as a function representing this characteristic.

  Here, T and K are coefficients, T> 0, and K> 0. K1 and K2 are normalization coefficients. When normalization is performed so that β (I (x, y)) falls within the range of 0 to 1 when 0 <I (x, y) <1, K1 and K2 are as described above.

  As the function, any function may be used as long as it represents the above characteristics, such as an upward convex quadratic function.

  Next, a method in which the light / dark part holding parameter determination unit 13 controls the light / dark part holding parameter β according to the visibility parameter α will be described with reference to FIG. When the visibility parameter α is large, the dark part noise and the bright part of the white part are easily noticeable. Therefore, the bright / dark part retention parameter determination unit 13 determines the bright / dark part retention parameter with the rising edge delayed and the falling edge advanced, as indicated by a broken line in FIG. By using the bright / dark part holding parameter, the bright / dark part holding level is strengthened. On the other hand, when the visibility parameter α is small, dark part noise and bright part whiteout are hardly noticeable. Therefore, as shown by the solid line in FIG. 6, the bright / dark part holding parameter determination unit 13 determines the bright / dark part holding parameter whose rise time is earlier than the broken light / dark part holding parameter. By using the bright / dark part holding parameter, the bright / dark part holding level is weakened. In this way, by controlling the intensity of the bright / dark part retention parameter β according to the intensity of the visibility parameter α, both better visibility and suppression of noise and overexposure can be achieved.

  The control of the intensity of the bright / dark part retention parameter β according to the intensity of the visibility parameter α as described above is expressed by the following expression using, for example, T in Expression 6 as a function of α.

Here, f (α)> 0 and K> 0. Further, since f (α) in the broken line in FIG. 6 is smaller than f (α) in the solid line in FIG. 6, f (α) is a decreasing function. However, the decreasing function in this case is a function that satisfies the condition “if α ₁ <α ₂ , f (α ₁ ) ≧ f (α ₂ )”. That is, as long as the function decreases as a whole, a portion where the same value continues may be included in the middle.

Therefore, f (α) is preferably expressed by the following equation, for example. This expression divides the visibility parameter α into cases. When the visibility parameter α is smaller than a certain value, f (α) is set to s _1, and when the visibility parameter α is larger than a certain value, f (α) is set. it is an s _2.

Alternatively, f (α) may be a continuous function. In this case, for example, a linear function with s ₃ as a slope and s ₄ as an intercept may be used as in the following equation.

  In the present embodiment, the visibility parameter is used as an example of the enhancement degree of the reflectance component of the original image. In addition, as an example of the bright and dark part holding degree, which is the holding degree of the dark part and the bright part of the original image, the bright and dark part holding parameter is used, and a determination unit that determines the bright and dark part holding degree using the original image and the enhancement degree As an example, a bright / dark part holding parameter determination unit 13 is provided. However, if the bright and dark area retention degree is high, the weight of the original image is higher than the reflectance estimated image in the light and dark areas, so the value of the bright and dark area retention parameter is small.

[Second Embodiment]
FIG. 7 is a block diagram illustrating a functional configuration example of the image processing apparatus 10 according to the second embodiment of the present invention. As shown in the figure, the image processing apparatus 10 according to the present embodiment includes an illumination estimation unit 11, a reflectance estimation unit 12, a dark part retention parameter determination unit 14, a bright part retention parameter determination unit 15, and an image reproduction unit 17. With. Here, the illumination estimator 11 and the reflectance estimator 12 are the same as those in the first embodiment, and thus the description thereof will be omitted. In the following, the dark part retention parameter determination unit 14, the bright part retention parameter determination unit 15, and the image reproduction are omitted. Only the unit 17 will be described. The basic idea of the second embodiment is to perform better reproduction by determining the dark part retention parameter and the bright part retention parameter with different functions.

  The image reproduction unit 17 performs processing for enhancing the reflectance component based on the reflectance estimation image generated by the reflectance estimation unit 12 and the original image. For example, a reproduction image in which the reflectance component is emphasized is generated by the following reproduction formula.

  Here, α is a visibility parameter for enhancing visibility, β is a dark part retention parameter determined by the dark part retention parameter determination unit 14, and γ is a bright part retention parameter determined by the bright part retention parameter determination unit 15. is there. I ^ (x, y) is the reflectance component itself when α = 1, β = 1, and γ = 1, and is the pixel value of the original image when α = 0, β = 0, or γ = 0. .

  The final reproduction parameter is a parameter obtained by multiplying α, β, and γ. The reproduction parameters include a characteristic that the dark portion retention parameter β increases in intensity as the visibility parameter α is increased, and a bright portion retention parameter γ increases in the brightness pixel as the visibility parameter α is increased. And hold. In this way, the final reproduction parameter approaches αβ for dark pixels and approaches αγ for bright pixels.

  In the present embodiment, a reproduced image is used as an example of an enhanced image in which the reflectance component of the original image is enhanced, and an image reproduction unit 17 is provided as an example of an enhanced image generation unit that generates an enhanced image. .

  Next, the dark part retention parameter determination unit 14 and the bright part retention parameter determination unit 15 will be described in detail. The dark part holding parameter determining unit 14 suppresses dark part noise while maintaining better visibility by changing the characteristic of β. In addition, the bright part holding parameter determination unit 15 suppresses overexposure of the bright part by changing the characteristic of γ while maintaining better visibility.

  For example, the function of Formula 11 can be considered as a function for determining the dark part retention parameter β.

  Here, T1 is a coefficient, and T1> 0.

  As shown in FIG. 8, the dark part retention parameter β decreases as the pixel becomes darker. Therefore, the contribution of the reflectance component is reduced, and the contribution of the original image is increased.

  Further, as a function for determining the bright portion retention parameter γ, the function of Expression 12 can be considered. Here, T2 and T3 are coefficients, and T2> 0 and T3> 0. The function of Expression 12 is based on a portion in which the original image pixel value of the function obtained by inverting the function of FIG. 8 around the axis of the dark area retention parameter corresponds to 0 to 1. However, since only a negative value is taken as it is, normalization is performed with the range T3 and the intercept +1.

  As shown in FIG. 9, the bright portion retention parameter γ decreases as the pixel becomes brighter. Therefore, the contribution of the reflectance component is reduced, and the contribution of the original image is increased.

  As a result, the relationship between the pixel value of the original image and the dark part retention parameter β × the bright part retention parameter γ is as shown by a thick line in FIG. That is, the relationship is similar to the relationship between the pixel value of the original image and the bright / dark part retention parameter β in the first embodiment.

  Here, the second embodiment is different from the first embodiment in that the dark part holding parameter β and the bright part holding parameter γ are independently controlled. For example, in the shape of FIG. 10, the dark part retention parameter β and the bright part retention parameter γ are determined to have the same strength. On the other hand, when it is desired to further suppress whiteout, T2 in Expression 12 may be reduced. By doing so, as shown in FIG. 11, the falling of the curve of the bright part retention parameter is accelerated. Then, using the characteristics of the bright area retention parameter and the characteristics of the dark area retention parameter of FIG. 8, the relationship between the pixel value of the original image and the dark area retention parameter β × the bright area retention parameter γ is indicated by a bold line in FIG. It will be as shown in. That is, the falling of β × γ at a bright pixel is earlier than the shape shown by the thick line in FIG. Therefore, whiteout is further suppressed. In addition, by controlling the dark part holding parameter and the bright part holding parameter independently as described above, noise suppression may be enhanced.

  Next, a method in which the dark part retention parameter determination unit 14 controls the dark part retention parameter β according to the visibility parameter α will be described.

  When the visibility parameter α is large, the dark part noise becomes more conspicuous. Therefore, the dark part holding parameter determination unit 14 determines the dark part holding parameter whose rise is delayed as shown by the broken line in FIG. By using the dark area retention parameter, the dark area retention level is enhanced. On the other hand, when the visibility parameter α is small, dark part noise is not noticeable. Therefore, the dark area retention parameter determination unit 14 determines a dark area retention parameter whose rise is earlier than the dashed dark area retention parameter, as indicated by the solid line in FIG. By using this dark part holding parameter, the dark part holding level is weakened. In this way, by controlling the strength of the dark part retention parameter β according to the strength of the visibility parameter α, better visibility and noise suppression can be achieved at the same time.

  Control of the intensity of the dark part retention parameter β according to the intensity of the visibility parameter α as described above is expressed by, for example, the following expression using T1 of Expression 11 as a function of α.

Here, f (α)> 0. Further, since f (α) in the broken line in FIG. 13 is smaller than f (α) in the solid line in FIG. 13, f (α) is a decreasing function. However, the decreasing function in this case is a function that satisfies the condition “if α ₁ <α ₂ , f (α ₁ ) ≧ f (α ₂ )”. That is, as long as the function decreases as a whole, a portion where the same value continues may be included in the middle.

Therefore, f (α) is preferably expressed by the following equation, for example. This expression divides the visibility parameter α into cases. When the visibility parameter α is smaller than a certain value, f (α) is set to s _1, and when the visibility parameter α is larger than a certain value, f (α) is set. it is an s _2.

Alternatively, f (α) may be a continuous function. In this case, for example, a linear function with s ₃ as a slope and s ₄ as an intercept may be used as in the following equation.

  Next, a method in which the bright part holding parameter determination unit 15 controls the bright part holding parameter γ according to the visibility parameter α will be described.

  When the visibility parameter α is large, the whiteout in the light portion is easily noticeable. Therefore, the bright part holding parameter determination unit 15 determines the bright part holding parameter whose falling edge is accelerated as indicated by a broken line in FIG. By using this bright part holding parameter, the bright part holding level is strengthened. On the other hand, when the visibility parameter α is small, the bright portion of the whiteout is not noticeable. Therefore, as shown by the solid line in FIG. 14, the bright part holding parameter whose falling is delayed from the broken line bright part holding parameter is determined. By using the bright part holding parameter, the bright part holding level is weakened. Thus, by controlling the intensity of the bright portion retention parameter γ according to the intensity of the visibility parameter α, better visibility and suppression of whiteout can be achieved at the same time.

  The control of the intensity of the bright part retention parameter γ according to the intensity of the visibility parameter α as described above is expressed by the following expression, for example, where T2 in Expression 12 is a function of α.

Here, f (α)> 0. Further, since f (α) in the broken line in FIG. 14 is smaller than f (α) in the solid line in FIG. 14, f (α) is a decreasing function. However, the decreasing function in this case is a function that satisfies the condition “if α ₁ <α ₂ , f (α ₁ ) ≧ f (α ₂ )”. That is, as long as the function decreases as a whole, a portion where the same value continues may be included in the middle.

Therefore, f (α) is preferably expressed by the following equation, for example. This expression divides the visibility parameter α into cases. When the visibility parameter α is smaller than a certain value, f (α) is set to s _1, and when the visibility parameter α is larger than a certain value, f (α) is set. it is an s _2.

Alternatively, f (α) may be a continuous function. In this case, for example, a linear function with s ₃ as a slope and s ₄ as an intercept may be used as in the following equation.

  In the present embodiment, the visibility parameter is used as an example of the enhancement degree of the reflectance component of the original image. Further, the dark part holding parameter is used as an example of the dark part holding degree that is the dark part holding degree of the original image, and the dark part holding parameter is used as an example of a determination unit that determines the dark part holding degree using the original image and the enhancement degree. A determination unit 14 is provided. However, when the dark portion retention degree is high, the weight of the original image is higher than the reflectance estimation image in the dark portion, and thus the dark portion retention parameter value is small. Further, as an example of the bright part holding degree, which is the bright part holding degree of the original image, the bright part holding parameter is used, and as an example of a determination unit that determines the bright part holding degree using the original image and the enhancement degree. A bright part holding parameter determination unit 15 is provided. However, when the bright portion holding degree is high, the weight of the original image is higher than the reflectance estimated image in the bright portion, and thus the value of the bright portion holding parameter is small.

  Further, in the present embodiment, the dark part holding parameter determining unit 14 and the bright part holding parameter determining unit 15 are provided, and the image reproduction unit 17 uses the dark part holding parameter and the bright part holding parameter in addition to the visibility parameter to reflect. Although a reproduced image in which the rate component is emphasized is generated, the present invention is not limited to this. Either one of the dark part holding parameter determination unit 14 and the bright part holding parameter determination unit 15 is provided, and the image reproduction unit 17 uses either one of the dark part holding parameter and the bright part holding parameter to reflect in addition to the visibility parameter. A reproduced image in which the rate component is emphasized may be generated. In this sense, the dark part holding parameter determination unit 14 and the bright part holding parameter determination unit 15 are an example of a determination unit that determines the holding degree of at least one of the dark part and the bright part of the original image.

[Third Embodiment]
FIG. 15 is a block diagram illustrating a functional configuration example of the image processing apparatus 10 according to the third embodiment of the present invention. As illustrated, the image processing apparatus 10 according to the present embodiment includes an illumination estimation unit 11, a reflectance estimation unit 12, a user parameter determination unit 16, and an image reproduction unit 17. Here, since the illumination estimation unit 11, the reflectance estimation unit 12, and the image reproduction unit 17 are the same as those in the second embodiment, a description thereof will be omitted, and only the user parameter determination unit 16 will be described.

  The user parameter determination unit 16 determines a visibility parameter, a dark part holding parameter, and a bright part holding parameter by user interaction. As a method for determining each parameter by user interaction, any method may be adopted. Here, two methods will be described. At that time, the user (operator) is instructed with a finger, but the finger is an example of an instruction means, and may be instructed with any other instruction means such as a touch pen.

  The first method is a method of determining each parameter according to the combination of directions designated by the user.

  For example, in FIG. 16, the user moves his / her finger in the left / right direction on the displayed original image to change the intensity of the dark area retention parameter. Specifically, when the user moves his / her finger to the left, the intensity of the dark area retention parameter is decreased, and when the user moves the finger to the right, the intensity of the dark area retention parameter is increased.

  In FIG. 16, the strength of the bright part holding parameter is changed by moving the finger up and down on the displayed original image. Specifically, when the user moves the finger up, the strength of the bright portion retention parameter is decreased, and when the user moves the finger down, the strength of the bright portion retention parameter is increased.

  Further, in FIG. 16, the strength of the visibility parameter is changed by the user performing a tap operation or a double tap operation on the displayed original image. Specifically, when the user performs a tap operation with a finger, the strength of the visibility parameter is increased, and when the user performs a double tap operation with a finger, the strength of the visibility parameter is decreased.

  Alternatively, although not shown in FIG. 16, both the dark part holding parameter and the bright part holding parameter may be controlled by the user moving his / her finger in an oblique direction on the displayed original image.

  Here, the visibility parameter, the dark part holding parameter, and the bright part holding parameter are independently determined by user interaction. However, the present invention is not limited to this. That is, only the visibility parameter may be determined by a tap operation or a double tap operation. The dark part holding parameter and the bright part holding parameter may be determined by the determined visibility parameter and Expressions 13 and 16 without moving the finger in the horizontal direction or the vertical direction.

  The second method is a method of determining each parameter according to selection of an option displayed on the UI such as a printer driver.

  For example, in FIG. 17, each of the item “visibility” corresponding to the visibility parameter, the item “noise suppression” corresponding to the dark portion retention parameter, and the item “whiteout suppression” corresponding to the bright portion retention parameter is divided into three stages. Intensity “High”, “Medium”, and “Small” are displayed as options. On the other hand, the user selects one of three levels of intensity for each item. When the strength is selected as shown in FIG. 17, the strength of the visibility parameter is increased, the strength of the dark portion retention parameter is set to medium, and the bright portion retention parameter is decreased. In this way, when the user selects one of the options for each parameter, both visibility and suppression of dark part noise and whiteout are achieved while reflecting the user's preference.

  Here, the visibility parameter, the dark part holding parameter, and the bright part holding parameter are independently determined by user interaction. However, the present invention is not limited to this. That is, only three visibility levels may be displayed as options for the visibility parameter, and only the visibility parameter may be determined by selecting the option. Then, the dark part holding parameter and the bright part holding parameter may be determined by the determined visibility parameter and Expressions 13 and 16 without displaying three levels of intensity as options.

  In the present embodiment, the user parameter determination unit 16 is provided as an example of a setting unit that sets the enhancement degree.

  In addition, in FIG. 15, the user parameter determination unit 16 determines the visibility parameter, the dark part holding parameter, and the bright part holding parameter by user interaction as a modification of the second embodiment. is not. As a modification of the first embodiment, the visibility parameter and the bright / dark part retention parameter may be determined by user interaction. In this case as well, the visibility parameter may be determined by user interaction, and the bright / dark portion retention parameter may be determined by the determined visibility parameter and Equation 7.

[Operation of the first to third embodiments]
FIG. 18 is a flowchart showing an operation example of the image processing apparatus 10 in the first to third embodiments of the present invention. In this operation example, a case will be described in which the bright / dark part holding parameter, the dark part holding parameter, the bright part holding parameter, and the like are controlled according to the visibility parameter.

  When the original image is input, first, the illumination estimation unit 11 generates a multilayer image including a plurality of smoothed images from the original image as shown in FIGS. 2 and 4, and these multilayer images Is used to generate an illumination estimation image (step 101).

  Next, the reflectance estimation unit 12 generates a reflectance estimation image based on the original image and the illumination estimation image generated in step 101 (step 102).

  Next, the image processing apparatus 10 determines a holding parameter for at least one of the dark part and the bright part (step 103).

  Here, the “holding parameter of at least one of the dark part and the bright part” is a bright and dark part holding parameter in the first embodiment. That is, in the first embodiment, the light / dark part holding parameter determination unit 13 determines the light / dark part holding parameter based on the original image and the visibility parameter.

  In addition, the “holding parameter of at least one of the dark part and the bright part” is a dark part holding parameter and a bright part holding parameter in the second embodiment. That is, in the second embodiment, the dark part retention parameter determination unit 14 determines the dark part retention parameter based on the original image and the visibility parameter, and the bright part retention parameter determination unit 15 determines the original image and the visual recognition. The bright part retention parameter is determined based on the sex parameter.

  Further, the “holding parameter of at least one of the dark part and the bright part” may be any of the bright and dark part holding parameter, the dark part holding parameter, and the bright part holding parameter in the third embodiment. However, the user parameter determination unit 16 determines a visibility parameter by user interaction, and any one of the bright and dark part holding parameter, the dark part holding parameter, and the bright part holding parameter is calculated by using the visibility parameter and the original image. To decide.

  Finally, the image reproduction unit 17 converts the original image, the reflectance estimation image generated in step 102, the visibility parameter, and the retention parameter of at least one of the dark part and the bright part determined in step 103. Based on this, a reproduced image is generated (step 104). Here, it is assumed that the original image is used on the assumption that the reproduction image is generated using Equation 4, but when the reproduction image is generated using Equation 5, the original image is not used in Step 104. May be.

[Fourth Embodiment]
This embodiment is an embodiment in which the same skin part retention parameter is applied to all pixels in a region representing human skin extracted from an original image (hereinafter referred to as “skin region”). Hereinafter, a skin region will be described as an example of a region for generating a pseudo contour, but the present invention is not limited to this.

  FIG. 19 is a block diagram illustrating a functional configuration example of the image processing apparatus 10 according to the fourth embodiment of the present invention. As illustrated, the image processing apparatus 10 according to the present embodiment includes an illumination estimation unit 11, a reflectance estimation unit 12, an image reproduction unit 17, a skin region extraction unit 18, and a skin part retention parameter determination unit 19. Is provided. Here, since the illumination estimation unit 11 and the reflectance estimation unit 12 are the same as those in the first embodiment, the description thereof will be omitted. Hereinafter, the skin region extraction unit 18, the skin part holding parameter determination unit 19, and the image reproduction unit will be described. Only 17 will be described.

  The skin region extraction unit 18 extracts a human skin region from the original image. Extraction of the skin area is done by converting the original image from RGB to HSV and extracting pixels close to typical skin values, or by using the information and color information by recognizing humans by object recognition and the like. Any method may be used such as

  In the present embodiment, a skin region is used as an example of a specific color portion that is predetermined as a color that generates a pseudo contour of the original image, and the skin region is used as an example of an extraction unit that extracts the specific color portion. An extraction unit 18 is provided.

  More specifically, in the present embodiment, an area in which a pseudo contour is likely to occur when Retinex processing is performed on the original image is specified, and a representative color in the specified area is called a “specific color”. Yes. Note that the pseudo contour generally refers to a phenomenon in which the density of an original image is emphasized by image processing to generate a step, and a pseudo line such as a contour appears.

  In a photographed image, a shadow is often generated on a part of the subject due to the relationship between the light source and the subject at the time of photographing. In such a case, although the photographed image is a subject that is essentially composed of colors, a large shade difference is generated in the photographed image. In this embodiment, a region in which a large shade is generated as a result of making a photographed image even though the color is originally close is a region in which pseudo contour is likely to occur in this embodiment.

  Here, there are various methods for determining a specific color, that is, a representative color in a region where a pseudo contour is likely to occur. Basically, a group of pixels belonging to a region where a pseudo contour is likely to occur in an original image. The “representative color” value is determined from (color values). Therefore, for example, when an area where pseudo contour is likely to occur is identified as a human face part, both the "lighted area" and the "shadowed area" are included in the area. In such a case, the color of the pixel belonging to either area can be extracted as a “specific color”. There are a plurality of methods for extracting which color value from the pixel group as a “representative color”. As an example, for example, “a region where a pseudo contour easily occurs” is referred to as a “skin region”. Predetermined, a color value that is likely to be a skin color is stored in advance, and a pixel having a color value closest to the stored color value indicating the skin color is designated as “representative”. It is also possible to extract it as a “natural color”.

  For example, due to human physical characteristics, it is easy to create shadows on the boundary between the contour of the face and the neck, the bent limbs, and the like. Therefore, in this embodiment, “skin area” is used as a “specific color” as a color representing these areas. Even if it is outside the skin region, a solid object or the like photographed under a specific environment such as sunlight is likely to be shaded. Therefore, if this three-dimensional object is light blue, for example, light blue is extracted as the specific color. .

  The skin part holding parameter determination unit 19 determines a skin part holding parameter according to the visibility parameter. Specifically, if the visibility parameter is α and the skin part retention parameter is δ (α), δ (α) may be expressed by, for example, Equation 19.

  FIG. 20 shows the relationship between the visibility parameter α and the skin part retention parameter δ (a) at this time. As the visibility parameter α is larger, the pseudo contour of the skin region is more conspicuous. Therefore, by reducing δ (α), the weight of the original image is increased. Any function may be used as long as δ (α) decreases as α increases.

  In the present embodiment, the visibility parameter is used as an example of the enhancement degree of the reflectance component of the original image. Further, as an example of the first holding degree that is the holding degree of the specific color portion of the original image, a skin part holding parameter is used, and a first determination unit that determines the first holding degree using the enhancement degree. As an example, a skin holding parameter determination unit 19 is provided. However, if the first holding degree is high, the weight of the original image is higher than that of the reflectance estimation image in the specific color, so the value of the skin part holding parameter becomes small.

  The image reproduction unit 17 changes processing between the extracted skin region and other than the skin region. The pixel value of the skin area is calculated by Equation 20 using the skin part holding parameter δ (α) determined by the skin part holding parameter determining unit 19. Pixel values other than the skin area are calculated by Equation 21 without using δ (α). However, 0 ≦ α ≦ 1 and 0 ≦ δ (α) ≦ 1.

Here, I (x, y) represents the pixel value of the original image, I _R (x, y) represents the pixel value of the reflectance estimated image, and I ^ (x, y) represents the pixel value of the reproduced image. Represent. Thus, the skin region is multiplied by a skin part retention parameter δ (α) of 1 or less in accordance with the visibility parameter α. Thereby, the weight of the pixel value of the reflectance estimation image is reduced, and the weight of the pixel value of the original image is increased, thereby suppressing the occurrence of the pseudo contour.

  In the present embodiment, a reproduced image is used as an example of an enhanced image in which the reflectance component of the original image is enhanced, and an image reproduction unit 17 is provided as an example of an enhanced image generation unit that generates an enhanced image. .

[Fifth Embodiment]
This embodiment is an embodiment in which skin part holding, dark part holding, and bright part holding are combined.

  FIG. 21 is a block diagram illustrating a functional configuration example of the image processing apparatus 10 according to the fifth embodiment of the present invention. As shown in the figure, the image processing apparatus 10 according to the present embodiment includes an illumination estimation unit 11, a reflectance estimation unit 12, a dark part retention parameter determination unit 14, a bright part retention parameter determination unit 15, and an image reproduction unit 17. And a skin region extraction unit 18 and a skin part retention parameter determination unit 19. Here, the illumination estimator 11 and the reflectance estimator 12 are the same as those of the first embodiment, and the dark part retention parameter determination unit 14 and the bright part retention parameter determination part 15 are the second embodiment. Since the skin area extraction unit 18 and the skin part retention parameter determination unit 19 are the same as those in the fourth embodiment, description thereof will be omitted, and only the image reproduction unit 17 will be described below.

  The image reproduction unit 17 changes processing between the extracted skin region and other than the skin region. The pixel value of the skin area is calculated by Expression 22 using the skin part holding parameter δ (α) determined by the skin part holding parameter determining unit 19. Pixel values other than the skin region are calculated by Equation 23 without using δ (α). However, 0 ≦ α ≦ 1, 0 ≦ β ≦ 1, 0 ≦ γ ≦ 1, and 0 ≦ δ (α) ≦ 1.

  Here, the difference between Formula 22 and Formula 23 will be described with reference to FIG. Since the processing other than the skin region is the same as that of the second embodiment, the function shape parameter such as a solid line is determined as the final reproduction parameter. On the other hand, since the skin region is multiplied by δ (α), the contribution of the original image is increased, and a function-shaped parameter such as a broken line is determined as a final reproduction parameter. Thereby, the pseudo contour of a skin part is also suppressed, holding a dark part and a bright part.

  FIG. 23 shows how the function shape of the parameter for the skin region in FIG. 22 changes when the visibility parameter α is increased. The function shape on the lower side of the two broken line function shapes in FIG. 23 is a function shape when α is increased. As α is higher, dark part noise, bright part whiteout, and pseudo contour of the skin part become more conspicuous, and the contribution of the original image is increased as shown by the arrows. Thereby, according to the visibility parameter (alpha), the pseudo contour of a skin part is also suppressed, holding a dark part and a bright part. Note that δ (α) may be calculated by a function that decreases as α increases as shown in Equation 19.

  In the present embodiment, a reproduced image is used as an example of an enhanced image in which the reflectance component of the original image is enhanced, and an image reproduction unit 17 is provided as an example of an enhanced image generation unit that generates an enhanced image. .

  In this embodiment, the dark part retention parameter determination unit 14 and the bright part retention parameter determination unit 15 are provided, and the image reproduction unit 17 includes the dark part retention parameter and the bright part retention parameter in addition to the visibility parameter and the skin part retention parameter. A reproduction image in which the reflectance component is emphasized is generated using, but this is not restrictive. In place of the dark part retention parameter determination unit 14 and the bright part retention parameter determination unit 15, the bright / dark part retention parameter determination unit 13 of the first embodiment is provided, and the image reproduction unit 17 uses the visibility parameter and the skin part retention parameter. In addition, a reproduction image in which the reflectance component is emphasized may be generated using the bright and dark part holding parameter. Alternatively, either one of the dark part holding parameter determination unit 14 and the bright part holding parameter determination unit 15 is provided, and the image reproduction unit 17 selects any one of the dark part holding parameter and the bright part holding parameter in addition to the visibility parameter and the skin part holding parameter. One of them may be used to generate a reproduction image in which the reflectance component is emphasized. In this sense, the dark part holding parameter and the bright part holding parameter are an example of a second holding degree that is a holding degree of at least one of the dark part and the bright part of the original image. The retention parameter determination unit 15 is an example of a second determination unit that determines the second retention degree using the original image and the enhancement degree.

[Operation of the fourth and fifth embodiments]
FIG. 24 is a flowchart showing an operation example of the image processing apparatus 10 in the fourth and fifth embodiments of the present invention. In this operation example, a case will be described in which the bright / dark part holding parameter, the dark part holding parameter, the bright part holding parameter, and the like are controlled according to the visibility parameter.

  When the original image is input, first, the illumination estimation unit 11 generates a multilayer image including a plurality of smoothed images from the original image as shown in FIGS. 2 and 4, and these multilayer images Is used to generate an illumination estimation image (step 151).

  Next, the reflectance estimation unit 12 generates a reflectance estimation image based on the original image and the illumination estimation image generated in step 151 (step 152).

  Next, the image processing apparatus 10 determines a holding parameter for at least one of the dark part and the bright part (step 153).

  However, in the fourth embodiment, step 153 is not executed.

  In addition, the “holding parameter of at least one of the dark part and the bright part” is a dark part holding parameter and a bright part holding parameter in the fifth embodiment. That is, in the fifth embodiment, the dark part holding parameter determination unit 14 determines the dark part holding parameter based on the original image and the visibility parameter, and the bright part holding parameter determination unit 15 determines the original image and the visual recognition. The bright part retention parameter is determined based on the sex parameter.

  Furthermore, “at least one holding parameter of the dark part and the bright part” is the same as that of the first embodiment instead of the dark part holding parameter determination unit 14 and the bright part holding parameter determination unit 15 in the fifth embodiment. In the modification in which the bright / dark part holding parameter determining unit 13 is provided, the bright / dark part holding parameter is used. That is, in this modification of the fifth embodiment, the light / dark part holding parameter determining unit 13 determines the light / dark part holding parameter based on the original image and the visibility parameter.

  Next, the skin region extraction unit 18 extracts a skin region from the original image (step 154).

  Next, the skin holding parameter determination unit 19 determines the skin holding parameter based on the visibility parameter (step 155).

  Finally, the image reproduction unit 17 includes the original image, the reflectance estimation image generated in step 152, the visibility parameter, and the retention parameter of at least one of the dark part and the bright part determined in step 153, For the skin region, a reproduction image is further generated based on the skin part retention parameter determined in step 155 (step 156). Here, it is assumed that the original image is used on the assumption that the reproduction image is generated using Expression 4, but when the reproduction image is generated using Expression 5, the original image is not used in Step 156. May be.

[Hardware configuration of image processing apparatus]
The image processing apparatus 10 according to the present embodiment can be realized, for example, as image processing software installed in a PC, but is typically realized as an image processing apparatus 10 that performs image reading and image formation.

  FIG. 25 is a diagram illustrating a hardware configuration example of such an image processing apparatus 10. As illustrated, the image processing apparatus 10 includes a central processing unit (CPU) 21, a random access memory (RAM) 22, a read only memory (ROM) 23, a hard disk drive (HDD) 24, and an operation panel 25. An image reading unit 26, an image forming unit 27, and a communication interface (hereinafter referred to as “communication I / F”) 28.

  The CPU 21 implements various functions to be described later by loading various programs stored in the ROM 23 and the like into the RAM 22 and executing them.

  The RAM 22 is a memory used as a working memory for the CPU 21.

  The ROM 23 is a memory that stores various programs executed by the CPU 21.

  The HDD 24 is, for example, a magnetic disk device that stores image data read by the image reading unit 26 and image data used for image formation in the image forming unit 27.

  The operation panel 25 is a touch panel that displays various types of information and receives operation inputs from the user. Here, the operation panel 25 includes a display on which various types of information are displayed, and a position detection sheet that detects a position designated by a finger, a stylus pen, or the like.

  The image reading unit 26 reads an image recorded on a recording medium such as paper. Here, the image reading unit 26 is, for example, a scanner, and a CCD system in which reflected light with respect to light irradiated from a light source to a document is reduced by a lens and received by a CCD (Charge Coupled Devices), or an LED light source is sequentially irradiated onto a document. It is preferable to use a CIS system in which reflected light with respect to the received light is received by a CIS (Contact Image Sensor).

  The image forming unit 27 forms an image on a recording medium. Here, the image forming unit 27 is, for example, a printer, and forms an image by transferring the toner attached to the photosensitive member to a recording medium to form an image, or ejecting ink onto the recording medium. An ink jet type may be used.

  The communication I / F 28 transmits / receives various information to / from other devices via the network.

  The program for realizing the present embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 11 ... Illumination estimation part, 12 ... Reflectance estimation part, 13 ... Bright / dark part holding parameter determination part, 14 ... Dark part holding parameter determination part, 15 ... Bright part holding parameter determination part, 16 ... User parameter determination , 17 ... Image reproduction unit, 18 ... Skin region extraction unit, 19 ... Skin part retention parameter determination unit

Claims (14)

A reflectance image generating means for generating a reflectance image having the reflectance component of the original image as a pixel value from the original image;
Determining means for determining a retention degree of at least one of a dark part and a bright part of the original image using the original image and the enhancement degree of the reflectance component of the original image;
Image processing comprising: an enhanced image generating unit configured to generate an enhanced image in which the reflectance component of the original image is enhanced using the reflectance image, the enhancement degree, and the retention degree. apparatus.   The image processing apparatus according to claim 1, wherein the determination unit determines a light / dark part holding degree that is a holding degree of a dark part and a bright part of the original image as the holding degree.   The determination means determines the degree of bright and dark portion retention so that the darker pixel of the original image has a higher degree of bright and dark portion retention, and the brighter pixel of the original image has a higher degree of bright and dark portion retention. The image processing apparatus according to claim 2, wherein:   The image processing apparatus according to claim 2, wherein the determination unit determines the bright and dark part holding degree so that the higher the enhancement degree is, the higher the bright and dark part holding degree is.   The determining means determines, as the holding degree, a dark part holding degree that is a holding degree of a dark part of the original image and a bright part holding degree that is a holding degree of a bright part of the original image. Item 8. The image processing apparatus according to Item 1.   The determining means determines the dark portion retention degree so that the darker pixel of the original image has a higher dark portion retention degree, and the brighter pixel of the original image has a higher bright portion retention degree. The image processing apparatus according to claim 5, wherein the degree of bright portion retention is determined.   The determination means determines the dark part retention degree so that the dark part retention degree increases as the enhancement degree increases, and the bright part retention degree increases as the enhancement degree increases. The image processing apparatus according to claim 5, wherein a part holding degree is determined.   The image processing apparatus according to claim 1, further comprising a setting unit that sets the degree of emphasis previously associated with an operation performed by an operator on the screen. A reflectance image generating means for generating a reflectance image having the reflectance component of the original image as a pixel value from the original image;
A first determination for determining a first holding degree, which is a holding degree of a specific color portion predetermined as a color for generating a pseudo contour of the original image, using the enhancement degree of the reflectance component of the original image. Means,
Emphasized image generating means for generating an enhanced image in which the reflectance component of the original image is enhanced using the reflectance image, the enhancement degree, and the first retention degree. An image processing apparatus. An extraction means for extracting the specific color portion from the original image;
The enhanced image generating means emphasizes the specific color portion of the original image using the first retention degree, and uses the first retention degree for portions other than the specific color portion of the original image. The image processing apparatus according to claim 9, wherein the emphasized image that is not emphasized is generated.   The image processing apparatus according to claim 9, wherein the first determination unit determines the first holding degree so that the higher the degree of emphasis is, the higher the first holding degree is. . Using the original image and the enhancement degree, further comprising a second determining means for determining a second holding degree that is a holding degree of at least one of a dark part and a bright part of the original image;
The image processing apparatus according to claim 9, wherein the enhanced image generation unit generates the enhanced image by further using the second holding degree. On the computer,
A function to generate a reflectance image having the reflectance component of the original image as a pixel value from the original image;
Using the original image and the enhancement degree of the reflectance component of the original image, a function of determining the retention degree of at least one of the dark part and the bright part of the original image,
A program for realizing a function of generating an enhanced image in which the reflectance component of the original image is enhanced using the reflectance image, the enhancement degree, and the retention degree. On the computer,
A function to generate a reflectance image having the reflectance component of the original image as a pixel value from the original image;
A function of determining a first holding degree that is a holding degree of a portion of a specific color that is predetermined as a color that generates a pseudo contour of the original image, using the enhancement degree of the reflectance component of the original image;
A program for realizing a function of generating an enhanced image in which a reflectance component of the original image is enhanced using the reflectance image, the enhancement degree, and the first retention degree.