JP2019169910A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

An object of the present invention is to provide a user with a highly reliable color measurement result from which the influence of noise has been removed. An apparatus for colorimetry, comprising: an acquisition unit for acquiring a captured image of an object; and a reference value setting unit for setting at least one color signal value representing a luminance component and a chromaticity component as a reference value. And color conversion means for converting a color signal value of a pixel constituting the captured image into a color signal value representing a luminance component and a chromaticity component; and Color difference information generation means for calculating a color difference from the reference value for a color signal value representing a luminance component and a chromaticity component, and generating color difference information, wherein the calculated color difference is When the difference is equal to or less than a predetermined threshold, the color difference is replaced with a constant value. [Selection diagram] FIG.

Description

  The present invention relates to an image processing technique for evaluating a color difference in an image.

  In recent years, measurement of an object using a digital imaging device such as a digital still camera or a digital video camera has been widely performed. As an example, there is a color measurement of a product, for example, there is a technique of measuring surface chromaticity such as coating unevenness by measuring the chromaticity of an object with a digital imaging device (Patent Document 1).

Japanese Patent Laid-Open No. 11-108759

  In the colorimetry technique as described above, when it is desired to measure the target article with high resolution, the light receiving area per pixel of the image sensor becomes small, so the influence of noise on the colorimetric value increases. In addition, even in a situation where shooting is unavoidable, such as when the shooting environment is dark, the colorimetric values are strongly influenced by noise. The colorimetric results obtained in these cases are a mixture of the characteristics of the subject itself and noise, but it is difficult for the user to determine the result.

An apparatus according to the present invention includes an acquisition unit that acquires a captured image of an object, a reference value setting unit that sets at least one color signal value representing a luminance component and a chromaticity component as a reference value, and the captured image. Color conversion means for converting color signal values of the constituent pixels into color signal values representing luminance components and chromaticity components, and the luminance and chromaticity components of the pixels constituting the captured image obtained by the conversion A color difference information generating unit that calculates a color difference from the reference value with respect to a color signal value that represents color difference information, and the color difference information generating unit has the calculated color difference equal to or less than a predetermined threshold value. The color difference is replaced with a specified value.

  According to the present invention, it is possible to provide a user with a highly reliable color measurement result from which the influence of noise has been removed.

Diagram showing an example of system configuration The figure which shows an example of the hardware constitutions of an image processing apparatus Functional block diagram showing the software configuration of the image processing apparatus The flowchart which shows the flow of the process which produces | generates the color difference information based on Embodiment 1. FIG. The figure which shows an example of UI screen for color difference map production | generation Diagram showing how the reference value is specified Flow chart showing details of color difference map generation processing (A) is a figure which shows an example of the one-dimensional color difference map by a conventional method, (b) is a figure which shows an example of the one-dimensional color difference map obtained by this embodiment. (A) is a figure which shows an example of the two-dimensional color difference map which showed the color difference in the dark part of a checkered pattern, (b) is a corresponding figure after color difference replacement. The flowchart which shows the detail of the threshold value setting process based on Embodiment 2. (A) And (b) is a figure which shows an example of the table as noise amount data. Flow chart showing details of threshold calculation processing The flowchart which shows the detail of the threshold value setting process based on Embodiment 3. (A) is a figure which shows an example of the histogram as noise distribution amount data, (b) is a figure which shows an example of a noise area. Diagram showing how two reference values are specified The flowchart which shows the detail of the color difference map production | generation process based on Embodiment 4. FIG. (A) And (b) is a figure which shows the one-dimensional color difference map before color difference replacement, (c) is a figure which shows the one-dimensional color map after color difference replacement. The figure which shows an example of UI screen for line profile generation based on Embodiment 5. 10 is a flowchart showing a flow of color difference information generation processing according to the fifth embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and all the combinations of features described in the present embodiment are not necessarily essential to the solution means of the present invention. In addition, about the same structure, the same code | symbol is attached | subjected and demonstrated.

Embodiment 1

  FIG. 1 is a diagram showing an example of a system configuration to which the present invention can be applied. The image processing apparatus 100 is, for example, a general PC (personal computer), acquires data of a captured image of the chart 120 as a subject from the digital imaging apparatus 110 together with imaging parameters such as ISO sensitivity at the time of imaging, Information representing the color difference in the captured image is generated.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 100. The image processing apparatus 100 includes a CPU 101, a RAM 102, an HDD 103, a general-purpose interface (I / F) 104, a monitor 108, and a main bus 109. The general-purpose I / F 104 connects a digital imaging device 110, an input device 106 such as a mouse and a keyboard, and an external memory 107 such as a memory card to a main bus 109. In the present embodiment, the color difference with respect to one reference value designated by the user is calculated for each pixel in the captured image of the chart 120, and the color difference equal to or less than a specified threshold is replaced with a constant value, thereby eliminating the influence of noise. Generate a map. FIG. 3 is a functional block diagram illustrating a software configuration of the image processing apparatus 100 that enables generation of such a color difference map, and includes an image input unit 201, a reference value setting unit 202, a threshold value setting unit 203, and a color difference map generation unit. 204 and a display control unit 206. The color difference map generation unit 204 further includes a color conversion unit 205, a color difference calculation unit 206, and a color difference replacement unit 207. These units are realized by the CPU 101 developing a predetermined program stored in the HDD 103 in the RAM 102 and executing the program. In each embodiment described below, a monochrome checkered chart 120 is used as an object to be measured. However, the checkered chart is for convenience in explaining the present embodiment, and an object having a three-dimensional shape (for example, an industrial product such as an automobile) may be used as the actual colorimetric object. is assumed.

  FIG. 4 is a flowchart showing the flow of color difference information generation processing according to this embodiment. This color difference information generation processing is executed based on, for example, a user operation via the UI screen shown in FIG. Hereinafter, description will be made along the flow of FIG. In the following description, the symbol “S” represents a step.

  In S401, captured image data of the chart 120 as a subject is acquired. When the user presses the “image input” button 501 in the UI screen 500 shown in FIG. 5, a sub window (not shown) for inputting the address of the storage location of the photographed image data is displayed. A photographed image is designated and an “OK” button 505 is pressed. Then, the captured image data to be processed is read and displayed in the image display area 504 in the UI screen 500. Now, in the image display area 504, an image obtained by photographing the checkered chart 120 is displayed. Note that the captured image data is read from the digital imaging device 110 or the external memory 107. At this time, the input captured image data may be temporarily stored in a storage device such as the HDD 103, and read out and acquired.

Next, in S402, a color signal value serving as a reference in calculating a color difference in the captured image acquired in S401, specifically, a color signal value (hereinafter referred to as “reference”, which is specified by a luminance component and a chromaticity component). Value)) is set. After the user presses the “reference value designation” button 502 in the UI screen 500 shown in FIG. 5, the user designates an arbitrary position by moving the pointer in the image display area 504 and the like, and the “OK” button 505 is designated. When is pressed, the color signal value at the designated position is set as the reference value. In general, since an image captured by the digital imaging device 110 is expressed by RGB values of the RGB color system, for example, by converting this into Lab values of the CIELAB color system, the luminance component and the chromaticity component are used. A reference value is set. In the present embodiment, the RGB value at the coordinate point at the position designated by the user is converted into the Lab value, and the reference value is set. The reference value information set in this way is stored in the RAM 102 or the like in association with the coordinate point information. FIG. 6 is a diagram illustrating how the reference value is designated. In this example, the position designated by the user is indicated by a cross, and data of Lab values (L ₀ , a ₀ , b ₀ ) at the coordinate point (x ₀ , y ₀ ) of the cross on the captured image is stored in the RAM 102. Saved in.

  In S403, a color difference map is generated based on the reference value set in S402. When the user presses a “color difference calculation” button 503 in the UI screen 500 shown in FIG. 5, generation of a color difference map is started. FIG. 7 is a flowchart showing details of the color difference map generation processing. Hereinafter, it will be described in detail along the flow of FIG.

  In S701, a threshold th for removing a noise component is set. In the present embodiment, a predetermined value (for example, color difference ΔE = 1.0) prepared in advance is read from the HDD 103 and set as the threshold th. At this time, a plurality of values may be prepared so that different thresholds th are set according to the reference value set in S402. For example, the threshold value is increased when the reference value is black, and the threshold value is decreased when the reference value is white.

In S702, a color conversion process is performed on the captured image acquired in S401. In the present embodiment, conversion is performed from a RGB color space depending on the digital imaging device 110 to a Lab color space that is a uniform color space by a conversion matrix. In the acquired photographed image, RGB values of 8 bits of the x pixel in the horizontal direction and the y pixel in the vertical direction from the upper left are respectively R (x, y), G (x, y), and B (x, y). Specific color conversion processing will be described below. First, each RGB value is normalized to a value between 0 and 1, and then multiplied by the γ characteristic of the digital imaging device 105 used to capture the input image to obtain luminance linear color data R ′, G ′, and B ′. . R ′, G ′, and B ′ are represented by the following formulas (1) to (3), respectively.
R ′ (x, y) = {R (x, y) / 255} ^ γ Expression (1)
G ′ (x, y) = {G (x, y) / 255} ^ γ Expression (2)
B ′ (x, y) = {B (x, y) / 255} ^ γ Expression (3)

  Next, the color data indicated by R′G′B ′ is converted into tristimulus values X, Y, and Z of the CIEXYZ color system. The XYZ value indicated by the CIEXYZ color system is a color signal value that is independent of the imaging device and imaging conditions, and is obtained by, for example, convolving a color matching function with a spectral radiance value measured by a spectral radiance meter. be able to. The R′G′B ′ / XYZ conversion matrix is, for example, a 3 × 3 matrix. The matrix product of the 3 × 3 matrix and the 3 × 1 matrix corresponding to the R′G′B ′ value indicates a 3 × 1 matrix corresponding to the XYZ value. Therefore, R′G′B ′ values can be converted into XYZ values by calculating the matrix product. Specifically, it is converted into X, Y, and Z by the following formulas (4) to (6).

X (x, y) = m11 × R ′ (x, y) + m12 × G ′ (x, y) + m13 × B ′ (x, y) (4)
Y (x, y) = m21 × R ′ (x, y) + m22 × G ′ (x, y) + m23 × B ′ (x, y) (5)
Z (x, y) = m31 × R ′ (x, y) + m32 × G ′ (x, y) + m33 × B ′ (x, y) (6)
In the above formulas (4) to (6), m11 to m33 are elements of the R′G′B ′ / XYZ conversion matrix M. For example, ITU-T BT. The color conversion matrix M defined in 709 is expressed by the following equation (7).

The color conversion matrix M may be obtained for each digital imaging device 110 and each photographing condition from the photographing result of the chart, instead of the value of the above equation (7). Further, the XYZ value is converted to the Lab value using the following formulas (8) to (10).
L (x, y) = 116 (Y (x, y) / Yw) ^ (1/3) -16 Expression (8)
a (x, y) = 500 ((X (x, y) / Xw) ^ (1/3)-(Y (x, y) / Yw) ^ (1/3)) (9)
b (x, y) = 200 ((Y (x, y) / Yw) ^ (1/3)-(Z (x, y) / Zw) ^ (1/3)) (10)

In addition, in said Formula (8)-Formula (10), Xw, Yw, Zw is a tristimulus value of a reference | standard white surface, for example in the definition of CIE (international lighting association), respectively, respectively the following formula | equation (11)- It is represented by Formula (13).
Xw = 95.039 / 100.0 Formula (11)
Yw = 1.0 Formula (12)
Zw = 108.880 / 100.0 Formula (13)

  Although the color conversion from the RGB color space to the Lab color space has been described here, the color conversion may be performed to a visual uniform color space (for example, YUV) other than the Lab color space. Further, instead of the conversion matrix, color conversion may be performed using, for example, a three-dimensional LUT.

In S703, a color difference is calculated for each pixel of the captured image acquired in S401. The Lab value as the reference value set in S402 is (L ₀ , a ₀ , b ₀ ), and the Lab value at any pixel position (x, y) in the captured image is (L _{(x, y)} , a _{( x, y)} , b _{(x, y)} ), the color difference ΔE at each pixel position is expressed by the following equation (14).

  In S704, a process of replacing the value of the color difference ΔE (x, y) for each pixel calculated in S703 using the threshold th set in S701 (color difference replacement process) is executed. Specifically, when the value of ΔE (x, y) is equal to or smaller than the threshold th, the value of ΔE (x, y) is replaced with a specified value const. When the value of ΔE (x, y) is larger than the threshold th, the color difference of the pixel remains unchanged. In this case, the specified value const is an arbitrary value that is equal to or smaller than the above-described threshold th, and is, for example, a value such as half of the threshold th or 0. In addition, it is assumed that the specified value const is read out from the HDD 103 and used in advance, but is not limited to this. For example, a “specified value” button (not shown) is separately provided in the UI screen 500 shown in FIG. 5, and a subwindow in which a specified value can be input is displayed when the button is pressed. The specified value const may be designated. When the color difference replacement process is completed for all the pixels in the captured image, a color difference map that visualizes the color difference value after the replacement is generated. FIG. 8A is a conventional one-dimensional color difference map showing the color difference when the color difference replacement process on the x0 line shown in FIG. 6 is not performed. Now, since the reference value is specified in the dark part of the checkered pattern, the color difference in the section corresponding to the dark part (black square) of the checkered pattern is small, and the color difference in the section corresponding to the bright part (white square) is large. . The jagged portion indicates that the color difference value is not constant due to the influence of noise. FIG. 9A is a two-dimensional color difference map showing the color difference in the dark portion (one black square) at this time. The smaller the color difference from the reference value, the closer to black, and the larger the color difference, the closer to white. . In other words, a portion close to gray in black represents noise. On the other hand, FIG. 8B is a one-dimensional color difference map generated in the present embodiment, showing the color difference when the color difference replacement process on the x0 line shown in FIG. 6 is performed. As a result of replacing the portion below the threshold among the color differences in the checkered dark part (black square) with the specified value const = 0, there is no jaggedness in the section corresponding to the dark part, and the color difference is constant. FIG. 9B is a two-dimensional color difference map showing the color difference after color replacement corresponding to FIG. 9A. Since there is no variation in color difference from the reference value, uniform black with no gray portion. It has become. In this way, the result of the color difference replacement process for each pixel of the captured image is displayed on the monitor 108. For example, when the subject is a checkered chart 120, the dark portion (black square) portion is displayed as a noise-free image as shown in FIG. 9B.

  The above is the content of the color difference information generation process according to the present embodiment. In the present embodiment, the reference value is set using the captured image, but the present invention is not limited to this. Instead of the photographed image, an image obtained by photographing the reference subject may be used, or the user interface may be configured so that the user can directly specify a specific color signal value. In this embodiment, the Lab color space is used as the color space after color conversion, and ΔE is used as the color difference, but the present invention is not limited to this. For example, when paying attention to the difference between physical values of colors as a color space after color conversion, an XYZ color space may be used. Alternatively, ΔE00 having higher visual consistency than ΔE may be used.

  As described above, according to the present embodiment, a color difference map from which noise is removed can be provided to the user.

Embodiment 2

  In the first embodiment, the threshold applied in the color difference replacement process is a fixed value prepared in advance. Next, a mode in which a more appropriate threshold value is dynamically set based on the noise characteristics of the captured image will be described as a second embodiment. Since the basic flow of color difference map generation is the same as the flow of FIG. 4 of the first embodiment, the following description will focus on threshold value setting processing that is a difference.

  FIG. 10 is a flowchart showing details of the threshold setting process of the present embodiment corresponding to S401 in the flow of FIG. Hereinafter, it will be described in detail along the flow of FIG. 10 with reference to the flow of FIG.

In S1001, information on the reference value set in S402, more specifically, the RGB value before color conversion at the pixel position specified by the user in the captured image is acquired. Hereinafter, the RGB values acquired in this step are represented by R ₀ , B ₀ , and G ₀ , respectively.

In S1002, data indicating the noise characteristics of the captured image acquired in S401, specifically, data relating to the amount of noise during shooting is acquired. FIG. 11A is a table associating ISO sensitivity and noise standard deviation as an example of noise amount data. In the table shown in FIG. 11A, the standard deviation value of noise corresponding to the ISO sensitivity value is stored for each RGB plane. Hereinafter, the standard deviations of RGB are expressed as S _R , S _G , and S _B. In the present embodiment, the noise standard deviation is used as the noise amount, but the present invention is not limited to this, and may be another index that can specify the noise amount. Such noise amount data is prepared in advance and stored in the HDD 103, for example, and is read and acquired.

  In S1003, based on the reference value (RGB value) acquired in S1001 and the noise amount acquired in S1002, a process for calculating a threshold (threshold calculation process) is executed. FIG. 12 is a flowchart showing details of the threshold value calculation process. Hereinafter, a description will be given along the flow of FIG.

In S1201, the noise interval is set for each RGB plane with reference to the noise amount data acquired in S1002. The noise section in each plane is expressed by the following equations (15) to (17).
R noise interval: [L _R = R ₀ −k × S _R , U _R = R ₀ −k × S _R ] (15)
Noise section of G: [L _G = G ₀ −k × S _G , U _G = G ₀ −k × S _G ] (16)
Noise section of B: [L _B = B ₀ −k × S _B , U _B = B ₀ −k × S _B ] Expression (17)

In the above formulas (15) to (17), k represents a constant that sets the size of the noise interval. L _R , L _G , and L _B each represent a lower limit value of the RGB noise interval, and U _R , U _G , and U _B each represent an upper limit value of the RGB noise interval. For example, when the constant k = 2 is set, 95% of the noise can be excluded from the color difference map.

In S1202, the noise section in the Lab color space is obtained by performing the color conversion processing described in S702 of the flow of FIG. 7 described above on the noise section of each RGB plane set in S1201. Now, let the result of color conversion for the lower limit values (L _R , L _G , L _B ) of the RGB noise interval be (L _L , a _L , b _L ), and the upper limit value (U _R , The result of color conversion for U _G , U _B ) is defined as (L _U , a _U , b _U ). At this time, the noise section in the Lab color space is expressed by the following equations (18) to (20).
L noise interval: [L _L , L _U ] (18)
Noise interval of a: [a _L , a _U ] (19)
Noise interval of b: [b _L , b _U ] (20)

In S1203, a threshold value is calculated based on the noise section in the Lab color space obtained in S1202. For example, the color difference ΔE _L between the lower limit value (L _L , a _L , b _L ) and the reference value in the Lab color space, and the color difference between the upper limit value (L _U , a _U , b _U ) and the reference value in the Lab color space. The larger value of ΔE _U is set as a threshold th. This is expressed by the following equation (21).

Here, the maximum value of ΔE _L and ΔE _U is taken as the threshold value th, but is not limited to this. For example, an average value of ΔE _L and ΔE _U , a linear sum thereof, or a minimum value may be taken. In the following, γ = 2.2 as the γ characteristic of the digital imaging device 110, the constant k = 2 in the above formulas (15) to (17), and (R, G, B) = (128) as the reference value. , 128, 128), a specific example of the threshold value obtained under the condition that is set will be described for each case.

(When ISO sensitivity is 100)
In this case, the noise standard deviation (S _R , S _G , S _B ) = (1.0, 1.3, 1.2) from the table of FIG. Then, the RGB noise interval is as follows from the above-described equations (15) to (17).
R noise interval: [126, 130]
G noise interval: [125.4, 130.6]
B noise interval: [125.6, 130.4]

The Lab noise interval obtained by color conversion processing is as follows from the above-described equations (18) to (20).
L noise interval: [43.5, 45.2]
Noise interval of a: [−4.7, −5.3]
Noise interval of b: [−8.6, −8.7]
Then, the threshold th = 0.92 is obtained from the above equation (21).

(When ISO sensitivity is 25600)
In this case, from the table of FIG. 11A, the standard deviation of noise (S _R , S _G , S _B ) = (14.2, 15.2, 13.2). Then, the RGB noise interval is as follows from the above-described equations (15) to (17).
R noise interval: [99.6, 156.4]
G noise interval: [97.6, 158.4]
Noise interval of B: [101.6, 154.4]

The Lab noise interval obtained by color conversion processing is as follows from the above-described equations (18) to (20).
L noise interval: [33.7, 54.3]
G noise interval: [−2.5, −7.4]
B noise interval: [-12.5, -5.4]

  Then, the threshold th = 11.6 is obtained from the above equation (21). The threshold value th thus determined is set as the threshold value th used in the color difference replacement process.

  The above is the content of the threshold setting process according to the present embodiment. In the present embodiment, the noise amount data in the device-dependent RGB color space is stored and converted into the Lab color space, and the threshold value is obtained. However, the present invention is not limited to this. For example, noise amount data in the Lab color space may be directly held, and a noise section in the Lab color space may be derived from the noise amount data. In this case, color conversion processing is not necessary.

<Modification>
In the above-described example, the data associated with the ISO sensitivity and the noise standard deviation is acquired as data representing the noise amount of the captured image. However, the amount of noise affects not only the shooting conditions but also the gradation of the subject. Therefore, a table (table in which the amount of noise is associated with both the ISO sensitivity and the gradation value) taking into account the gradation of the subject as shown in FIG. 11B is prepared. An amount may be obtained. Note that only the standard deviation of noise corresponding to some combinations of ISO sensitivity and gradation value is stored in the table and stored, for example, by linear interpolation according to the RGB value set as the reference value. The amount of noise may be determined. As a result, the threshold th takes into account the luminance dependence of camera noise, and a more suitable color difference map can be obtained.

  As described above, according to the present embodiment, a threshold corresponding to the amount of noise at the time of shooting is set, so that it is possible to obtain a more suitable color difference map that excludes color differences caused by noise.

Embodiment 3

  In the second embodiment, the aspect in which the threshold value to be applied in the color difference replacement process is dynamically determined based on the data regarding the noise amount at the time of shooting has been described. Next, an embodiment in which the threshold value is determined with reference to data indicating the noise distribution status of the photographed image so as to cope with a case where the noise distribution is not a normal distribution but has a more complicated shape will be described as a third embodiment. explain. In addition, suppose that description about the content which is common in Embodiment 1 and 2 is abbreviate | omitted, and suppose that it sets centering on the setting process of the threshold value which is a difference below.

  FIG. 13 is a flowchart showing details of the threshold setting process of the present embodiment, corresponding to the flow of FIG. 10 of the second embodiment. Hereinafter, it will be described in detail along the flow of FIG. 13 with reference to the flow of FIG. 4 which is also common to the present embodiment.

  In S1301, information on the reference value set in S402 (RGB value before color conversion at the pixel position specified by the user in the captured image) is acquired. In subsequent S1302, noise distribution data is acquired based on the reference value acquired in S1301. FIG. 14A is a noise histogram in the R plane as an example of noise distribution amount data, and shows the frequency of noise for each gradation value. In the case of the histogram in FIG. 14A, noise is distributed asymmetrically with respect to the reference value. Since it is only necessary to know the noise distribution status, for example, a mathematical expression approximating the noise distribution may be used as the noise distribution data instead of the histogram. Such noise distribution data is prepared in advance and stored in the HDD 103, for example, and is read and acquired.

  In S1303, a threshold value calculation process is executed based on the reference value acquired in S1301 and the noise distribution data acquired in S1302. Since the flow is the same as the threshold value calculation process in the second embodiment, description will be made along the flow of FIG.

In S1201, the noise interval is set for each RGB plane with reference to the noise distribution data acquired in S1302. The noise intervals [L _R , U _R ], [L _G , U _G ], and [L _B , U _B ] for each of RGB are determined so that the sum of frequencies in the noise interval becomes a constant value. Specifically, it is obtained by the following equation (22).

  In the above equation (22), c represents a variable for setting the frequency of the section. Furthermore, histR (t), histG (t), and histB (t) represent RGB noise distributions in the gradation t, and satisfy the relationship of the following equation (23).

FIG. 14B shows the noise section [L _R , U _R ] of the R plane when the value of c in the above equation (22) is 0.025 in the state of the noise distribution shown in FIG. ing. In FIG. 14 (b), L _R is the interval [0, L _R] as noise frequency is the entire 2.5% in, U _R is the noise frequency in the interval [UR, 255] of the total 2.5 % Is set to be 95% of the entire noise.

  Subsequent S1202 and S1203 are the same as those described in the second embodiment, and thus description thereof is omitted. In the present embodiment, the noise distribution data in the device-dependent RGB color space is held, and the threshold value is obtained by converting it into the Lab color space. However, the present invention is not limited to this. For example, noise distribution data in the Lab color space may be directly held, and a noise section in the Lab color space may be derived from the noise distribution data. In this case, color conversion processing is not necessary.

  As described above, according to the present embodiment, even when the noise distribution shape is complicated, it is possible to dynamically determine the appropriate threshold th in the color difference replacement process, and to obtain a more suitable color difference map. .

Embodiment 4

  In the embodiments so far, the reference value to be set is one, but the number of reference values that can be set in each embodiment is not limited to one. Next, an embodiment in which a plurality of reference values are set based on the first embodiment will be described as a fourth embodiment. In addition, description is abbreviate | omitted or simplified about the part which is common in Embodiment 1, and it shall demonstrate below centering on difference.

  Since the basic flow when generating the color difference map as the color difference information is the same as the flow of FIG. 4 of the first embodiment, the following description will be made along the flow of FIG.

In S401, captured image data to be subjected to color difference map generation processing is acquired. In S402, a plurality of reference values for calculating the color difference in the captured image acquired in S401 are set. The method of setting the reference value is basically the same as in the first embodiment. For example, after the user presses the “reference value designation” button 502 on the UI screen 500 in FIG. 5 described above, a plurality of reference values are designated by popping up a sub window for designating the number of reference values to be set. You can do it. Here, the following description will be given by taking as an example the case where two reference values are set. For example, as shown in FIG. 15, the user designates two points represented by x marks in the image display area 504. Thereby, two Lab values ((L ₀ , a ₀ ) corresponding to the designated point 1 of the coordinates (x0, y0) and the designated point 2 of the coordinates (x0, y1) indicated by the two X marks on the photographed image. , B ₀ ) and (L ₁ , a ₁ , b ₁ )) are stored in the RAM 102.

  In step S403, a color difference map is generated based on the two specified reference values. FIG. 16 is a flowchart showing details of the color difference map generation processing when a plurality of reference values are set according to the present embodiment. This flow corresponds to the flow of FIG. 7 of the first embodiment. Hereinafter, it demonstrates along the flow of FIG.

  In S1601, similarly to S701, a threshold th (for example, color difference ΔE = 1.0) for removing a noise component is set. In the subsequent step 1602, similarly to S702, a color conversion process is performed on the captured image acquired in S401, and the RGB color space is converted to the Lab color space as in the first embodiment.

  In S1603, one reference value to be noticed is determined from all the reference values set in S402. In subsequent S1604, as in S703, the color difference ΔE (x, y) in each pixel of the captured image acquired in S401 is calculated using the above-described equation (14).

  In S1605, a color difference replacement process is executed in which the value of the color difference ΔE (x, y) from the reference value of interest calculated for each pixel calculated in S1604 is replaced with the specified value const using the threshold th set in S1601. At this time, the value of the specified value const may be changed for each reference value or may be the same. In the present embodiment, the first reference value corresponding to the designated point 1 is set to the specified value const = 0, and the second reference value corresponding to the designated point 2 is set to the first reference value and the second reference value. The difference value is defined as a specified value const.

  In S1606, it is determined whether or not the color difference replacement process has been completed for all the reference values. If there is an unprocessed reference value, the process returns to S1603 and the process is continued. On the other hand, if the color difference replacement process has been completed for all the reference values, this process ends. Here, it demonstrates using a specific example. FIG. 17A is a one-dimensional color difference map in which the horizontal axis is the x0 line in FIG. 15 and the vertical axis is the color difference ΔE (x, y) when the target reference value is the first reference value. is there. FIG. 17B is a one-dimensional color difference in which the x0 line in FIG. 15 is taken on the horizontal axis and the color difference ΔE (x, y) is taken on the vertical axis when the target reference value is the second reference value. It is a map. FIG. 17C is a one-dimensional color difference map after the color difference is replaced with respect to the color difference from the first reference value. In FIG. 17A, the pixels below the threshold th (black square pixels) are replaced with the specified value const (= 0), and the pixels below the threshold (white square pixels) in FIG. The difference value between the reference value and the second reference value is replaced. Thereby, noise is removed, and a color difference map in which both the bright and dark portions of the checkered pattern are flattened is obtained. Compared with the first embodiment, noise in the bright part can also be removed, so that a more suitable color difference map is obtained.

  As described above, according to the present embodiment, it is possible to provide a user with a color difference map obtained by removing noise components with respect to a plurality of reference values with respect to a photographing result including camera noise.

Embodiment 5

  In the first to fourth embodiments, it is assumed that a two-dimensional color difference map is generated and displayed by performing each of color difference calculation and color difference replacement for all pixels constituting a captured image. Next, an embodiment in which a color difference calculation process and a color difference replacement process are performed on a specific line in a captured image to generate and display a one-dimensional line profile as color difference information will be described as a fifth embodiment. In addition, description is abbreviate | omitted or simplified about the part which is common in other embodiment, and suppose that it demonstrates centering on difference below.

  FIG. 18 is an example of a UI screen used by the user in this embodiment. FIG. 19 is a flowchart showing the flow of line profile generation processing according to the present embodiment, corresponding to the flow of FIG. 4 of the first embodiment. Hereinafter, it demonstrates along the flow of FIG.

  In S1901, similarly to S401, captured image data to be subjected to line profile generation processing is acquired. When the user presses an “image input” button 1801 in the UI screen 1800 shown in FIG. 18, a sub window (not shown) for inputting an address indicating the storage location of the captured image data is displayed. The photographed image is designated and an “OK” button 1806 is pressed. Then, the captured image data to be processed is read and displayed in the image display area 1805 in the UI screen 1800.

  Next, in S1902, a reference value for calculating a color difference in the captured image acquired in S1901 is set as in S402. When the user presses a “reference value designation” button 1802 in the UI screen 1800 and designates an arbitrary position by moving a pointer in the image display area 1805 or the like and presses an “OK” button 1806, The color value at the designated position is set as the reference value.

  In step S1903, the specific line in the captured image acquired in step S1901 is set based on a user operation as a line for calculating the color difference. When the user presses the “color difference calculation line setting” button 1803 in the UI screen 1800, specifies two points with the mouse or the like in the image display area 1805, and presses the “OK” button 1806, the specified 2 A line connecting points is set as a target line for color difference calculation. A double arrow 1810 in the image display area 1805 indicates the color difference calculation line set in this way. Note that the line to be specified does not have to be horizontal with respect to the captured image, and may be vertical or oblique.

  In S1904, the color difference calculation process is executed based on the reference value set in S1902 for the color difference calculation line set in S1903. When the user presses a “color difference calculation” button 1804 in the UI screen 1800 shown in FIG. 18, generation of a line profile for the set color difference calculation line is started. The content of the color difference calculation process here is basically the same as the flow of FIG. 7 of the first embodiment. However, the processing target is limited to the color difference calculation line. In this way, a one-dimensional color difference map (= line profile) as shown in FIG. 8B showing the color difference on a specific line in the captured image is obtained. Then, the line profile is displayed on the monitor 108 as a color measurement result.

  The above is the content of the line profile generation process according to the present embodiment. As described above, the subject of the present invention may be the entire captured image or a part thereof.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 image processing apparatus 201 reference value setting unit 204 color difference map generating unit 205 color converting unit 206 color difference calculating unit 207 color difference replacing unit

Claims (13)

Acquisition means for acquiring a captured image of the object;
A reference value setting means for setting at least one color signal value representing a luminance component and a chromaticity component as a reference value;
Color conversion means for converting color signal values of pixels constituting the captured image into color signal values representing luminance components and chromaticity components;
Color difference information generating means for calculating a color difference from the reference value for a color signal value representing the luminance component and chromaticity component of the pixels constituting the captured image obtained by the conversion, and generating color difference information;
With
The color difference information generation unit replaces the color difference with a specified value when the calculated color difference is equal to or less than a predetermined threshold value.   The apparatus according to claim 1, wherein the predetermined threshold is a predetermined fixed value.   The apparatus according to claim 1, further comprising threshold setting means for setting the predetermined threshold based on the reference value and a noise characteristic of the captured image. The threshold setting means includes
Means for obtaining data representing the amount of noise as the noise characteristics;
Means for setting a predetermined noise interval from the acquired noise amount data;
The apparatus according to claim 3, further comprising: means for determining a threshold value based on a value representing a color difference between a lower limit value and an upper limit value in the predetermined noise section and the reference value.   5. The apparatus according to claim 4, wherein the data representing the amount of noise is data in which an ISO sensitivity is associated with a standard deviation of noise.   5. The apparatus according to claim 4, wherein the data representing the amount of noise is data in which ISO sensitivity and a gradation value of a subject are associated with a standard deviation of noise. The threshold setting means includes
Means for obtaining data representing a noise distribution as the noise characteristics;
Means for setting a predetermined noise interval from the acquired noise distribution data;
The apparatus according to claim 3, further comprising: means for determining a threshold value based on a value representing a color difference between a lower limit value and an upper limit value in the predetermined noise section and the reference value.   8. The apparatus according to claim 7, wherein the data representing the noise distribution is a histogram or an approximate expression indicating a noise distribution state.   9. The apparatus according to claim 8, wherein the predetermined noise interval is set by subtracting a certain ratio from an upper part and a lower part of the histogram.   The threshold setting means sets a maximum value, a minimum value, or a linear sum of values representing a color difference between a lower limit value and an upper limit value and the reference value in the predetermined noise section as a threshold value. The apparatus according to 4 or 7.   The apparatus according to claim 1, wherein the color difference information generation unit generates the color difference information for a part specified by a user in the captured image. Obtaining a captured image of the object;
Setting at least one color signal value representing a luminance component and a chromaticity component as a reference value;
Converting color signal values of pixels constituting the captured image into color signal values representing luminance components and chromaticity components;
Calculating a color difference from the reference value for a color signal value representing the luminance component and chromaticity component of the pixels constituting the captured image obtained by the conversion, and generating color difference information;
Including
In the step of generating the color difference information, when the calculated color difference is equal to or less than a predetermined threshold, the color difference is replaced with a constant value.   The program for operating a computer as an apparatus of any one of Claims 1 thru | or 11.

Images