JP2006287585A - Image processing apparatus, image processing method, computer program, and computer-readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that reduces an estimation error in spectral reflectance restoration processing by performing input / output image processing without performing spectral reflectance estimation processing when spectrally reproducing a subject color. To do.
An image processing apparatus for spectrally reproducing the color of a subject, patch output means for outputting a patch image for creating a lookup table, and a multiband image for capturing the patch image in a multiband manner An acquisition unit; a lookup table creation unit that creates a lookup table that associates an output signal of the patch output unit with a pixel value obtained from the multiband image; and an input multiband based on the lookup table Image conversion means for converting an image into an output signal.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus using multiband image input / output technology.

  In recent years, due to digitalization of images, images acquired by input devices capable of inputting images, such as digital cameras and scanners, are displayed on image display devices such as monitors and projectors, or images such as inkjet printers and laser printers are displayed. Opportunities for printing on output devices are increasing. Generally, in a color reproduction technique for reproducing the color of an object, an image input device acquires color information by separating an original color into three primary colors using RGB three-color filters.

  On the other hand, image display devices use RGB three-color illuminants and perform additive color mixing to reproduce the original color. Inkjet printers and laser printers use CMYK four-color color materials to perform subtractive color mixing. This reproduces the original color.

  Currently, images are input / output using various devices under various environments. For example, an image of a subject is acquired with a digital camera outdoors under sunlight, and a printed matter printed out indoors is frequently observed under a fluorescent lamp.

  On the other hand, there is an increasing demand for faithful reproduction of original colors, particularly in professional photographers, designers and other professional use sites. However, as described above, it is necessary to strictly manage color information in order to faithfully reproduce the original color under a situation where the input device / environment is different from the output device / environment.

  In particular, when the observation light source at the time of input is different from that at the time of output, for example, when an image is taken under sunlight with a color temperature of about 5000K and a printed matter is observed indoors under a light bulb with a color temperature of about 3000K, In the colorimetric color reproduction using the three primary colors, it is extremely difficult to faithfully reproduce the original color.

  This is because, in colorimetric color reproduction, tristimulus values (XYZ values or Lab values) including information on the light source are matched, so that if the light source at the time of input is different from that at the time of output, the same color This is because the relationship no longer holds.

  Therefore, in recent years, a technique called spectral color reproduction has attracted attention as a technique for faithfully reproducing original colors under various environments. Spectral color reproduction is not to reproduce the color of an object by matching the tristimulus values, but also to match the reflectance (spectral reflectance) for each wavelength of light to create an original under any light source. It is a technology that faithfully reproduces the colors of.

  For example, in Patent Document 1, a display device that uses six types of filters, acquires a 6-band image, estimates the spectral reflectance of a subject using Wiener estimation or the like, and is closest to the spectral reflectance. Are converted into signal values for each of the RGB channels.

  Also in Patent Document 2, a multiband image of a subject is acquired using a multi-channel filter, the spectral reflectance of each pixel is calculated, and converted to the RGB value of the output device.

As represented by the color conversion processing disclosed in Patent Documents 1 and 2, in the conventional spectral color reproduction technique, the spectral reflectance is estimated once from image data acquired using a multi-channel multiband camera. Then, the signal value of the output device is converted.
JP 2002-221931 A JP 2000-333186 A

  However, for example, when spectral reflectance data is sampled at an interval of 10 nm in the range of 380 nm to 730 nm, 36-dimensional data must be handled for each color, and the CPU and memory required for calculation become enormous. There was a problem. In addition, there is a problem that an estimation error always occurs in the spectral reflectance estimation processing using Wiener estimation or principal component analysis, and in the conversion processing from the spectral reflectance to an output signal.

  The present invention has been made to solve such a problem. When spectrally reproducing the color of a subject, the input / output image processing is performed without performing the spectral reflectance estimation processing. It is an object of the present invention to provide an image processing apparatus that reduces an estimation error in reflectance restoration processing.

  In order to solve the above-described problems, an image processing apparatus according to the present invention is an image processing apparatus for spectrally reproducing the color of a subject, and includes a patch output unit that outputs a patch image for creating a lookup table. Multiband image acquisition means for taking a multiband image of the patch image; lookup table creation means for creating a lookup table that associates an output signal of the patch output means with a pixel value obtained from the multiband image; Image conversion means for converting an input multiband image into an output signal based on the look-up table.

  An image processing method according to the present invention is an image processing method for spectrally reproducing the color of a subject, a patch output step for outputting a patch image for creating a look-up table; A multi-band image acquisition step for performing band shooting, a lookup table creation step for creating a lookup table for associating the output signal obtained in the patch output step with the pixel value obtained from the multi-band image, and the lookup An image conversion step of converting an input multiband image into an output signal based on the table.

  Further features of the present invention will become apparent from the best mode for carrying out the present invention and the accompanying drawings.

  According to the present invention having the above-described configuration, in performing spectral color reproduction of a subject color, input / output image processing is performed without performing spectral reflectance estimation processing, thereby performing spectral reflectance restoration processing. The estimation error can be reduced.

A. First Embodiment Hereinafter, a first embodiment according to the present invention will be described in detail with reference to the drawings.

<Image processing configuration>
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the first embodiment. DESCRIPTION OF SYMBOLS 1 is the image processing apparatus which is embodiment of this invention, 2 is a multiband camera etc., such as a multiband camera, The multiband image acquisition apparatus for acquiring the color of an object using a multi-primary color filter, 3 controls the multiband image acquisition part 2 And a multiband image input unit for inputting the acquired multiband image to the image processing apparatus 1.

  4 is a grid data creation unit for creating output signals at equal intervals Grid, and 5 is a multiband image input by the multiband image input unit 3 for calculating an average value of pixel values in a predetermined area. The average pixel value calculation unit 6 is an LUT creation unit for creating an LUT representing the correspondence between the pixel value of each band of the multiband image and the output signal.

  7 is an LUT storage unit for storing the LUT created by the LUT creation unit 6, and 8 is a LUT stored in the LUT data storage unit 7 for the multiband image input by the multiband image input unit 3. An image conversion unit 9 for converting the output signal into an output signal, and an image output unit 9 for outputting an output signal which is image data created by the image processing apparatus 1.

  Reference numeral 10 denotes an image output unit that actually outputs an image based on an output signal output from the image output unit 8 such as an inkjet printer, and 11 denotes a filter configuration mounted on the multiband image acquisition device 5 and an image. A filter / color material configuration setting unit for setting the configuration of the color material used in the output device 9, and 12 stores the filter set by the filter / color material configuration setting unit 11 and the configuration of the color material. This is a filter / color material configuration storage unit for storing.

<Overall processing in image processing apparatus>
Here, FIG. 2 is a flowchart of image input / output processing performed in the image processing apparatus 1, and FIG. 3 is an example of a user interface for performing a filter / color material configuration. Hereinafter, the process of the first embodiment according to the present invention will be described in detail.

  First, in step S201, the filter configuration installed in the multiband image acquisition device 2 and the color material configuration used in the image output device 10 are displayed on the filter / color material configuration setting unit 11 by the user in FIG. This is set using a user interface as shown and stored in the filter / color material configuration storage unit 12.

  In step S202, it is determined whether an LUT corresponding to the filter / color material configuration set in step S201 has already been created and stored in the LUT data storage unit 7. If stored, the process proceeds to step S209. If not stored, the process proceeds to step S203.

  In step S203, grid data is generated from the plurality of LUT data stored in the LUT data storage unit 7 and corresponding to the filter / color material configuration closest to the filter / color material configuration set in step S201. The filter 4 and the color material that have been changed are read into the unit 4 and the multiband image input unit 8 and stored in the filter / color material configuration storage unit 12 as changed filter information and changed color material information.

  In step S204, all combinations (hereinafter referred to as grid data) tables in which the grid data creation unit 4 changes the signal values of the primary colors used in the image output apparatus 10 at equal intervals as shown in FIG. Create

  In step S205, only the grid whose change color material output value stored in the filter / color material configuration storage unit 12 is not 0 is selected from the grid data created in step S203, and the image output unit 9 is selected. Then, as shown in FIG. 5, a patch image in which output signals of each grid are arranged in a patch shape is created and output by the image output device 10.

  In step S206, the multiband image acquisition apparatus 2 captures the patch image output in step S205. However, photographing at this time is performed using only the change filter stored in the filter / color material configuration storage unit 12.

  In step S207, the average pixel value calculation unit 5 calculates the average pixel value of each patch for each band for the patch image photographed in step S206.

  In step S208, the LUT creation unit 6 associates the grid data created by the grid data creation unit 6 with the average pixel value calculated in step S207. In the LUT data read in step S203, Only the portion relating to the changed filter or color material is updated and stored in the LUT data storage unit 7.

  In step S209, a multiband image of a subject to be spectrally reproduced in color is acquired by the multiband image acquisition apparatus 2, and is read into the image processing apparatus 1 by the multiband image input unit 3.

  In step S210, based on the LUT data stored in the LUT data storage unit 7, the pixel value of the multiband image read in step S209 is converted into an output signal by the image conversion unit 8. In step S <b> 211, the image data converted in step S <b> 207 is output by the image output unit 9 and output from the image output device 10.

<Effect of the first embodiment>
As described above, according to the first embodiment, in the spectral image processing for reproducing the spectral reflectance of the subject, the input multiband image can be output without being converted into the spectral reflectance. Therefore, since no restoration error is caused when restoring the spectral reflectance at high speed, it is possible to perform spectral color reproduction by the output device with high accuracy.

  Further, even when the filter configuration of the multiband camera to be used or the color material used for output is changed, the entire image is not updated by changing only the portion related to the changed portion instead of updating the entire LUT. The LUT can be updated only by acquiring images.

<Another Form of Filter / Color Material Configuration Setting Method>
In the first embodiment, the filter / color material configuration is manually set by the user using the user interface as shown in FIG. 3, but for example, the filter / color material in FIG. The configuration setting unit 11 and the multiband image acquisition device 2 or the image output device 10 may be connected on-line to automatically read the mounted filter / color material configuration.

B. Second Embodiment Hereinafter, a second embodiment according to the present invention will be described in detail with reference to the drawings.

<Configuration of image processing apparatus>
FIG. 6 is a block diagram illustrating a configuration of an image processing apparatus according to the second embodiment. Reference numeral 601 denotes an image processing apparatus according to an embodiment of the present invention, 602 denotes a multiband image acquisition apparatus for acquiring a subject color using a multi-primary color filter, such as a multiband camera, and 603 controls a multiband image acquisition unit 602. And a multiband image input unit for inputting the acquired multiband image to the image processing apparatus 601.

  Reference numeral 604 denotes a grid data generation unit for generating output signals at equal intervals Grid, and reference numeral 605 denotes an average value of pixel values in a predetermined region in the multiband image input by the multiband image input unit 603. An average pixel value calculation unit 606 is an LUT creation unit for creating an LUT representing a correspondence relationship between a pixel value of each band of a multiband image and an output signal.

  Reference numeral 607 denotes an LUT storage unit for storing the LUT created by the LUT creation unit 606, and reference numeral 608 denotes an LUT stored in the LUT data storage unit 607 for the multiband image input by the multiband image input unit 603. 609 is an image output unit for outputting an output signal that is image data created by the image processing apparatus 601.

  Reference numeral 610 denotes an image output unit that actually outputs an image based on an output signal output from the image output unit 608, such as an inkjet printer, and 611 denotes a filter configuration mounted on the multiband image acquisition apparatus 602, and an image The filter / color material configuration setting unit 612 for setting the color material configuration used in the output device 610 stores the filter and color material configuration set by the filter / color material configuration setting unit 611. This is a filter / color material configuration storage unit for storing.

  Reference numeral 613 denotes a filter mounted on the multiband image acquisition device 602, and a filter / color material optimization unit for optimizing the color material used in the image output device 610. Reference numeral 614 denotes an optimum filter / color material optimization unit 613. An input / output error calculation unit 615 for calculating an input / output error in the case of using an integrated filter and color material is a filter / color material database that stores characteristics of the filter and the color material.

<Overall processing of image processing apparatus>
Here, FIG. 7 is a flowchart of image input / output processing performed by the image processing apparatus 601. Hereinafter, a second embodiment according to the present invention will be described in detail with reference to the drawings.

  First, in step S701, the filter / color material optimization unit 611 optimizes a filter mounted on the multiband image acquisition apparatus 602 (details will be described later).

  In step S702, the color material used in the image output apparatus 610 is optimized (details will be described later).

  In step S703, the filter optimized in step S701 and the color material optimized in step S702 are mounted on the multiband image acquisition device 602 and the image output device 610, and information on the filter and color material is stored. This is set by the filter / color material configuration setting unit 611.

  In step S704, the grid data creating unit 604 creates grid data as in the first embodiment.

  In step S <b> 705, a patch image is created by the image output unit 609 based on the grid data created in step S <b> 704 and output by the image output device 610.

  In step S706, the multiband image acquisition apparatus 602 captures the patch image output in step S705.

  In step S707, the average pixel value of each patch is calculated for each band by the average pixel value calculation unit 605, as in the first embodiment, for the patch image taken in step S706.

  In step S708, the LUT creation unit 606 associates the grid data created in step S704 with the average pixel value calculated in step S707, and stores them in the LUT data storage unit 607.

  In step S <b> 709, a multiband image of a subject to be spectrally reproduced in color is acquired by the multiband image acquisition device 602, and is read into the image processing device 601 by the multiband image input unit 603.

  In step S710, based on the LUT data stored in the LUT data storage unit 607, the pixel value of the multiband image read in step S709 is converted into an output signal by the image conversion unit 608.

  In step S711, the image data converted in step S710 is output by the image output unit 609 and output from the image output device 610.

<Filter configuration optimization processing>
Next, details of the filter configuration optimization process in step S701 will be described with reference to FIG. FIG. 8 is a flowchart of the filter configuration optimization process performed in step S701.

  In step S801, the filter / color material configuration optimization unit 613 reads an arbitrary filter as an initial value from the filter / color material database as a filter initial value.

  In step S <b> 802, arbitrary sample spectral reflectance data, for example, Macbeth Color Checker 24 color spectral reflectance data is set as the sample spectral reflectance data.

In step S803, the input value (sensor response) v = [v1, v2,..., Vn] ^t (n is the number of filters) is estimated by (Equation 1).

Where F is a product of the spectral transmittance of the filter, the spectral distribution of the light source, the spectral transmittance of the lens, and the spectral characteristics of the sensor, and o is the sample spectral reflectance data.
In step S804, an estimation matrix G for estimating the sample spectral reflection from the input value is calculated from (Equation 2).

  In step S805, spectral reflectance is estimated by (Equation 3).

In step S806, an error between the sample spectral reflectance data o and the estimated spectral reflectance data o ′ is calculated.
In step S807, it is determined whether or not the error calculated in step S806 is minimum. If it is minimum, the current filter configuration is updated as the optimal filter configuration in step S808, and the process proceeds to step S809. If it is not the minimum, nothing is done and the process proceeds to step S809.

  In step S809, it is determined whether an end condition (for example, whether the RMS error between the sample spectral reflectance and the estimated spectral reflectance is a threshold value or less) is satisfied. In S810, the filter configuration is updated, and the process returns to Step S803.

<Color material optimization processing>
Next, details of the color material configuration optimization process in step S702 will be described with reference to FIG. FIG. 9 is a flowchart of the color material configuration optimization process performed in step S702.

  In step S901, the filter / color material configuration optimization unit 613 reads an arbitrary color material as an initial value from the filter / color material database as a color material initial value.

  In step S902, arbitrary sample spectral reflectance data, for example, Macbeth Color Checker 24-color spectral reflectance data o is set as the sample spectral reflectance data.

  In step S903, the output spectral reflectance o 'is estimated using a printer output estimation model such as Neugebauer equation or Kubelka-Munk equation.

  In step S904, an error between the sample spectral reflectance data o and the estimated spectral reflectance data o ′ is calculated.

  In step S905, it is determined whether or not the error calculated in step S904 is minimum. If it is minimum, the current color material configuration is updated as the optimal color material configuration in step S906, and the process proceeds to step S906. If it is not the minimum, nothing is done and the process proceeds to step S907.

  In step S907, it is determined whether an end condition (for example, whether the RMS error between the sample spectral reflectance and the estimated spectral reflectance is a threshold value or less) is satisfied. In step S908, the color material configuration is updated, and the process returns to step S903.

<Effects of Second Embodiment>
According to the second embodiment as described above, in setting the filter / color material configuration for obtaining and reproducing the target color, the candidate filter / color material characteristics are set, and the set By estimating the reproduction color using the filter / color material and determining the filter / color material to be used based on the determination result of the error amount between the target color and the reproduction color, the color material that best reproduces the target color is obtained. Can be selected.

<Filter / colorant composition candidate>
In the second embodiment, as filters to be optimized and ink candidates in the filter configuration optimization in step S701 and the color material configuration optimization processing in step S702, actual filters / color material data are It may be stored in the filter / color material database 615 and the filter / color material data may be read, or may be modeled by a normal distribution as shown in (Equation 4).

C. Other Embodiments In the present invention, a storage medium recording software program codes for realizing the functions of the embodiments is provided to a system or apparatus, and the computer of the system or apparatus (or CPU or MPU) is stored in the storage medium. This is also achieved by reading and executing the stored program code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code Includes a case where the function of the above-described embodiment is realized by performing part or all of the actual processing.

  Furthermore, after the program code read from the storage medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function is based on the instruction of the program code. This includes the case where the CPU of the expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  In addition, by distributing the program code of the software that realizes the functions of the above-described embodiments via a network, the program code is stored in a storage unit such as a hard disk or memory of a system or apparatus or a storage medium such as a CD-RW or CD-R Needless to say, this can also be achieved by the computer (or CPU or MPU) stored in the system or apparatus reading and executing the program code stored in the storage means or the storage medium.

1 is a block diagram illustrating an image processing apparatus according to a first embodiment. 3 is a flowchart for explaining a processing operation in the image processing apparatus according to the first embodiment. FIG. 5 is a diagram illustrating an example of a user interface for setting a filter / color material configuration in the first embodiment. It is a figure which shows an example of the Grid data produced in 1st Embodiment. It is a figure which shows an example of the Grid patch image produced in 1st Embodiment. It is a block diagram which shows the image processing apparatus of 2nd Embodiment. It is a flowchart for demonstrating the processing operation in the image processing apparatus of 2nd Embodiment. It is a flowchart for demonstrating a filter structure optimization process process. It is a flowchart for demonstrating a color material optimization process.

Claims (16)

An image processing apparatus for spectrally reproducing the color of a subject,
Patch output means for outputting a patch image for creating a lookup table;
Multiband image acquisition means for performing multiband imaging of the patch image;
Lookup table creating means for creating a lookup table for associating the output signal of the patch output means with the pixel value obtained from the multiband image;
Image conversion means for converting an input multiband image into an output signal based on the lookup table;
An image processing apparatus comprising: Furthermore, the information acquisition unit includes information relating to the filter configuration and the color material configuration of the image input device, and a determination unit that determines whether a lookup table corresponding to the information has been created,
The image processing apparatus according to claim 1, wherein when the determination unit determines that the corresponding lookup table has not been created, the corresponding lookup table is created.   3. The image according to claim 1, wherein, when part or all of the filter and the color material configuration are changed, the lookup table creation unit updates only the grid data relating to the changed part. 4. Processing equipment.   The image processing apparatus according to any one of claims 1 to 3, wherein the image input device is an input device that acquires color information of a subject using a filter of four or more primary colors.   The image processing apparatus according to claim 1, wherein the image output device is an image output device that performs color reproduction using color materials of four or more primary colors.   6. The image processing apparatus according to claim 1, further comprising a filter / color material configuration setting unit that sets a filter configuration and a color material configuration of the image input device.   The image processing apparatus according to claim 6, wherein the filter / color material configuration setting unit optimizes an error so as to reduce an error when reproducing an arbitrary spectral reflectance data group. An image processing method for spectrally reproducing the color of a subject,
A patch output process for outputting a patch image for creating a lookup table;
A multiband image acquisition step of taking a multiband image of the patch image;
A lookup table creating step of creating a lookup table that associates the output signal obtained in the patch output step with the pixel value obtained from the multiband image;
An image conversion step of converting an input multiband image into an output signal based on the lookup table;
An image processing method comprising: Furthermore, it includes a determination step of acquiring information about the filter configuration and color material configuration of the image input device and determining whether a lookup table corresponding to the information has been created,
2. The image processing method according to claim 1, wherein when the determination step determines that the corresponding lookup table has not been created, the corresponding lookup table is created.   10. The image according to claim 8, wherein, when part or all of the filter and color material configuration is changed, the lookup table creation step updates only grid data relating to the changed part. 11. Processing method.   The image processing method according to claim 8, wherein the image input device is an input device that acquires color information of a subject using a filter of four primary colors or more.   The image processing method according to any one of claims 8 to 10, wherein the image output device is an image output device that performs color reproduction using color materials of four or more primary colors.   The image processing method according to claim 8, further comprising a filter / color material configuration setting step of setting a filter configuration and a color material configuration of the image input device.   The image processing method according to claim 13, wherein the filter / color material configuration setting unit optimizes an error so as to reduce an error when reproducing an arbitrary spectral reflectance data group.   15. A computer program for executing the image processing method according to claim 8. A computer-readable storage medium storing the computer program according to claim 15.

Images