[go: up one dir, main page]

WO2018160847A1 - Modification d'image couleur avec fonction d'approximation - Google Patents

Modification d'image couleur avec fonction d'approximation Download PDF

Info

Publication number
WO2018160847A1
WO2018160847A1 PCT/US2018/020468 US2018020468W WO2018160847A1 WO 2018160847 A1 WO2018160847 A1 WO 2018160847A1 US 2018020468 W US2018020468 W US 2018020468W WO 2018160847 A1 WO2018160847 A1 WO 2018160847A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
color
color space
data
approximation
Prior art date
Application number
PCT/US2018/020468
Other languages
English (en)
Inventor
Alexander PARTIN
Kimball Darr THURSTON III
Georgios GEORGIADIS
Original Assignee
Dolby Laboratories Licensing Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dolby Laboratories Licensing Corporation filed Critical Dolby Laboratories Licensing Corporation
Priority to EP18708895.0A priority Critical patent/EP3566429B1/fr
Priority to CN201880015431.1A priority patent/CN110383802B/zh
Priority to US16/489,662 priority patent/US10713765B2/en
Publication of WO2018160847A1 publication Critical patent/WO2018160847A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H04N1/603—Colour correction or control controlled by characteristics of the picture signal generator or the picture reproducer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/67—Circuits for processing colour signals for matrixing
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/06—Colour space transformation

Definitions

  • An image processing pipeline commonly refers to a sequence of operations applied to an image, and the pipeline can start at the image source (e.g., a camera) and end at the image Tenderer (e.g., a television).
  • the image data often undergoes manipulations which require one or more non-linear conversions between spaces.
  • Figure 1A shows an example of an image processing pipeline which includes such a conversion.
  • an input 10 containing image data in a general intermediate color space (denoted as color space ABC) is provided as an input to the converter 12 which uses a non-linear function Ho to convert image data in the ABC color space to image data 14 in another color space, denoted as color space JKL, which can be an RGB -type color space.
  • the image data 14 can be further processed (e.g., in operations 16), resulting in an input 19 (in the RGB linear light color space-RGB L) that is provided to a transfer function 21 which is the electro-optical transfer function (EOTF) of the target display.
  • EOTF Electro-Optical Transfer function
  • Function 21 produces an output 23 that can drive the target display.
  • IPT color space refers to a Luma (I) -Chroma (P and T) -like space (like YCbCr).
  • IPT space was described in "Development and testing of a color space (ipt) with improved hue uniformity", by F. Ebner and M.D. Fairchild, in Proc. 6th Color Imaging Conference: Color Science, Systems, and Applications, IS&T, Scottsdale, Arizona, Nov. 1998, pp. 8-13 (to be referred to as the Ebner paper), which is incorporated herein by reference in its entirety.
  • IPT perceptual quantizer
  • ICtCp is a color-opponent based encoding scheme intended to separate luma from chroma information.
  • ICtCp offers constant intensity (CI) representation.
  • Figure 1A shows an example of an image processing pipeline which includes a non- linear function.
  • Figure IB shows an example of an image processing pipeline which includes a unit or component that trims (e.g., adjusts) the image data in the JKL color space.
  • Figure 2 shows an architecture of an image processing pipeline in which color trims are applied to image data in the ABC color space.
  • Figure 3A shows an example of an image processing pipeline that uses back projection to obtain trimmed image data in the ABC color space from trimmed image data in the JKL color space.
  • Figure 3B shows a generalized example of a display management system that uses one or more approximation functions to create trimmed image data.
  • Figure 4 shows an example of a display management system that can support images that have high dynamic range and wide color gamut.
  • Figure 5 shows an example of a display management system that includes back projection to obtain trimmed image data in a first color space (e.g., an IPT-type color space) from trimmed image data in a second color space (e.g., an RGB-type color space).
  • a first color space e.g., an IPT-type color space
  • a second color space e.g., an RGB-type color space
  • Figure 6 shows an example of a display management system that uses a set of approximation functions according to one embodiment described herein.
  • Figure 7 is a flowchart that shows a method according to one or more
  • Figure 8 is a flowchart that shows another method according to one or more embodiments of the invention.
  • Figure 9 shows an example of a display management system according to one or more embodiments described herein.
  • Figure 10 shows an example of a data processing system which can be used by a colorist to create color trims.
  • references in the specification to "one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the invention.
  • the appearances of the phrase “in one embodiment” in various places in the specification do not necessarily all refer to the same embodiment.
  • the processes depicted in the figures that follow are performed by processing logic that comprises hardware (e.g. circuitry, dedicated logic, etc.), software, or a combination of both. Although the processes are described below in terms of some sequential operations, it should be appreciated that some of the operations described may be performed in a different order. Moreover, some operations may be performed in parallel rather than sequentially.
  • the embodiments described herein relate to the use of one or more approximation functions, created from one or more back projections of trimmed image data from a downstream point in an image processing pipeline to an upstream point in the pipeline.
  • the trimmed image data can be created by a colorist at the downstream point, and then the approximation function(s) can be derived (e.g., through back projection and regression) and then provided to a display management system on one or more target display systems.
  • the approximation function(s) can be used to match between two sets of data or variables that are related by one or more complicated non-linear transfer or transformation functions.
  • Color grading or color correction generally refers to the process of manipulating image content to enhance image quality on a target display.
  • Various color grading tools such as six- vector grading, a Color Decision List (CDL) as defined by the American Society of Cinematographers (ASC), and others are available for use by colorists, and the choice of a specific tool or method can depend on the desired results.
  • Color correction is usually applied in the RGB or HSV color spaces which are more intuitive due to human perception of color in these spaces.
  • Figure IB shows an example of an image processing pipeline that uses color correction, referred to as "trims" (in the case of the CDL defined by ASC), in a common color space that is used to make color corrections, such as RGB or HSV.
  • Input image data 51 can be in a general intermediate color space, denoted as ABC, which can in one embodiment be an IPT-type color space.
  • the input image data 51 is converted, by non-linear function Ho, from image data in the ABC color space to image data 55 in the JKL color space.
  • JKL represents one of the common color spaces in which color correction trims are usually applied (e.g., RGB color space for ASC-CDL grading or HSV color space for six- vector grading).
  • trim unit 57 one or more color trims To are applied to the image data 55 to create trimmed image data 59 in the JKL color space (labeled as JKLT).
  • the trimmed image data 59 can be further processed (e.g., in operations 61), resulting in an input 63 (in the RGB linear light color space) that is provided to the target display's EOTF 65.
  • the target display's EOTF 65 provides an output 67 that drives the target display 69 to cause the target display 69 to display images (e.g., a movie or television show or video game, etc.) on the screen of the target display.
  • the trim is deferred until the image data is in the JKL color space, but deferring trims to some point near the target display can require expensive hardware (at a display management system of the target display) to implement the trims.
  • the one or more embodiments described herein provide a solution that does not defer the trim in this way.
  • FIG 2 shows an example of an architecture of an image processing pipeline in which the color trims Ti are performed in the ABC color space (on image data 101) before the image data is converted into image data in the JKL color space by the color space converter 107 (which uses the non-linear function Ho to convert the ABCT image data).
  • image data 101 in the ABC color space is adjusted with color trims Ti in the trim unit 103 which produces trimmed image data 105 (ABCT) in the ABC color space.
  • This trimmed image data 105 is then converted into trimmed image data 109 (JKLT) by the color space converter 107.
  • the trimmed image data 109 can be further processed (e.g., in operations 111), resulting in an input 113 (in the RGB linear light-RGBL color space) that is provided to the target display's EOTF 115.
  • the output from the target display's EOTF 115 drives the target display 119 using the input 117 (in RGBE) to the target display 119.
  • One advantage of the architecture of the pipeline in Figure 2 is that it may avoid expensive hardware modifications to display management systems that drive target displays but still provide for the use of existing color grading techniques such as the CDL as defined by the ASC.
  • the embodiments described herein enable the functionality of the pipeline shown in Figure 2 and use the projection of color trims To (in Figure IB) into color trims Ti.
  • Figure 3 A shows a general example of this projection in a display management system; in one embodiment, this projection can be performed on a colorist's system that creates the color trims, and this projection can be referred to as a back projection.
  • image data 151 in the ABC color space is converted, by color space converter 153, into image data 155 in the JKL color space.
  • color trims To are applied to image data 155 by the trim unit 157 to produce trimmed image data 159 (JKLT) in the JKL color space.
  • This trimmed image data 159 can be further processed in operations 163 to derive image data 171 (in RGB L color space) which is transformed by the target display's EOTF to create image data 175 that drives the target display 177.
  • image data 171 in RGB L color space
  • the output of the image on the target display 177 allows a colorist (or other user) to view the image on the target display 177 and decide whether the image has been properly or correctly trimmed (adjusted), and the colorist can interactively change the trims to see the resulting change in the image on the target display 177.
  • the trimmed image data 159 (JKL T ) can be back projected, through back projection 161, by feeding the trimmed image data 159 as an input to a function Ho "1 which is an inverse function of the function Ho; the inverse function Ho "1 in converter 165 produces trimmed image data (ABC T ) 167 in the ABC color space based upon the trimmed image data 159 (JKL T ) in the JKL color space.
  • the trim unit 157 is bypassed by the back projection 161 as shown in Figure 3A.
  • the display management system in Figure 3 ⁇ has a set of ABC T values (image data in ABC color space) and a corresponding set of ABC values (original image data in ABC color space) that are related through one or more non- linear transfer functions.
  • the back projection 161 in one embodiment maps the image data between the color spaces.
  • the non-linear relationship between the set of ABC T values and the set of ABC values can be mathematically expressed as:
  • a T f A (A, B, C, ⁇ ti,..., t n ⁇ j),
  • B T f B (A, B, C, ⁇ ti,..., t n ⁇ K ),
  • ti,..., t n can be the trims in the ASC-CDL color correction format (including, for example, trim values for slope, offset, and power (S, O, P)).
  • an approximation method such as linear regression, neural network training, or support vector machine can be used to generate a function such as a linear approximation function (e.g., F A , F B , FC) that fits a model or approximation function to the two sets of data.
  • a linear approximation function e.g., F A , F B , FC
  • BT FB (A, B, C, ⁇ ti,..., t n ⁇ K ),
  • CT Fc (A, B, C, ⁇ ti,..., t n ⁇ L ).
  • F A , F B , and Fc can be derived from the two sets of image data (ABC and ABC T ) on the colorist's system, and the derived functions (or metadata such as coefficients in the functions) can be saved and then transmitted or otherwise distributed to one or more display management systems that drive target displays, and these display
  • image data 203 in the ABC color space is provided as an input to three approximation functions 207, 209, and 211 within the approximation unit 205; the approximation function 207 computes the trimmed output A T for the A coordinate, the approximation function 209 computes the trimmed output B T for the B coordinate, and the approximation function 211 computes the trimmed output C T for the C coordinate.
  • the trim values selected by the colorist for the image are represented by coefficients in each of the approximation functions.
  • a first set of coefficients in this embodiment specifies a color trim for a first set of one or more images, and the three approximation functions 207, 209, and 211 use this first set of coefficients to complete the trimmed image data (ABC T ) 215 for the first set of images;
  • a second set of coefficients specifies a color trim for a second set of one or more images, and the three approximation functions 207, 209, and 211 use this second set of coefficients to compute the trimmed image data 215 for the second set of images.
  • the trimmed image data 215 is provided as an input to the non- linear function Ho in color space converter 217 which converts the ABC T image data in the ABC color space into trimmed image data 219 (JKL T ) in the JKL color space.
  • the trimmed image data 219 (JKL T ) can then be further processed (e.g., in operation 221), resulting in image data 223 (in an RGB L color space) that is applied as an input to the target display's EOTF 225.
  • the target display's EOTF 225 applies the EOTF for the target display 229 to display one or more images in the content.
  • the pipeline 201 can reduce a need to modify hardware in the display management system while providing a way to use color trims which in effect were derived from color trims applied in the JKL color space.
  • Figures 3A and 3B show generalized examples of one or more embodiments of the invention while
  • Figures 5 and 6 show a more specific example. Prior to describing Figures 5 and 6, some information about the architecture of the pipelines in Figures 5 and 6 will be provided in conjunction with Figure 4.
  • the image processing pipeline 301 in Figure 4 is an example of a display management system that drives a target display that may have the capability of displaying high dynamic range images and also display images with a wide color gamut.
  • the display management system can be designed to receive image data created with a high quality (e.g., "reference") source display and map the image data so that it can properly be displayed on a target display that may not be capable of the full dynamic range or color gamut of the reference display.
  • a high quality e.g., "reference”
  • the pipeline 301 can be part of a display management system which supports Dolby Vision technology from Dolby Laboratories, and this technology can permit the distribution of high dynamic range and wide color gamut content, based on reference systems that can create such content, to target systems that have smaller dynamic ranges or smaller color gamuts while retaining high quality images. Further information about such technology can be found in published US Patent Application No. US 2016/0005349 which is incorporated herein by reference.
  • the pipeline 301 can include a color space converter 305, a tone mapper and color volume mapper 307, a set of one or more converters 309 to convert the mapped image data from the maps 307 into an output color space, a target display transfer function 311, and a target display 314 that is driven by the image data in the output from the target display transfer function.
  • the color space converter 305 receives image data 303 representing one or more images.
  • the image data 303 can be an RGB -type color space
  • the converter 305 can convert that image data 303 into image data in an IPT-type color space (e.g., IPT or ICtCp color spaces) which is then mapped in the mapper 307.
  • IPT-type color space e.g., IPT or ICtCp color spaces
  • the tone mapper in one embodiment, in the mapper 307 can map intensity (or luminance like) values, such as the I values in image data from converter 305 in the IPT-type color spaces, in the input image data (which can have the high dynamic range of the source display) to intensity (or luminance like) values in the dynamic range of the target display (which may be a smaller dynamic range relative to the source display).
  • the color volume mapper in the mapper 307 can, in one embodiment, perform color volume mapping from the input image data (which can have the wide color gamut of the source display) to image data in the color gamut of the target display 314.
  • the output 308 from the mapper 307 can be provided as an input to one or more color space converters 309 which convert the mapped image data in the output 308 to the output space of the target display 314; the output 310 from the one or more color space converters 309 is transformed by the target display transfer function 311 to derive an output that drives the target display 314.
  • the pipeline 301 can also include other components used in Dolby Vision, such as a detail preservation unit that restores high frequency details in the intensity channel that may be lost due to tone mapping.
  • Figure 5 shows a specific example of a pipeline that is similar to pipeline 301 but is used by a colorist (or other user) to generate approximation functions (or data, such as coefficients, to generate the approximation functions); thus, the pipeline in Figure 5 can include some of the same components as the pipeline 301 but includes a target display 315 which can be a reference display used by a colorist to view adjusted (trimmed) images and select one or more color trims using the trim unit 363.
  • the pipeline in Figure 5 can, in one embodiment, be part of a Dolby Grading Tool from Dolby Laboratories.
  • the color space converter 304 in Figure 5 receives the image data 303, which can be in an RGB-type color space, and converts that image data 303 into image data 351 which can be in an IPT-type color space.
  • the image data 351 is then tone mapped and color volume mapped by mapper 307 in Figure 5, which can be the same as mapper 307 in Figure 4.
  • Figure 5 provides mapped image data 353 (in an IPT P Q color space in that the IPT image data has undergone a perceptual quantization transformation).
  • This mapped image data 353 is then transformed through a series of three color space conversions (by color space converters 355, 357, and 359) into image data in the LMS P Q color space and then into image data in the LMS linear light color space and finally into an RGB linear light space at the output of color converter 359.
  • a traditional colorist system would normally output the RGB linear image data, through the target display transfer function 366, to the target display 315.
  • the pipeline in Figure 5 has been modified by the addition of three blocks (PQ transformation 361, trims unit 363, and PQ "1 transformation 365) that allow the colorist to generate trims far downstream (near the target display) and then back project those generated trims back to trimmed image data in the IPT color space at the end of back projection path 377.
  • the modifications by these three blocks in effect creates trims like trims To in Figure IB and then back projects the trimmed image data (using reverse or inverse transformations along the back projection paths 369, 373, 375, and 377) to derive trimmed image data ( ⁇ ) in the IPT color space.
  • This trimmed image data ( ⁇ ) is then used with the mapped image data 353 to derive the approximation functions such as the approximation functions shown in trim unit 405.
  • the PQ transformation 361 performs a perceptual quantization (PQ)
  • the trims unit 363 can then modify the image data based on color trims, where the color trims can be based on inputs 371 which can be inputs from a colorist who is viewing the results of the color trims on the target display 315; the colorist, in one embodiment, can interactively change the inputs 371 to see the results of the change displayed on the target display 315 in order to decide which color trims to select.
  • the trimmed image data 364 modified by the trips unit 363 is provided as an output from trims unit 363 to the inverse PQ (PQ 1 ) transformation 365 which performs an inverse PQ transformation on the trimmed image data 364 (in the RGB PQ color space) to produce an output of trimmed image data (in RGB linear light color space) which is then processed by the target display transfer function 366 to yield a set of image data (e.g., a frame of image data) that drives the source display 315.
  • the inverse PQ transformation 365 also provides an output that is the beginning of the back projection path 369.
  • the back projection paths include a set of inverse transformations which convert the trimmed image data (from the output of inverse PQ transformation 365) into image data in the LMS linear color space (after an RGB linear to LMS linear conversion in converter 359) and then convert the trimmed image data into image data in the LMS PQ color space and then convert the trimmed image data into image data in the IPT PQ color space to produce the output ⁇ at the end of back projection path 377.
  • the trims unit 363 is similar to trims unit 157 in Figure 3A in that the color trims selected by a user (e.g. a colorist controlling inputs 371) result in trimmed image data that is back projected to derive a set of data (e.g.
  • ABC T or IPT T that can be used with another set of data for the same image (e.g., ABC or IPT) to create a set of approximation functions that can be used on a target display's display management system to trim the image on the target display.
  • the approximation function(s) solver 379 generates one or more approximation functions (e.g., the approximation functions used in trim unit 405) which fit the two sets of data (mapped image data 353 (IPT P Q) and IPT T data at the end of the back projection path 377) to a mode (e.g., one or more equations) that represents the non-linear relationship between the two sets of data.
  • approximation functions e.g., the approximation functions used in trim unit 405
  • a mode e.g., one or more equations
  • the approximation function solver 379 can use any one of a variety of techniques to perform the fit of the two sets of data to a model; for example, the solver 379 can use linear regression (e.g., multivariate linear regression) or neural network training or support vector machine or other techniques known in the art to fit or match two sets of data to the model used by the solver 379 to generate the approximation functions, such as the approximation functions in trim unit 405 in Figure 6.
  • linear regression e.g., multivariate linear regression
  • neural network training or support vector machine or other techniques known in the art to fit or match two sets of data to the model used by the solver 379 to generate the approximation functions, such as the approximation functions in trim unit 405 in Figure 6.
  • the approximation function(s) solver 379 provides output 381 which can include a full description of the approximation functions, such as one or more equations and a set of coefficients that is used in the equations, or the output 381 can include just the coefficients for a known model or set of equations used by trim units (e.g., trim unit 405) in display management systems.
  • T T F T (I, P, T, ⁇ S, O, P ⁇ B ), wherein IPT are represented by the mapped image data 353 (IPTPQ), and IT is represented by the image data in the I channel in ⁇ at the end of back projection 377, PT is represented by the image data in the P channel in ⁇ at the end of back projection 377, and TT is represented by the image data in the T channel in ⁇ at the end of back projection 377.
  • IPTPQ mapped image data 353
  • IT is represented by the image data in the I channel in ⁇ at the end of back projection 377
  • PT is represented by the image data in the P channel in ⁇ at the end of back projection 377
  • TT is represented by the image data in the T channel in ⁇ at the end of back projection 377.
  • each one of the data components in ⁇ (IT, PT, TT) is a function of the components I, P, and T and also the S, O, P (Slope, Offset, and Power) trim values
  • the trim values can be selected by the inputs 371 which can be controlled by the colorist using the pipeline shown in Figure 5.
  • a multivariate linear regression is used to determine three mathematical models (one for each of Fi, Fp, and FT) to express each of the ⁇ data components (IT, PT, TT) as functions of I, P, T (represented by mapped image data 353) and the SOP trim values.
  • a third order linear regression model :
  • the 20 coefficients (ao, ..., ⁇ 19) depend on the SOP trims and are provided, in one embodiment, per image frame as metadata along with the image data of the frame; in other words, the SOP trims are embedded in the 20 coefficients because the regression fitted the IPT and ⁇ data sets for a single given set of SOP trims (specified by inputs 371).
  • the solver 379 can, in one embodiment, generate the set of 20 coefficients for use with the model in Equation 4 for each of the equations 1-3 and provide this set of coefficients as an output 381 (for the associated image) for distribution (along with the image data) to one or more display management systems that drive target displays. Display management systems which receive these coefficients can then derive the set of approximation functions such as the approximation functions in trim unit 405.
  • Figure 6 shows an example of such a display management system, in one embodiment, which receives image data and metadata in the form of the coefficients (e.g., 20 coefficients) used to generate the approximation functions in trim unit 405.
  • the display management system shown in Figure 6 can, in one embodiment, be a system which supports the Dolby Vision technology from Dolby Laboratories, and thus it is similar to the image processing pipeline 301 shown in Figure 4.
  • the color space converter 401 can be similar to the color space converter 305
  • the tone mapper and color volume mapper 403 can be similar to the tone mapper and color volume mapper 307.
  • the display management system shown in Figure 6 receives image data 303 which represents one or more images; in one embodiment, the image data can be in the RGB color space, and the converter 401 converts the image data in the RGB color space into image data in the IPT color space. Then, the tone mapper and color volume mapper 403 can perform tone mapping and color volume mapping operations as described herein, such as the operations described relative to the mapper 307 in Figure 4. The mapper 403 produces mapped image data in the IPT color space which is then provided as an input to the three approximation functions shown in the trim unit 405, which produces trimmed IPT values for each pixel in the image based upon the trims specified inherently in the coefficients used in the equations of the approximation functions within the trim unit 405.
  • the display management system shown in Figure 6 receives the image data along with metadata describing the color trims, and this metadata can be the 20 coefficients determined by the approximation function solver 379 on a colorist's system in one embodiment.
  • the metadata can apply to one or more images that have been trimmed by the colorist, and another set of images can use a different set of color trims specified by another set of different metadata.
  • the metadata may be a full description of a set of one or more approximation functions; this may be used in situations in which the model or approximation functions can change as may be desirable depending upon image content or availability of different hardware, etc.
  • trim unit 405 in this case does not use a static, predetermined set of approximation functions or models.
  • the trimmed image data from trim unit 405 is provided as an output to a set of one or more color converters, such as color converters 409, which can be the same as the first, second, and third stage color converters in Figure 5.
  • the color converters 409 produce, in one embodiment, an output of image data in the RGB linear light color space which is applied as an input to the target display transfer function 411 which produces image data as an input to the target display 415 which then displays the image on the target display 415.
  • Figure 7 shows an example of a method according to one embodiment of the invention, and this method can be performed on a system such as that shown in Figure 3A or the system shown in Figure 5, which systems can be operated by a colorist or other user who is causing color trims to be generated.
  • the method shown in Figure 7 can begin in operation 451, and this operation can include converting image data in a first color space into image data in a second color space.
  • operation 451 can be performed by color space converter 153 which transforms image data in the ABC color space into image data in the JKL color space.
  • operation 451 may be performed by a set of color converters such as the color converters 355, 357, and 359.
  • operation 451 may be preceded by other operations, such as a color space conversion operation (e.g., a conversion done by color space converter 304) and may also be preceded by mapping operations, such as a tone mapping and a color volume mapping operation by, for example, the mapper 307.
  • a color space conversion operation e.g., a conversion done by color space converter 304
  • mapping operations such as a tone mapping and a color volume mapping operation by, for example, the mapper 307.
  • the system receives inputs that indicate trim data for the image data in the second color space.
  • the inputs may be inputs from a colorist or other user who is viewing a target display, such as target display 315.
  • the trim data in one embodiment, can be the SOP data described herein.
  • the system can apply the trim data to the image data in the second color space to obtain trimmed image data in the second color space.
  • this trimmed image data in the second color space can be the output from the inverse PQ transformation 365 which provides an input to the back projection path 369 as shown in Figure 5.
  • the trimmed image data is back projected through, for example, reverse transformations, to corresponding image data in the first color space.
  • this trimmed image data in the first color space is the input provided by the end of the back projection path 377 shown in Figure 5.
  • processing logic such as, for example, the approximation function solver 379 shown in Figure 5, determines data for use with a set of approximation functions that transform image data in the first color space into trimmed image data in the first color space based on the applied trim data.
  • the data which was determined in operation 459 can be distributed to one or more display management systems that drive one or more target displays.
  • the data which was determined in operation 459 can be distributed to one or more display management systems that drive one or more target displays.
  • metadata describing a set of coefficients for the approximation functions can be distributed along with the image data to a display
  • Figure 8 shows an example of a method according to one embodiment of the invention which can be performed by a display management system that drives a target display, such as the display management system shown in Figure 3B or the display management system shown in Figure 6.
  • operation 501 first image data representing an image is received and converted from first image data in a first color space into second image data in a second color space.
  • the color space converter 401 can perform operation 501.
  • operation 503 a tone mapping function and a color volume mapping function are applied to image data based on the second image data to produce mapped image data in the second color space.
  • the mapper 403 can perform operation 503 to produce the mapped image data in the second color space.
  • the system can receive data that specifies one or more color trims, and this data can be used in one or more approximation functions that approximates one or more non-linear
  • this data can be a set of coefficients that is used in the approximation functions, such as the approximation functions shown in the trim unit 405 in Figure 6.
  • the approximation function is used on the image data with the data that specifies the color trims to produce trimmed image data in the second color space.
  • the result, in one embodiment, of operation 507 is the output of the trimmed image data in the second color space from the trim unit 405 shown in Figure 6.
  • one or more color space transformation functions are applied on the image data to convert the trimmed image data into third image data for use in driving a target display.
  • the set of converters 409 shown in Figure 6 can perform operation 509 to provide image data for use in driving a target display through a target display transfer function, such as the target display transfer function 411 shown in Figure 6.
  • operations 501 and 503 can be optional, such as the embodiment shown in Figure 3B which does not include the color space conversion of operation 501 and does not include the tone mapping function and the color volume mapping function.
  • Figure 9 shows an example of components of a display management system, such as the display management system shown in Figure 6.
  • This example in Figure 9 typically includes hardware processing systems, such as either general purpose processor(s) or some special purpose processor(s) or both, and memory and input and output interfaces.
  • the general purpose processor(s), if present, can be programmable by software instructions which execute on the one or more general purpose processors.
  • the special purpose processors, if present, may also be configured by software or hardware logic to perform operations, such as color space conversions or other processing operations.
  • the input interface 601 can be a network interface, such as an Ethernet or WiFi interface or a cable TV interface or a satellite TV interface, and the input interface 601 receives, in one embodiment, the input image represented by image data 303 shown in Figure 6.
  • the color space converter 602 can be implemented in either general purpose programmable hardware or special purpose hardware configured to perform color space conversions (or a combination of both) and contained within the processing system 603, which is coupled to memory 604.
  • the memory 604 can store images and metadata, such as metadata describing the color trims (e.g., coefficients in the approximation functions).
  • the color space converter 602 can perform the color space transformation in operation 501 of Figure 8.
  • the processing system 603 can perform, in one embodiment, the tone mapping operations and the color volume mapping operations as well as perform the color trims by using the set of approximation functions, such as the approximation functions shown in the color trim unit 405 in Figure 6.
  • the color space converter 605 can then convert the image data that has been trimmed based upon the approximation functions into an output color space to drive the target display 609 through an output interface 607.
  • the output interface 607 can be a hardware interface, such as an HDMI interface or other known display interfaces which are coupled to the target display 609, which can be the same as the target display 415 shown in Figure 6.
  • Figure 10 shows an example of a data processing system that can be used by a colorist.
  • the systems and methods described herein can be implemented in a variety of different data processing systems and devices, including general-purpose computer systems, special purpose computer systems, or a hybrid of general purpose and special purpose computer systems.
  • Data processing systems that can use any one of the methods described herein include desktop computers, laptop computers, tablet computers, embedded electronic devices, or other electronic devices.
  • Figure 10 is a block diagram of data processing system hardware according to an embodiment. Note that while Figure 10 illustrates the various components of a data processing system, it is not intended to represent any particular architecture or manner of interconnecting the components as such details are not germane to the present invention. It will also be appreciated that other types of data processing systems that have fewer components than shown or more components than shown in Figure 10 can also be used with one or more embodiments of the present invention.
  • the data processing system includes one or more buses 1309 that serve to interconnect the various components of the system.
  • One or more processors 1303 are coupled to the one or more buses 1309 as is known in the art.
  • Memory 1305 may be DRAM or non- volatile RAM or may be flash memory or other types of memory or a combination of such memory devices. This memory is coupled to the one or more buses 1309 using techniques known in the art.
  • the data processing system can also include non- volatile memory 1307, which may be a hard disk drive or a flash memory or a magnetic optical drive or magnetic memory or an optical drive or other types of memory systems that maintain data even after power is removed from the system.
  • the non- volatile memory 1307 and the memory 1305 are both coupled to the one or more buses 1309 using known interfaces and connection techniques.
  • a display controller 1322 is coupled to the one or more buses 1309 in order to receive display data to be displayed on a source display device 1323.
  • the source display device 1323 can include an integrated touch input to provide a touch screen.
  • the data processing system can also include one or more input/output (I/O) controllers 1315 which provide interfaces for one or more I/O devices, such as one or more mice, touch screens, touch pads, joysticks, and other input devices including those known in the art and output devices (e.g. speakers).
  • the colorist can use the one or more input devices to adjust color trims for one or more images.
  • the input/output devices 1317 are coupled through one or more I/O controllers 1315 as is known in the art.
  • Non- volatile memory 1307 and the memory 1305 are coupled to the one or more buses directly rather than through a network interface
  • the present invention can utilize non- volatile memory that is remote from the system, such as a network storage device which is coupled to the data processing system through a network interface such as a modem or Ethernet interface.
  • the buses 1309 can be connected to each other through various bridges, controllers and/or adapters as is well known in the art.
  • the I/O controller 1315 includes one or more of a USB (Universal Serial Bus) adapter for controlling USB peripherals, an IEEE 1394 controller for IEEE 1394 compliant peripherals, or a Thunderbolt controller for controlling Thunderbolt peripherals.
  • USB Universal Serial Bus
  • one or more network device(s) 1325 can be coupled to the bus(es) 1309.
  • the network device(s) 1325 can be wired network devices (e.g., Ethernet) or wireless network devices (e.g., WiFi, Bluetooth) that receive images from a camera, etc.
  • one or more embodiments of the present invention may be embodied, at least in part, in software. That is, the techniques may be carried out in a data processing system in response to its one or more processor(s) executing a sequence of instructions contained in a storage medium, such as a non-transitory machine- readable storage medium (e.g. DRAM or flash memory).
  • a storage medium such as a non-transitory machine- readable storage medium (e.g. DRAM or flash memory).
  • a storage medium such as a non-transitory machine- readable storage medium (e.g. DRAM or flash memory).
  • a storage medium such as a non-transitory machine- readable storage medium (e.g. DRAM or flash memory).
  • a storage medium such as a non-transitory machine- readable storage medium (e.g. DRAM or flash memory).
  • hardwired circuitry may be used in combination with software instructions to implement the present invention.
  • the techniques are not limited to any specific combination of hardware circuitry and
  • EEEs enumerated example embodiments
  • EEE 1 A method for processing one or more images, the method comprising:
  • approximation function that approximates one or more non- linear transformations of image data
  • EEE 2 The method of EEE 1, the method further comprising:
  • EEE 3. The method of any preceding EEE, wherein the data represents coefficients in the approximation function and the approximation function is a linear approximation function.
  • EEE 4. The method of any preceding EEE, wherein the second color space is an IPT-type color space and the first color space and the third color space are an RGB-type color space.
  • EEE 5. The method of any preceding EEE, wherein the data that specifies one or more color trims was created by performing a color trim in the third color space and then back projecting the result of the color trim to produce trimmed IPT image data.
  • EEE 6. The method of EEE 5, wherein the linear approximation function is derived through a multivariate linear regression which uses the trimmed IPT image data and corresponding IPT data for the image to produce the linear approximation function.
  • EEE 7 The method of any preceding EEE, wherein the tone mapping function maps image data from a high dynamic range used on a source display to a lower dynamic range used on the target display and wherein the color volume mapping function maps image data from a large color gamut on the source display to a smaller color gamut on the target display.
  • EEE 8 A non-transitory machine readable medium storing instructions which when executed by a display management system cause the system to perform a method as in any one of EEEs 1-7.
  • EEE 9 A display management system which includes a memory and a processing system configured to perform a method as in any one of EEEs 1-7.
  • EEE 10 A method for processing one or more images, the method comprising:
  • trim approximation data for use in an approximation function that
  • EEE 11 The method of EEE 10, the method further comprising:
  • EEE 12 The method of any one of EEEs 10 to 11, wherein the first color space is an RGB -type color space, the second color space is an IPT-type color space, and the third color space is an RGB -type color space.
  • EEE 13 The method of any one of EEEs 10 to 12, wherein the one or more user inputs is from a colorist who is viewing the display.
  • EEE 14 The method of any one of EEEs 10 to 13, wherein the tone mapping function maps image data from a high dynamic range used on a source display to a lower dynamic range used on the display and wherein the color volume mapping function maps image data from a large color gamut on the source display to a smaller color gamut on the display.
  • EEE 15 The method of any one of EEEs 10 to 14, wherein the trim approximation data represents coefficients in the approximation function and the approximation function is a linear approximation function.
  • EEE 16 The method of any one of EEEs 10 to 15, wherein determining the trim approximation data involves using multivariate regression to derive the coefficients in the approximation function.
  • EEE 17 The method of EEE 14, wherein projecting back involves using reverse transformations to reverse the sequence of transformations between the mapped image data and the second trimmed image data.
  • EEE 18 The method of EEE 15, wherein the projecting backs occur in response to a user input by a user after viewing the display.
  • EEE 19. A non-transitory machine readable medium storing instructions which when executed by a data processing system cause the data processing system to perform a method as in any one of EEEs 10-18.
  • EEE 20 A data processing system comprising a memory and a processing system coupled to the memory, the processing system configured to perform a method as in any one of EEEs 10-18.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

Des données d'ajustement de couleur sont utilisées dans une fonction d'approximation pour approximer une ou plusieurs transformations non linéaires de données d'image dans un pipeline de traitement d'image. Les données d'ajustement de couleur sont dérivées, dans un mode de réalisation, par l'intermédiaire d'une rétroprojection sur un système de coloriste, et les données d'ajustement de couleur sont utilisées au moment du rendu d'une image sur un système de gestion d'affichage.
PCT/US2018/020468 2017-03-03 2018-03-01 Modification d'image couleur avec fonction d'approximation WO2018160847A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP18708895.0A EP3566429B1 (fr) 2017-03-03 2018-03-01 Modification d'image couleur avec fonction d'approximation
CN201880015431.1A CN110383802B (zh) 2017-03-03 2018-03-01 利用逼近函数的彩色图像修改方法
US16/489,662 US10713765B2 (en) 2017-03-03 2018-03-01 Color image modification with approximation function

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201762466963P 2017-03-03 2017-03-03
EP17159216.5 2017-03-03
US62/466,963 2017-03-03
EP17159216 2017-03-03

Publications (1)

Publication Number Publication Date
WO2018160847A1 true WO2018160847A1 (fr) 2018-09-07

Family

ID=58231478

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/020468 WO2018160847A1 (fr) 2017-03-03 2018-03-01 Modification d'image couleur avec fonction d'approximation

Country Status (1)

Country Link
WO (1) WO2018160847A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080166043A1 (en) * 2007-01-05 2008-07-10 Silicon Optix Inc. Color and geometry distortion correction system and method
US9077994B2 (en) 2011-12-06 2015-07-07 Dolby Laboratories Licensing Corporation Device and method of improving the perceptual luminance nonlinearity-based image data exchange across different display capabilities
US20160005349A1 (en) 2013-02-21 2016-01-07 Dolby Laboratories Licensing Corporation Display Management for High Dynamic Range Video
WO2016118395A1 (fr) * 2015-01-19 2016-07-28 Dolby Laboratories Licensing Corporation Gestion d'affichage pour vidéo à plage dynamique élevée
US20160381335A1 (en) * 2015-06-24 2016-12-29 Samsung Electronics Co., Ltd. Tone mastering system with creative intent metadata

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080166043A1 (en) * 2007-01-05 2008-07-10 Silicon Optix Inc. Color and geometry distortion correction system and method
US9077994B2 (en) 2011-12-06 2015-07-07 Dolby Laboratories Licensing Corporation Device and method of improving the perceptual luminance nonlinearity-based image data exchange across different display capabilities
US20160005349A1 (en) 2013-02-21 2016-01-07 Dolby Laboratories Licensing Corporation Display Management for High Dynamic Range Video
WO2016118395A1 (fr) * 2015-01-19 2016-07-28 Dolby Laboratories Licensing Corporation Gestion d'affichage pour vidéo à plage dynamique élevée
US20160381335A1 (en) * 2015-06-24 2016-12-29 Samsung Electronics Co., Ltd. Tone mastering system with creative intent metadata

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
F. EBNER; M.D. FAIRCHILD: "Proc. 6th Color Imaging Conference: Color Science, Systems, and Applications, IS&T", November 1998, article "Development and testing of a color space (ipt) with improved hue uniformity", pages: 8 - 13
HIGH DYNAMIC RANGE ELECTRO-OPTICAL TRANSFER FUNCTION OF MASTERING REFERENCE DISPLAYS, 16 August 2014 (2014-08-16)

Similar Documents

Publication Publication Date Title
US10713765B2 (en) Color image modification with approximation function
EP2498499B1 (fr) Interpolation de gamme de couleurs pour affichage sur un affichage cible
US10313687B2 (en) Saturation processing specification for dynamic range mappings
CN101243680B (zh) 硬件加速色彩数据处理的方法和系统
US10798347B2 (en) Information processing device, information processing method, and program
CN101461250A (zh) 将色度变换从输入色彩空间转换到输出色彩空间
TW202139137A (zh) 用於在高動態範圍信號中色彩轉換的方法、設備及非暫態電腦可讀儲存媒體
US10832636B2 (en) Image processing apparatus, image processing method, and program
EP2324636A1 (fr) Procédé et système de fourniture de contenu
EP3070925B1 (fr) Mise en correspondance de couleurs pour systèmes d'imagerie
CN113851066B (zh) 在具有三维（3d）功能的显示器上呈现宽色域、二维（2d）图像
CN105122795A (zh) 用于数字图像创建和显示的图像外观框架和应用
EP3566430A1 (fr) Ajustement de saturation de couleur dans un espace de couleur non uniforme
US7764832B2 (en) System and method for processing image data for color space conversion
EP3087725A1 (fr) Procédé de mappage de couleurs sources d'images d'un contenu vidéo à la gamme de couleurs cibles d'un dispositif de couleur cible
CN115294945A (zh) 对象展示、生成颜色查找表的方法及装置
JP6711275B2 (ja) 画像表示制御装置、送信装置、および画像表示制御方法、並びにプログラム
WO2018160847A1 (fr) Modification d'image couleur avec fonction d'approximation
JP2016534603A (ja) 色調整モニター、色調整システム及び色調整方法
WO2018160883A1 (fr) Ajustement de saturation de couleur dans un espace de couleur non uniforme
US20160156812A1 (en) Method and apparatus for interactive color manipulation
US10542207B2 (en) Image processing apparatus and image processing method
JP2016208430A (ja) 映像表示装置およびシステム
US20140218410A1 (en) System And Method For Efficiently Generating Device-Dependent Anaglyph Images
JP2024538575A (ja) メタデータを使用した複数意図画像およびビデオのエンコードおよびデコード

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18708895

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 2018708895

Country of ref document: EP

Effective date: 20190806

NENP Non-entry into the national phase

Ref country code: DE