[go: up one dir, main page]

WO2018039665A1 - Remodelage de la chrominance sur la base de différences à peine perceptibles - Google Patents

Remodelage de la chrominance sur la base de différences à peine perceptibles Download PDF

Info

Publication number
WO2018039665A1
WO2018039665A1 PCT/US2017/048860 US2017048860W WO2018039665A1 WO 2018039665 A1 WO2018039665 A1 WO 2018039665A1 US 2017048860 W US2017048860 W US 2017048860W WO 2018039665 A1 WO2018039665 A1 WO 2018039665A1
Authority
WO
WIPO (PCT)
Prior art keywords
chroma
channel
computer
codewords
bins
Prior art date
Application number
PCT/US2017/048860
Other languages
English (en)
Inventor
Cheng-Chi Li
Guan-Ming Su
Original Assignee
Dolby Laboratories Licensing Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dolby Laboratories Licensing Corporation filed Critical Dolby Laboratories Licensing Corporation
Priority to US16/327,758 priority Critical patent/US10602110B2/en
Priority to EP17758995.9A priority patent/EP3504875B1/fr
Publication of WO2018039665A1 publication Critical patent/WO2018039665A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/64—Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor
    • H04N1/648—Transmitting or storing the primary (additive or subtractive) colour signals; Compression thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/648—Video amplifiers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/68—Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits

Definitions

  • Fig.1 illustrates an example of a test pattern.
  • Fig.2 illustrates an example of a forward reshaping function.
  • Fig. 3 illustrates an exemplary system for an encoder according to the methods of the present disclosure.
  • Fig. 4 illustrates an exemplary flowchart for one embodiment of the methods of the present disclosure.
  • Fig.5 depicts an exemplary embodiment of a target hardware for implementation of an embodiment of the present disclosure.
  • This disclosure relates to methods and systems for chroma reshaping.
  • the method comprises receiving at least one image or video frame.
  • the color space of the at least one image or video frame is partitioned in M 1 x M 2 x M 3 non-overlapping bins.
  • For each bin it is determined whether it is a valid bin, for which the at least one image or video frame has at least one pixel with a color value falling within said bin.
  • a required number of codewords is calculated for representing two color values in said valid bin that have consecutive codewords for the respective chroma channel without a noticeable difference.
  • At least one content-aware chroma forward reshaping function is generated based on the calculated required numbers of codewords and applied to the at least one image or video frame.
  • the disclosure further relates to a computer program product having instructions which, when executed by a computing device or system, cause said computing device or system to perform any of the methods of this disclosure.
  • DETAILED DESCRIPTION [0011] The present disclosure describes methods to reshape chroma based on just noticeable differences (JNDs). As known to the person of ordinary skill in the art, a just noticeable difference refers to the human perception of small differences in video processing parameters.
  • the hue or intensity of a color may be changed slightly, and the difference may or may not be perceived by a viewer, depending on the numerical value of the change in the parameters, and on the quality of perception of the viewer.
  • a change in a video parameter can be considered as just noticeable, for example, when a viewer will detect it at least half the time.
  • EDR extended dynamic range
  • HDR high dynamic range
  • frames with an extended dynamic range refer to frames which have a dynamic range which is broader than a standard dynamic range.
  • the term 'dynamic range' may relate to a capability of the human visual system (HVS) or a display to perceive or display a range of intensity (e.g., luminance, luma) in an image, e.g., from darkest blacks (darks) to brightest whites (highlights).
  • HVS human visual system
  • high dynamic range relates to a dynamic range breadth that spans the some 14-15 or more orders of magnitude of the human visual system (HVS).
  • EDR may relate to a DR that spans 5 to 6 orders of magnitude; thus while perhaps somewhat narrower in relation to true scene referred HDR, EDR nonetheless represents a wide DR breadth and may also be referred to as HDR.
  • the present disclosure relates to JNDs in the perception of colors.
  • the required codeword to distinguish two colors by a JND, in the chroma channel is not uniform and can change with the color range and channel.
  • a non-linear reshaping function can be used to equalize the non-uniform JND.
  • the reshaping function may be single-channel or multi-channel.
  • Forward reshaping may refer to the process of mapping (or quantizing) an image from its original bit depth to an image of a lower or the same bit depth, to allow compressing the image using existing coding standards or devices.
  • Forward reshaping may be applied in an encoder.
  • a receiver or decoder
  • the receiver may apply an inverse reshaping function.
  • the present disclosure describes how to generate a reshaping function based on methods to calculate a just-noticeable-difference (JND) metric, for determining a reshaping function for chroma, using the ICtCp color space.
  • JND just-noticeable-difference
  • the ICtCp color space is a constant intensity color space based on opposing colors, and is intended to separate the luma and chroma information.
  • the Recommendation ITU-R BT.2100-0 07/2016
  • Image parameter values for high dynamic range television for use in production and international programme exchange ITU, July, 2016 explains how to convert from RGB to ICtCp, and is incorporated herein by reference in its entirety.
  • the methods described herein may be applied to other color spaces as well, such as the IPT color space.
  • each chroma channel can be partitioned into multiple non-overlapping bins.
  • a color histogram of an image can be produced first by discretization of the colors in the image into a number of bins, and counting the number of image pixels in each bin. For each bin, it is possible to calculate the required number of codewords necessary to observe a difference in color in each bin. The required number of codewords represents the bit depth necessary to avoid quantization distortion and color artifacts. [0017] After the bit depth measurement, it is possible to generate a content-aware chroma reshaping function by checking the color distribution, building a codeword bit depth look-up table (LUT), and form a forward reshaping LUT.
  • LUT codeword bit depth look-up table
  • the reshaping function is generated based on the image quantization.
  • two exemplary approaches can be taken according to the hardware available to implement the video encoding algorithms. However, other embodiments may be based on different approaches.
  • the algorithm comprises the following steps. After the bit depth measurement, which establishes the required bit depth to avoid color artifacts, the content- aware chroma reshaping function is generated by checking the color distribution, building the codeword bit depth LUT, and forming the forward reshaping LUT. The backward reshaping LUT can be obtained via inverse scanning of the forward reshaping LUT.
  • the backward reshaping LUT can be approximated by a multi-piece 2 nd order polynomial function.
  • the reshaping function is multi-channel. This process can be termed multi-channel reshaping (to be referred to as MMR).
  • MMR multi-channel reshaping
  • a 3D-LUT is generated, based on the JND, to map multiple input channels to the target color channel, to obtain a first reshaped signal. Subsequently, the 3D-LUT can be approximated, based on MMR.
  • the backward reshaping function is obtained via applying MMR again based on the targeted reshaped signal and original extended dynamic range (EDR) signal.
  • EDR extended dynamic range
  • the partition may comprise M 1 x M 2 x M 3 parallelepipeds (e.g., 16 x 32 x 24) instead of cubes.
  • the color space may be partitioned into M1 x M2 x M3 cuboid bins.
  • the number of codewords in each parallelepiped bin in each direction is
  • the color cube index (i, t, p) can be used, where 0 ⁇ i ⁇ M; 0 ⁇ t ⁇ M; 0 ⁇ p ⁇ M.
  • the center of each cube in each direction is
  • the JND measurement can be carried out for each chroma channel.
  • two measurements can be carried out for a first and second chroma channel.
  • CT channel For the first chroma channel (CT channel), the following algorithm can be used:
  • V[] 0;
  • First block has color
  • V[k] (observe two blocks have different colors) ? 1 : 0
  • V[] 0;
  • First block has color
  • V[k] (observe two blocks have different colors) ? 1 : 0
  • the JND measurements ⁇ p (i,t, p ) ⁇ and ⁇ t (i,t, p ) ⁇ can serve as the minimal required bit depth when constructing the reshaping function.
  • Fig.1 illustrates an example for the test patterns. In this example, the ICtCp color space will be used.
  • the JND measurements ⁇ p (i,t, p ) ⁇ and ⁇ t (i,t, p ) ⁇ are expressed in units of bit depth, these measurements do not need to correspond to integer values.
  • k loops over -K to K in steps of 0.5, so that the resulting ⁇ p (i,t, p ) ⁇ and ⁇ t (i,t, p ) ⁇ may also be a fraction.
  • N p min and N t min represent the required number of codewords for the P and T chroma channel respectively.
  • the values for I, Ct, and Cp are fixed for the background (105), while only the value in one color channel is changed in each rectangle (107).
  • the codeword value difference (between the rectangles and the background) is increased positively for the top row (110) of rectangles, while the codeword value difference is decreased negatively for the bottom row (115) of rectangles.
  • a minimal absolute codeword value difference which can be expressed as minimal bit depth, is selected to show the color difference, which is the JND. In other words, the bit depth is adjusted to the appropriate value for the JND in color.
  • the largest value (amongst the minimal bit depths) is taken as JND for value (I,Ct,Cp), for one particular color direction.
  • the bit depth of the (I,Ct,Cp) value in the rectangles is varied to find the JND.
  • the largest bit depth (among the minimal bit depths) required for the JND in the rectangles is taken as the overall necessary bit depth required for the JND.
  • 16 x 16 x 16 pictures can be used to measure the JND in each color channel. For example, the top left and bottom left rectangles in Fig. 1 could start at the same codeword value difference, or at different codeword value differences.
  • the reshaping chroma function is calculated for a baseline profile of a codec.
  • a single channel function is generated. More specifically, the baseline profile only allows single-channel multiple-piece polynomials in each channel. Therefore, no cross-channel reshaping method can be used.
  • To calculate the forward reshaping function the following steps can be applied: Denote the k th pixel in the I, Ct, Cp channel from the input extended dynamic range (EDR) at frame j as and Denote the k th pixel in the I, Ct, Cp channel for the output reshaped
  • the pixel count is A color cube with a non-zero pixel
  • the maximum among the calculated JND bit depths is found, for each of the M values of p and t respectively.
  • the maximum among the calculated JND bit depths can be searched along the C p channel direction with the following algorithm:
  • the maximum is found using valid bins, that is, bins where the pixel count h is greater than zero.
  • the values n the algorithms above are the
  • the JND factors that allow adjustment of the JND. These factors can be termed adjustment factors.
  • the not just noticeable difference is used (N-JND).
  • the value ⁇ p can be calculated as:
  • bit depth adjustment factor ⁇ t can be calculated as 1 / log2(Nt).
  • the resulting quantities Qt and Qp are content-aware, as they depend on the specific counts for the bins of the specific frames being analyzed and processed.
  • the forward reshaping function can be constructed. In the following, the method is described with reference to each chroma channel. Without loss of generality, index m is used in the following to denote either p in the P channel or t in the T channel. [0039] Considering that there are M bins, the M just noticeable differences (JNDs) for these M bins can be mapped to M required normalized codewords.
  • the normalized required codeword can be defined as:
  • B T is the bit depth in the target domain (container for the output signal, such as 10 bits for HEVC 10 compression).
  • Dm 0.5, for the normalized EDR domain.
  • ⁇ di ⁇ is the lower bound required codeword
  • W is the bin width. Any quantization curve should satisfy this lower bound to avoid generating other color artifacts.
  • the resulting curve can be smoothened via, for example, simple moving average. The smoothing process can be carried out multiple times to ensure that the curve is smooth and that the reverse curve is easy to be approximated by a multi-piece 2 nd order polynomial. Therefore, in some embodiments, a multi-piece 2 nd order polynomial can be used.
  • a simple moving average filter can be used for smoothing. To ensure the resultant curve is smooth enough, it can be necessary to cover two bin widths, that is, 2L.
  • An exemplary moving average filter is: where In some embodiments, a typical value of W allows averaging over at
  • each bin contains 4096 codewords and 2W should be no less than 4096 in order to cover one bin. In other embodiments, W is no less than 1024.
  • the resulting smooth curve can be denoted as To build the reshaping curve, the cumulative sum of can be calculated as:
  • the resultant reshaping curve can be built starting from 0. To maintain a good coding efficiency, it may be better to move to the center point:
  • Fig. 2 illustrates an exemplary forward reshaping function, where the output reshaped Ct codewords are obtained from the input EDR Ct codewords. It can be noted that the lower values (205) and the higher values (210) are clipped, as the reshaping is reducing the number of codewords.
  • the backward reshaping function can be built by searching the reverse tracking of the forward reshaping function.
  • the JND-based chroma reshaping function can be generated allowing cross-color channel reshaping. There are several methods to implement multi- dimensional reshaping to take advantage of a non-uniform JND.
  • Multi-dimensional reshaping can be implemented, for example, by constructing a three- dimensional JND curve.
  • the JND measurement represents the maximal quantization step size possible.
  • the JND measurement also represents the differential value when building the 1D- LUT for reshaping.
  • the 1D-LUT can be built via the cumulative JND value.
  • the JND cube in 3D represents the difference value in 3D space.
  • the 3D-LUT can be also built via constructing a cumulative JND cube. For example, the following calculation can be carried out to generate the 3D-LUT:
  • the adjustment factors ⁇ t and ⁇ p can be taken into account for calculating C t and Cp as well, by multiplying ⁇ t and ⁇ p by ⁇ t and ⁇ p respectively.
  • the calculation of Ct and Cp then becomes:
  • the 3D-LUT as calculated by the method above can be highly non-linear.
  • a 3D low pass filter may be needed to smoothen it.
  • 3D convolution can be applied for the smoothing step.
  • For each input(i,t, p ) there are correspondingC t (i,t, p ) andC p (i,t, p ) .
  • This problem can be formulated as a polynomial regression to find the mapping from (i,t, p ) to C t (i,t, p ) and C p (i,t, p ) via, for example, a multi-channel reshaping (MMR) technique.
  • MMR multi-channel reshaping
  • polynomial terms may be a function of variables from several channels.
  • polynomial terms may be a function of a first and second chroma channel, a function of a chroma channel and a luma channel, or both chroma channels and the luma channel, depending on the embodiment.
  • the MMR vector can be constructed based on For a 2 nd order MMR, the
  • the vector comprises 15 coefficients, while for a 3 rd order MMR, the vector comprises 22 coefficients.
  • the MMR vectors are described, for example, in US Patent Publication No. 8,811,490 B2.
  • the vector may be expressed as
  • the two chroma channels mapped via C t (i,t, p ) and C p (i,t, p ) can be denoted as:
  • the chroma value predicted via MMR can be denoted as:
  • the MMR matrix can be obtained by minimizing a cost
  • the cost function ensures that the predicted standard dynamic range (SDR) value
  • the least square method can be used, to obtain the optimal chroma forward reshaping function
  • an image or video can be reshaped by a three dimensional LUT calculated via a cumulative sum.
  • An MMR matrix can be generated via cost minimization between the chroma values predicted by the MMR matrix and the chroma values mapped by the three dimensional LUT.
  • the method above can be extended to a scene based solution.
  • the backward reshaping function can be generated, to map it back to the original signal:
  • channels in the original EDR frames can be termed as:
  • the chroma value predicted via MMR can be termed as:
  • the purpose is to obtain the MMR backward reshaping matrix
  • the JND method described above has several advantages over other methods, such as the uniform quantizer.
  • the JND method requires a smaller number of codewords than the uniform quantizer.
  • increasing the JND to 48, thereby reducing the codewords to about 30, can display color artifacts.
  • the methods described above can be carried out in an encoder. As described above, the entire reshaping process contains 1) forward reshaping and 2) backward reshaping. Forward reshaping can be carried out at the encoder side.
  • the backward reshaping function can be generated at the encoder side and then transmitted through a metadata bitstream.
  • a decoder receives the metadata, and backward reshaping can be carried out at the decoder side via that metadata.
  • forward reshaping can be carried out in LUT format.
  • the function can be compressed to save bandwidth.
  • the backward reshaping function can be compressed by using a multi piece 2 nd order polynomial.
  • Fig. 3 illustrates an exemplary system for an encoder according to the methods of the present disclosure.
  • an input image or video frame is processed in a forward reshaping function module (305).
  • the image or frame can then be processed for any further coding (310), while a backward shaping module (315) can generate a backward shaping function.
  • the modules of Fig. 3 can be implemented as separate processors or as a single processor in an encoder.
  • An output image or video frame is then transmitted, for example to a decoder.
  • the actual backward shaping can be applied to an image at a decoder based on metadata generated by the backward shaping module (315) which generates the backward shaping function at the encoder. Therefore, a forward reshaped signal can be encoded at the encoder, and transmitted with the relevant metadata to the decoder.
  • Fig. 4 illustrates an exemplary flowchart for one embodiment of the methods of the present disclosure. After receiving an image or video frame (405), the color space is partitioned (410), a color histogram is generated (415), the required number of codewords is generated for valid bins (420), the maximal bit depth to avoid color artifacts is found (425), and the forward reshaping function is generated (430). [0073] Fig.5 is an exemplary embodiment of a target hardware (10) (e.g., a computer system) for implementing the embodiments of Figs. 1-4.
  • a target hardware 10 (e.g., a computer system) for implementing the embodiments of Figs. 1-4.
  • This target hardware comprises a processor (15), a memory bank (20), a local interface bus (35) and one or more Input/Output devices (40).
  • the processor may execute one or more instructions related to the implementation of Figs.1- 4, and as provided by the Operating System (25) based on some executable program (30) stored in the memory (20). These instructions are carried to the processor (15) via the local interface (35) and as dictated by some data interface protocol specific to the local interface and the processor (15).
  • the local interface (35) is a symbolic representation of several elements such as controllers, buffers (caches), drivers, repeaters and receivers that are generally directed at providing address, control, and/or data connections between multiple elements of a processor based system.
  • the processor may be fitted with some local memory (cache) where it can store some of the instructions to be performed for some added execution speed. Execution of the instructions by the processor may require usage of some input/output device (40), such as inputting data from a file stored on a hard disk, inputting commands from a keyboard, inputting data and/or commands from a touchscreen, outputting data to a display, or outputting data to a USB flash drive.
  • the operating system (25) facilitates these tasks by being the central element to gathering the various data and instructions required for the execution of the program and provide these to the microprocessor.
  • the operating system may not exist, and all the tasks are under direct control of the processor (15), although the basic architecture of the target hardware device (10) will remain the same as depicted in Fig. 5.
  • a plurality of processors may be used in a parallel configuration for added execution speed.
  • the executable program may be specifically tailored to a parallel execution.
  • the processor (15) may execute part of the implementation of Figs.1-4, and some other part may be implemented using dedicated hardware/firmware placed at an Input/Output location accessible by the target hardware (10) via local interface (35).
  • the target hardware (10) may include a plurality of executable programs (30), wherein each may run independently or in combination with one another. [0074]
  • a method comprises:
  • the at least one image or video frame comprises a plurality of pixels, each having a color value.
  • the color value comprises a luma value, a first chroma value and a second chroma value.
  • Each bin of the color space partitioning spans a range of luma values in a luma direction, a range of first chroma values in a first chroma direction and a range of second chroma values in a second chroma direction.
  • the required number of codewords corresponds to the minimum number of codewords necessary for representing the chroma range of the respective bin without noticeable color artefacts.
  • the number L is in general greater than necessary to represent the chroma channel: no color difference will be noticeable between colors having a chroma value represented by two consecutive codewords.
  • the method of embodiments of this disclosure in general reduces the number of codewords used for each bin.
  • the method calculates a minimum number of codewords for each valid bin that still ensures that no noticeable difference occurs between two color values in said valid bin that have consecutive codewords for the respective chroma channel.
  • the calculated minimum number of codewords can be used to re-quantize the input chroma codewords to (a typically smaller number of) output chroma codewords without a noticeable difference in color.
  • the required number of codewords correspond to a required number of quantization levels in a valid bin necessary to represent values neighboring in color space without a noticeable difference.
  • At least one content-aware forward reshaping function is calculated.
  • a single channel chroma forward reshaping function is generated for each chroma channel.
  • the at least one content-aware chroma forward reshaping function comprises a multi channel chroma forward reshaping function.
  • the M 1 x M 2 x M 3 non-overlapping bins of the partitioning of the color space comprise M1 arrays of M2 x M3 bins in a luma direction, M2 arrays of M1 x M3 bins in a first chroma direction and M3 arrays of M1 x M2 bins in a second chroma direction.
  • the method further comprises: calculating, by the computer, for each of the M2 arrays in the first chroma direction, a first maximum value among the calculated required numbers of codewords of the valid bins in the respective array; and calculating, by the computer, for each of the M3 arrays in the second chroma direction, a second maximum value among the calculated required numbers of codewords of the valid bins in the respective array.
  • the bins can be identified by a bin index (i luma , i first chroma , i second chroma ), wherein 0 ⁇ iluma ⁇ M1 is an index for the luma direction, 0 ⁇ ifirst chroma ⁇ M2 is an index for the first chroma direction and 0 ⁇ i second chroma ⁇ M 3 is an index for the second chroma direction.
  • the color space is the ICTCP color space
  • (i luma , i first chroma , i second chroma ) (i, t, p).
  • the method may further comprise: building a codeword look-up table.
  • the method may further comprise: calculating a cumulative sum of the calculated required numbers of codewords over the M1 x M2 x M3 non-overlapping bins to obtain a three dimensional look- up table.
  • this embodiment may calculating C t (i, t, p) and C p (i, t, p) as described above.
  • the method may comprise calculating a multi-channel forward reshaping matrix by minimizing a cost function between chroma values of the at least one image or video frame predicted by the multi-channel forward reshaping matrix and chroma values obtained by mapping the at least one image or video frame using the three dimensional look-up table.
  • this embodiment may calculate as described above.
  • the method may further comprise generating, by the computer, at least one backward reshaping function based on the at least one content-aware chroma forward reshaping function.
  • the method may comprise approximating, by the computer, the at least one backward reshaping function by a polynomial function, such as a multi-piece 2 nd order polynomial function.
  • the method may comprise generating a look-up table via inverse scanning of the at least one content-aware chroma forward reshaping function.
  • generating the at least one backward reshaping function may comprise calculating a multi-channel backward reshaping matrix by minimizing a cost function between chroma values of the at least one image or video frame predicted by the multi-channel backward reshaping matrix and chroma values resulting from applying the multi-channel chroma forward reshaping function to the at least one image or video frame.
  • the step of calculating, by the computer, for the respective chroma channel the required number of codewords for each valid bin comprises:
  • a content independent partitioning of the color space for said respective chroma channel that has M1 x M2 x M3 non-overlapping bins, with M1 rows of M2 x M3 bins in the luma direction, M 2 rows of M 1 x M 3 bins in the first chroma direction and M 3 rows of M1 x M2 bins in the second chroma direction, wherein each bin is linked to a required number of codewords.
  • a pre-determined M 1 x M 2 x M 3 array is accessed, each element of the array storing a required number of codewords.
  • the array is content independent, and may be obtained by performing tests, e.g.
  • the method step of calculating a required number of codewords for each valid bin then comprises copying the required number of codewords stored in the corresponding element of the pre-determined M 1 x M 2 x M 3 array.
  • the methods and systems described in the present disclosure may be implemented in hardware, software, firmware or any combination thereof. Features described as blocks, modules or components may be implemented together (e.g., in a logic device such as an integrated logic device) or separately (e.g., as separate connected logic devices).
  • the software portion of the methods of the present disclosure may comprise a computer-readable medium which comprises instructions that, when executed, perform, at least in part, the described methods.
  • the computer-readable medium may comprise, for example, a random access memory (RAM) and/or a read-only memory (ROM).
  • the instructions may be executed by a processor (e.g., a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable logic array (FPGA), a graphic processing unit (GPU) or a general purpose GPU).
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable logic array
  • GPU graphic processing unit
  • general purpose GPU e.g., a graphics processing unit
  • EEE 2 The method of EEE 1, wherein generating, by the computer, a content-aware chroma forward reshaping function comprises building a codeword bit depth look-up table.
  • EEE 3 The method of EEE 1, wherein the content-aware chroma forward reshaping function is single channel.
  • EEE 4 The method of EEE 3, further comprising generating, by the computer, a backward reshaping function based on the content-aware chroma forward reshaping function.
  • EEE 5 The method of EEE 2, wherein the content-aware chroma forward reshaping function is multi channel and the codeword bit depth look-up table is three dimensional.
  • EEE 6 The method of EEE 4, further comprising approximating, by the computer, the backward reshaping function by a polynomial function.
  • EEE 7 The method of EEE 5, wherein the polynomial function has terms dependent on quantities of a luma channel and a first and second chroma channels.
  • EEE 8 The method of EEE 4, wherein generating, by the computer, the backward reshaping function comprises generating a look-up table via inverse scanning of the content-aware chroma forward reshaping function.
  • EEE 9 The method of EEE 6, wherein the polynomial function is a multi-piece 2 nd order polynomial function.
  • EEE 10 The method of EEE 1, wherein the at least one chroma channel comprises two chroma channels and the required number of codewords is 50 codewords for each chroma channel.
  • EEE 11 The method of EEE 10, wherein the two chroma channels belong to the ICtCp color space.
  • EEE 12 The method of EEE 1, wherein partitioning, by the computer, the at least one chroma channel comprises partitioning, by the computer, a color space in M1 x M2 x M3 non-overlapping parallelepipeds.
  • EEE 13 The method of EEE 1, wherein calculating, by the computer and for each valid bin of the plurality of valid bins, a required number of codewords comprises generating a test pattern.
  • EEE 14 The method of EEE 13, wherein generating a test pattern comprises:
  • EEE 15 The method of EEE 12, wherein generating, by the computer, a content-aware chroma forward reshaping function comprises calculating, for the at least one image or video frame, a maximal value and a minimal value in a luma channel of the color space, a maximal value and a minimal value in a first color channel of the color space, a maximal value and a minimal value in a second color channel of the color space, and a pixel count for each non-overlapping cube.
  • EEE 16 The method of EEE 1, further comprising smoothing the content-aware chroma forward reshaping function.
  • EEE 17 The method of EEE 16, wherein smoothing is by moving average.
  • EEE 18 The method of EEE 1, wherein obtaining a minimum bit depth is for each valid bin, and generating, by the computer, a content-aware chroma forward reshaping function comprises summing the minimum bit depth over the plurality of non-overlapping bins to obtain a look-up table.
  • EEE 19 The method of EEE 10, wherein obtaining a minimum bit depth is for each valid bin, and generating, by the computer, a content-aware chroma forward reshaping function comprises summing the minimum bit depth over the plurality of non-overlapping bins to obtain a three dimensional look-up table.
  • EEE 20 The method of EEE 19, wherein generating, by the computer, a content-aware chroma forward reshaping function further comprises calculating a multi-channel forward reshaping matrix by minimizing a cost function between first chroma values of the at least one image or video frame predicted by the multi-channel forward reshaping matrix and second chroma values of the at least one image or video frame mapped by the three dimensional look-up table.
  • EEE 21 The method of EEE 6, further comprising generating metadata for the approximated backward reshaping function, and transmitting the metadata to a decoder.
  • EEE 22 The method of EEE 1, wherein: partitioning, by the computer, the at least one chroma channel in a plurality of non- overlapping bins comprises partitioning a first chroma channel in a first plurality of non- overlapping bins and a second chroma channel in a second plurality of non-overlapping bins,
  • calculating a pixel count is for the first and second pluralities of non-overlapping bins, calculating a required number of codewords comprises calculating a first number of codewords for each valid bin of the first plurality of valid bins, and a second number of codewords for each valid bin of the second plurality of valid bins,
  • calculating a maximal bit depth based on a maximum value among the required number of codewords comprises calculating a first maximal bit depth based on the first number of codewords and a second maximal bit depth based on the second number of codewords, and generating, by the computer, a content-aware chroma forward reshaping function is based on the first and second maximal bit depths.
  • EEE 23 The method of EEE 1, wherein calculating a maximal bit depth necessary to avoid color artifacts in the at least one image or video frame is further based on an adjustment factor.
  • EEE 24 The method of EEE 22, wherein calculating a first maximal bit depth is further based on a first adjustment factor ⁇ t , and calculating a second maximal bit depth is further based on a second adjustment factor ⁇ p.
  • EEE 25 The method of EEE 17, wherein smoothing comprises averaging over a width of two bins and taking a cumulative sum over the plurality of valid bins.
  • EEE 26 The method of EEE 1, wherein the at least one image or video frame comprises a plurality of images or video frames forming one scene in a video.
  • EEE 27 The method of EEE 5, further comprising generating, by the computer, a backward reshaping function based on the content-aware chroma forward reshaping function.
  • EEE 28 The method of EEE 20, further comprising generating, by the computer, a backward reshaping function based on the content-aware chroma forward reshaping function.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne des procédés et des systèmes de remodelage de la chrominance qui sont appliqués à des images ou à des trames vidéo. Le procédé consiste à recevoir au moins une image ou une trame vidéo. L'espace colorimétrique de l'image ou des images ou de la trame vidéo ou des trames vidéo est divisé en M1 x M2 x M3 segments non chevauchants. Pour chaque segment, on détermine s'il s'agit d'un segment valide, pour lequel l'image ou les images ou la trame vidéo ou les trames vidéo comportent au moins un pixel dont la valeur de couleur se trouve dans ledit segment. Pour chaque canal de chrominance, un nombre requis de mots de code est calculé pour représenter deux valeurs de couleur dans ledit segment valide qui comportent des mots de code consécutifs pour le canal de chrominance respectif sans différence perceptible. Au moins une fonction de remodelage avant de la chrominance sensible au contenu est générée sur la base des nombres requis et calculés de mots de code et est appliquée à l'image ou aux images ou à la trame vidéo ou aux trames vidéo.
PCT/US2017/048860 2016-08-26 2017-08-28 Remodelage de la chrominance sur la base de différences à peine perceptibles WO2018039665A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US16/327,758 US10602110B2 (en) 2016-08-26 2017-08-28 Chroma reshaping based on just noticeable differences
EP17758995.9A EP3504875B1 (fr) 2016-08-26 2017-08-28 Remodelage de chrominance sur la base de différences juste perceptibles

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201662379930P 2016-08-26 2016-08-26
EP16185961.6 2016-08-26
US62/379,930 2016-08-26
EP16185961 2016-08-26

Publications (1)

Publication Number Publication Date
WO2018039665A1 true WO2018039665A1 (fr) 2018-03-01

Family

ID=56851436

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/048860 WO2018039665A1 (fr) 2016-08-26 2017-08-28 Remodelage de la chrominance sur la base de différences à peine perceptibles

Country Status (1)

Country Link
WO (1) WO2018039665A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113647111A (zh) * 2019-03-13 2021-11-12 交互数字Vc控股公司 环内重整形自适应重整形方向

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8811490B2 (en) 2011-04-14 2014-08-19 Dolby Laboratories Licensing Corporation Multiple color channel multiple regression predictor
US20160134872A1 (en) * 2013-06-17 2016-05-12 Dolby Laboratories Licensing Corporation Adaptive Reshaping for Layered Coding of Enhanced Dynamic Range Signals

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8811490B2 (en) 2011-04-14 2014-08-19 Dolby Laboratories Licensing Corporation Multiple color channel multiple regression predictor
US20160134872A1 (en) * 2013-06-17 2016-05-12 Dolby Laboratories Licensing Corporation Adaptive Reshaping for Layered Coding of Enhanced Dynamic Range Signals

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
KEROFSKY LOUIS ET AL: "Recent developments from MPEG in HDR video compression", 2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), IEEE, 19 August 2016 (2016-08-19), pages 879 - 883, XP033016651, DOI: 10.1109/ICIP.2016.7532483 *
PU F ET AL: "Comments on Reshaping for HDR/WCG compression", 113. MPEG MEETING; 19-10-2015 - 23-10-2015; GENEVA; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. m37267, 21 October 2015 (2015-10-21), XP030065635 *
TAORAN LU ET AL: "ITP Colour Space and Its Compression Performance for High Dynamic Range and Wide Colour Gamut Video Distribution", 5 February 2016 (2016-02-05), XP055282146, Retrieved from the Internet <URL:http://wwwen.zte.com.cn/endata/magazine/ztecommunications/2016/1/articles/201603/P020160311294995790791.pdf> [retrieved on 20160620] *
ZICONG MAI ET AL: "Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression", IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 20, no. 6, 1 June 2011 (2011-06-01), pages 1558 - 1571, XP011411814, ISSN: 1057-7149, DOI: 10.1109/TIP.2010.2095866 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113647111A (zh) * 2019-03-13 2021-11-12 交互数字Vc控股公司 环内重整形自适应重整形方向
US12081744B2 (en) 2019-03-13 2024-09-03 Interdigital Vc Holdings, Inc. In-loop reshaping adaptive reshaper direction

Similar Documents

Publication Publication Date Title
KR102230776B1 (ko) 이미지/비디오 처리를 위한 선형 인코더
EP3203442B1 (fr) Contenu block-restauration adaptative pour images hdr
US10506243B2 (en) Image decoding method and device therefor, and image encoding method and device therefor
CN104322072B (zh) 分层vdr编码方法以及编码器
US10291939B2 (en) Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method
CN112913237A (zh) 使用深度神经网络的人工智能编码和人工智能解码方法和设备
US20130287312A1 (en) Image encoding device, image decoding device, image encoding method, and image decoding method
US11341624B2 (en) Reducing banding artifacts in HDR imaging via adaptive SDR-to-HDR reshaping functions
EP3510772A1 (fr) Codage vidéo à plage dynamique élevée à l&#39;aide d&#39;un remodelage basé sur des segments
EP3198556A1 (fr) Codage et décodage de contenu vidéo quantifié perceptivement
US10699671B2 (en) Pixel processing with color component
JP2016213528A (ja) 画像符号化装置、画像処理装置、画像符号化方法
KR20140103928A (ko) 상이한 디스플레이 능력들을 통한 지각적 휘도 비선형성―기반 이미지 데이터 교환을 향상시키는 디바이스 및 방법
JP2010183402A (ja) 映像信号符号化装置及び、映像信号符号化方法
WO2018049335A1 (fr) Codage vidéo à plage dynamique élevée à l&#39;aide d&#39;un remodelage basé sur des segments
EP3659339B1 (fr) Mappage inverse à canal unique pour un traitement d&#39;image/vidéo
WO2019023202A1 (fr) Mappage inverse monocanal pour traitement d&#39;image/vidéo
US10602110B2 (en) Chroma reshaping based on just noticeable differences
CN108347607B (zh) 固定码率且基于行的嵌入式视频压缩方法和图像处理设备
US12177459B2 (en) Rate-control-aware reshaping in HDR imaging
WO2018039665A1 (fr) Remodelage de la chrominance sur la base de différences à peine perceptibles
KR20240056752A (ko) 모바일 애플리케이션들에서의 hdr 비디오를 위한 텐서-곱 b-스플라인 예측
JP2015103969A (ja) 画像符号化装置及び画像符号化方法
HK1202742B (en) Method for hierarchical vdr encoding and encoder
JP2015115735A (ja) 画像符号化装置、画像符号化方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17758995

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017758995

Country of ref document: EP

Effective date: 20190326