HK1210558B - Global display management based light modulation - Google Patents

Description

Light modulation based on global display management

Cross Reference to Related Applications

This application claims priority from U.S. provisional patent application No.61/756713 filed on 25/1/2013, which is incorporated herein by reference in its entirety.

Technical Field

The present invention generally relates to image data. More particularly, one embodiment of the invention relates to light modulation based on global display management.

Background

Some existing displays have relatively narrow dynamic ranges (e.g., SDR, LDR, etc.) compared to High Dynamic Range (HDR) displays. These displays include projectors or display systems based on rec.709 and the Digital Cinema Initiative (DCI) specification. The dynamic range of these display systems is limited in part by the display capabilities that dominate when defining those standards (such as CRT displays) or by the minimum and maximum brightness levels specified in those standards and due to other technical limitations in the system.

The images initially captured by HDR or extended dynamic range cameras may have a scene dependent Dynamic Range (DR) that is significantly larger than the narrow dynamic range supported by the aforementioned display devices. Display manufacturers attempt to avoid this problem by adjusting global display parameters, such as minimum brightness level, maximum brightness level, and gamma value, to create a high dynamic range compression (compression). However, these adjustments cannot increase the simultaneous (or instantaneous) contrast in the image and can best maintain the same relative contrast in the image from the input signal based on the narrow dynamic range information decoded from the input image. In addition, these techniques are expected to produce poor quality when rendering narrow dynamic range versions of HDR images, and introduce additional perceptual artifacts such as clipping, union, color shift, and the like.

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Accordingly, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, issues identified with respect to one or more methods should not be considered to have been identified in any prior art based on this section.

Drawings

The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

FIG. 1 illustrates an exemplary aspect of a display system that supports presenting an input image with limited dynamic range;

FIGS. 2A and 2B illustrate an exemplary mapping from an input image with a wide dynamic range to an output image with a limited dynamic range;

fig. 3 compares LDR display systems with or without global light modulation capability;

fig. 4 illustrates exemplary processing and light modulation paths in an LDR display system with global light modulation capability;

fig. 5 shows two exemplary LDR images rendered by an LDR display system;

FIG. 6 shows an exemplary sequence of dynamic range windows for a sequence of input VDR images in a scene;

FIG. 7 illustrates an exemplary display system;

FIG. 8 illustrates an exemplary process flow; and

FIG. 9 illustrates an exemplary hardware platform on which a computer or computing device described herein may be implemented.

Detailed Description

Example embodiments are described herein that relate to light modulation based on global display management. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It may be evident, however, that the subject invention may be practiced without these specific details. In other instances, well-known structures and devices are not described in detail to avoid unnecessarily obscuring, or obscuring the present disclosure.

Exemplary embodiments are described herein according to the following outline:

1. general overview

2. Non-perceptual adjustment of absolute brightness levels

3. Perceptual adjustment of absolute brightness levels

4. Comparison of display systems

5. Exemplary processing and light modulation paths

6. Non-perceptual adjustment of absolute brightness levels

7. Exemplary display System

8. Exemplary Process flow

9. Implementation mechanisms-hardware overview

10. Equivalents, extensions, substitutions, or the like

1. General overview

This summary gives a basic description of some aspects of embodiments of the invention. It should be noted that this summary is not an extensive or exhaustive overview of the various aspects of the embodiments. Moreover, it should be noted that this summary is not intended to be construed as identifying any particularly important aspect or element of the embodiment, nor as particularly outlining any aspect of the embodiment, nor as limiting the invention as a whole. This summary is provided to represent some concepts related to the exemplary embodiments in a simplified and abstract format only, and should be understood as being a conceptual prelude to the more detailed description of the exemplary embodiments that follows.

According to the techniques described herein, an input image is mapped to an output image through windows having different dynamic ranges (e.g., different instances having limited dynamic ranges through adjustments to absolute minimum and maximum luminance levels, etc.) generated based on a luminance level distribution of the input image.

In one example, when an input image shows one or more salient (salient) objects in a dark luminance level distribution, the input image is mapped to an output image through a dark dynamic range window. For a large number of relatively dark pixels in the input image, a large portion of the dark dynamic range window is used to produce the same or substantially the same (e.g., within 5%, 10%, 20%, etc. of the JND) luminance level. The bright side of the dark dynamic range window is used to master (host) the compressed luminance levels of the remaining relatively bright pixels in the input image.

In another example, when the input image shows one or more salient objects in a bright luminance distribution, the input image is mapped to the output image through a bright dynamic range window. For a large number of relatively bright pixels in the input image, a large portion of the bright dynamic range window is used to produce the same or substantially the same brightness level. The dark side of the bright dynamic range window is used to master (host) the compressed luminance levels of the remaining relatively dark pixels in the input image.

An input image of the video input images may be perceptually encoded. The techniques described herein map a large number of input pixels that capture salient content of a wide dynamic range input image to the same or substantially the same brightness levels in a corresponding output image with a limited dynamic range. The techniques described herein allow pixels having luminance levels outside or at the edges of a dynamic range window to be maximally compressed via display management, rather than merely clipped. The method according to the techniques described herein preserves the perceptual quality of the input image to a greater extent in the output image than other methods.

The methods described herein may also be used to individually globally modulate one or more primary channels (e.g., red, green, blue, or other) of an employed color space while mapping a wide dynamic range input image to an output image having a limited dynamic range. The separate modulation of the main channel re-adjusts the color balance in the color gamut and is beneficial in cases where certain colors (hue and saturation regions in the image) are more dominant than others. For example, in a scene with blue dominated by red and only a few shades (muted), the blue channel LEDs are more attenuated than the other channels, while remaining able to perceive correctly expressing the shades blue, since a large bluish color gamut is not required.

A limited dynamic range display system with global light modulation capability may be cost-effectively implemented relative to a wide dynamic range display system. According to the techniques described herein, a limited dynamic range display system may achieve deep black and rich highlights in an input image from an Extended Dynamic Range (EDR), visual dynamic range (VDR, as described below), or HDR input image. Furthermore, from a statistical point of view, the dynamic range of many input images is located in some dynamic range windows or some instances with limited dynamic range by adjustments to absolute minimum and maximum luminance levels. In this approach, only some pixels may need to be mapped or compressed by display management. Since the dynamic range of the human field of view tends to decrease when both very bright and very dark elements are visible due to glare in the eye, luminance compression for remote luminance levels can be performed without unduly affecting the perceived quality of the output image.

The techniques described herein may also be used to achieve other benefits including, but not limited to, conserving energy, extending the life of the light source elements and electronic components (e.g., LEDs) involved.

In some embodiments, the mechanisms described herein form part of a media processing system that includes, but is not limited to, handheld devices, gaming machines, televisions, laptop computers, tablet computers, notebook computers, cellular radiotelephones, projectors, movie systems, electronic book readers, point of sale terminals, desktop computers, computer workstations, computer kiosks, and various types of terminals and media processing units.

Various modifications to the preferred embodiments and the generic principles and features described herein will be readily apparent to those skilled in the art. Thus, the present disclosure is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.

2. Non-perceptual adjustment of absolute brightness levels

Fig. 1 illustrates an exemplary aspect of a display system that supports rendering input images of a low dynamic range or LDR (102). Examples of LDRs are (without limitation) defined in rec.709 or the Digital Cinema Initiative (DCI) specification. The relatively narrow dynamic range (e.g., 104) of such display systems results from a minimum brightness level and a maximum brightness level (e.g., 0.1 nit and 140 nit, respectively) that may be set simultaneously at a given time (e.g., for an image frame).

Problems with relatively narrow dynamic ranges such as dynamic range (140) may be alleviated by varying the absolute minimum and maximum luminance levels up and down over time based on the overall luminance level of the image. At any given time, both the absolute minimum and maximum brightness levels may be moved up or down together while maintaining the relative brightness level between them.

As shown in fig. 1, the absolute minimum and maximum luminance levels (0.1 nit and 140 nit, respectively) of such a display system may be moved up to the dynamic range (140-1) with the higher absolute minimum and maximum luminance levels, down to the dynamic range (140-3) with the lower absolute minimum and maximum luminance levels, or remain the same, unbiased dynamic range (104-2) as the dynamic range (104). The relative luminance level in the input image may be represented in a particular luminance space (e.g., 8-bit luminance space, 10-bit luminance space, etc.) at a certain bit depth. Changing the absolute minimum and maximum brightness levels in the display system by using a global iris, aperture, light focusing or dispersing components, light modulation devices, etc. in the light path does not change the relative brightness levels represented in the input image. The same relative code values are used to drive the rendering of the input image in the display system.

In some embodiments, the relative brightness level may be represented by an 8-bit code value. A code value (e.g., 0) having a first setting of an absolute minimum brightness level and a maximum brightness level may refer to a first absolute brightness level (e.g., 0.001 nit) at a first time for a first image frame, while the same code value (0) having a second different setting of an absolute minimum brightness level and a maximum brightness level may refer to a second different absolute brightness level (e.g., 0.1 nit) at a second different time for a second different image frame.

Human vision may not be able to perceive the difference between the two brightness levels if they do not differ significantly from each other. In contrast, human vision perceives differences only when the luminance levels differ by no less than Just Noticeable Differences (JNDs). Due to perceptual non-linearity of human vision, the amounts of the respective JNDs are not uniformly sized or scaled over a range of luminance levels, but differ from one another with respect to different respective luminance levels.

In some embodiments, image data in a video signal input to a display system may or may not be perceptually encoded. Even if the input image data is perceptually encoded, after the absolute minimum and maximum luminance levels are raised or lowered, the relative luminance levels in the input image data result in absolute luminance levels that also move up or down. The shifted absolute luminance levels typically do not match the perceptual non-linearity of human vision at the shifted absolute minimum and maximum luminance levels (e.g., 104-1 or 104-3) of fig. 1, except between 0.1 nit and 140 nit as defined in the rec.709 or DCI specification. The mismatch between the perceptual non-linearity of human vision for different ranges of absolute minimum and maximum brightness levels cannot be simply corrected by adjusting global parameters of the input image, such as gamma values.

Thus, adjusting absolute minimum and maximum brightness levels in a display system by using a global iris, aperture, light focusing or dispersing components, light modulation devices, etc. in the light path causes image appearance changes and visual artifacts. Those appearance changes and visual artifacts include, but are not limited to, any of the following: clipping, loss of original perceptible resolution, loss of correct color perception, banding, false contouring, color shift in dark areas of an image, and the like.

3. Perceptual adjustment of absolute brightness levels

In accordance with the techniques described herein, an LDR display system (e.g., an SDR display system, etc.), a mobile device, a tablet computer, etc., is configured to receive a video signal input that provides a relatively wide dynamic range of images that is much wider than the instantaneous dynamic range supported (or expected) by the LDR display system. For example, the instantaneous dynamic range supported by the display system may be comparable to or nearly equal to the dynamic range defined in rec.709, DCI specification, etc., while the dynamic range supported by the video signal input may be HDR, VDR, etc. In some embodiments, the video signal input may carry image data based on perceptual quantization coding techniques developed by Dolby Laboratories, inc.

As shown in fig. 2A, the wide dynamic range (202) encoded in the video signal input received by an LDR display system can cover the wide dynamic range that a wide-range, high-end contemporary display system (having a much larger dynamic range than the LDR of the display system) can support. Examples of wide dynamic range may be, but are not limited to, 001 to 600 nits. As used herein, "VDR" or "visual dynamic range" may refer to a dynamic range (e.g., represented by 10-bit code values, 12-bit code values, 14-bit code values, etc.) that is wider than a standard dynamic range (e.g., represented by 8-bit code values, 10-bit code values, etc.) and may include, but is not limited to, a wide dynamic range up to the instantaneous perceptible dynamic range and color gamut that human visual moments are maximally perceptible. The supported brightness levels in the VDR can be assigned in such a way that they are optimally spaced or quantized to match the perceptual non-linearity of human vision.

Using a wide dynamic range (202) video signal input, as shown in FIG. 2A, all three example ranges 222-1, 222-2, and 222-3 of input code values may be represented. However, the dynamic range of the display system is limited to a suitable subset of the wide dynamic range (202) at any given time. For example, as shown in FIG. 2A, the dynamic range of the display system covers a dynamic range window comparable to a portion (204) of the wide dynamic range (202) by a particular setting of the global light modulation (e.g., a setting of the global iris, etc.). As used herein, the term "dynamic range window" refers to an instance of limited dynamic range generated at a given time at a particular setting of global light modulation; the luminance levels in the dynamic range window are typically generated from the output code values at a particular setting of the global light modulation; the dynamic range window (e.g., significantly) generated by the LDR display system at any given time is narrower than the wide dynamic range (e.g., 202) of the video input signal. Thus, where the dynamic range window is comparable to the portion 204 of the wide dynamic range (202), the upper code values (222-1 and 222-2) in the wide dynamic range (202) may be displayed, while the lowest code value (222-3) in the wide dynamic range (202) is outside the dynamic range and will be clipped.

In some embodiments, LDR display systems according to the techniques described herein are configured with global light modulation capabilities. The display systems described herein are configured to determine a particular setting of global light modulation to produce a (e.g., optimal, etc.) dynamic range window for a particular VDR input image. The dynamic range window includes specific absolute minimum and maximum brightness levels corresponding to specific settings of the global light modulation. The display system produces different dynamic range windows for different VDR input images. In one example, for a first VDR input image 206-1, the display system sets a first setting for global light modulation and produces a limited dynamic range represented by a first dynamic range window. In another example, for a second VDR input image 206-2, the display system sets a second setting for global light modulation and produces a limited dynamic range represented by a second dynamic range window.

The VDR input image may cover the entire wide dynamic range (202), but need not. Even though the two VDR input images (e.g., 206-3 and 206-4) cover the same portion of the wide dynamic range (202), one of the VDR images (e.g., 206-3) may contain more highlights (highlights) than the other VDR image (e.g., 206-4).

Some VDR input images (e.g., 206-1 and 206-2) may occupy only a portion of the wide dynamic range (202) that respectively fits within a particular dynamic range window produced by a particular setting of global light modulation. As shown in fig. 2a (c), the first VDR input image (206-1) can occupy a first portion of the wide dynamic range (202) that fits within a first dynamic range window produced by the first setting of the global light modulation. Similarly, the second VDR input image (206-2) can occupy a second portion of the wide dynamic range (202) that fits within a second dynamic range window produced by a second setting of the global light modulation. The display systems described herein are configured to convert input code values in a VDR input image (in this example, the first VDR input image 206-1 or the second VDR input image 206-2) to output code values (or system-specific code values) such that the output code values at a corresponding setting of global light modulation (in this example, the first setting or the second setting of global light modulation) produce the same or substantially the same brightness level as the input code values perceptually encoded in the VDR input image, although the numerical values of the output code values may be different from the numerical values of the corresponding input code values in some embodiments.

As used herein, "input code values" refer to code values of one or more channels of the color space employed in the VDR input image; the input code values may be represented by values in a code space of relatively high bit length (e.g., 10 bits, 12 bits, 14 bits, etc.). As used herein, "output code values" refer to code values (e.g., standards-based, system-specific, etc.) for one or more channels of an employed color space in an LDR image (which may be the same or different than the color space employed for the VDR input image); the output code values may be represented by values in a code space (e.g., standard-based, system-specific, etc.) of relatively low bit length (e.g., 8 bits, 10 bits, etc.).

Some VDR input images (e.g., 206-3,206-4, etc.) may occupy a large portion of the wide dynamic range (202) that does not fit within any dynamic range window produced by any setting of the global light modulation. As shown in fig. 2a (d), the third VDR input image (206-3) can occupy the entire wide dynamic range (202) that does not fit within the third dynamic range window optimally selected by the display system for the third VDR input image (206-3) or any dynamic range window produced by any setting of global light modulation. Similarly, the fourth VDR input image (206-4) can occupy a wide dynamic range (202) that does not fit within the fourth dynamic range window (not shown) optimally selected by the display system for the fourth VDR input image (206-4) or any dynamic range window produced by any setting of global light modulation.

It should be noted that although the third VDR input image (206-3) may cover the same dynamic range as the fourth VDR input image (206-4), the dynamic range windows determined by the display system for the two VDR input images (206-3 and 206-4) may be different. For example, the third VDR input image (206-3) may contain more highlights and less dark regions than the fourth VDR input image (206-4) as a whole, and the third dynamic range window selected for the third VDR input image (206-3) may cover a brighter portion of the wide dynamic range (202) than the portion of the wide dynamic range (202) covered by the fourth dynamic range window selected for the fourth VDR input image (206-4).

In some embodiments, the display system is configured to partition the dynamic range of a VDR input image (e.g., 206-3,206-4, etc.) into a range of input code values for perceptual preservation (perceptual preservation) and zero or more ranges of input code values for display management. The number, location, and size of these ranges set for the VDR input image (e.g., 206-3,206-4, etc.) are determined based on the particular image data of the VDR input image (e.g., 206-3,206-4, etc.) and may vary from image to image.

Referring to FIG. 2A (d), the third dynamic range window determined for the third VDR input image (206-3) may cover an upper portion of the wide dynamic range (202). In some embodiments, the display system is configured to partition the third dynamic range into an input code value range (e.g., 208-1) for perceptual retention located at an upper portion of the wide dynamic range (202) and an input code value range (210-1) for display management below the input code value range (208-1). The number, location and size of these ranges (208-1 and 210-1) set for the third VDR input image (206-3) are determined based on the image data of the VDR input image (206-3).

Similarly, the fourth dynamic range window determined for the fourth VDR input image (206-4) of fig. 2A may cover a lower portion of the wide dynamic range (202). In some embodiments, the display system is configured to partition the fourth dynamic range into an input code value range (e.g., 208-2) for perceptual retention located at a lower portion of the wide dynamic range (202) and an input code value range (210-1) for display management above the input code value range (208-2). The number, location, and size of these ranges (208-2 and 210-2) set for the fourth VDR input image (206-4) are determined based on the image data of the fourth VDR input image (206-4).

Pixels in a VDR input image (e.g., 206-3,206-4, etc.) having input code values within the input code range for perceptual preservation are referred to as "in-range" pixels and are given output code values (or system-specific code values) that produce the same or substantially the same brightness level as the input code values perceptually encoded in the VDR input image (e.g., 206-3,206-4, etc.) under a particular setting of global light modulation for a particular dynamic range window, but the values of the output code values may differ from the values of the corresponding input code values. Pixels in a VDR input image (e.g., 206-3,206-4, etc.) that have input code values within zero or more input code value ranges outside of the input code range used for perceptual retention are referred to as "out-of-range" pixels, and are given output code values (or system-specific code values) that may not produce the same brightness level as the input code values perceptually encoded in the VDR input image (e.g., 206-3,206-4, etc.) under a particular setting of global light modulation for a particular dynamic range window. The display management operation may be performed for out-of-range pixels.

In some embodiments, the output code values (or system-specific code values) may have a different density than the input code values of the wide dynamic range (202). The "in-range" pixels may be given a value that is perceptually accurately adjusted. For example, generating an output code value closest to a luminance level for a luminance value represented by an input code value in an LDR display system may be selected to represent the input code value in a rendering operation of the LDR display system.

As shown in fig. 2B, at t0, a VDR input image having a VDR luminance (horizontal) distribution is received. The VDR luminance level distribution includes a prominent portion located below and a small highlight portion located above. This VDR input image is converted to an SDR output image through a dynamic range window determined based on the VDR brightness level distribution, according to the techniques described herein (e.g., by Dolby global dimming (dimming), etc.). Most (or significant) of the VDR luma level distribution at lower luma levels corresponding to pixels within the range is approximately mapped to SDR luma levels in the SDR output image. The small portion of the VDR luma level distribution (or highlight) at the highlight level corresponding to out-of-range pixels is compressed to the SDR luma level through display management (e.g., tone mapping, etc.).

At t1, different VDR input images with different VDR luminance (horizontal) distributions are received. The different VDR brightness level distributions comprise a significant part in the middle, a small highlight part above and a small dark part below. According to the techniques described herein (e.g., by Dolby global dimming, etc.), this different VDR input image is converted to a different SDR output image through different dynamic range windows (lower than the dynamic range window for the VDR input image at t 0) determined based on the VDR brightness level distribution. A large portion (or significant portion) of the VDR luma level distribution at the mid-range luma level is approximately mapped into the SDR luma level in the SDR output image. Both small highlight and small dark portions of the VDR luminance level distribution at high and low luminance levels are compressed into the SDR luminance level by display management (e.g., tone mapping, etc.).

According to the techniques described herein, pixel values of in-range pixels in a VDR input image are perceptually accurately maintained or adjusted in a perceptual quantization space, an LDR image is output that includes output code values that drive rendering operations of an LDR display system, preserving a majority of the original perceptual appearance of the VDR input image, even if the output LDR image has a limited dynamic range that can be supported by the LDR display system.

4. Comparison of display systems

Fig. 3 compares an LCD display system with global light modulation capability to an LCD display system without global light modulation capability. Not all VDR input images use the entire wide dynamic range (e.g., 202) of the video input signal, such as the full VDR range. As shown in fig. 3, the fifth VDR input image (206-5) has a luminance level in the fifth portion of the wide dynamic range (202), for example between 0.5 and 4000 nits. For example, the sixth VDR input image (206-6) has a luminance level in a sixth portion of the wide dynamic range (202), e.g., between 0.005 and 8.0 nits. For display systems without global light modulation, both the fifth VDR input image (206-5) and the sixth VDR input image (206-6) are shadowed to the same limited dynamic range (304) of the LDR display system, resulting in an undesirable perceptible appearance change and visual artifacts relative to the VDR input images (206-5 and 206-6). Note that such LDR display systems optionally may still have local dimming capabilities; however, the absolute minimum and maximum values of the dynamic range of LDR display systems are fixed.

In contrast, for a display system with global light modulation, each of the fifth (206-5) and sixth (206-6) VDR input images is mapped to a particular dynamic range window (306-1 or 306-2) of the LDR display system determined based on the range of luminances represented in that VDR input image. Furthermore, the output code values used by the rendering operation of the display system are perceptually accurately adjusted/changed to produce the same or substantially the same brightness levels as represented in the VDR input image, resulting in a faithful reproduction relative to the perceivable appearance of the VDR input image (206-5 and 206-6). Note that such LDR display systems optionally may still have local dimming capabilities; however, the absolute minimum and maximum values of the dynamic range of an LDR display system are not fixed, but may be adjusted to generate different dynamic range windows, suitably according to global light modulation.

The dynamic range of the VDR input image can be split by an LDR display system with global modulation capability into a range of input code values for perceptual retention and zero or more ranges of input code values for display management. The range of input code values reserved for perception may be assigned to a significant portion of the VDR input image (e.g., the middle portion of the VDR image, the portion in which motion is detected, etc.), or a relatively large number of pixels in range. Even if the entire dynamic range of the VDR input image cannot fit within the optimal dynamic range window by any setting of the global light modulation for the VDR input image, a relatively large number of pixels or significant portions of the VDR input image are perceptually accurately preserved/adjusted in the corresponding LDR image to be rendered by the LDR display system. Thus, a relatively large amount of perceptual information in the VDR input image is maintained in the LDR image as rendered by the LDR display system, as compared to other methods including, but not limited to, without global light modulation capability.

5. Exemplary processing and light modulation paths

Fig. 4 illustrates exemplary processing and light modulation paths in an LDR display system with global light modulation capability. In some embodiments, the processing path includes a global modulation driver (402) and a display management module (404) that are respectively configured to control a light generation component, a light modulation component (410), a light control component (412), and the like in the projection path to render the LDR image on a display screen (406). The light modulation assembly (410) may be, but is not limited to, a Digital Light Processing (DLP)/liquid crystal on silicon (LCoS)/Liquid Crystal Display (LCD) based light modulation assembly. The light control component (412) may be, but is not limited to, a global aperture, a global iris, etc., and is controlled in part by settings for global light modulation. An LDR image is derived largely by perceptually accurately adjusting input code values in a VDR input image receivable in a wide dynamic range (202) of video signal input.

In the processing path, a VDR input image in the video signal input is analyzed by a global modulation driver (402) to determine a brightness level distribution (e.g., histogram, table, etc.) of the VDR input image, and to determine an optimal dynamic range window (which is an instance of an LDR at a particular setting of global light modulation) to which input code values in the VDR input image are mapped. The determination of the optimal dynamic range window includes determination of absolute minimum and maximum luminance levels to be generated by a global light source module (408) and/or a global light modulation component (such as a global aperture, a global iris, etc.). The global modulation driver (402) may be configured to perform light source control operations and to perform control operations of the global light modulation component (410), and to modulate a global amount of light illuminating the one or more local modulation layers for the purpose of rendering a LDR image (corresponding to a VDR input image) on the display screen (406). In some embodiments, the global modulation driver (402) may also be configured to perform laser modulation control as part of global or local light modulation.

The display management module (404) of fig. 4 may be configured to continuously update its input parameters (such as the minimum brightness level and the maximum brightness level of the optimal dynamic range window). In some embodiments, the minimum brightness level and the maximum brightness level of the optimal dynamic range window vary from image to image according to the setting of the global light modulation. The particular setting of global light modulation depends on the image data of the particular VDR input image and is used to place the light source module (408) and light modulation component (410) in particular states to produce particular minimum and maximum brightness levels and an optimal dynamic range.

The display management module (404) of fig. 4 may be configured to perform continuous adjustment between input code values of a VDR input image and output code values in a corresponding LDR image. The display management module (404) can be configured to perceptually map input code values of the VDR input image into a particular optimal dynamic range window determined based on the VDR input image. The pixel adjustments generated or determined by the display management module (404) can be used to control a pixel-level or pixel block-level light modulation component (410) to present a perceptually correct LCD image corresponding to the VDR input image on the display screen (406).

To avoid "pumping" artifacts (e.g., unexpected ringing or sudden shift of absolute minimum and maximum brightness levels in a continuous dynamic range, etc.), temporal suppression (temporal warping) may be applied such that two different dynamic range windows may relatively transition each other gradually in perception (e.g., within a time interval of 0.5 seconds, 1 second, 3 seconds, etc.) rather than abruptly.

The display system can be configured to determine/select a dynamic range window for the VDR input image (e.g., 206-. For example, the display system may determine a brightness level distribution of the VDR input image, select a dynamic range window to cover as much of the brightness level distribution as possible, and determine a particular setting of global light modulation based on the global light modulation capabilities of the display system to produce the dynamic range window. The luminance levels in the luminance level distribution may be weighted differently. Luminance levels having a relatively large number of pixels are assigned a relatively high weight compared to other luminance levels having a relatively small number of pixels. The display system may be biased to select a dynamic range window to cover more brightness levels with relatively many pixels. In addition, the display system may use the luminance level distribution to identify a range of input code values in the dynamic range window for perceptual preservation.

The display system may be configured to minimize the number of "out-of-range" pixels that are outside the range of input code values used for perceptual preservation, and/or to minimize the number of levels at which luminance compression is required. The display system may be configured to minimize the number of brightness levels that require brightness compression (e.g., display management operations via tone mapping, display management operations developed by dolby laboratories, inc., including but not limited to San Francisco, California, etc.).

The display system can perceptually and accurately adjust the input code values of in-range pixels to output code values and map the input code values of out-of-range pixels to output code values having compressed luminance levels by tone mapping or the like.

The operation for selecting the optimal dynamic range window to cover at least a significant portion of the VDR input image and the operation for setting the settings of the global light modulation are associated. A feedback loop may be implemented between the display management module (404) and the global modulation driver (402) to continuously select the dynamic range window and set the settings for the global light modulation. As a result, a perceptually correct image can be maintained even if the overall brightness level of the VDR input image changes over time.

The VDR luma level of pixels within the range of the VDR input image can be perceptually correctly preserved by the LDR luma level in one or more portions of the dynamic range window that are preserved for perceptual preservation. Depending on the dynamic range of the VDR input image received by the display system, some VDR brightness levels of the VDR input image remain outside of the selected dynamic range window and therefore remain clipped or compressed. Clipping and compression of some VDR luma levels may be perceptually hidden by mapping those VDR luma levels to LDR luma levels in some portions of the dynamic range window that are reserved for display management. At any given time, zero or more portions of the dynamic range window reserved for display management and one or more portions of the dynamic range window reserved for perceptual reservation constitute the entire dynamic range window.

There are several processes in the human visual system that support the efficiency of the techniques described herein. For example, the perceivable dynamic range of the human visual system is often reduced when both very bright and very dark elements are visible at the same time due to glints in the eye (which may occur similarly in a camera lens). While the contrast may improve the perception of black, to some extent independent of the absolute black and absolute white levels of the display system.

Fig. 5 shows two exemplary LDR images rendered by an LDR display system. The dark image on the left is derived from the first VDR input image with a narrow set of low luminance levels (or dark black levels) without many bright highlights. The bright image on the right is derived from a second VDR input image having a wide set of both low and high luminance levels.

The techniques described herein may be used to generate a left dark image from a first VDR input image without significantly compressing the brightness levels of the pixels of the scene. The aperture or iris that controls the minimum and maximum brightness levels may be set to a relatively small opening or may be closed for a relatively long period of time compared to the period of time that the aperture or iris is open. Thus, the optimal dynamic range window determined based on the first VDR input image creates a fully addressable black level for all or some of the full VDR luminance levels in the first VDR input image.

The sun in the bright image on the right creates a strong highlight. Glare caused by the presence of the sun causes the black level of the second input VDR image to rise. Thus, the base jet black level of the LDR of the display system is raised when rendering the LDR image corresponding to the second VDR input image. The raising of the jet black level is partly achieved by shifting the LDR from a relatively dark dynamic range window selected from the first input VDR image up a relatively bright dynamic range window selected from the second input VDR image. The aperture or iris that controls the minimum and maximum luminance levels may be set to a relatively large opening or may be opened for a relatively long period of time compared to the period of time that the aperture or iris is closed.

The techniques described herein may also be used to generate a bright LDR image on the right side by accurately maintaining/adjusting the mid-or high VDR luma level perception of the second VDR input image in the perceptually-preserved input code values to the output code values in the perceptually-preserved portion of the optimal dynamic range window.

The relatively few pixels in the second input VDR image may be potential outliers relative to the optimal dynamic range window determined based on the second input VDR image and may be mapped, compressed, and/or even clipped into the optimal dynamic range window by the display management module (404). Some low VDR luminance levels of the second input VDR image may be compressed into the zero or some portion reserved for luminance compression of the optimal dynamic range window.

As shown in fig. 5, the perceptual fidelity of the VDR input image may be relatively better preserved than otherwise according to the techniques described herein. The dark image on the left largely preserves the entire dynamic range of the first input VDR image, while the bright image on the right largely preserves a significant portion (salient portion) of the second VDR image due to the presence of the more visible sun. Since human vision inherently loses some ability to distinguish low luminance levels when faced with simultaneous strong glare, the perceived fidelity of the low luminance levels in the right bright image is not sacrificed much by the luminance compression of the low luminance levels in the second input VDR image and remains satisfactory/acceptable.

6. Non-perceptual adjustment of absolute brightness levels

Fig. 6 shows an exemplary sequence of dynamic range windows of an input VDR image sequence in a scene. The dynamic range windows and their maximum and minimum luminance values are a function of the VDR input image. The VDR input image in the video signal input covers the input dynamic range 602 (base 10 absolute luminance level in logarithmic domain) along the vertical axis in fig. 6. The LDR display system may select/set the global modulation setting to generate respective narrow dynamic range windows at respective times that collectively cover the relatively wide output dynamic range 604 in time.

A VDR input image in a scene is analyzed by a global modulation driver (402) in an LDR display system to determine a dynamic range window from the VDR input image. Parameters such as maximum luma level (on the maximum luma value curve 606), minimum luma level (on the minimum luma value curve 608), etc., define each of the determined dynamic range windows and are shared by a display management module (404) of the LDR display system, which can map input code values in the VDR input image to output code values in the corresponding LDR image through the dynamic range window.

Light source components (408), such as a backlight unit (BLU), and light control components (412), such as a global aperture, a global iris, etc., in the display system can be controlled on an image basis by specific settings to modulate the global light output of the LDR image to which the VDR input image is mapped.

For purposes of illustration, from frame 1 to frame 7 (as shown by the horizontal axis in FIG. 6), via a global modulation driver (40)2) The determined dynamic range window spans approximately from 10^-1To 10²cd/m². The dynamic range window determined by the global modulation driver (402) spans approximately 10 from frame 9 to frame 12^-0.25To 10^2.75cd/m². Even though the VDR input image (e.g., frame 10, frame 11, etc.) may have a value higher than 10^2.75cd/m²Is better identified, the maximum brightness level of the physical dynamic range of a display system (e.g., DLP, LCD, etc.) with a full power light source and maximum light output limit is still clipped to 10^2.75cd/m²Which is the maximum value of the exemplary display system that is not configured to produce.

The techniques described herein may be used for various purposes in achieving display operation. These purposes include, but are not limited to: maximizing the perceived dynamic range and increasing the perception accuracy in a display system with global light modulation, reducing power consumption, reducing heat generation, preventing overheating of LEDs, etc. In one embodiment, a certain amount of energy usage per hour cannot be exceeded, and the techniques described herein can be used to dim a light source in an LDR system to a suitable intensity level in a given environment while maintaining the perceived fidelity of the VDR input image as large as possible.

In another embodiment, it may be desirable to prevent the LED light source in an LDR display system from being overdriven for an extended period of time to prevent the LDE light source from being thermally related permanently damaged. The techniques described herein may be used to slowly dim (or temporarily suppress) the intensity level of an LED light source if the LED light source is overdriven for a preconfigured time. The resulting reduction in brightness in a given environment can be perceptually compensated for using the techniques described herein while minimizing as much as possible the adverse effects on the perceived fidelity of the VDR input image.

As used herein, the terms "luminance level" or "luminance value" may be used interchangeably and may refer to a quantized luminance level in a particular dynamic range. As used herein, the term "nit" or its abbreviation "nt "may refer synonymously or interchangeably to units of image intensity, brightness, luma, and/or luminance, equal to or equal to 1 candela (or cd/m) per square centimeter²)。

For purposes of illustration, it has been described that the relatively wide dynamic range is VDR. However, the present invention may not be limited thereto. This technique can be used to preserve the perceptual fidelity of non-VDR dynamic range (input) images in display systems that support relatively narrow dynamic ranges. The relatively wide dynamic range may be represented by (input) code values in a color space located at 10 bits, 12 bits, 13 bits, 14 bits or higher, whereas the relatively narrow dynamic range may be represented by (LDR) code values in a color space located at 14 bits, 13 bits, 12 bits, 10 bits, 8 bits or lower.

For purposes of illustration, it has been described that input code values of a relatively wide dynamic range are mapped to output code values of a relatively narrow dynamic range. It should be noted that the code values described herein may be values in a luminance channel of a color space, but may also be pixel values in other channels than the luminance channel. According to the techniques described herein, code values, whether directly representing luminance levels or partially representing contributions to luminance levels, may be perceptually adjusted, mapped, etc., e.g., in order to preserve the perceptual fidelity of the color. Additionally, alternatively or in lieu, a color space (e.g., RGB +, etc.) other than the color space (e.g., YCbCr, etc.) that includes the luminance channel (or encoded luminance information) may be used to encode the relatively wide dynamic range image and/or the relatively narrow dynamic range image to which the relatively wide dynamic range image is mapped. The techniques described herein may be used to individually globally modulate one or more color channels, other than the luminance channel, of an employed color space when generating output images of different dynamic range windows based on corresponding input images of a wide dynamic range. The individual modulation of the color channels may take into account the respective luminance contribution from each color channel and re-adjust the color balance in the color gamut. If certain colors (e.g., hue or saturated regions in the image, etc.) are more dominant than others, for example in a scene with dominant red and only some dim blue, the blue channel LEDs may be more dimmed relative to the other channels. The individual modulation of the color channels provides a greater level of distinct significant portions of each color channel (in this example, the dimmed blue) while maintaining the ability to perceive the correct rendering of each color channel and the entire image on the display system.

Additionally or alternatively, the techniques described herein may be used in conjunction with local dimming. For example, local dimming may be used to control individual light sources, generate different local brightness levels in different regions of an image, and provide a relatively wide dynamic range for a display system. Global light modulation techniques can be used to adjust global maximum and minimum brightness levels and produce different instances of relatively wide dynamic range generated by local dimming to cover even wider dynamic range of the video input image. The perceptual mapping and display management techniques described herein may be used to maintain the perceptual quality of images rendered in local dimming display systems having dynamic ranges narrower than the dynamic range of the video input signal.

7. Exemplary display System

FIG. 7 illustrates an exemplary display system including a display screen (406) and a display controller (704), the display controller (704) including a global modulation driver (402) and a display management module (404), according to one embodiment. The display controller 704 may be configured to control one or more light sources, and system components that modulate the light output for rendering an LDR image on a display screen (406). The display controller 704 may be operatively coupled with an image data source (706) and configured to receive image data from the image data source (706). The display controller (704) may be configured to receive a relatively wide dynamic range input image in image data from an image data source (706). The image data may be provided to the display system by an image data source (706) in a variety of ways including radio, set-top box, networked server and/or storage medium coupled to the display system.

The image data received by the display system may be initially in any of a variety of formats (standard-based, proprietary, extensions thereof, etc.) and/or may be derived from any of a variety of image sources (cameras, image servers, tangible media, etc.). Examples of image data to be encoded include, but are not limited to, raw or other high bit depth images. The raw or other high bit-depth image may be obtained from a camera, a presentation system, an art guidance system, another upstream image processing system, a graphics server, a content database, and so forth. The image data may include, but is not limited to, data for digital photographs, video image frames, 3D images, non-3D images, computer generated graphics, and the like. The image data may include scene-related images, device-related images, or images having various dynamic ranges. Examples of image data may include a high quality version of an original image. The original or other high bit-depth image may have a high sampling rate used by professionals, art studios, broadcasters, high-end media production entities, and the like. The image data may also be fully or partially computer generated and may even be obtained based in whole or in part on existing data sources, such as old movies and documentaries.

The image data may include floating point or fixed point image data and may be in any color space. In one exemplary embodiment, the input image may be located in an RGB color space. In another exemplary embodiment, the input image may be located in the YCbCr color space. In one embodiment, each pixel in an image described herein includes floating point pixel values for all channels defined in a color space (e.g., red, green, and blue channels in an RGB color space). In another example, each pixel in an image described herein includes fixed point pixel values for all channels defined in a color space (e.g., fixed point pixel values of 16 bits or higher/lower bit numbers for red, green, and blue channels in an RGB color space). Each pixel may optionally and/or alternatively include pixel values that are downsampled for one or more of the channels in the color space relative to other channels in the color space.

8. Exemplary Process flow

FIG. 8 illustrates an exemplary process flow according to an embodiment of the present invention. In some embodiments, one or more computing devices or components may perform this processing flow. In block 802, a display system receives a plurality of input images in an input audio signal of a wide dynamic range.

In block 804, the display system determines a particular setting of global light modulation based on a particular input image of the plurality of input images. A particular setting of global light modulation results in a particular dynamic range window. The output luminance level range represented by this particular dynamic range window is a suitable subset of the luminance level range represented by the wide dynamic range.

In block 806, the display system converts the plurality of input code values in the particular input image to a plurality of output code values in a particular output image corresponding to the particular input image. The plurality of output code values produce a luminance level that is the same or substantially the same as the luminance level represented by the plurality of input code values. A plurality of output code values are within the particular dynamic range window.

In one embodiment, the plurality of input code values represents one or more salient portions of a particular input image.

In one embodiment, the particular input image is perceptually encoded by input code values independent of the display system.

In one embodiment, the display system is further configured to convert one or more remaining input codes in the particular input image into one or more output code values in the particular output image. The one or more output code values produce a different luminance level than the luminance level represented by the one or more input code values. The one or more output code values are within the particular dynamic range window.

In one embodiment, at least one of the wide dynamic range or the specific dynamic range window represents a dynamic range with an upper limit of the following values: less than 500 nits; between 500 nits and 1000 nits, and 500 nits and 1000 nits are included; between 1000 nits and 5000 nits, and 1000 nits and 5000 nits are included; 5000 nits and 10000 nits, and 5000 nits and 10000 nits are included; 10000 nit and 15000 nit, and 10000 nit and 15000 nit are included; or greater than 15000 nit.

In one embodiment, the dynamic range window represents a dynamic range with a lower limit of the following values: less than 0.001 nit; between 0.001 nit and 0.1 nit, and 0.001 nit and 0.1 nit are included; between 0.1 nit and 1 nit, and 0.1 nit and 1 nit are included; between 1 nit and 10 nit, and 1 nit and 10 nit are included; between 10 nits and 100 nits, and 10 nits and 100 nits are included; or greater than 100 nits.

In one embodiment, the display system is further configured to present the particular output image on the display screen through a particular setting of the global light modulation.

In one embodiment, the display system is further configured to determine a second particular setting of global light modulation based on a second particular input image of the plurality of input images, the particular setting of global light modulation producing a second particular dynamic range window, the second particular dynamic range window being different from the particular dynamic range window; converting a second plurality of input code values in the second particular input image to a second plurality of output code values in a second particular output image corresponding to the second particular input image, the second plurality of output code values producing a same or substantially the same luminance level as represented by the second plurality of input code values.

In one embodiment, the specific input image includes the same maximum and minimum luminance levels as the second specific input image; the particular dynamic range window includes a maximum luminance level and a minimum luminance level that are different from a maximum luminance level and a minimum luminance level of a second particular input image.

In one embodiment, the video input signal comprises image data encoded in one of: high resolution High Dynamic Range (HDR) image formats, RGB color space associated with the academy color coding specification (ACES) standard of the Academy of Motion Picture Arts and Sciences (AMPAS), the P3 color space standard of the digital cinema initiative, the reference input media metric/reference output media metric (RIMM/ROMM) standard, the sRGB color space, RGB color space associated with the bt.709 recommendation standard of the International Telecommunication Union (ITU), the CIE color space defined by the international commission on illumination (CIE), the CIELAB color space, the CIELUV color space, the YCbCr color space, the IPT uniform color space, the LCh color space, or color spaces related to spectral coding, etc.

In one embodiment, the specific settings for the global light modulation include one or more specific settings for one or more of the light source modules or the global light modulation modules.

In one embodiment, the light illumination on the pixel-level or pixel block-level light modulation layer according to a specific setting of the global light modulation is uniform. In various embodiments, the light illumination on the pixel-level or pixel-block-level light modulation layer according to a particular setting of the global light modulation is non-uniform.

In one embodiment, the wide dynamic range includes input code values of a code space having a bit depth of at least one of: less than 12 bits; between and including 12 bits and 14 bits; at least 14 bits; or 14 bits or more.

In one embodiment, the particular dynamic range window is represented by an output code value of a code space having a bit depth of at least one of: less than 8 bits; between 8 bits and 12 bits, and inclusive of 8 bits and 12 bits; or 12 bits or more.

In some embodiments, the particular input image comprises an image-specific dynamic range that fits within the particular dynamic range window. In a different embodiment, the particular input image comprises an image-specific dynamic range that does not fit within the particular dynamic range window.

In various embodiments, an apparatus, computing system, display system, or the like performs any or a portion of the foregoing methods.

9. Implementation mechanisms-hardware overview

According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special purpose computing device may be hardwired to perform the techniques, or include digital electronics such as one or more Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs) permanently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques according to program instructions in firmware, memory, other storage devices, or a combination. Such special purpose computing devices may also combine special purpose hardwired logic, ASICs, or FPGAs with custom programming to implement the techniques. The special purpose computing device may be a desktop computer system, portable computer system, hand-held device, networked device, or any other device that incorporates hardwired and/or program logic to implement the techniques.

For example, FIG. 9 is a block diagram that illustrates a computer system 900 upon which an exemplary embodiment of the invention may be implemented. Computer system 900 includes a bus 902 or other communication mechanism for communicating information, and a processor 904 coupled with bus 902 for processing information. The hardware processor 904 may be, for example, a general purpose microprocessor.

Computer system 900 also includes a main memory 906, such as a Random Access Memory (RAM) or other dynamic storage device, coupled to bus 902 for storing information and instructions to be executed by processor 904. Main memory 906 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 904. Such instructions, when stored in a non-transitory storage medium accessible to processor 904, present computer system 900 as a special-purpose machine customized to perform the operations specified in the instructions.

Computer system 900 also includes a Read Only Memory (ROM)908 or other static storage device coupled to bus 902 for storing static information and instructions for processor 904. A storage device 910, such as a magnetic disk or optical disk, is provided and coupled to bus 902 for storing information and instructions.

Computer system 900 may be coupled via bus 902 to a display 912, such as a liquid crystal display, for displaying information to a computer user. An input device 914, including alphanumeric and other keys, is coupled to bus 902 for communicating information and command selections to processor 904. Another type of user input device is cursor control 916, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 904 and for controlling cursor movement on display 912. Such input devices typically have two degrees of freedom in two axes, a first axis (e.g., x, horizontal) and a second axis (e.g., y, vertical), that allows the device to specify positions in a plane.

Computer system 900 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and/or program logic that is combined with the computer system to cause computer system 900 or program computer system 900 to function as a special purpose machine. According to one embodiment, the techniques herein may be performed by computer system 900 in response to processor 904 executing one or more sequences of one or more instructions contained in main memory 906. Such instructions may be read into main memory 906 from another storage medium, such as storage device 910. Execution of the sequences or instructions contained in main memory 906 causes processor 904 to perform the process steps described herein. In alternative embodiments, hard-wired logic may be used in place of or in combination with software instructions.

The term "storage medium" as used herein may refer to any non-transitory medium that stores data and/or instructions that cause a machine to operate in a specific manner. Such media may include non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 910. Volatile media includes dynamic memory, such as main memory 906. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from, but usable with, transmission media. Transmission media participate in the transfer of information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 902. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infrared data communications.

Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 904 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 900 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 902. Bus 902 carries the data to main memory 906, from which main memory 906 processor 904 retrieves instructions and executes the instructions. The instructions received by main memory 906 may optionally be stored on storage device 910 either before or after execution by processor 904.

Computer system 900 also includes a communication interface 918 coupled to bus 902. Communication interface 918 provides a two-way data communication coupling to a network link 920, network link 920 connects to a local network 922. For example, communication interface 918 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 918 may be a Local Area Network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 918 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

Network link 920 typically provides data communication through one or more networks to other data devices. For example, network link 920 may provide a connection through local network 922 to a host computer 924 or to data equipment operated by an Internet Service Provider (ISP) 926. ISP 926 in turn provides data communication services through the global packet data communication network (now commonly referred to as the "internet") 928. Local network 922 and internet 928 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 920 that carry the digital data to and from computer system 900 through communication interface 918 are exemplary forms of transmission media.

Computer system 900 can send messages and receive data, including program code, through the network(s), network link 920 and communication interface 918. In the Internet example, a server 930 might transmit a requested code for an application program through Internet 928, ISP 926, local network 922 and communication interface 918.

The received code may be executed by processor 904 as it is received, and/or stored in storage device 910, or other non-volatile storage for later execution.

10. Equivalents, extensions, alternatives, and other various forms

Example embodiments are thus described that relate to forensically detecting upmixes in multi-channel audio content based on analysis of the content. In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (24)

1. A method for global display management based light modulation, comprising:

receiving a plurality of input images in an input video signal of a wide dynamic range;

determining a particular setting of global light modulation for each color channel based on a particular input image of the plurality of input images, the particular setting of global light modulation producing a particular dynamic range window, an output luminance level range represented by the particular dynamic range window being a suitable subset of the luminance level range represented by the wide dynamic range, wherein perceptual non-linearities of the particular input image for human vision are perceptually encoded by input code values independent of a display system;

converting, for a particular setting of the global light modulation for a first color channel, a plurality of input code values in the particular input image to a plurality of output code values in a particular output image corresponding to the particular input image, the plurality of output code values producing a same or substantially a same brightness level as represented by the plurality of input code values, and the plurality of output code values being within the particular dynamic range window; and

converting, for a particular setting of the global light modulation for a first color channel, one or more remaining input code values in the particular input image to one or more output code values in the particular output image, the one or more output code values yielding a different brightness level than represented by the one or more input code values, and the one or more output code values being within the particular dynamic range window.

2. The method of claim 1, wherein the plurality of input code values represent one or more salient portions of the particular input image.

3. The method of claim 1, wherein at least one of the wide dynamic range or the specific dynamic range window represents a dynamic range with an upper limit of the following values:

less than 500 nits of the total weight of the composition,

between 500 nits and 1000 nits, and including 500 nits and 1000 nits,

between 1000 nits and 5000 nits, and including 1000 nits and 5000 nits,

between 5000 nit and 10000 nit, and 5000 nit and 10000 nit are included,

between 10000 nit and 15000 nit, and 10000 nit and 15000 nit are included, or

Greater than 15000 nit.

4. The method of claim 1, wherein the dynamic range window represents a dynamic range having a lower limit of:

less than 0.001 nit, and the content of the active carbon,

between 0.001 nit and 0.1 nit, and including 0.001 nit and 0.1 nit,

between 0.1 nit and 1 nit, and including 0.1 nit and 1 nit,

between 1 nit and 10 nit, and including 1 nit and 10 nit,

between 10 nitre and 100 nitre, and including 10 nitre and 100 nitre, or

Greater than 100 nits.

5. The method of claim 1, further comprising rendering the particular output image on a display screen through a particular setting of the global light modulation.

6. The method of claim 1, further comprising:

determining a second particular setting of global light modulation based on a second particular input image of the plurality of input images, the particular setting of global light modulation producing a second particular dynamic range window, the second particular dynamic range window being different from the particular dynamic range window;

converting a second plurality of input code values in the second particular input image to a second plurality of output code values in a second particular output image corresponding to the second particular input image, the second plurality of output code values producing a same or substantially the same luminance level as represented by the second plurality of input code values.

7. The method of claim 6, wherein the particular input image comprises a same maximum and minimum brightness level as the second particular input image, and wherein the particular dynamic range window comprises a maximum and minimum brightness level different from the maximum and minimum brightness level of the second particular input image.

8. The method of claim 1, wherein the input video signal comprises image data encoded in one of: a high resolution High Dynamic Range (HDR) image format, an RGB color space associated with the academy color coding specification (ACES) standard of the Academy of Motion Picture Arts and Sciences (AMPAS), a P3 color space standard of the digital cinema initiative, a reference input media metric/reference output media metric (RIMM/ROMM) standard, an sRGB color space, an RGB color space associated with the bt.709 recommendation standard of the International Telecommunications Union (ITU), a CIE color space defined by the international commission on illumination (CIE), a CIELAB color space, a CIELUV color space, a YCbCr color space, an IPT uniform color space, an LCh color space, or a color space related to spectral coding.

9. The method of claim 1, wherein the particular settings of global light modulation comprise one or more particular settings for one or more of a light source assembly or a global light modulation assembly.

10. The method of claim 1, wherein light illumination on a pixel-level or pixel block-level light modulation layer according to a particular setting of the global light modulation is uniform.

11. The method of claim 1, wherein light illumination on a pixel-level or pixel-block-level light modulation layer according to a particular setting of the global light modulation is non-uniform.

12. The method of claim 1, wherein the wide dynamic range comprises input code values of a code space having a bit depth of at least one of:

less than 12 bits;

between and including 12 bits and 14 bits;

at least 14 bits;

14 bits or more.

13. The method of claim 1, wherein the particular dynamic range window is represented by output code values of a code space having a bit depth of at least one of:

less than 8 bits;

between 8 bits and 12 bits, and inclusive of 8 bits and 12 bits;

12 bits or larger.

14. The method of claim 1, wherein the particular input image includes an image-specific dynamic range that fits within the particular dynamic range window.

15. The method of claim 1, wherein the particular input image includes an image-specific dynamic range that does not fit within the particular dynamic range window.

16. An apparatus for global display management based light modulation, comprising a processor configured to perform the method of any of claims 1-15.

17. A computing device for global display management based light modulation, comprising one or more processors and one or more storage media storing a set of instructions that, when executed by the one or more processors, cause performance of the method recited in any of claims 1-15.

18. An apparatus for global display management based light modulation, comprising:

means for receiving a plurality of input images in an input video signal of wide dynamic range;

means for determining, based on a particular input image of the plurality of input images, a particular setting of global light modulation for each color channel that produces a particular dynamic range window, an output luminance level range represented by the particular dynamic range window being a suitable subset of the luminance level range represented by the wide dynamic range, wherein perceptual non-linearities of the particular input image for human vision are perceptually encoded by input code values independent of a display system;

means for converting, for a particular setting of the global light modulation for a first color channel, a plurality of input code values in a particular input image to a plurality of output code values in a particular output image corresponding to the particular input image, the plurality of output code values producing a same or substantially the same brightness level as represented by the plurality of input code values and the plurality of output code values being within the particular dynamic range window; and

means for converting, for a particular setting of the global light modulation for a first color channel, one or more remaining input code values in the particular input image into one or more output code values in the particular output image, the one or more output code values yielding a brightness level that is different from a brightness level represented by the one or more input code values, and the one or more output code values being within the particular dynamic range window.

19. The apparatus of claim 18, wherein the plurality of input code values represent one or more salient portions of the particular input image.

20. The apparatus of claim 18, wherein at least one of the wide dynamic range or the specific dynamic range window represents a dynamic range with an upper limit of:

less than 500 nits of the total weight of the composition,

between 500 nits and 1000 nits, and including 500 nits and 1000 nits,

between 1000 nits and 5000 nits, and including 1000 nits and 5000 nits,

between 5000 nit and 10000 nit, and 5000 nit and 10000 nit are included,

between 10000 nit and 15000 nit, and 10000 nit and 15000 nit are included, or

Greater than 15000 nit.

21. The apparatus of claim 18, wherein the dynamic range window represents a dynamic range having a lower limit of:

less than 0.001 nit, and the content of the active carbon,

between 0.001 nit and 0.1 nit, and including 0.001 nit and 0.1 nit,

between 0.1 nit and 1 nit, and including 0.1 nit and 1 nit,

between 1 nit and 10 nit, and including 1 nit and 10 nit,

between 10 nitre and 100 nitre, and including 10 nitre and 100 nitre, or

Greater than 100 nits.

22. The apparatus of claim 18, further comprising means for rendering the particular output image on a display screen through a particular setting of the global light modulation.

23. The apparatus of claim 18, further comprising:

means for determining a second particular setting of global light modulation based on a second particular input image of the plurality of input images, the particular setting of global light modulation producing a second particular dynamic range window, the second particular dynamic range window being different from the particular dynamic range window;

means for converting a second plurality of input code values in the second particular input image to a second plurality of output code values in a second particular output image corresponding to the second particular input image, the second plurality of output code values producing a same or substantially the same luminance level as represented by the second plurality of input code values.

24. A non-transitory computer-readable storage medium storing software instructions that, when executed by one or more processors, cause performance of the method recited in any one of claims 1-15.