EA040379B1 - METHOD AND DEVICE FOR ENCODING, DECODING VIDEO SIGNAL IN HIERARCHY OF QUALITY LEVELS - Google Patents

Description

State of the art

Numerous techniques are known in the art for compressing and decompressing multidimensional signals or signals that evolve over time. This applies to audio signals, video signals, and other multi-dimensional signals such as surround signals used in scientific and medical applications.

To achieve high compression ratios, these methods involve the use of spatial and temporal correlation within the signal. For example, conventional methods identify a reference and attempt to determine the signal difference between the current location and the given reference. This is done both in the spatial domain, where the reference is a section (eg, block or macroblock) of a previously received and decoded spatial plane, and in the time domain, where a single temporal signal instance (eg, a video frame in a sequence of frames) is used as a reference in during a certain period of time. This is relevant, for example, to MPEG (Moving Pictures Expert Group) compression algorithms, where previously decoded macroblocks are used as reference in the spatial domain, and I-frames and P-frames are used as reference for subsequent P-frames in the time domain.

Known methods involve the use of spatial correlation and temporal correlation in a variety of ways, using several different methods to identify, simplify, encode and communicate differences. In accordance with traditional methods, in order to use the spatial correlation of the residuals in the corresponding block of pixels, a domain transformation (for example, to the frequency domain) is performed, and then the lossy transformed information is removed and quantized, usually introducing, to one degree or another, blocking artifacts. In contrast, in the time domain, conventional methods involve transmitting the quantized difference between the current sample and the motion-compensated reference sample. To achieve maximum similarity between samples, encoders try to estimate the modifications of the reference signal that occur over time. In traditional coding methods (eg, MPEG family technologies, VP8, etc.), this is called motion estimation and compensation.

Modern CPUs (central processing units) and GPUs (graphic processing units) are usually very powerful; a single GPU can include several hundred processing cores for parallel processing of information. With modern technology equipment, very large portions of an image may be stored in the processor's cache for processing. The need to fragment images into a large number of small blocks, which was a driving factor during the creation of JPEG and MPEG, since the processors of that era could process very small pieces of video data at once and, again, only sequentially, is no longer for modern CPUs and GPUs. Thus, the available processing power can, for the most part, be left unused in the implementation of MPEG type encoding/decoding, while blocking artifacts are unnecessarily introduced into the signal.

Also, compared to when MPEG was developed, today's applications typically require much higher definition video encoding and much higher overall playback quality. In high definition (e.g. fullHD, UltraHD) and high quality (e.g. where artifacts are relatively inconspicuous relative to the original signal) video signals, there is a much greater difference between areas of low detail (potentially even out of focus) and areas of very high detail. This makes the use of frequency domain transforms, such as those used in the JPEG-based and MPEG-based methods, even less suitable for image processing and reproduction, since the range of relevant frequencies is greatly expanded.

In addition, higher resolution images include a higher degree of camera noise and/or film grain, i. highly detailed high-frequency pixel transitions that require a large number of bits to be encoded, but which may be completely irrelevant for viewing compared to similar high-frequency pixel transitions of object boundaries.

Another aspect that is not given importance in the prior art, except for a few attempts, is the requirement for quality scalability. The scaling coding method allows a single version of the compressed signal to be encoded and delivers at different levels of quality, bandwidth availability, and decoder complexity. Scalability has been taken into account in known methods such as MPEG-SVC and JPEG2000, but has not yet been put to good use due to computational complexity and generally inefficient compression compared to non-scaling methods.

Previously, as a scalable alternative to the JPEG/MPEG standards for encoding/decoding, so-called image Laplace pyramids were used for encoding/decoding purposes. For example, traditional Laplace pyramid systems created lower resolution images using Gaussian filters and then built a pyramid of differences between

- 1,040,379 images upsampled by a hard-coded decoder back from lower resolution levels to the original level. The use of encoding with traditional Laplace pyramids was abandoned due to the inefficiency of compression.

Residual domain transforms used so far by prior art coding methods (eg, Fourier transforms, discrete cosine transforms, Hadamard transforms, wavelet transforms, etc.) suffer from a number of disadvantages.

First of all, the choice of the frequency domain transform itself does not allow correct use of the correlation between large sections of the signal (ie, sections with a large number of samples for each of the measurements), since real world signals usually exhibit a limited degree of periodicity. As a result, frequency domain transforms are performed on blocks that are both too large and too small: too large to be computationally simple, too small to sufficiently exploit high-resolution signal correlation. For example, to exploit the correlation of a large enough set of samples while managing computational complexity, traditional image and video coding methods operate on blocks of 8x8, 16x16, or 32x32 elements: obviously too small to fully capture image pattern correlation in a high-definition image (for example, with 8 millions of pixels), but large enough to absorb significant processing power.

Secondly, known methods using frequency domain transforms implicitly assume that humans are sensitive to harmonics (e.g., color transition frequencies) regardless of the direction of the transition, while a number of studies have shown that people perceive the clarity of the transition much better than precise transition direction/angle, especially when observing complex shapes.

Thirdly, known lossy coding methods operate by quantizing the results of the transformation, inevitably creating two problems:

(1) blocking artifacts between one block and adjacent blocks to be corrected by relatively complex anti-block image processing techniques;

(2) the inability to easily control the maximum error in the encoded image, because the actual pixel values are the result of the inverse transformation of the dequantized parameters, as a result, the quantization errors in the quantized block parameters are combined with each other in such a way that they are difficult to manage without several repeated encoding operations and/ or extremely complex quantization schemes.

Preventing blocking artifacts and ensuring maximum error control are extremely important features, especially in applications such as medical imaging or professional photography/videography.

Short description

The embodiments discussed herein differ from conventional systems and methods by providing new ways of quantizing and/or transforming element information (e.g., residual data) to reconstruct an image representation, providing encoding methods characterized by efficient entropy coding, low computational complexity, parallelism, scalability, control mistakes, etc. The methods and embodiments discussed herein present an innovative approach to achieve efficient entropy coding results with low computational complexity, concurrency, scalability, and error management.

According to its broadest aspect, the embodiments discussed herein include a method for converting chip sets (ie, suitable sequences of values) into transformed chip sets for efficient entropy coding. The embodiments illustrated herein are particularly focused on a non-limiting use case in which said signal elements correspond to residual data to be added to a predictive (i.e., provisional) image representation to obtain a target representation of the same image, said predictive representation being obtained, at least in part by upsampling and/or processing a representation of the same image at a lower quality level (eg, with a lower level of fidelity compared to the representation of the original signal). The embodiments illustrated here are particularly focused on non-limiting cases where a lower quality level means lower resolution (i.e., a coarser sampling grid) in one or more of the signal dimensions, although those skilled in the art can readily adapt the same methods to cases where the signal at a lower quality level is a signal with the same resolution and a lower level of detail (eg, without limitation, coarser quantization of element values).

The embodiments discussed herein are useful along with traditional coding methods and useful along with layered coding methods, in other patent applications those

- 2 040379 same authors described many other new ways.

For simplicity, the non-limiting embodiments illustrated herein refer to a signal as a sequence of multidimensional samples (i.e., sets of one or more elements organized as arrays in one or more dimensions, e.g., by way of non-limiting example, sets of picture elements organized as 2D images). In the description, the terms image or plane (meaning in its broadest sense a hyperplane, i.e. an array of elements with any number of dimensions and a given sampling grid) will often be used to identify a digital representation of a signal sample along a sequence of samples, with each plane having a given resolution for each of its dimensions (for example, X and Y, in the case of two dimensions), and contains a set of elements of the plane (or element or pel, for two-dimensional images often referred to as pixels, for volumetric images often referred to as voxels, etc.) characterized by one or more values or settings (for example, by way of non-limiting examples, color settings in a suitable color space, settings indicating density level, settings indicating temperature levels, settings indicating audio tone, settings indicating amplitude, etc.) . Each element of the plane is identified by a suitable set of coordinates indicating the integer positions of said element in the image sampling grid. The signal dimensions may include only spatial dimensions (eg in the case of an image) or also a temporal dimension (eg in the case of a signal evolving over time).

By way of non-limiting examples, the signal can be an image, an audio signal, a multichannel audio signal, a video signal, a multiview video signal (e.g., 3D video), a plenoptic signal, a multispectral signal, a surround signal (e.g., medical imaging, scientific imaging, holographic imaging, etc.), surround video, or even signals with more than four dimensions.

For simplicity, the non-limiting embodiments illustrated here often refer to signals that are displayed as 2D adjustment planes (e.g., 2D images in a suitable color space), such as an image or video signal (i.e., a sequence of 2D images evolving over time). However, those skilled in the art can easily apply the same principles and techniques to any other type of multidimensional signal (eg, surround signals, video signals, 3D video signals, plenoptic signals, etc.). The non-limiting embodiments illustrated here also often illustrate cases of signals encoded in a layered hierarchy of quality levels (LOQ), with higher quality levels in the layered hierarchy having progressively higher resolution (i.e., an increased number of samples for one or more of the signal dimensions) . For simplicity, the non-limiting embodiments illustrated here often refer to the simple case where the resolution is increased by a scale factor of two for each of the signal measurements, but those skilled in the art can easily apply the same techniques to cases with different (and potentially anisotropic) scaling factors. When dealing with signals encoded in a multilevel hierarchy of quality levels, residuals that allow reconstructing the settings of a given LOQ element driven by (eg, co-aligned with) the same element of a lower LOQ exhibit important correlations that are effectively exploited by the non-limiting embodiments illustrated here. For simplicity, we will refer to the plane control element (or control pad) of a given plane element set at a higher LOQ (managed pads) as the parent element at a lower LOQ that is aligned with a given plane element set at a higher LOQ.

The non-limiting embodiments further illustrated herein provide for transforming the residual data (i.e., the adjustment information to apply to the chips to obtain a suitable representation of the signal at a given quality level) to use both the correlation between the residuals themselves and the correlation that exists between the residuals and signal presentation at a lower quality level. In particular, in order to efficiently encode residual data into a digital bitstream, two operations are needed: quantization (ie, assigning each value a symbol belonging to a given finite alphabet of symbols) and entropy coding. Converting characters to entropy encoding reduces the variability of the encoded characters, thereby increasing the efficiency of the entropy encoder. Using a signal representation at a lower quality level allows a portion of the residual information to be predicted based on the information already available at the decoder, further improving coding efficiency. Residual block transformation while using signal representation at a lower quality level, i.e. efficient implementation of multi-scale transformation, also allows the use of signal correlation over a portion of the signal that is wider than the block itself: thus, it is possible to keep the block size extremely small (for example, even 2x2), while using correlation over a much wider area, achieving both the advantages of computational complexity and benefits of coding efficiency.

- 3 040379

The methods illustrated herein essentially allow the residuals to be transformed to minimize the number of different symbols in the set of transformed quantized residual data sets and thus reduce the bitstream size required to encode and transmit said set of sets. In some non-limiting embodiments described herein, the quantization step is performed after the transform step on the encoding side (and therefore before the inverse transform step on the decoding side), in a manner similar to how it has been done with conventional encoding methods so far. In other novel non-limiting embodiments described herein, the quantization step is performed prior to the transform step on the encoding side (and hence after the inverse transform step on the decoding side), effectively implementing a lossless transform method. The implementation of the quantization step prior to the transformation step allows you to manage (and, if necessary, limit) the errors introduced by lossy coding, even before the more computationally intensive parts of the coding. The non-restrictive embodiment described here also allows efficient use of known inter-symbol correlations obtained through a lossless transform.

The non-limiting embodiment illustrated here often illustrates decomposition transformations applied to 2x2 sets of residues. It should be recognized as non-restrictive, and those skilled in the art can easily apply the same techniques to different size residual sets, including different sizes for each dimension (e.g., 2 x 1 residual sets) or more than two dimensions (e.g., LxNxM sets residuals in a volumetric image or in a video signal, where the third dimension represents time).

In the non-limiting embodiment described here, the first signal processor (encoder) is configured to encode a signal in a quality layer hierarchy, while the second signal processor (decoder) is configured to decode a representation of said signal in a quality layer hierarchy.

The encoder receives a signal at a first quality level and creates a representation of the signal at a second quality level, the second quality level being lower than the first quality level. The encoder then processes the signal representation at the second quality level, producing a predictive representation of the signal at the first quality level. The encoder then generates residual data, said residual data corresponding to the difference between the signal at the first quality level and the predicted representation of the signal at the first quality level. The encoder then transforms the residual data by applying a decomposition transform, producing parameters (components or residual components). Decomposing residuals to obtain residual components allows the encoder to better exploit the correlation between residuals; in addition, some of the components correlate better than the residuals with the signal representation at the second quality level, which makes it easier to transmit them than the untransformed residuals. The encoder then processes said components, said predictive signal representation at the first quality level, and said signal representation at the second quality level, producing transformed residuals. The encoder then encodes said second quality level signal representation and said transformed residuals into the corresponding encoded data.

The decoder receives said encoded data and reconstructs, based at least in part on said encoded data, the signal representation at the second quality level and the reconstructed transformed residuals. The decoder processes the signal representation at the second quality level, producing a predictive signal representation at the first quality level, the first quality level being higher than the second quality level. The decoder processes said predictive signal representation at the first quality level, said signal representation at the second quality level, and said reconstructed transformed residuals to produce reconstructed components. The decoder applies an inverse decomposition transformation to the reconstructed components, producing reconstructed residuals. The decoder combines the predicted representation of the signal at the first quality level with the reconstructed residuals, producing a representation of the signal at the first quality level.

In the non-limiting embodiment described here, the residuals are obtained by calculations containing the sums and/or differences between the residuals, and the residuals are obtained by calculations containing the sums and/or differences between the residuals. In a non-limiting embodiment, the operations performed to encode the transformed residuals include selecting a subset of the transformed residuals corresponding to a subset of adjacent residuals in the residual plane at a first quality level, generating a value at least in part based on a signal representation at a second (lower) ) quality level, and combining said value with at least one of the residual components, creating a subset of transformed residuals corresponding to said subset of residuals.

In other non-limiting embodiments described here, the decoder processes the

- 4 040379 coded data, generating a plurality of residuals corresponding to adjustments to be made into elements of a predictive signal representation at a first quality level to create a signal representation at a first quality level, said predictive signal representation at a first quality level being generated by a decoder of at least based in part on representing the signal at a second quality level, the second quality level being lower than the first quality level. The decoder receives encoded data corresponding to the transformed residuals and generates, at least in part based on said encoded data, reconstructed residual components. The decoder then generates a set of reconstructed residuals by applying an inverse decomposition to the reconstructed residual components.

In a non-limiting embodiment, the decoder generates said reconstructed residual components also based at least in part on the representation of the signal at the second (lower) quality level. In a non-limiting embodiment, for each subset of chips at the first quality level, the decoder generates a set of parameters (reconstructed transformed residuals). The decoder generates a value corresponding to said set of parameters, said value being at least partially based on the representation of the signal at a second (lower) quality level.

The decoder then combines one or more parameters with said value, producing a set of reconstructed residuals corresponding to said subset of chips at the first quality level. The decoder then processes the reconstructed residuals by applying inverse decomposition, producing a plurality of reconstructed residuals. In a non-limiting embodiment, each reconstructed residual in the set is obtained by calculating the sums and/or differences between the reconstructed residual components.

In some non-limiting embodiments described herein, the reshaped transformed residues are dequantized prior to processing to create the reshaped residual components, i.e. dequantization is carried out before applying the inverse decomposition. In a non-limiting embodiment, the dequantization operations applied to each reconstructed transformed residue (or each reconstructed residual component) are changed based on the relative position of the corresponding transformed residue in the set of transformed residues.

In other non-limiting embodiments described herein, dequantization operations are applied after the reverse decomposition, dequantizing the reconstructed residues.

In some non-limiting embodiments described herein, said value, combined with a set of reconstructed transformed residuals to create a corresponding set of residuals, corresponds to the mean of the residuals in the set (predicted mean). In a non-limiting embodiment, a predicted mean of a plurality of residuals is generated by averaging the plurality of predicted signal representation bins at the first (higher) quality level and subtracting the corresponding signal representation bin at the second (lower) quality level.

In some non-limiting embodiments described here, the decomposition of the residuals at the first (higher) LOQ takes advantage of the fact that the subset of residuals corresponding to a given subset of plane elements (driven pads) correlates with the control pad value at the lower LOQ, since the sum of the average of the predicted pads (known to the decoder) and the average of the residuals (reconstructed by the decoder) corresponds to the control cell value (also known to the decoder). In fact, in the absence of quantization operations, the average of the residuals (or equivalently, the sum of the residuals) can be accurately determined from information already known at the decoder side prior to beginning to provide residual data. There are several ways to use the ability to calculate the predicted average of the residuals to improve the efficiency of encoding and decoding the residual. In a non-limiting embodiment, the residual plane at the first LOQ is divided into 2x2 blocks. For each 2x2 block, one of the residuals is not encoded (i.e., not transmitted to the decoder) because the decoder reconstructs it from the other three and from the predicted average residuals (i.e., the decoder has the information it needs to generate the missing residuals based on three other remnants of each 2x2 block). When the residual data is quantized, this embodiment concentrates all accumulated quantization errors on one of the four residuals of each 2x2 block. Other non-limiting embodiments avoid this problem (still retaining the efficiency advantage of using predicted mean information) by applying a decomposition transformation to each block of residuals.

In some non-limiting embodiments described herein, the encoder applies a directional decomposition transform to each block of 2x2 residuals of a two-dimensional residual plane corresponding to a signal representation at a first quality level. In such embodiments, the four residual components correspond to the four directions of variability for the remainder.

- 5 040379 fine 2x2 block: mean (A corresponding to intensity slope, in principle orthogonal to the plane of the 2x2 block, sum of residuals divided by four), horizontal slope (H corresponding to horizontal difference of residuals, sum of residuals on the left minus sum of residuals on the right), the vertical slope (V corresponding to the vertical difference of the residuals, the sum of the residuals above minus the sum of the residuals below) and the diagonal slope (D corresponding to the remaining differences not explained by a simple directional transition in 2x2 residual blocks).

The difference between the mean A component and the predicted mean (eg, without limitation, computed as the sum of the mean of the predicted items minus the control on the lower LOQ) is called the mean delta or 5 _a . δ _a is often zero for real world signals and, as a result, it is often preferable to calculate δ _a and entropy encode 5 _a rather than the A component. are likely to be locally correlated, which again improves the efficiency of entropy coding.

Other non-limiting embodiments employ other directional decomposition methods, wherein at least one of the transformed residues in a given transformed residue set corresponds to a directional slope of the values of the residue set that corresponds to said given transformed residue set. A non-limiting embodiment computes the transformed residuals by performing calculations that include image processing operations (eg, directional transforms, spline calculations, Sobel filter, gradient calculations, etc.).

In the non-limiting embodiment described here, for each block of NxM residuals (eg, without limitation, for each block of 2x2 residuals) in the residual plane, the encoder generates a set of parameters corresponding to the transformed residuals. The encoder then divides the parameters into subsets - each of said subsets corresponds to the same class of transformed residuals, with each element of the subset corresponding to a block of residuals - and individually encodes each of the subsets (i.e., the subsets are encoded separately, and the parameters used for entropy coding of each of the subsets do not depend on the entropy coding of other subsets).

In some non-limiting embodiments described herein, the encoder performs directional decomposition on 2x2 blocks of residuals, and individually (i.e., individually) encodes the four subsets of parameters δ _a , H, V, and D.

In other non-limiting embodiments, at least one encoded parameter corresponds to multiple transformed residues of the same block of residues. In a non-restrictive embodiment, the encoder performs directional decomposition on 2x2 blocks of residuals, and for each block of 2x2 residuals creates a single parameter (i.e., a single symbol) corresponding to the four parameters δ _a , H, V, D, with said parameter being the only encoded parameter corresponding to the mentioned block of residuals. In other non-limiting embodiments, the parameter subsets δ _a and D parameters are encoded separately, while the H and V parameters are processed to create a single parameter subset in which each parameter corresponds to the H parameter and the V parameter of a given block of 2x2 residuals.

In other non-limiting embodiments described herein, the decoder receives data corresponding to the transformed residuals for the controlled pels (ie, for the representation elements at a given LOQ of a signal decoded in a multilevel LOQ hierarchy). To reconstruct the residuals based on the transformed residuals, the decoder combines the transformed residuals with parameters based at least in part on representing the signal at a lower LOQ than said given LOQ. In a non-limiting embodiment, the decoder combines one of the converted residuals with the difference between the control pad setting at the lower LOQ and the average of the predicted settings (ie, settings before adding residues) of the control pads at that LOQ. In a non-limiting embodiment, each transformed residual corresponds to a block of residuals and affects the values of all residuals that belong to its corresponding block.

In other non-limiting embodiments described here, the encoder selects a block of residuals in the plane of residuals at the first (higher) LOQ and creates a set of parameters based at least in part on the residuals in said block and based on the signal representation at the second (lower) LOQ .

In other non-limiting embodiments described here, the decoder receives a set of parameters corresponding to a block of residuals in the plane of residuals at the first (higher) LOQ and creates a set of residuals based at least in part on said set of parameters and based on the reconstructed signal representation on the second ( lower) LOQ.

According to additional non-limiting embodiments, the input signal is encoded and decoded according to a layered hierarchical coding method, and

- 6 040379 residual bones for adjacent samples in the time dimension are created at least partially based on the same set of elements (reference plane), by compensating for the movement of said reference plane and by combining it with relative residuals, according to the methods described in other patent applications by the same authors. In a non-limiting embodiment, the converted relative residuals are based at least in part on said reference plane. In a non-limiting embodiment, the reference plane and relative residual planes are encoded and decoded according to the methods described herein.

Note that the embodiments discussed herein may be implemented in software or hardware, or may be implemented using a combination of software and hardware, and may include the configuration of one or more computerized devices, routers, networks, workstations, PDAs, laptops, tablets. , mobile phones, game consoles, set-top boxes, etc., to perform and/or support any or all of the method steps disclosed herein. In other words, one or more computerized devices or processors may be programmed and/or configured to operate in the manner explained herein to carry out various embodiments.

In addition to the techniques discussed above, other embodiments discussed herein include software programs to perform the steps and operations summarized above and detailed below. One such embodiment comprises a computer-readable hardware storage resource (ie, non-transitive computer-readable media) on which logic, instructions, etc. are encoded. a computer program that, when executed on a computerized device having a processor and associated memory, programs and/or directs the processor to perform any of the operations disclosed herein. Such configurations may be provided in the form of software, code, and/or other data (e.g., data structures) hosted or encoded in a computer-readable medium, such as an optical medium (e.g., CD-ROM, DVD-ROM, or BLU-RAY), a flash memory card, a floppy or hard disk, or any other medium capable of storing computer-readable instructions, such as firmware or microcode, in one or more ROM or RAM or PROM chips or in the form of an application specific integrated circuit (ASIC). Software or firmware or other such configuration may be installed on the computerized device to cause the computerized device to carry out the methods explained here.

Accordingly, one particular embodiment of the present disclosure relates to a computer program product that includes a computer-readable hardware storage medium that stores instructions to support signal processing operations.

For clarity, the order of the steps has been added. These steps can be performed in any suitable order.

Other embodiments of the present disclosure include software programs, firmware, and/or related hardware for performing any of the steps and operations of a method embodiment summarized above and detailed below.

In addition, it should be understood that the system, method, apparatus, instructions in computer-readable storage media, etc. discussed herein are may be implemented strictly as a program implemented in software, as a set of software, firmware, and/or hardware, or only in hardware, such as in a processor, or in an operating system, or in an application program, etc.

As discussed above, the techniques disclosed herein are well suited for use in software, firmware, and/or hardware applications that process signals and generate coded data bitstreams, or that process coded data bitstreams and create signal representations. However, it should be noted that the embodiments discussed here are not limited to use in such applications, and that the methods discussed here are well suited for other applications.

Additionally, note that while each of the various features, configuration methods, etc. described herein may be considered at different places in this disclosure, it is contemplated that each of the principles may be performed independently of or in conjunction with each other. Accordingly, one or more of the present inventions, embodiments, etc. described herein can be implemented and viewed in many different ways.

Also, note that this preliminary review of the embodiments discussed herein does not specify every embodiment and/or newer aspect of the present disclosure or the claimed invention(s). On the contrary, this brief description represents only general embodiments and corresponding novelties compared to conventional methods. Additional details and/or possible perspectives (permutations) of the invention(s) reader

- 7 040379 can be found in the Detailed Description section and the corresponding figures of this disclosure, which are discussed further below.

Brief description of the drawings

The above and other objects, features and advantages of the invention will appear from the following detailed description of the preferred embodiments discussed herein, illustrated in the accompanying drawings, in which like reference numerals refer to the same parts throughout the various views. The drawings are not necessarily drawn to scale, but emphasis is placed on illustrating embodiments, concepts, principles, etc.

Fig. 1A, 1B, 1C, 1D, and 1E are illustrative diagrams describing encoding and decoding systems according to the non-limiting embodiments illustrated herein.

Fig. 2A and 2B are exemplary diagrams describing multi-layer hierarchical decoding and coding according to the non-restrictive embodiments illustrated herein.

Fig. 3A, 3B, 3C, 3D, 3E, and 3F illustrate decomposition and decompression of residues according to the non-limiting embodiments illustrated herein.

Fig. 4A, 4B, 4C, and 4D are illustrative diagrams describing inverse decomposition according to the non-limiting embodiments illustrated herein.

Fig. 5 illustrates operations performed by a decoder according to the non-limiting embodiment illustrated here.

Fig. 6 illustrates a block diagram of an encoder performing directional decomposition according to the non-limiting embodiments illustrated herein.

Fig. 7 illustrates a flow diagram of a decoder performing reverse decomposition according to the non-limiting embodiments illustrated herein.

Fig. 8 is an illustrative diagram of a computer system that provides data processing in accordance with the embodiments described herein.

Fig. 9 is an exemplary information encoding scheme according to the embodiments discussed herein.

Fig. 10 and 11 are exemplary flowcharts of operations of exemplary methods according to the embodiments discussed herein.

Detailed Description and Additional Summary of Embodiments

The methods illustrated here are suitable for any type of multi-dimensional signal, including, but not limited to, audio signals, multi-channel audio signals, images, 2D images, video signals, multi-view video signals, 3D video signals, surround signals, surround video signals, medical imaging signals, signals with more than four dimensions, etc.

For simplicity, throughout the description, the illustrated embodiments typically provide for the case of using 2D images (e.g., either single images or frames/fields of a video signal), with each element (in such a non-limiting exemplary case usually referred to as a pixel, more generally referred to as a pel) characterized by a set of color or other parameter settings in a suitable color space (eg YUV, RGB, HSV, etc.). Different color planes (eg, a luminance plane, Y, and two chrominance planes, U and V) are often encoded separately and often at different resolutions (due to the lower sensitivity of the human eye to chrominance information).

The methods and embodiments illustrated herein may be used in conjunction with each other and/or with other methods. Many of the preferred embodiments illustrated herein describe methods and algorithms for achieving compression, i. encoding a suitable representation of the signal with a minimum number of bits. This is also a non-limiting example: other non-restrictive embodiments pursue other goals, such as multiscale encoding and decoding, adaptive streaming, reliable and efficient filtering, signal denoising (e.g., image denoising, video denoising, etc.), signal enhancements ( eg, signal resampling, deinterleaving, etc.), generation of signal quality metrics, content identification, machine vision, signal encryption (eg, secure communication), etc.

In FIG. 1A is an exemplary diagram describing non-limiting embodiments of encoding and decoding systems using the methods illustrated herein.

Encoder 110 receives a signal 100 (eg, a two-dimensional image) and encodes it into a data stream 115 . Decoder 120 receives data stream 115 and generates reconstructed signal 130. Reconstructed signal 130 may be identical or only substantially similar to original signal 100 depending on whether the encoding/decoding is lossy or lossless.

In one embodiment, the encoder 111 takes the source signal 140 (a video signal, ie, a sequence of several frames of two-dimensional images) and encodes it into a data stream 116 transmitted over the appropriate communication line to the target recipient. The decoder 121 on the receiver side receives the data stream 116 and generates a reconstructed signal 150 in which the reconstructed images 150-1, 150-2,...,150-n are reconstructed representations corresponding to the original images 140-1, 140 -2,...,140-n. The reconstructed images 150-1 may be identical or substantially similar to the original images 140-1 depending on whether the corresponding encoding and/or decoding is lossy or lossless; the reconstructed images 150-2 may be identical or substantially similar to the original images 140-2 depending on whether the corresponding encoding and/or decoding is lossy or lossless; the reconstructed images 150-3 may be identical or substantially similar to the original images 140-3 depending on whether the corresponding encoding and/or decoding is lossy or lossless; etc.

In accordance with additional options for implementation, the encoder 112 receives the original signal 160 (surround signal, which includes several elements) and encodes it into the stream 117 data. Decoder 122 receives data stream 117 and creates a reconstructed signal 170 (eg, a representation of the original signal).

In FIG. 1B is a non-restrictive illustrative diagram illustrating a multi-scale encoding and decoding system according to the embodiments discussed herein.

In one embodiment, encoder 111 takes the original signal 140 and encodes it into a multiscale data stream 116. In this exemplary embodiment, decoder 121-2 receives data stream 116 and creates a reconstructed signal at a lower LOQ 151 (eg, including frames 151-1, 151-2, etc.) at a first (lower) level quality than the original signal and/or signal 150.

Additionally, in this exemplary embodiment, decoder 121-1 receives data stream 116 and creates a reconstructed signal at a higher LOQ 150 (eg, second quality level). In this exemplary embodiment, the second quality level is higher than the first.

In a non-limiting embodiment, the second quality level has a higher resolution (spatial and/or temporal) than the first quality level. In another non-limiting embodiment, decoder 121-2 uses MPEG-based decoding techniques (eg, MPEG2, MPEG4, h.264, etc.) to decode data stream 116. In yet another non-limiting embodiment, decoder 121-1 generates the reconstructed signal at the lower LOQ 151 and then reconstructs or outputs the reconstructed signal at the higher LOQ 150 at least in part based on the first playback of the reconstructed signal at the lower LOQ 151. In others in other words, decoder 121-1 uses the first portion of data encoded in stream 116 to reproduce representations of signal 151 (eg, 151-1, 151-2, etc.). Thereafter, decoder 121-1 uses the second chunk of data in data stream 116 to upsample and modify signal representations 151 into signal representations 150.

In a non-limiting embodiment, the second quality level has a resolution obtained by upsampling with a scale factor of two in each spatial dimension of the first quality level. For example, data stream 116 may be configured to include a first chunk of reconstruction data indicating how to reproduce the original signal representation element at a first quality level. Additional reconstruction data in data stream 116 may indicate how to transform an element at the first quality level into multiple elements at the second quality level. By way of non-limiting example, multiple elements at the second quality level provide a higher resolution of the original element at the first quality level. By repeating these steps, low resolution images can be converted to higher resolution images. Higher resolution images provide more image detail (because it includes many more image elements) and thus arguably be more attractive to the viewer.

In other non-limiting embodiments, the second quality level has a resolution obtained by upsampling with a specific scale factor (including non-integer scale factors and/or scale factors equal to 1) of each of the signal measurements.

By way of example, in a non-limiting illustrative embodiment, the decoding signal processor of a television decoder (eg, without limitation, a set-top box) is programmed to implement the method as shown in FIG. 1B, with data stream 116 corresponding to the received broadcast signal. Thus, conventional decoders receive the same data stream 116 but simply ignore the additional data, allowing the signal representations to be reconstructed at a higher quality level. As mentioned, a higher quality level may be a higher resolution signal representation.

In other non-limiting embodiments, the streaming server processes the encoded data stream 116 and, in accordance with the characteristics of the decoder and/or bandwidth congestion, generates a version of the data stream that includes only the encoded data needed to decode the signal, to a given quality level (as opposed to the highest possible quality level). In such an example, no bandwidth is wasted on transmitting data that will not be used to reconstruct the representation of the original signal.

Fig. 1C illustrates a non-restrictive exemplary embodiment of multi-level hierarchical encoding and decoding of a two-dimensional image, with scaling factors of two in both dimensions between any subsequent quality levels, according to the embodiments discussed here.

In this exemplary embodiment, image 130-3 is a representation of an image at a given quality level, and specifically has a resolution of 8x8 pixels. Image 130-4 is a representation of the same image at the next lower quality level and has a resolution of 4x4 elements.

The lower LOQ element 130-4-1 is aligned with the higher LOQ elements 130-3-1, 130-3-2, 130-3-3 and 130-3-4. In this exemplary embodiment, element 130-4-1 is thus the control element (or control panel or parent element) of elements 130-3-1, 130-3-2, 130-3-3, and 130-3- 4 (managed elements or sub-elements).

Other non-limiting embodiments apply the same techniques on images with more than two dimensions, creating layered hierarchies with appropriate scale factors for each level, including non-integer scale factors and/or different scale factors for different dimensions and/or different scale factors across the hierarchy .

Fig. 1D illustrates another non-restrictive exemplary embodiment of layered hierarchical encoding and decoding of a 2D image, with scaling factors of two in both dimensions between LOQ N-1 and LOQ N, according to the embodiments discussed here.

The parent element 130-(N-1)-1 with LOQ N-1 (control element) is aligned with the corresponding four sub-elements 130-N-1 with LOQ N (control elements). In general, parent element 130-(N-1)-1 is used as the basis for reproducing (eg, by upsampling followed by adjustments using residual data) the respective four sub-elements 130-N-1.

Fig. 1E illustrates another non-restrictive exemplary embodiment of layered hierarchical encoding and decoding of a 2D image, with non-integer scaling factors of 3/2 in both dimensions between LOQ N-1 and LOQ N, according to the embodiments discussed here.

Parent block 130-(N-1)-2 with LOQ N-1 (group of controls, since in this case no element with LOQ N-1 is exactly aligned with multiple elements with LOQ N) is aligned with sub-elements 130-N- 2 with LOQ N (Many Controlled Elements). In this non-limiting example, calculating the predicted average of the residuals corresponding to subelements 130-N-2 may require processing both the predictive representation of subelements 130-N-2 and the parent block 130-(N-1)-2 (as opposed to a single parent element).

For simplicity and for purposes of further illustration, the following portions of the disclosure include examples using scale factors of two from one quality level to the next. However, those skilled in the art can easily adapt all of the methods described here for multi-level hierarchies to any of several different types of scale factors, including anisotropic and/or non-integer scale factors.

In FIG. 2A is an exemplary diagram describing signal decoding in a layered hierarchy of quality levels according to the embodiments discussed herein.

In this exemplary embodiment, decoder 210 receives reconstruction data 200-1 and creates a decoded image 245-(N-1) with LOQ N-1 (in this example, lower quality level or lower quality level).

The prediction calculator for the next LOQ 220-1 processes the image 245-(N-1) with the operations indicated by the corresponding parameters for the operations 215-1, producing a prediction image 225-1. Assuming a 2x2 zoom factor, calculator 220-1 creates a predictive image 225-1 indicating settings for an 8x8 block.

Decoder 230 receives reconstruction data 200-2 and creates residuals 240 (ie, reconstruction data or residual data). In this exemplary embodiment, the residuals 240 may be an 8x8 matrix of element setting information consistent with the number of elements in the image 245-N.

The predictive image 225-1, also generated by calculator 220, may also be an 8x8 matrix of element setting information consistent with the number of elements in image 245-N.

In one embodiment, the upsampling of image 245-(N-1) into predictive image 225-1 is just base images including 8x8 elements. The predicted picture 225-1 and the residuals 240 are combined, on an element-by-element basis, to create a decoded image 245-N, which is a signal representation with LOQ N (eg, 8x8 elements). In one embodiment, residuals 240 include a plurality of 8x8 elements that are added or combined with elements in predictive image 225-1 to create image 245-N. Applying residuals 240 to predictive image 225-1 creates a more accurate representation of the signal at an appropriate quality level. In other words, the signal representation 245-N is a more accurate representation of the original signal than the predicted image 225-1.

Signal representations at each ascending LOQ may be created according to the same method: generating, based on the representation at the current LOQ, a predictive representation at the next ascending LOQ; decoding residuals; combining the predicted representation at the higher LOQ with the decoded residuals, producing a representation of the signal at the higher LOQ. The merging may include, for each element in the array, selecting the position of the element in the predictive image 225-1 and the residuals 240 and summing the values to create a value for the corresponding element in the image 245-N.

In a non-limiting embodiment, decoder 210 uses, to decode reconstruction data 200-1, a decoding method that is different from the decoding method used by decoder 230. In one non-limiting illustrative embodiment, decoder 210 implements an H.264 decoding method.

In other non-limiting embodiments, decoder 230 creates residuals 240 based on a set of encoded data (reference plane encoded data) that is also used to generate residuals for an adjacent image in the image sequence.

Thus, any suitable upsampling algorithm can be used to convert a baseband signal or image at the first quality level to a higher resolution predictive image at the next quality level in ascending order.

In FIG. 2B is an exemplary diagram describing an encoder encoding a signal in a layered hierarchy of quality levels according to the embodiments discussed herein.

The signal processor configured as an encoder receives as an input image 130-0 with LOQ #0, for example, a higher resolution image. Image 130-0 is downsampled 250-0 to create image 130-1 with LOQ #-1. Image 130-1 is processed by the downsampler 251-1 to create image 130-2 at the next in descending order LOQ #-2. This downsampling process is repeated until a 130-B image is created at a lower LOQ #B (an even lower quality level).

After creating the image 130-B at the lowest quality level, the encoder begins to create reconstruction data (residual data or correction values) that will be required on the decoding side (i.e., at the decoder) to reconstruct the representation of the original signal at different quality levels (or, more precisely, a multi-level hierarchy of signal representations).

In this illustrative embodiment, for this purpose, the encoder creates reconstruction data for the lower LOQ (the initial LOQ for signal reconstruction), and then simulates all the operations that will be performed by the decoder for gradual reconstruction, based on the lower LOQ, to each next LOQ in ascending order. . Creating adjustment values for each element as you move up the hierarchy allows the appropriate decoder to reproduce a representation of the original signal at each of several quality levels.

In one embodiment, the lowest quality level is the lowest resolution. Successively higher quality levels represent higher resolution representations of the signal.

In this exemplary embodiment, image 130-B is processed by residual calculator 260+B to produce residual data R ^B 270+B. The 270+V residual data is processed by the 280+V reconstruction data generator to generate 290+V reconstruction data.

The reconstruction data 290+B is processed by the simulation decoder 2100+B to form a signal representation 245-B at the lower LOQ #B and a predictive representation 225-(B+1) with LOQ #(B+1). The predicted views at any given LOQ #N are processed by the encoder in conjunction with the corresponding picture 130-N to create residual data.

Residual calculator 261 receives image 130-1 with LOQ #-1 and predictive representation 225-(-1) with LOQ #-1, producing residual data 271. In one embodiment, residual data 271 (or reconstruction data) includes correction values to be applied to the received predictive representation 225-(-1) to reproduce the signal representation at the corresponding quality level #-1. The residual data 271 is processed by the reconstruction data generator 281 to generate reconstruction data 291 .

The reconstruction data 291 is processed by the simulation decoder 2101 to form a pre

- 11 040379 setting signal 245-1 with LOQ #-1 and predictive presentation 225-0 with LOQ #0.

Residual calculator 260 receives original image 130-0 with LOQ #0 and predictive representation 225-0 with LOQ #0, producing residual data 270. In one embodiment, residual data 270 (or reconstruction data) includes correction values to be applied to the received predictive representation 225-0 to reproduce the signal representation at the corresponding quality level #0. The residual data 270 is processed by the reconstruction data generator 280 to generate reconstruction data 290 .

The reconstruction data 290 is processed by the simulation decoder 2100 to form a signal representation 245-0, which is the highest LOQ signal representation in the representation hierarchy to be generated on the decoding side.

In a non-limiting embodiment, the encoder makes sure that the representation 2450 is appropriately similar to the original image 130-0 according to the quality metric. If the quality metric value falls below the threshold, the encoder reprocesses the reconstruction data for one or more LOQs (eg, adjusting the appropriate encoding parameters) until the quality metric for similarity between 245-0 and 130-0 exceeds the threshold.

In another non-restrictive embodiment, the encoder makes sure that the amount of bitstream required to encode reconstruction data 290, 291,...,290+B is below a maximum threshold. If the required bitstream value is above a threshold, the encoder reprocesses the reconstruction data for one or more LOQs (eg, adjusting the appropriate encoding parameters) until the required amount of data to be transmitted in the bitstream is below the threshold.

Fig. 3A and 3B illustrate an example of directional decomposition according to the embodiments discussed here. Directional decomposition is applied to groupings of 2x2 blocks of residuals, in the case of 2D upsampling with a scale factor of two in both dimensions (i.e. for each parent element with LOQ #(N-1) there are four managed pals (i.e. picture elements ) on the next in ascending order LOQ #N, and therefore four residues need to be decoded in order to properly reconstruct said controlled pads.

Directional decomposition provides a way to encode reconstruction data (eg, residual data) in a useful way.

In this example, it is assumed that residual block 300 is a portion of the residual data plane at a given LOQ #N that is processed by a decoder and combined with a predicted signal representation at a given LOQ #N to create a signal representation at said given LOQ #N. In particular, the block of residues 300 is formed by four residues ri,j, ri,j ₊₁ , r _i+1 ,j and r _i+1 ,j ₊₁ . The remainder ri,j is an adjustment value indicating how to modify the corresponding element pi,j in the prediction image to reproduce the representation of the element xi,j; the reconstruction data ri,j ₊₁ is an adjustment value indicating how to modify the corresponding element pi,j ₊₁ in the prediction image to reproduce the representation of the element xi,j ₊₁ ; reconstruction data r _i+1 ,j is a correction value indicating how to modify the corresponding element P _i+1 ,j in the prediction image to reproduce the representation of element x _i+1 ,j; reconstruction data r _i+1 ,j ₊₁ is an adjustment value indicating how to modify the corresponding element P _i+1 ,j ₊₁ in the prediction image to reproduce the representation of element x _i+1 ,j ₊₁ .

In particular, the block of residuals 300 corresponds to the block 305 of pixel elements xi,j, xi,j ₊₁ , x _i+1 ,j, x _i+1 ,j ₊₁ c LOQ #N controlled by the parent element y ¹ 306 at the lower LOQ #(N-1), which was generated by the encoder based on the pixels with LOQ #N.

In this non-restrictive embodiment, the parent element settings y ¹ were generated by quantizing the weighted average of the element settings corresponding to block 300 according to one of several downsampling algorithms, such as a bilinear filter kernel, and with the parent element sampling position being at the center of the block (i.e. .in the non-restrictive case of bilinear filtering, the parent element corresponds to the simple average of the controlled pels). As a result, the sum of the residuals of the controlled pels (i.e. the sum of ry, ri,j ₊₁ , r _i+1 ,j and r _i+1 ,j ₊₁ ) depends at least in part on the setting of the parent element y ¹ ; in fact, in the absence of quantization operations, it will be exactly determined by the parent element, which is already known to the decoder from the start time of block 300 reconstruction.

To make proper use of such information, the encoder discussed here performs a decomposition transformation that takes advantage of two types of correlation together:

(1) autocorrelations between block 300 residuals, and (2) correlations between block 300 residuals, predictors corresponding to block 300, and the parent element.

As mentioned, decomposition is a way of encoding residues 300 into different formats (for example, directed components 320 and then converted residues 350 that are more suitable

- 12 040379 for transmission over the link to a remote target device that decodes the data to reproduce the image).

Specifically, in this exemplary embodiment, in order to exploit autocorrelation between residuals, the encoder applies a directional decomposition transform to the residuals, producing four directional components A, H, V, and D:

A - the average of the residues g _c , Hz+1, η+ _υ , Γί+ _υ +1;

H is the horizontal/latitudinal slope of said residues;

V is the vertical/longitudinal slope of said residues;

D is the diagonal/oblique slope of said remnants. In general, the H, V, and D values include sufficient decoding information to convert the A value to more specific settings for each of the residual components ry, ry ₊ i, r _i+1 ,j, ri+y+i.

From a geometric point of view, the transformation is equivalent to replacing the four samples in space with their representation given by the plane being on the z-axis with mean of the samples and slope, respectively, for a 4-sample reconstruction as shown in FIG. 3A.

As mentioned above, the transformation has two goals: exploiting the correlation between the residuals (e.g., directional correlation of lines and typical image patterns) and reducing the information entropy corresponding to the average A, which is easily predictable at the decoder side, at least in part based on the predicted values.

On the encoder side, the encoder knows that the parent has been computed according to formula 3.12 in FIG. 3B. The difference δ _a between y and y ¹ is due to errors generated in the quantization/dequantization processes. According to equation 3.14 in FIG. 3B, by subtracting from y ¹ the average of the predictions p for the control panel settings x, one can generate a prediction (PA, for the predicted average) for the average of the four residues 300 such that the difference between A and PA is δ _a .

In this exemplary embodiment, the encoder thus generates the transformed residuals 350 as follows: processes the residuals 300 with the directional decomposition block 310, producing directional components 320. In particular, the directional components are computed according to the calculations illustrated in formula 3.11:

creates a predicted average of 340;

processes the directional components 320 and the predicted mean 340 with the transformed residual calculator 330 to produce the transformed residuals 350.

In a non-limiting embodiment, δ _a is calculated according to the formula δ _a =A-PA=A-(y' _k ,hAvgP), where AvgP is the average value of the predictions p for the settings x of the controlled pels.

In a non-limiting embodiment, when calculating the directional components 320, the encoder does not divide by 4 as indicated in formula 3.11 in FIG. 3B in order to reduce the computation required for the inverse decomposition process required at the decoding side.

In a non-restrictive embodiment, the decoder receives the transformed residuals δ _a , H, V, D (calculated by the encoder as described above) and computes the residuals ri,j, ry ₊₁ , r _i+1 ,j, ri+1,j+1 , by first creating the predicted mean RA and reconstructing the A, H, V, D components, and then solving the equation in Equation 3.11 for ri,j, ry ₊ i, r _i+1 ,j, r _i+1 ,j _{+ 1} .

In another non-restrictive embodiment, the decoder receives the transformed residuals δ _a , H, V, D and calculates A according to the formula A=δ _a +(y' _k , _h -AvgP), where AvgP is the average value of the predictions p for the settings x of the controlled pels . From the point of view of information entropy, the advantage of transmitting/receiving δ _a (i.e. the difference between the actual mean of the residuals and the predicted mean of the residuals) instead of the value of A (average) is based on the fact that δ _a is very likely to be zero (or close to to zero), so the null character becomes more likely to occur in the sequence of encoded transformed residues, which improves coding efficiency. In other words, passing the value δ _a, as opposed to passing the value A, saves on the number of data bits that need to be encoded remainders 300. Having calculated A in this example, the decoder calculates the remainders ri,j, ri,j ₊₁ , r _i+1 , j, r _i+1 ,j ₊₁ , solving the equation in 3.11 for ri,j, ri,j ₊₁ , r _i+1 ,j, ri+1j+1 generating transformed residuals 350 takes advantage of the correlation between residuals. In a non-limiting embodiment, decomposition allows the residues to be separated into a constant portion and one or more variable portions. Some of the transformed elements may be predictive, making them easier to convey. In one embodiment, the converted residuals 350 are obtained based on a combination of sums and/or differences between the individual components.

As the equations show, starting with a vector r (containing the four remainders 300, such as ri,j), the encoder computes a vector l, where the vector l contains four parameters derived from mean A, horizontal slope H, vertical slope V, and diagonal slope D. In particular, , the vector l is calculated by multiplying the vector r by the matrix M. The decoder receives parameters that allow it to reconstruct the vector l (in particular, in the non-restrictive embodiment described above,

- 13 040379 instead of receiving the parameter A, the decoder receives the parameter δ _ω indicating the difference between A and the predicted average of the residuals PA). Having reconstructed l, the decoder reconstructs the vector r by multiplying the matrix inverse of the matrix M by the vector l. As follows from the following equations in Fig. 3B, in a non-restrictive embodiment, matrix M ^-1 is identical to matrix M:

I - M - g

' eleven 1 1 Ί ^r oo 4v -eleven 1 -1 'll eleven -1 -1 % J- ¹ -1 Cib

In another non-limiting embodiment, different residuals in the set of transformed residuals are quantized according to different quantization parameters. This is due to the different relative importance of the directional components to perceived quality, which allows certain transformed residuals to be quantized differently than others without noticeable differences in quality. In other words, as mentioned, the reconstruction data or residual data at each respective quality level to which the data belongs is responsible for adding detail back to the predicted signal representation, whereby the generated signal representation more accurately reflects the original signal for the corresponding quality level.

For example, the transformed residuals corresponding to the mean or diagonal components are quantized more aggressively than others. In one embodiment, the decoder dequantizes different residuals according to different dequantization parameters. In another embodiment, different quantization parameters make it possible to improve the accuracy of the vertical tilt and horizontal tilt of the residuals (to which the human eye is particularly sensitive) independently of the accuracy of the diagonal/tilt information, so that, in the case of quantization artifacts (e.g., image errors that arise due to filtering for saving data bits) contributed by compression, particular components of the residuals are favored with respect to bit rate allocation.

In another non-limiting embodiment, the H and V parameters are quantized differently according to an estimate of the amounts of horizontal vs. vertical correlation in the signal (e.g., a field in interlaced video often has higher horizontal correlation than vertical correlation, allowing it to be efficiently coded by quantizing H and V by in different ways).

In another non-restrictive embodiment, for the last quality level, the decoder takes only the parameters corresponding to H and V and reconstructs the full set of directional components (and then the residuals) by setting A and D to default values.

Other non-limiting embodiments employ similar residual transformation techniques to encode and/or decode residuals for control pad sets having more than two dimensions (e.g., by way of non-limiting example, residues for sets of eight control pads obtained by a 3D upsampling operation), and/or different scaling factors from the lower LOQ to the next in ascending LOQ order.

Those skilled in the art can easily derive the transformation matrix to be used in other embodiments by performing upsampling operations involving more than two dimensions (e.g., by way of non-limiting example, operations involving two spatial dimensions and a temporal dimension, all scaled by a factor of two, reconstruction of the 8 higher LOQ controlled pods for each lower LOQ controlled pal). By way of non-limiting example, volume upsampling with a scale factor of two in all dimensions requires an 8x8 matrix transformation of the residual decomposition computing 8 directional gradients/residual slopes. In a non-limiting embodiment, for each block of a 2x2x2 signal, the encoder calculates (and the decoder receives and decodes) one parameter corresponding to the mean residual value and 7 parameters corresponding to the 7 residual slope values.

In another non-limiting embodiment, both the encoder and the decoder operate separately in temporal and spatial dimensions. Temporal tilt is transmitted and decoded first, and then the remaining 6 values (3 for each spatial plane) are transmitted as spatial tilt values.

In another non-restrictive embodiment, the encoder and decoder use a full directional decomposition method in two dimensions (e.g., at the encoder, for each block of 2x2 residuals: calculating mean A, horizontal slope H, vertical slope V, and diagonal slope D of residuals, and then encoding, instead of mean A, the difference δ _a between mean A and predicted mean PA, at the decoder: receiving the difference δ _α between mean A and predicted mean PA, horizontal slope H, vertical slope V, and diagonal slope D, and calculation from such parameters four residuals), and then a partial directional decomposition in the third dimension (eg, in time, calculating, encoding and decoding only the slope between the two averages of the 2x2 blocks of residuals).

In another non-limiting embodiment, the encoder may choose to avoid transmitting one residue for each set of residues corresponding to the same control pad at a lower LOQ. This is equivalent to quantizing 5a to zero, regardless of its actual value.

In other non-limiting embodiments, the decoder receives complete sets of residuals and exploits redundancy in the residual information for error correction or error detection purposes, which improves transmission reliability.

In other non-limiting embodiments, with scaling factors among LOQs other than 2 for one or more directions, the encoder and decoder calculate a predicted mean over multiple residuals based at least in part on the values of multiple signal bins at the lower LOQ.

In other non-limiting embodiments, the encoder applies decomposition transformations into particular sets of residues in order to maximize the number of transformed residues that are assigned a value of zero. This is because entropy encoding methods are more efficient at compressing a stream of numbers that include a higher percentage of zero values. In a non-limiting embodiment, the parameters corresponding to the applied decomposition transforms are transmitted to the decoder, along with the reconstruction data.

In a non-limiting embodiment, the encoder selects multiples of different picture residuals on the same LOQ in a picture sequence, said residual sets corresponding to the same position (i.e., set of coordinates) across multiple consecutive pictures (e.g., multiple sampling positions in time ). For each given position in the residual sets, the encoder selects the residuals of the sets that are at that position (e.g., the temporal sequence of the residuals over time) and generates a sequence of transformed residuals starting with a value corresponding to the average of said residuals and then, according to each data remainder, values corresponding to the stage between the previous remainder and this remainder. Thus, the encoder very effectively represents a sequence of several values that are constant for a while, then change to another value, and then remain constant again: in fact, such a sequence is represented by only two non-zero values (i.e., the value corresponding to the average, and the value corresponding to the pad where the stage change occurs). In such an embodiment, to reconstruct the original residuals, the decoder processes the transformed residuals and computes the residuals by multiplying the transformed residuals by, for example, the following matrices (depending on the number of residuals in the sequence):

Μ2

32 1

-132 1

-1-221

-1 -2 -3 1

-1 -2 -3 -4 _? , etc.

In FIG. 3F is an exemplary diagram illustrating reconstruction data generation according to the embodiments discussed herein.

As shown, the computer processing equipment of FIG. 3F is configured to generate reconstruction data according to the embodiments discussed herein. As shown, when encoding, the downsampling algorithm 371 receives and processes a block 305 including X11, X12, X21, X22. Each of these values X11, X12, X21, X22 represents a corresponding setting associated with the corresponding element.

In one embodiment, the downsampling algorithm 371 receives and processes the array X at a first quality level and converts the array of x values (X11, X12, X21, X22) into a single y value (eg, representative value 361). The quantizer algorithm 371 transforms

- 15 040379 the value of y in y ¹ (for example, the quantized representative value of 362, also indicated as 306 in Fig. 3B). The upsampling algorithm 373 associated with the encoder, by processing the signal representation at the lower LOQ #2, initiates upsampling of the value 3 62 (eg, y ¹ ) into the predicted representation (P11, P12, P21, P22) corresponding to the original signal representation ( X11, X12, X21, X22). The difference function 376 creates reconstruction data or residual data 300 as the difference between the original signal indicated by X11, X12, X21, X22 and the predicted representation P11, P12, P21, P22 to create correction values R11, R12, R21, R22. In one embodiment, the correction value R11=X11-P11; correction value R12=X12-P12; correction value R21=X21-P21; and correction value R22=X22-P22.

Processor resource 377 receives a first set of adjustment values (eg, residual data 300 or adjustment values R11, R12, R21, R22). The first set of adjustment values indicates adjustments to be made to the predictive representation (P11, P12, P21, P22) of the signal generated at the corresponding level #1 for reconstructing the representation of the signal (X11, X12, X21, X22) at the first quality level. Processor resource 377 processes the first set of adjustment values R11, R12, R21, R22. Processor resource 377 outputs a second set of correction values (or converted residuals 350 or 5 _a , H, V, D) based on the first set of correction values R11, R12, R21, R22 and a signal representation of, for example, y ¹ at a second quality level.

In this example, quality level #2 is below quality level #1. The signal may be an image that includes several elements. By way of non-limiting example, quality level #1 may be the resolution of the first image; quality level #2 may be the resolution of the second image. The resolution of the second image is lower in resolution than the resolution of the first image.

Note, again, that the signal encoded by the encoder hardware in FIG. 3F may be any suitable signal type. For example, the signal may be a two-dimensional image; the signal may be a video signal; the signal can be a three-dimensional image; etc.

In another specific embodiment, processor resource 377 includes a reconstruction data processor 396 . The reconstruction data processor 396 converts the residual data 300 into A, H, V, and D component values. As mentioned, the A, H, V, and D values are an encoded representation of the residuals 300. Based at least in part on the value of y ¹ and the directional components of A , H, V, and D, reconstruction data converter 397 generates correction values 5 _a , H, V, D (transformed residual data 350).

In one non-limiting exemplary embodiment, processor resource 377 outputs a plurality of correction values 5 _a , H, V, D (transformed residual data 350) by processing a plurality of correction values R11, R12, R21, R22 to create multiple correction components A, H, V and D, which represent a plurality of adjustment values R11, R12, R21, R22. The reconstruction data converter 397 creates a set of correction data δ _a , H, V, D (transformed residual data 350) based on a combination of at least one of several correction components, for example an A value, with at least one setting associated with the signal representation, for example, ¹ , at quality level #2.

As mentioned, one or more of the correction values δ _a , H, V, D (transformed residual data 350) can be calculated based on a linear combination of a plurality of correction values R11, R12, R21, R22 with a signal representation (for example, y ¹ ) at the level quality #2. For example, the value δ _a can be calculated based on the difference between the average A (average of X11, X12, X21, X22) generated by the quantizer algorithm 372 and the predicted average generated at least in part based on the average of the elements (P11, P12 , P21, P22) at block 393 (eg, [P11+P12+P21+P22]/4).

By way of example, suppose that the data in block 305 (corresponding to the bin plane area of the bins that are downsampled) includes: X11=10, X12=20, X21=25, X22=30. In this example, the downsampling algorithm 371 generates a value of y=21.25 using the average elements X11, X12, X21, X22. A single y=21.25 value at quality level #2 is a representative setting value associated with elements X11, X12, X21, X22.

Assume that the quantizer algorithm 372 quantizes the value y=21.25 to create y'=21. In one embodiment, quantization includes comparing a received value, such as y=21.25, with multiple ranges. In this example, it is assumed that any values in the range A1 between 21 and 21.999 are assigned the value 21. The value y=21.25 falls within the range A1 and is thus quantized to a value of 21 as a result of quantization by the quantizer algorithm 372.

As previously discussed, the embodiments discussed herein include upsampling a quantized value y'=21 (also based at least in part on its neighbors) into a set of predictive values P11, P12, P21, P22. In this example

- 16 040379 it is assumed that the application of the upsampling algorithm 373 creates settings P11=10,

P12=15, P21=20, P22=25. These P values approximate the settings of the original X values (X11, X12,

X21, X22), but needs some correction.

The embodiments discussed herein include applying the difference function 376 to generate R(R11, R12, R21, R22) values. For example, the difference function 376 sets R11=X11P11 (R11=10-10=0); difference function 376 sets R12=X12-P12 (R12=20-15=5); difference function 376 sets R21=X21-P21 (R21=25-20=5); the difference function 376 sets R22=X2-P22 (R22=30-25=5).

In accordance with the previous equations, the reconstruction data processor 396 creates settings for A, H, V, and D based on the settings of R11, R12, R21, R22 at quality level #1. For example, according to the corresponding matrix, the reconstruction data processor 396 creates directional components 320 as follows:

A=[Rl1+R12+R21+R22]/4=[0+5+5+5]/4=3.75

H=[Rl1-R12+R21-R22]/4=[0-5+5-5]/4=1.25

V=[Rl1+R12-R21-R22]/4=[0+5-5-5]/4=1.25

D=[Rl1-R12-R21+R22]/4=[0-5-5+5 ]/4=-1.25

Based on these values for A, H, V, and D, reconstruction data converter 397 generates transformed residuals 300 (δ _a , H, V, D) as follows:

5 _a = mean - predicted mean = {3.75 - [y ¹ - (Pl1 + P12 + P21 + P22) / 4]} =

5 _a \u003d {3.75- [21- (10 + 15 + 2 0 + 25) / 4] } \u003d 3.75- (21-17.5) \u003d 3.75-3.5 \u003d 0.25

In this example, the processor resource 377 creates the transformed residual data 350 as follows:

b _a =0.25

H=1.25

V=l,25

D=-l,25

Accordingly, the encoder hardware as discussed here receives a signal representation (X11, X12, X21, X22) at quality level #1. The encoder downsampling algorithm 371 in FIG. 3F creates a signal representation y at quality level #2 based on the signal representation (X11, X12, X21, X22) at quality level #1. The upsampling algorithm 373 creates a predictive representation of the signal P11, P12, P21, P22 at quality level #1 from the representation of the signal at quality level #2. For example, the quantizer algorithm 372 creates y ¹ from y. The upsampling algorithm 373 converts y ¹ into P11, P12, P21, P22 through quality level #2 signal representation processing. As mentioned, the difference function 376 creates a set of correction values R11, R12, R21, R22 as the difference between the signal representation R11, R12, R21, R22 at quality level #1 and the predicted signal representation P11, P12, P21, P22 at quality level #1 . The reconstruction data processor 396 creates a formatted set of adjustment values (eg, directional components 320) from the set of adjustment values R11, R12, R21, R22. As its name suggests, reconstruction data converter 397 converts the formatted correction value set into a correction value set δ _a , H, V, D.

The embodiments discussed herein may further include additional encoder hardware and software for encoding a plurality of correction values δ _a , H, V, D and representing signal y ¹ at quality level #2 into corresponding encoded information. The transmitter resource transmits the corresponding encoded information over the link to the decoder processing equipment (see FIG. 5 for a non-limiting example of decoding).

In accordance with FIG. 5, the decoder processing equipment receives the encoded information. From the encoded information, the decoder processing equipment obtains (by decoding) reconstruction data, for example, the value y1=21. The value of y ¹ indicates how to reconstruct the signal representation for block 306 (FIG. 3B) at quality level #2. From the received encoded information, the decoder processing equipment obtains (by decoding) a set of correction values δ _a , H, V, D.

The decoder processing equipment reproduces the predictive representation of the signal P11=(element 500-1-1), P12 (element 500-1-2), P21 (element 500-1-3), P22 (element 500-1-4) at the quality level #1 based at least in part on the y value of ¹ in the resulting reconstruction data. The decoder processing equipment outputs a correction value (predicted average or PA), for example, a value of 430-5, based on the processing of the settings associated with the predicted signal representation, and the obtained reconstruction data (for example, y1=21). In this exemplary embodiment, block 520 generates an adjustment value 430-5 equal to y ¹ minus the average of AvgP predictions (eg, [(P11+P12+P21+P22)/4). Recall that the upsampling algorithm 373 (also used by the decoder processing equipment) applied to y ¹ and its neighbors in the decoder processing equipment produces the values P11=10, P12=15, P21=20, P22=25. Note that the particular algorithm to be used for upsampling the value of y ¹ may be indicated in the data received by the processing equipment of the decoder. In particular, for a given element of signal pre-representation at a higher quality level, creating settings for said given element may comprise: receiving metadata associated with upsampling operations to be used to create, based on the signal representation at a lower quality level, preliminary presentation of the signal at a higher quality level; identifying an upsampling operation (eg, without limitation, upsampling kernel coefficients) as indicated by the metadata; and applying the operation to the subset of the lower quality level signal presentation elements to output the settings for said given signal preview element at the higher quality level.

Thus, processing unit 520 creates y ¹ -AvgP=21-[ (10 + 15+20 + 25)/4]=21-17.5=3.5

Function 562 sums the delta 445-1 (e.g., δ _α =0.25) obtained by decoding as discussed above) and the value of PA=3.5 to create a signal 430-5, e.g., a value of 3.75 (i.e., the average A=3.75). Thus, the decoder processing equipment uses the derived correction value (e.g., PA=3.5) as the basis for modifying the value _5a =0.25 into the correction value A=3.75)

The data obtained by decoding the received information also includes obtaining values for H, V, and D. In this example, the values are: H=1.25, V=1.25, and D=-1.25.

The decoder processing equipment then applies the modified set of correction values (eg, A, H, V, D) to the reproduced predictive signal representation P11, P12, P21, P22 to create signal representation 550-1 at quality level #1. For example, the decoder processing hardware uses a row of adders associated with matrix values 532 to create:

element 550-1-1=element 500-1-1+(A 455-1+H 455-2+V 455-3+D 455-4) = 10+(3.75+(-1.25)+ (-1.25)+(-1.25))=10=X11;

element 550-1-2 = element 500-1-2+(A 455-1-H 455-2+V 455-3-D 455-4) = 15+(3.7 5-(-1.25) + (-1.25) - (-1.25)) \u003d 20 \u003d X12;

element 550-1-3=element 500-1-3+(A 455-1-H 455-2+V 455-3-D

455-4) \u003d 20 + (3.75 + (-1.25) - (-1.25) - (-1.25)) \u003d 25 \u003d X21; and element 550-1-4=element 500-1-4+(A 455-1-H 455-2-V 455-3+D

455-4) \u003d 2 5 + (3.7 5- (-1.25) - (-1.25) + (-1.25)) \u003d 3 0 \u003d X12.

Accordingly, by obtaining the residual data, for example, y1=21 and values _5a =.25, H=-1.25, V=-1.25, and D=-1.25, the decoder processing equipment can reproduce the original values X11, X12 , X21 and X22.

In accordance with additional embodiments in FIG. 9, the encoder processing hardware discussed herein may be configured to generate multiple reconstruction data sets 910. For example, the first portion of the signal, such as X elements, may be processed and encoded to create a reconstruction data set 910-1 as previously discussed. Similarly as discussed previously, the encoder may be configured to generate a reconstruction data set 910-2 based on elements Z11, Z12, Z21, and Z22.

As discussed, each reconstruction data set 910 may include settings for multiple components, e.g., A (first component), H (second component), V (third component), and D (fourth component) by means of the transformed value δa (the first transformed value ), H (second converted value), V (third converted value), and D (fourth converted value). The first converted value, for example, 5 _a (delta mean) in each of the sets corresponds to the first attribute class; the second converted value in each of the sets corresponds to the second attribute class, for example, horizontal (H); the third transformed value in each of the sets corresponds to the third attribute class, for example, vertical (V); the fourth transformed value in each of the sets corresponds to the fourth attribute class, for example, diagonal (D).

- 18 040379

In one embodiment, the encoder processing hardware creates a first grouping 9251 to include a first component value from each of multiple reconstruction data sets 910. For example, grouping 925-1 includes component values S _a1 , δ _a 2,... .

The encoder processing hardware creates a grouping 925-2 to include a second component value, such as H, from each of the multiple reconstruction data sets 910. For example, grouping 925-2 includes component values H1, H2,... .

The encoder processing hardware creates a grouping 925-3 to include a third component value, such as V, from each of the multiple reconstruction data sets 910. For example, grouping 925-3 includes component values V1, V2,... .

The encoder processing hardware creates a grouping 925-4 to include a third component value, such as D, from each of the multiple reconstruction data sets 910. For example, grouping 925-4 includes component values D1, D2,... .

In one embodiment, the encoder processing hardware encodes the groupings independently of each other. For example, the encoder processing equipment entropy encodes the first grouping 925-1 into encoded information 965-1 independently of entropy encoding the second grouping 925-2 into encoded information 965-2.

In other non-limiting embodiments, groupings of transformed values are obtained by grouping the transformed values corresponding to the same signal tile (eg, tile), and thus individually encoding the transformed residuals corresponding to each tile. In other non-limiting embodiments, transformed values corresponding to the same residual data set 910 are assigned to a given symbol, and the encoder processing hardware encodes a sequence of symbols, with each of the symbols corresponding to one of the residual data sets 910.

Fig. 3C illustrates a non-restrictive embodiment of directional decomposition applied to groupings of two residuals, in the case of 2D upsampling with a scale factor of two in the horizontal dimension and one in the vertical dimension (i.e., for each parent element with LOQ #(N-1) there are two managed pals on the next in ascending order LOQ #N, and therefore two residues need to be decoded to properly reconstruct said managed pals) according to the embodiments discussed here.

The encoder calculates, according to formula 3.21, the two directional components A (for the mean) and T (for the slope), and then creates the transformed residuals δ _a and T, where δ _a is calculated as the difference between A and the predicted mean PA, and PA is calculated as the difference between the value of y ¹ element 308 and the average AvgP of the predicted values corresponding to two elements 307.

Fig. 3D illustrates a non-restrictive embodiment of directional decomposition applied to 2x2 block groupings of residuals, in the case of 2D upsampling with a scale factor of two in both dimensions (i.e., for each parent element with LOQ #(N1) there are four managed pals in the next in order increasing LOQ #N, and therefore four residues need to be decoded in order to properly reconstruct said controlled pals) according to the embodiments discussed here.

Residual block 300 is a portion of the residual data plane at a given LOQ #N that is processed by a decoder and combined with a predicted signal representation at a given LOQ #N to create a signal representation at said given LOQ #N. Specifically, the block of residues 300 is made up of four residues ry, ri,j ₊₁ , r _i+1 ,j and r _i+1 ,j ₊₁ .

Residuals 300 are processed by a directional decomposition block 310 according to Formula 3.11 to form directional components 320. Directional components 320 and predicted mean 340 are then processed by a transformed residuals calculator 330 to form transformed residuals 350. The transformed residuals 350 are then quantized by quantizer 360 to form quantized transformed residuals. 370.

In a non-limiting embodiment, the quantized transformed residuals 370 are then further processed and encoded into encoded reconstruction data. In a non-limiting embodiment, the quantized transformed residuals for the total signal are grouped into separate groupings, which are individually entropy encoded into separate reconstruction encoded data sets. In a non-limiting embodiment, the individual groupings correspond to individual classes of transformed residuals (eg, without limitation, quantized transformed residuals corresponding to H components for the total signal versus quantized transformed residuals corresponding to V components for the total signal, etc.). In another non-limiting embodiment, the individual groupings correspond to individual sections of the signal (eg, without limitation, signal tiles).

In accordance with FIG. 3D directional decomposition block 310 creates a formatted set

- 19 040379 correction values, eg directional components 320 of the plurality of residues 300 (eg correction values). The calculator 330 converts the formatted set of adjustment values, eg, directional components 320, into transformed residuals 350 (eg, adjustment values). The encoder then applies the quantizer algorithm 360 to the set of correction values (transformed residuals 350) to create a set of quantized correction values, such as quantized transformed residual 370.

In one embodiment, the set of correction values δ _a , H, V, D is a vector including multiple reconstruction data components (eg, 5a component, H component, V component, D component). Quantizer 360 quantizes each of the correction components to create a quantized set of correction values, such as residuals 370.

Different components in the transformed residues 350 can be quantized according to different rules. For example, the first set of ranges can be used to quantize the 5a component in the respective residuals 370. Each range can be assigned a corresponding value. Depending on which of the ranges the value of component 5a falls in, the corresponding value assigned to the range is then assigned to component 5a as the quantized value for the component.

The second set of ranges may be used to quantize the H component in the respective residuals 370. Each range in the second set may be assigned a corresponding quantized value. Depending on which of the ranges in the second set the value of the H component falls in, the corresponding value assigned to the range is then assigned to the H component as the quantized value for the component.

Thus, a variety of different quantization ranges can be used to generate quantized values for each of the different component types 5a, H, V, D. As mentioned, quantization is used to generate similar settings for otherwise different component settings. Quantization reduces the amount of data required to capture a signal.

Accordingly, the embodiments discussed herein include computer processor hardware for quantizing the first correction data component in vector 5a, H, V, D based on first quantization rules. The computer processor hardware quantizes the second component of the correction data in the vector based on the second quantization rules. The second quantization rules are different from the first quantization rules.

Fig. 3E illustrates another non-restrictive embodiment of quantizing and transforming a 2x2 block of residuals 300, with the residuals being quantized prior to transform according to the embodiments discussed herein. It should be emphasized that pre-transform quantization is an important and additional new step provided by this invention, since, until now, traditional coding methods have applied post-transform quantization, for reasons that include the possibility of quantizing certain transformed parameters more heavily than others, i.e., That is, generally speaking, giving the higher priority parts of the transform a higher bit rate.

As shown in FIG. 3E, in this non-limiting embodiment, the quantization of the residual is performed prior to proceeding to the transform step, and in particular, the quantizer 380 processes the residuals 300 to produce quantized residuals 309.

The quantized residuals 309 are processed by a lossless directional decomposition block 315, which produces lossless directional components 325 (ldc) by processing a vector of four quantized residuals 309 (qr) according to the following formula:

ldc- M dg

It should be emphasized that the number of bits required for the lossless directional components 325 is greater than the number of bits used to describe the quantized residuals 309, since the lossless directional components 325 are obtained by performing addition and subtraction operations on the quantized residuals 309.

The lossless directional components 325 and the predicted sum 345 are processed by the transformed residual calculator 335 to form the transformed residuals 355.

In a non-limiting embodiment, the quantization of residues 300 in quantizer 380 is based at least in part on parameters generated by processing the entire plane of residues to which residues 300 belong (ie, based on the total signal).

In a non-limiting embodiment, the quantization of residues 300 is performed by quantization

- 20 040379 an optimization method that ensures that the difference between the reconstructed element (i.e., the combination of the predicted element and its corresponding residual) and the corresponding element in the original signal is below a given maximum error threshold effectively guarantees error control of the representation of the signal reconstructed at decoder. In some non-limiting embodiments, the quantization of the residuals 300 is based on parameters that are generated by processing the substantially complete signal and/or the entire residual plane that contains the residuals 300. In a non-limiting embodiment, this is achieved by processing the total signal and the entire residual plane, generating a priority map that assigns a priority value to each residual in the residual plane; the residuals are then quantized at least in part based on said priority values.

In a non-limiting embodiment, the transformed residuals are symbols selected in character sets having a different number of bits (i.e., total number of symbols) based on the relative position of each transformed residual in the vector of transformed residuals 355.

As discussed above, if desired, quantization can be performed directly on the residues 300 before decomposing into appropriate values. For example, quantizer algorithm 380 receives a first set of correction values (residuals 300) and creates a quantized correction value set, such as quantized residual 309. In one embodiment, directional decomposition block 315 applies one or more lossless and reversible operations on the received quantized correction set. values to create an intermediate set of adjustment values, such as S, V ¹ , H ¹ , D ¹ . The calculator 335 receives a value (eg, a predicted sum) derived at least in part based on the signal representation at the second quality level and the predicted signal representation at the first quality level. Calculator 335 uses the value (eg, predicted sum 345) to convert an intermediate set of adjustment values S, V ¹ , H ¹ , D ¹ into a converted remainder 355 (eg, 5s, H ¹ , V ¹ , D ¹ ).

In one embodiment, the settings of the quantized adjustment value set (eg, quantized residuals 309) are created such that the differences between the settings in the set of quantized adjustment value sets (eg, quantized residuals 309) and the corresponding settings in the adjustment value set (eg, settings 300 ) below the error threshold.

Additional Embodiments

As discussed above, the correction values may be quantized at any suitable time during the encoding process. Note that, in accordance with other additional embodiments, the set of correction values refers to the reconstruction of some particular position in the plane of several elements that are encoded at different quality levels. Each of the different portions of the elements in the plane can be quantized with different volumes depending on the specific location mentioned where the corresponding elements are located. For example, in relatively homogeneous areas of the signal, the corresponding area can be quantized to a lesser extent to maintain good signal quality of such areas; areas of high contrast can be quantized to a lesser extent to maintain good signal quality of such areas; areas in the signal where a specific object, such as a person or text, is detected, can be quantized to a lesser extent to maintain good signal quality of such areas; etc. Accordingly, the embodiments discussed herein may include quantizing the correction value depending on the attributes of the signal to which the plurality of correction values relate. According to further embodiments, the set of adjustment values may be a vector including multiple reconstruction data components (eg, δ component, H component, V component, D component). When encoding, two or more of these components may be combined and represented as a single encoded symbol.

As mentioned, each of the multiple regions of the original signal may be encoded using a corresponding set of correction values (eg, δ, H, V, D). For example, the first section or area of the signal can be encoded by the first set of correction values (eg, δ ₁ , H ¹ , V ¹ , D ¹ ); the second portion or region of the signal may be encoded with a second set of correction values (eg, 52, H ² , V ² , D ² ); the third portion or region of the signal may be encoded by a third set of correction values (eg, δ ₃ , H ³ , V ³ , D ³ ); etc. The embodiments discussed herein may include encoding each of at least two of several reconstruction data sets (eg, δ, H, V, D) or portions thereof using a common symbol. Accordingly, a single symbol may be used to represent settings for multiple sets of adjustment values.

Fig. 4A illustrates a non-restrictive embodiment of reverse decomposition performed by a decoder apparatus according to the embodiments discussed herein.

- 21 040379 niya.

The backward decomposition block 410 receives five feature planes with XxY resolution: a 400-5 predicted mean plane and four transformed residual planes: a 400-1 δ plane, a 400-2 H plane, a 400-3 V plane, and a 400-4 D plane.

Block 410 processes the received planes according to the backward decomposition, producing a residual plane with a resolution of 2Xx2Y 420.

In a non-restrictive embodiment, block 410 selects the appropriate elements in the five input planes, and for each grouping of the corresponding elements, back-decomposes by reconstructing the directional components and then adding and subtracting the directional components appropriately to create a set of residuals corresponding to the grouping.

Fig. 4B illustrates another non-limiting embodiment of reverse decomposition according to the embodiments discussed herein. In particular, the diagram illustrates the operations performed by a signal processor configured as a decoder to reconstruct a set of residues 480.

The four quantized parameters Q(δ _A ) 430-1, Q(H) 430-2, Q(V) 430-3 and Q(D) 430-4 are processed respectively by dequantizer 440-1, dequantizer 440-2, dequantizer 440-3 and dequantizer 440-4. The dequantizer 440-1 creates a dequantized delta parameter 445-1, which is processed by the combiner 450 and combined with the predicted mean 430-5 to create the dequantized mean component 455-1. Dequantizer 440-2 creates a dequantized horizontal component 4552. Dequantizer 440-3 creates a dequantized vertical component 455-3. The dequantizer 440-4 creates a dequantized diagonal component 455-4.

Parameters 455-1, 455-2, 455-3, and 455-4 are processed by the reverse decomposition block 470, according to calculations corresponding to matrix M 460, to form residues 480.

In a non-limiting embodiment, the matrix M 460 corresponds to sixteen coefficients, and the remainders 480 are obtained by adding and subtracting parameters 455-1, 455-2, 455-3, 455-4, according to the mentioned coefficients. In a non-limiting embodiment, residues 480 are a set of four residues, and each of said residues is obtained by multiplying each of parameters 455-1, 455-2, 455-3, 455-4 by a matrix factor M 460 and then summing the results of said multiplications. In another non-limiting embodiment, the matrix M 460 is a 4x4 matrix and the residual vector 480 is obtained by multiplying the parameter vector 455-1, 455-2, 455-3, 455-4 by the matrix M 460.

In a non-limiting embodiment, dequantizers 440-1, 440-2, 440-3, and 440-4 each perform quantization operations according to specific quantization parameters (e.g., dequantization operations performed on parameter 430-1 are different from dequantization operations performed on parameter 430-2).

As shown in FIG. 4B, different dequantization operations can be applied to the received quantized data, such as Q( _δA ) 430-1, Q(H) 430-2, Q(V) 430-3, and Q(D) 430-4. The dequantizer 440-1 applies the first dequantization operation to reproduce the first correction value, such as the dequantized delta 445-1; the dequantizer 440-2 applies the second dequantization operation to reproduce the second correction value 455-2; the dequantizer 440-3 applies the third dequantization operation to reproduce the third correction value 455-3; the dequantizer 440-4 applies the fourth dequantization operation to reproduce the fourth correction value 455-4. Each dequantization operation may be unique with respect to each other.

Fig. 4C illustrates another non-restrictive embodiment of reverse decomposition performed by a signal processor configured as a decoder to reconstruct a set of four residues 480 according to the embodiments discussed here.

The embodiment shown in FIG. 4C performs operations that are similar to those carried out according to the embodiment illustrated in FIG. 4B, except that in this case the decoder only receives two quantized parameters Q(H) and Q(V) to reconstruct the 2x2 block of four residues 480. This is because the information carried by parameters 430-1 and 430-4 on fig. 4B is generally less important to the perception of quality, therefore, due to bandwidth limitations, the decoder shown in FIG. 4C is able to reconstruct residuals 480 based on 430-2 and 430-3 only by substituting a default value (eg, without limitation, zero) for 430-1 and 430-4.

Parameters 430-2 and 430-3 are processed by dequantizers 440-2 and 440-3 creating parameters 4552 and 455-3. Block 470 receives a predicted mean 430-5, a dequantized horizontal component 455-2, a dequantized vertical component 455-3, and a zero value 440-6, producing residuals 480 according to the calculations corresponding to matrix M 460.

Fig. 4D illustrates a non-restrictive, lossless inverse decomposition performed by a signal processor configured as a decoder to reconstruct a set of four residues 483 according to the embodiments discussed herein. An embodiment illustrates the inverse decomposition to form a block of four quantized residues 481 transformed by a lossless decomposition transformation, i.e. using quantization operations performed before the transformation, and not after the transformation.

The transformed residual R ^t ₁ , 432-1, the parameter formed by 2 bits, is received by the combiner 451 and combined with the predicted sum 432-5, the parameter formed by n+2 bits, to form the sum S of quantized residuals 457-1, the parameter formed by n +2 bits.

The converted residuals R/ 432-2, a parameter formed by n+1 bits, Rt ₃ 432-3, a parameter formed by n+1 bits, and R/ 432-4, a parameter formed by n bits, are received and processed by the redundant bit reconstructor 495 together with the sum S of quantized residues 457-1. The redundant bit reconstructor 495 uses the known redundancy of information generated by the lossless decomposition that was performed on the encoder side and creates additional bits for 4322, 432-3 and 432-4 by using the least significant bit of 457-1 as the least significant bit and as the second after the least significant bit added to 432-4 is the exclusive OR (XOR) result of the corresponding bits 457-1, 432-2 and 432-3. Thus, redundant bit reconstructor 495 creates four parameters S 457-1, H ¹ 457-2, V ¹ 457-3 and D ¹ 457-4 formed by n+2 bits and characterized in that they all have the same the least significant bit and the second after the least significant bit, as a result, the number of ones in said second after the least significant bit is an even number.

Parameters 457-1, 457-2, 457-3, and 457-4 are processed by reverse decomposition 472 according to operations corresponding to the parameters included in the matrix M 460 to form a set of four quantized residues 482, each of said quantized residues being formed by n bits.

The quantized residues 482 are then processed by dequantizer 442 to form residues 483.

The described operations performed by redundant bit reconstructor 495 for reconstructing redundant information in parameters 457-1, 457-2, 457-3 and 457-4 (which is a well-known consequence of the lossless transformation that was performed on the encoder side) are just a non-restrictive embodiment. . Those skilled in the art can suggest several other ways of extracting said redundant information on the encoder side (thus avoiding encoding and transmitting redundant information) and reconstructing it on the decoder side.

As shown, the set of adjustment values may be a vector of several adjustment components R ^t ₁ , R/, R/, R' ₄ . By applying a matrix 460 and appropriate processing to a vector of multiple correction components, the decoder processing hardware outputs a set of quantized correction values, such as residuals 482. With a dequantizer 442, the decoder processing hardware applies a dequantization algorithm to a set of quantized correction values, such as residuals 482, to create a plurality of dequantized correction values, eg, residuals 483 (ie reconstruction data). In one embodiment, the decoder processing equipment applies a plurality of the dequantized correction value set, eg residuals 483, to the pre-representation of the signal to create the secondary signal representation discussed herein.

In one embodiment, to output a plurality of dequantized correction values, the decoder processing hardware may be configured to use combiner 451 to modify a particular correction component in the vector to create a modified vector from multiple correction components (e.g., residual 457-1, residual 4322, residual 432 -3, remainder 432-4). One or more of the multiple correction components in the modified set may have different bit lengths. The decoder processing equipment applies the first algorithm to the modified vector (e.g., residual 457-1, residual 432-2, residual 4322, residual 432-4) of multiple adjustment components to properly adjust the bit lengths of the components in the modified set to a total length, such as n+ 2 as in this non-restrictive illustrative embodiment.

Fig. 5 illustrates a non-restrictive embodiment of the operations performed by a decoder according to the embodiments discussed herein. The decoder reconstructs, based on the set of four predictive elements 500-1 with LOQ #1, on the corresponding signal representation element with lower LOQ #2 and on the four parameters 445-1, 455-2, 455-3 and 455-4, four signal representation element 550-1 with LOQ #1.

Block 520 processes the four elements 500-1 and the corresponding representation element 510 with LOQ #2, producing a predictive mean (PA) 430-5.

The predicted mean of 430-5 is combined with the delta of 445-1 to create the A of 4551.

Parameters 455-1, 455-2, 455-3 and 455-4 are combined with elements 500-1-1, 500-1-2, 500-1-3 and 500-23 040379

1-4 according to the operations (addition and/or subtraction) shown in FIG. 5, creating elements 550-1-1,

550-1-2, 550-1-3 and 550-1-4. For example, element 550-1-1 is obtained by adding 455-1, 455-2, 455-3 and 455-4 to 500-1-1; element 550-1-2 is obtained by adding 455-1 and 455-3 and subtracting 455-2 and 455-4 to 500-1-2; etc.

In a non-restrictive embodiment, elements 500-1-1, ..., 500-1-4 have n bits, 455-1, ..., 455-4 have n+2 bits, and for each element 550-1- n addition and subtraction shown in FIG. 5 are first performed on parameters 455-1, ..., 455-4 (forming a result of n+2 bits), then the result is divided by 4 (i.e., the last two bits are excluded), forming a parameter of n bits, and then said parameter of n bits is added to the corresponding element 500-1-n, creating an element 5501-n.

Accordingly, the decoder processing equipment shown in FIG. 5 receives setting information or a value of y ¹ associated with a signal representation 510 at quality level #2. The decoder processing hardware receives a plurality of correction values (eg, delta 445-1, H 4552, V455-3, and D 455-4). The decoder processing equipment generates a signal preview (eg, signal 500-1) at quality level #1 based on the setting information of signal presentation elements 510 at quality level #2. The decoder processing hardware creates a second signal representation, such as a 550-1 representation at quality level #2, based on a plurality of adjustment values (eg, deltas 445-1, H 455-2, V 455-3, and D 455-4) and the pre-representation signal 510 at quality level #2. As shown, creating a second representation of a signal, such as signal 550-1, may include: modifying a set of delta 445-1, H 455-2, V 455-3, and D 455-4 correction values based at least in part on 550-1 signal preview at quality level #1. The decoder processing equipment applies the modified set of correction values A 455-1, H 455-2, V455-3, and D 455-4 to the pre-signal representation 500-1 to create the second signal representation 550-1.

According to further embodiments, the setting information, for example y ¹ , can be considered as the first setting information. Modifying a plurality of adjustment values, such as delta 445-1, H 455-2, V455-3, and D 455-4, may include: at processing block 520, receiving first information at ¹ settings. The first information at settings ¹ is related to signal preview 500-1. The processing unit 520 receives the second setting information, for example, the settings of element 500-1-1 (P11), element 500-1-2 (P ₁₂ ), element 500-1-3 (P ₂₁ ), and element 500-1-4 ( R ₂₂ ). The second setting information is related to the representation of the signal at quality level #1. A preview of the signal 500-1 is derived at least in part based on the first information y ¹ settings. The processing unit 520 applies a mathematical algorithm to the combination of the first setting information of ¹ and the second setting information to generate a modification value, for example, a value of 430-5. Function 562 applies modification value 430-5 to the adjustment value set (eg, delta 445-1) to create a modified adjustment value set that includes value A 455-1.

In one embodiment, function 562 receives a data component, eg, delta 445-1 from a set of adjustment values including delta 445-1, H 455-2, V455-3, and D 455-4. In one embodiment, one or more of the multiple components in the correction value set includes directional slope information.

Function 562 adds the modification value 430-5 and the extracted delta component 445-1 to modify the delta component back to the mean 455-1. By applying the matrix values 532 (i.e., by performing addition and subtraction operations according to the matrix values 532), the decoder processing hardware modifies the settings of each of the multiple elements in the second quality level signal pre-representation using the modified delta component 455-1. In other words, the decoder processing equipment shown in FIG. 5 uses the value 430-5 to modify each of the preliminary signal representation 500-1 to create a second signal representation 550-1.

As discussed, decoding may be performed at each of several levels up a hierarchy of several levels. Note that the second signal representation 550-1 can then be used as the basis for creating a signal preview at the next quality level in ascending order.

In addition, as discussed earlier, the representation of signal 510 at quality level #2 may be the first quality level and represents the resolution of the first image. Quality level #1 may be the resolution of the second image, which is lower than the resolution of the first image. Thus, the representation of signal 510 at one quality level includes a first plurality of pixels (eg, without limitation, one element y ¹ ). The signal representation, such as signal 500-1 and 550-1, includes a second plurality of pixels (eg, without limitation, four elements x11, x ₁₂ , x ₂₁ and x ₂₂ ).

As mentioned above, the reconstruction data, including reconstruction data 510, for example, y ¹ and correction values δ _A 445-1, H 455-2, V455-3 and D 455-4, may additionally

- 24 040379 include metadata. The decoder processing equipment shown in FIG. 5 may be configured to receive metadata associated with a plurality of adjustment values. Through the metadata processing, the decoder processing equipment identifies one or more operations to be applied to the reconstruction data to reproduce the signal representation 550-1 using the metadata. For example, in one embodiment, the decoder processing equipment applies the identified one or more operations to the set of correction values and/or the value y ¹ to derive the second representation of the signal 550-1 from the preliminary representation of the signal 500-1.

Additionally, embodiments discussed herein may include receiving metadata associated with a plurality of adjustment values. The decoder processing hardware may be configured to identify one or more of the operations indicated by the metadata. The decoder processing equipment applies one or more of the identified operations to the set of correction values to derive a second signal representation 550-1 from the preliminary signal representation 500-1.

One embodiment discussed herein involves linearly applying a modified set of correction values to a signal pre-representation. For example, applying matrix values 532 may include generating a second representation of signal 550-1 by linearly applying (eg, using the sum and difference functions shown) a modified set of adjustment values, such as mean A 455-1, H 455-2, V 455-3 and D 455-4 to signal pre-submission 500-1.

Thus, as shown with the matrix values 532, the decoder processing equipment creates a second representation of the signal 550-1 by applying a modified set of mean correction values A 455-1, H 455-2, V455-3, and D 455-4 to the pre-representation signal 500-1. The modified set of adjustment values includes a vector of weighted values (indicated by addition functions and subtraction functions) applied to signal preview 500-1 in accordance with matrix values 532.

In accordance with another illustrative embodiment, the vector corresponding to the average A 455-1, H 455-2, V455-3 and D 455-4 represents a modified set of adjustment values. The decoder processing hardware may be configured to create a secondary representation of the signal 550-1 by applying a modified set of correction values to the preliminary representation of the signal 500-1. The correction value vector may be applied by the decoder processing hardware to the pre-representation to generate a secondary representation of the signal 550-1 in accordance with the weighting factors indicated by the matrix values 532.

Matrix values can be changed and selected from a group of possible matrices containing:

p 5 4 3 21 h -1 4 3 21

-1 -2 3 21 'j 1 -1' -2-32I h -1 -2 -3 -41 (1 _2 -3 -4 -5j

As discussed previously, reconstruction data including reconstruction data 510, for example, y ¹ and correction values delta 445-1, H 455-2, V455-3, and D 455-4, may be received as encoded information.

The decoder processing equipment receiving such data may be configured to decode the encoded information into multiple symbols. The decoder processing equipment may be configured to extract a symbol from multiple symbols and, depending on the encoding, producing any one or more components in the set of delta 445-1, H 455-2, V455-3, and D 455-4 adjustment values from the extracted symbol.

In FIG. 6 is a flowchart showing a non-restrictive embodiment implementing directed decomposition in a layered hierarchical coding loop according to the embodiments discussed herein.

A signal processor configured as an encoder receives the signal at a given quality level (highest LOQ), performs downsampling operations until a lower LOQ representation of the signal is created (with said lower LOQ being lower than the highest LOQ), encodes the lower LOQ representation of the signal by a suitable encoding method, generating encoded reconstruction data, and then processes said encoded reconstruction data

- 25 040379 to simulate operations performed on the decoding side, creating a decoded representation of the signal at the lower LOQ (step 6.01 in Fig. 6).

Having created a decoded representation of the signal at the lower LOQ, the encoder enters a processing loop that repeats for each successive LOQ in ascending order, processing the LOQ hierarchy level by level up to the highest LOQ.

In particular, the encoder processes the decoded representation of the signal at a given LOQ (lower LOQ) and creates a predicted representation of the signal at the next LOQ in ascending order. The encoder then generates residual data, said residual data corresponding to the difference between the original signal representation at the next ascending LOQ and the predicted signal representation at the next ascending LOQ (step 6.02).

The encoder then separates the residual data into groups of residuals (eg, without limitation, adjacent 2x2 groupings of residuals) and for each grouping performs a directional decomposition transformation, producing directional components (step 6.03).

The encoder then creates four separate directional component planes (middle, horizontal, vertical, diagonal), with the corresponding elements of said four planes corresponding to the same residual grouping (step 6.04).

The encoder then creates a predicted component means plane, where each of the elements of said plane is the predicted mean for the respective residual grouping, calculated based on the difference between the mean of the respective predictive elements in the next ascending LOQ and the corresponding element in the lower LOQ. The encoder creates a delta component plane, with each element of the plane calculated as the difference between the corresponding mean component and the corresponding predicted mean component. The delta plane is replaced by the mean plane, resulting in a set of four transformed directional delta components, H, V and D (step 6.05).

The encoder then quantizes the planes of the transformed directional components, producing planes of quantized transformed directional components (step 6.06).

The quantized transformed directional components are then encoded to form encoded reconstruction data (step 6.08).

Then, to move to the next level in the hierarchy, the encoder simulates the decoding operations that would be performed on the decoding side, producing a decoded representation of the signal at the next LOQ in ascending order. To this end, the encoder dequantizes the planes of the quantized transformed directional components; combines the plane of dequantized deltas with the plane of predicted means, reconstructing the plane of dequantized mean components; calculates the residual data by performing, for each grouping of the four residuals, a reverse decomposition of the respective directional components A, H, V, and D; combines the residual data with the predicted signal representation at the next ascending LOQ, producing a decoded signal representation at the next ascending LOQ (step 6.07).

Here, if the next LOQ in ascending order is not the highest LOQ, the encoder proceeds to the next level in the hierarchy, returning to step 6.02 where, for the next level in the hierarchy, the representation of the signal at the next LOQ in ascending order that has just been decoded , becomes the decoded representation of the signal at the lower LOQ.

In other non-limiting embodiments, the quantization operations applied to each plane of the transformed directional components are based on parameters that are specific to each class of transformed directional components (e.g., without limitation, the quantization level applied to the diagonal components is stronger than the quantization level applied to horizontal and/or vertical components).

In other non-limiting embodiments, the encoder, instead of creating and encoding four separate planes of transformed components, creates and encodes groupings of four transformed components, each of said groupings corresponding to a grouping of four residues in the plane of residues.

In other non-limiting embodiments, the encoder performs quantization operations in step 6.02, creating quantized residuals. In such embodiments, the decomposition and inverse decomposition operations are lossless operations, i. no additional quantization is applied to the directional components and/or the transformed directional components.

In other non-limiting embodiments, the lower LOQ representation of the signal is encoded using an MPEG-based method; thus, conventional decoders that are not configured to process and reconstruct higher LOQs in the hierarchy are at least capable of decoding lower LOQ signal representations.

In FIG. 7 is a flowchart showing a non-restrictive embodiment implementing reverse decomposition in a layered hierarchical decoding loop according to the embodiments discussed herein.

- 26 040379

The signal processor, configured as a decoder, receives and decodes encoded data corresponding to the lower quality level, producing a decoded representation of the signal at said lower LOQ (step 7.01).

Having created a decoded representation of the signal at the lower LOQ, the decoder enters a decoding loop that repeats for each successive LOQ in ascending order, reconstructing the LOQ hierarchy level by level up to the highest LOQ.

In particular, the decoder processes the decoded representation of the signal at a given LOQ (lower LOQ), decodes reconstruction data corresponding to the next LOQ in ascending order, and creates four quantized parameter planes. The decoder then performs suitable dequantization operations on said quantized parameter planes, producing dequantized transformed parameters (step 7.02).

The decoder then creates a predictive representation of the signal at the next LOQ in ascending order, by using upsampling operations and/or other signal processing operations indicated by the received parameters (step 7.03).

For each grouping of four dequantized transformed parameters (delta, H, V, D) corresponding to a grouping of four residuals, the decoder creates a predictive mean parameter calculated based on the difference between the corresponding signal bin at the lower LOQ and the average of the four corresponding bins of the predicted signal representation on the next LOQ in ascending order. for each grouping, the predicted mean is combined with the dequantized delta component to create the dequantized mean component (step 7.04).

Then, for each grouping, the decoder inverses the corresponding dequantized components of the mean, H, V, and D, producing four residuals. In particular, each of the residuals is obtained by performing a linear combination (i.e., sums and/or differences, with appropriate coefficients) of the dequantized components of the mean, H, V, and D. Thus, the decoder creates a residual plane at the next in ascending order LOQ ( step 7.05).

The decoder then combines the predicted signal representation at the next ascending LOQ with the generated residuals, reconstructing the decoded signal representation at the next ascending LOQ (step 7.06).

Here, if the next LOQ in ascending order is not the highest LOQ, the decoder proceeds to decode the next level in the hierarchy, returning to step 7.02, in which, for the next LOQ in the hierarchy, the signal representation at the next LOQ in ascending order, which has just was decoded becomes the decoded representation of the signal at the lower LOQ.

In other non-limiting embodiments, the dequantization operations are performed after the inverse transformation, dequantizing the residuals instead of dequantizing the transformed parameters.

In FIG. 8 is a block diagram of a computer system 800 that provides computer processing according to the embodiments discussed herein.

Computer system 800 may be, or at least include, a computerized device, such as a personal computer, a processing circuit, a television, a playback device, an encoder, a workstation, a portable computing device, a mobile phone, a tablet, a console, a set-top box , network terminal, processing device, network device acting as a switch, router, server, client, etc.

Note that the following discussion provides a basic embodiment indicating how to implement the functionality associated with the previously discussed signal processor. However, it should be noted that the actual configuration for carrying out the operations described here may vary depending on the respective application.

As shown, the computer system 800 of the present example includes a communication bus 811 that allows communication with computer-readable storage media 812, such as non-transitive computer-readable storage media, etc., in which digital information can be stored and retrieved.

Computer system 800 may further include a processor 813, an I/O interface 814, and a communication interface 817.

In one embodiment, I/O interface 814 provides connectivity to storage 880 and, if present, to an on-screen display, speakers, peripherals 816 such as a keyboard, computer mouse, and so on.

As briefly mentioned above, computer-readable storage media 812 can be any suitable device and/or equipment, such as memory, optical storage, solid state storage, hard drive, floppy disk, and so on. In one embodiment, storage media 812 are non-transient storage media (ie, non-carrier wave media) configured to store instructions associated with signal processor application 840-1. Instructions are executed by the appropriate resource, e.g.

- 27 040379 processor 813, to perform any of the operations discussed here.

Communication interface 817 allows computer system 800 to communicate over network 890 to retrieve information from remote sources and communicate with other computers, switches, clients, servers, and so on. I/O interface 814 also allows processor 813 to retrieve or attempt to retrieve stored information from store 880.

As shown, computer-readable storage media 812 can be encoded by signal processor application 840-1 executed by processor 813 as process 840-2.

Note that computer system 800 can be implemented to include computer-readable storage media 812 for storing data and/or logical instructions.

Code associated with the signal processor application 840-1 may implement the functionality of the signal processors illustrated here and/or other resources necessary to implement the invention discussed here.

In one embodiment, processor 813 accesses computer-readable storage media 812 via communication bus 811 to execute, execute, execute, interpret, or otherwise implement signal processor application 840-1 instructions stored in computer-readable storage media 812. Execution of signal processor application 840-1 creates processing functionality on processor 813. In other words, signal processor process 840-2 associated with processor 813 represents one or more aspects of the execution of signal processor application 840-1 at or on processor 813 in a computer system. 800.

Those skilled in the art will appreciate that computer system 800 may include other processes and/or software and hardware components, such as an operating system or other software, that controls the allocation and use of hardware processing resources to execute signal processor application 840-1. .

In accordance with various embodiments, note that a computer system may be any of various types of devices, including, but not limited to, a personal computer system, a desktop computer, a laptop computer, a laptop computer, a netbook computer, a mainframe computer system, a pocket computer. , workstation, network computer, application server, mass storage device, consumer electronic device such as a camera, handheld camera, set-top box, mobile device, smartphone, tablet, video game console, handheld video game device, peripheral device such as a switch, modem, router, or, more generally, any type of computing or electronic device.

The functionality supported by the various resources will be illustrated below in the flowcharts shown in FIG. 10-11. Note that the steps depicted in the flowcharts described below may be performed in any suitable order.

In FIG. 10 is a flowchart 1000 illustrating an exemplary method according to the non-limiting embodiments described herein. Note that this is somewhat in line with the principles discussed above.

In processing block 1010, the computer processor hardware receives a first set of adjustment values, the first set of adjustment values indicating adjustments to be made to the predicted signal representation generated at the first quality level in order to reconstruct the signal representation at the first quality level.

In processing block 1020, the computer processor hardware processes the first set of adjustment values.

In processing block 1030, the computer processor hardware outputs a second set of correction values based on the first set of correction values and based on the representation of the signal at the second quality level. The second quality level is lower than the first quality level.

In FIG. 11 is a flowchart 1100 illustrating an exemplary method according to the non-limiting embodiments described herein. Note that this is somewhat in line with the principles discussed above.

In processing block 1110, the computer processor hardware receives setting information related to representing the signal at the first quality level.

In processing block 1120, the computer processor hardware receives a plurality of adjustment values.

In processing block 1130, the computer processor hardware creates a second quality level signal preview based on the setting information. The second quality level is higher than the first quality level.

In processing block 1140, the computer processor hardware creates a secondary representation of the signal at the second quality level based on the plurality of adjustment values and the preliminary representation of the signal at the second quality level.

- 28 040379

Note, again, that the methods disclosed here are well suited for encoding and decoding in hierarchical systems. However, it should be noted that the embodiments discussed here are not limited to use in such applications, and that the methods discussed here are well suited for other applications.

Based on the description set forth here, numerous specific details are set forth in order to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will appreciate that the claimed subject matter may be practiced without these specific details. In other cases, methods, devices, systems, etc. known to those skilled in the art are not described in detail so as not to obscure the claimed subject matter. Some portions of the detailed description have been presented in relation to algorithms or symbolic representations of operations on bits of data or binary digital signals stored in the memory of a computing system, such as computer memory. These algorithmic descriptions or representations are examples of methods used by data scientists to convey the subject matter of their work to other technicians. The algorithm described here is generally considered to be a self-consistent sequence of operations or similar processing leading to the desired result. In this context, operations or processing involve the physical manipulation of physical quantities. Typically, though not necessarily, such quantities may take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, or otherwise manipulated. It is sometimes convenient, mainly for reasons of common use, to consider such signals as bits, data, values, settings, elements, characters, characters, terms, numbers, digits, and the like. However, it should be understood that all of these and similar terms are to be associated with the proper physical quantities and are merely convenient labels.

Unless specifically stated to the contrary, as follows from the following discussion, it is obvious that throughout this description of the invention the use of terms such as processing, calculation, calculation, creation, definition, and the like. refers to the activities or processes of a computing platform, such as a computer or similar electronic computing device, that manipulates data or transforms data represented as physical electronic or magnetic quantities in memory units, registers or other information storage devices, transmission devices or display devices of the computing platform .

While this invention has been specifically shown and described with reference to its preferred embodiments, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the spirit and scope of the present application as defined by the following claims. Such changes are to be included within the scope of this application. Therefore, the above description of the embodiments of the present application is not to be limited. On the contrary, any limitations of the invention are set forth in the following claims.

Claims (16)

CLAIM 1. A method for encoding a video signal, comprising, by means of computer processing equipment, receiving (6.01) a video signal plane with a first resolution, the plane containing a pixel plane; downsampling (6.01) the video plane to generate a representation of the plane with a second resolution, the second resolution being lower than the first resolution; generating (6.01, 6.02) a predicted first resolution signal plane by obtaining a second resolution coded signal plane and upsampling the simulated decoding of the second resolution coded signal plane; creating (6.02) residual data as a difference between the received video plane with the first resolution and the predicted video plane with the first resolution; processing (6.03, 6.04) of the residual data using directional decomposition to generate sets of directional components (320), with directional decomposition applied to 2x2 blocks of residuals to generate, for each 2x2 block, four directional components: A - the average value of the block of residuals, H - horizontal difference within the block of residuals, V is the vertical difference within the block of residuals and D - diagonal difference within the block of residuals; deriving (6.05) the sets of transformed residuals (350) from the sets of directional components (320) by adjusting the average directional component of each set of directional components (320), including determining the predicted mean value for each 2x2 block as the difference between the quantized pixel value at the second resolution , representing the block, and the average - 29 040379 of the corresponding block in the predicted signal plane, and subtracting the predicted mean from the mean directional component for the block of residuals; quantization (6.06) of sets of transformed residues (350); and encoding (6.08) the quantized transformed residual sets to output encoded reconstruction data for transmission to the decoder processing equipment for decoding. 2. The method of claim 1, further comprising transmitting encoded reconstruction data to the decoder processing equipment. 3. The method according to claim 1, in which each set of transformed residues refers to the reconstruction of a specific position in the plane of the video signal with a first resolution; and in which the sets of transformed residuals are quantized depending on the attributes of the image at particular positions. 4. The method of claim 1, wherein the sets of transformed residuals are quantized depending on the image attributes of portions of the video signal frame that correspond to the sets of transformed residuals, wherein one or more of uniform image regions, sharp contrast image regions, and object detected image regions are quantized. less. 5. The method of claim 1, comprising encoding at least two directional components using one symbol. 6. A method for decoding a video signal, comprising, by means of computer processing equipment, receiving (7.01) decoded pixel values associated with a video signal plane with a first resolution; receiving (7.02) encoded reconstruction data for the video signal; decoding (7.02) the encoded reconstruction data to obtain quantized sets of transformed residuals; dequantization (7.02) of the quantized sets of transformed residues to obtain sets of transformed residues; creating (7.03) a second resolution preview of the video plane by upsampling said pixel values, the second resolution being higher than the first resolution; generating (7.04, 7.05) a residual plane from the transformed residual sets by performing a backwards decomposition, where each transformed residual set contains four directional components corresponding to corresponding 2x2 blocks of the residual plane, the directional components containing δ - the adjusted mean of the residual block, H - horizontal difference within the block of residuals, V is the vertical difference within the block of residuals and D is the diagonal difference within a block of residuals, and generating (7.06) a second resolution video plane secondary representation by combining the residual plane and a second resolution video plane pre-representation, the method further comprising generating (7.05) a predicted mean parameter for each 2x2 block , derived from the set of transformed residuals, wherein the predicted mean is calculated as the difference between the corresponding pixel value for the first resolution block and the mean of the corresponding block in the second resolution video plane preview; and performing combining by adding the predicted averages with the corresponding adjusted average directional components at the time of creating the video plane secondary representation to recover the average directional components for the residual plane. 7. The method of claim 6, wherein performing combining the predicted means includes summing each predicted mean and the corresponding adjusted mean directional component to modify the adjusted mean directional component. 8. The method of claim 7, further comprising modifying the settings of each of the plurality of pixel elements in the second resolution video plane preview based on the modified directional component. 9. The method of claim 6, wherein the second resolution video plane preview is a predicted second resolution video plane representation; and wherein the secondary video plane representation at a second resolution is used as the basis for generating a preliminary video plane representation at a third resolution. - 30 040379 resolution, the third resolution being higher than the second resolution. 10. The method of claim 6, further comprising receiving metadata associated with the encoded reconstruction data; identifying an upsampling operation that is specified by the metadata; and applying an upsampling operation to derive a preview of the second resolution video plane. 11. The method of claim 6, further comprising generating a secondary representation of the video plane with a second resolution by linearly applying the residual plane to the preliminary representation of the video plane. 12. The method of claim 6, wherein the directional components are represented as a vector of values, and each block of residuals is extracted from the directional components using weights that are defined by a matrix. 13. The method of claim 6, further comprising receiving encoded reconstruction data; decoding the encoded reconstruction data into a plurality of symbols; extracting a symbol from the plurality of symbols; and generating quantized transformed residuals from the extracted symbol. 14. The method of claim 6, further comprising applying a first dequantization operation to reproduce the first converted residue; and applying a second dequantization operation to reproduce the second transformed residue, wherein the first dequantization operation is different from the second dequantization operation. 15. A video encoder/decoder comprising a processor and a memory in which program instructions are placed for the processor to implement the method of any one of claims 114. 16. A computer-readable storage medium containing instructions for the processor of the video encoder/decoder to perform the operations of the method according to any one of claims 1-14.