CN101710987B - Configuration method of layered B forecasting structure with high compression performance - Google Patents

Abstract

The invention discloses a configuration method of a layered B forecasting structure with high compression performance, which comprises the following steps: configuring the layered B forecasting structure by using the recursive dichotomy; dividing an image group into two sub-image groups by the image B of the first layer; and then, respectively dividing each sub-image group of which the length is greater than 1 into two smaller sub-image groups by the image B of the lower layer, and carrying out recursion until the whole dividing process is finished, wherein the image B for each dividing process is selected through a simple expression. The invention is suitable for any image group length, and the configuring difficulty and the dividing regularity of the configuration method of the invention are equivalent to that of of the traditional configuration method. However, the configuration method in the invention obtains an optimal forecasting structure through dynamic programming search on the basis of mathematical modeling on the compression performance of the layered B forecasting structure; and the experimental result indicates that under various image group lengths, the configuration method of the invention has better average coding rate distortion performance.

Description

A Hierarchical B Prediction Structure Allocation Method with High Compression Performance

technical field

The invention relates to the field of video coding, in particular to a method for configuring a layered B prediction structure with high compression performance.

Background technique

Compared with analog video, digital video has many advantages such as high quality, easy processing, easy correction, large capacity, and many programs. With the continuous improvement of people's demand for video quality and benefiting from the advancement of technology, digital video has been more and more widely used today.

However, the amount of information contained in digital video is very large, which requires high storage capacity and bandwidth of transmission network. In order to make digital video more effective, compression coding is the first problem to be solved. For this reason, academia and industry have carried out extensive and in-depth research in the field of video compression coding technology. At present, the main methods of video compression coding are waveform-based coding and content-based coding. Since the 1980s, the International Standards Organization (ISO) and the International Telecommunication Union Telecommunication Standardization Division (ITU-T) have successively introduced a series of international standards for digital video compression coding, which has greatly promoted the development of video communication and digital TV broadcasting. Most of these standards adopt a hybrid coding framework mainly based on predictive coding, transform coding and entropy coding, and belong to waveform-based coding methods.

In March 2003, ITU-T/ISO officially announced the video coding standard H.264/AVC. H.264/AVC not only significantly improves the compression performance, but also has good network affinity, and is known as a new generation of video coding standards. Compared with previous video coding standards, the H.264/AVC standard also adopts a hybrid coding framework, but adds many features. Among these features, the flexibility of the image level is also one of the reasons why H.264/AVC achieves efficient performance.

The digital video signal is formed successively by successive images distributed in discrete time. Since there is a certain correlation between the scenes in the adjacent images, the encoding method of inter-frame prediction can be used. The prediction method is a simple and practical compression coding method. Inter-frame prediction divides the currently encoded image into several blocks or macroblocks, and tries to find out the corresponding position of each block or macroblock in the adjacent encoded image to obtain the predicted value. After encoding, only the difference between the predicted value and the actual value needs to be transmitted to achieve the effect of compression. A picture that does not use inter-frame predictive coding is called an I picture, a picture that uses only one reference picture for coding is called a P picture, and a picture that uses multiple reference pictures for coding is called a B picture. When using the H.264/AVC standard for encoding, it is very different from previous video encoding standards in that the encoding order and display order of images can be configured arbitrarily, and any image including bidirectionally referenced B pictures can also be used as a reference . H.264/AVC also provides a powerful reference image management function, which can provide prediction accuracy through multi-reference frame prediction in motion estimation. These flexible features allow arbitrary selection of prediction structures during video coding, including the layered B prediction structure with excellent compression performance.

The hierarchical B-prediction structure adopts several levels of B-pictures in a group of pictures, and these B-pictures adopt bidirectional prediction, and the high-level B-pictures can be used as references for low-level B-pictures. The first image of the video sequence is coded with an I picture, followed by key images at equal intervals. The key image can be coded by I-picture or P-picture, and generally I-picture is used for coding. For the key image, the display sequence of the image whose coding sequence is earlier must also be earlier. Images located between these key images are encoded in a hierarchical B-picture manner. Each B-picture uses the most recent picture whose forward and backward directions are higher than its own level as a reference picture. If the key image is regarded as the highest level, the B picture of the first level can only use the two key pictures before and after as reference pictures, and the B picture of the lowest level does not need to be used as a reference. A key picture and all B-pictures before it and after the previous key picture form a group of pictures (GOP). A GOP of length L has L-1 B-pictures.

Compared with other prediction structures in the time direction, the hierarchical B structure can significantly improve the compression performance. Hierarchical B prediction structure is not only very important in traditional video applications, but also plays an important role in some cutting-edge video applications and research fields, such as scalable video coding and multi-view video coding. Scalable video coding is aimed at the growing Internet business, the types and contents of video services required by different users are not the same, so the video coding technology needs to be able to encode with different bit rates and video under the condition of only one coding quality to meet different application requirements. Each picture level in the hierarchical B prediction structure provides precisely temporally different resolutions. Multi-viewpoint video coding is a key technology in the research field of 3D stereoscopic video technology. The multi-viewpoint video signal is a group of camera arrays that shoot the scene from different viewpoints at the same time, and use one or more viewpoint information to synthesize virtual viewpoint information to achieve the purpose of providing stereoscopic perception and free switching of any viewpoint. Massive data of multi-viewpoint video signals put forward higher demands on compression technology. Due to the superior compression performance of the layered B structure, it is used as the prediction structure in the time direction in the multi-view video coding standard.

首先将所有的B图标注为最低层级；而后将距离该GOP之前的关键图像的间隔为2¹的正整数倍的B图作为倒数第二层级；如此类推，将距离该GOP之前的关键图像的间隔为2^lv-2的正整数倍的B图作为第二层级；最后将距离该GOP之前的关键图像的间隔为2^lv-1的正整数倍的B图作为第一层级。以上两种分层B预测结构的配置方法，均为二分法，即首先用一个B图将GOP划分为两个部分，该B图采用GOP首尾端的图像作为参考；而后又将这两个部分分别用一个B图划分为两个部分，分别采用各自部分首尾端的图像作为参考；如此往下进行直至无可划分。Scholars at HHI first proposed the hierarchical B prediction structure, and analyzed its compression performance, encoding delay, and storage capacity requirements. At present, there are already two preset hierarchical B prediction structures supported by the reference software JM of H.264/AVC. The method used is to set the parameter HierarchicalCoding to 1 and 2 respectively in the configuration file of JM. The configuration method of the hierarchical B prediction structure with HierarchicalCoding equal to 1 is to classify the B pictures in the GOP according to the parity of the display order. B-pictures whose distance from the key picture before the GOP is an odd number are not used as references, and the two pictures immediately before and after it are used as references; B pictures whose distance from the key picture before the GOP is an even-numbered picture are used as references. Reference, and the hierarchy descends from front to back in the order of display. The configuration method of the hierarchical B prediction structure with HierarchicalCoding equal to 2 is, if the length of the GOP is L, divide the number of layers of the B picture

First, mark all the B-pictures as the lowest level; then take the B-picture whose distance from the key image before the GOP is a positive integer multiple of 2 ¹ as the second-to-last level; and so on, the distance from the key picture before the GOP The B-picture whose interval is a positive integer multiple of 2 ^lv-2 is used as the second level; finally, the B-picture whose interval is a positive integer multiple of 2 ^lv-1 from the key image before the GOP is used as the first level. The configuration methods of the above two hierarchical B prediction structures are both dichotomous methods, that is, first divide the GOP into two parts with a B picture, and the B picture uses the images at the beginning and end of the GOP as a reference; Divide a picture B into two parts, and use the images at the beginning and end of each part as a reference; proceed until there is no division.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art and provide a high compression performance layered B prediction structure configuration method.

High compression performance hierarchical B prediction structure configuration method is:

A recursive dichotomy is used to configure the hierarchical B prediction structure, that is, a B picture is first used to divide the image group into two sub-image groups. Divide a B picture into two smaller sub-image groups, respectively use the images at the beginning and end of each part as reference, and proceed until there is no division;

代表向下取整，若L≥2ⁿ×3，则m＝n+1，否则m＝n，该B图到首尾图像中另一个图像的距离为D₂＝L-D₁。If the length of the image group or sub-image group is L, and L>1, then the image group or sub-image group is divided into two smaller sub-image groups, and the distance between the picture B of the image group or sub-image group is one of the first and last images The distance should be an integer power of 2 D ₁ =2 ^m , m is a positive integer and the value method is: let

Represents rounding down, if L≥2 ⁿ ×3, then m=n+1, otherwise m=n, the distance between the picture B and the other image in the first and last images is D ₂ =LD ₁ .

Compared with the two existing configuration methods, the configuration method of the hierarchical B prediction structure with high compression performance proposed by the present invention adopts the dichotomy method for recursive division, and the difficulty of configuration and the regularity of division are comparable. However, the configuration method in the present invention is based on the mathematical modeling of the compression performance of the hierarchical B prediction structure, and the optimal prediction structure obtained through dynamic programming search. Experimental results show that the present invention's The profiled approach has better average rate-distortion performance.

Description of drawings

Figure 1 is a schematic diagram of a typical hierarchical B prediction structure (the length of the image group is 8, with 4 image levels);

Figure 2(a) is the comparison between the actual normalized code rate NR value of the news sequence and the value estimated by the linear model;

Figure 2(b) is the comparison between the actual normalized code rate NR value of the basket sequence and the value estimated by the linear model;

Fig. 3 is a step-by-step schematic diagram of a hierarchical B prediction structure mapped to a binary tree with an image group length of 8;

Fig. 4 is a schematic diagram of another hierarchical B prediction structure mapped to a binary tree with a group of pictures length of 8;

Fig. 5 is the recursive structure of the optimum binary tree of 8 for image group length;

FIG. 6 is a schematic diagram of an optimal prediction structure configuration with an image group length of 10.

Detailed ways

High compression performance hierarchical B prediction structure configuration method is:

A recursive dichotomy is used to configure the hierarchical B prediction structure, that is, a B picture is first used to divide the image group into two sub-image groups. Divide a B picture into two smaller sub-image groups, respectively use the images at the beginning and end of each part as reference, and proceed until there is no division;

代表向下取整，若L≥2ⁿ×3，则m＝n+1，否则m＝n，该B图到首尾图像中另一个图像的距离为D₂＝L-D₁。If the length of the image group or sub-image group is L, and L>1, then the image group or sub-image group is divided into two smaller sub-image groups, and the distance between the picture B of the image group or sub-image group is one of the first and last images The distance should be an integer power of 2 D ₁ =2 ^m , m is a positive integer and the value method is: let

Represents rounding down, if L≥2 ⁿ ×3, then m=n+1, otherwise m=n, the distance between the picture B and the other image in the first and last images is D ₂ =LD ₁ .

To analyze and compare the performance of different hierarchical B prediction structures, and to seek a prediction structure with high compression performance, it is an effective way to conduct mathematical modeling on it first. The impact of video coding prediction structure on compression performance mainly depends on the exploration and utilization of the correlation between adjacent images in inter-frame prediction. Generally speaking, when there is no sudden change of scene or periodic repetition of scenes in the active video image sequence, the correlation between adjacent images is related to its time interval. The closer the interval, the higher the correlation. The better the performance too. Therefore, expressing the compression rate of an inter-frame predictive coded image as a function of the reference interval can guarantee its validity in a statistical sense. For a periodically repeated prediction structure in the form of a group of pictures (GOP), its overall compression performance can be expressed as the sum of the code rates of all pictures in a group of pictures.

The present invention adopts following two-parameter model to represent the compression rate of the B picture that contains two reference images:

R _B ＝R _I ×(θ ₁ +θ ₂ log(D ₁ ·D ₂ )) (1)

D ₁ and D ₂ represent the two reference intervals of Figure B, and θ ₁ and θ ₂ are parameters to be estimated. R _B and R _I are the output code rates obtained by performing B-picture coding and I-picture coding respectively on the same picture under the same coding conditions. After using the H.264/AVC encoder to obtain data under different reference intervals, the parameters are estimated by the least square method. The experimental results show that the model is very consistent with the change law of the compressed code rate of the B picture, as shown in Figure 2. The solid line marked NR in the figure represents the actual data of the normalized B image compression rate R _B /R _I , and the dotted line marked LSE represents the value generated by the linear model obtained by least squares estimation. The selected sequences are news with a resolution of 352x288 and basket with a resolution of 720x576. The abscissa represents various combinations of the two reference intervals D ₁ and D ₂ when the GOP length L=2, 3...8.

For a hierarchical B prediction structure, its overall compression performance can be expressed as the sum of the code rates of all B pictures in a group of pictures (GOP) plus the code rate of the I picture. The overall compression performance of the hierarchical B prediction structure with GOP length L is as follows:

$R_{\mathrm{GOP}}=R_I\&times;(1+{\mathrm{\&theta;}}_1(L-1)+{\mathrm{\&theta;}}_2\log (\underset{i=1}{\overset{L-1}{\mathrm{\&Pi;}}}{\mathrm{<figure-callout\; id="1"\; label="D.\; i"\; filenames="H2009101558832E00011.png,H2009101558832E00023.png"\; state="\{\{state\}\}">D.</figure-callout>}}_i\&Center\; Dot;{\mathrm{D.}}_{i2}))$ (2)

In the formula, D _i1 and D _i2 represent the reference interval of the i-th (i=1...L-1) B-picture respectively.

If you only focus on the prediction structure itself, and compare the performance of different hierarchical B prediction structures under the same GOP length, you can omit the parameters in (2), and only keep the last item, that is, all B The product of graph reference intervals, as shown in the following formula:

$R_{\mathrm{GOP}}=\underset{i=1}{\overset{L-1}{\mathrm{\&Pi;}}}{\mathrm{<figure-callout\; id="1"\; label="D.\; i"\; filenames="H2009101558832E00011.png,H2009101558832E00023.png"\; state="\{\{state\}\}">D.</figure-callout>}}_i\&Center\; Dot;{\mathrm{D.}}_{i}2$ (3)

Using formula (3), it is very convenient to analyze and compare the compression performance of any hierarchical B prediction structure. And using it as a cost, a dynamic programming method can be used to find the optimal prediction structure under any GOP length.

Since the function of the B-picture compression rate is monotonously increasing, adopting the dichotomy method to configure the hierarchical B prediction structure has better compression performance. That is, first divide the GOP into two parts with a B picture, and the B picture uses the images at the beginning and end of the GOP as a reference; For reference; and so on until no divisions can be made. If the dichotomy method is not used, it is assumed that in a certain division, the part is divided into three parts with two B pictures, and the two B pictures use the images at the beginning and end of the part as references. Then it can be transformed into a bisection, so that one of the B-pictures has a shorter reference interval in a smaller part, and better compression performance is obtained. So the hierarchical B prediction structure for the best compression performance only needs to be found in the dichotomous configuration structure.

The hierarchical B prediction structure configured by dichotomy can be mapped as a binary tree with one-to-one correspondence. The mapping method is to use the length of the GOP as the root node of the tree. After a B-graph divides a GOP into two parts, the length of each part is represented by two child nodes of the root. Continue to divide in this way, and continue to use the child nodes of the child nodes to represent the length of each part. Fig. 3 shows a step-by-step schematic diagram of mapping a hierarchical B prediction structure with a GOP length of 8 into a binary tree. Fig. 4 shows a schematic diagram of mapping another hierarchical B prediction structure with a GOP length of 8 into a binary tree.

种分法中找出最好的一个，即得到GOP长度为L的最优二叉树以及最优预测结构。因为对数相加等于相乘再取对数，在计算代价时可直接使用乘法。令C_i为GOP长度为i时最优树的代价，对于GOP长度为L的最优树具体的推算方法如下：On the binary tree, the value of each node is not only the length of each subpart, but also the reference interval of each B-graph. Using formula (3), the compression performance of the hierarchical B prediction structure can be expressed by multiplying the values of all nodes except the root on the binary tree and then taking the logarithm. On this basis, using the method of dynamic programming, the optimal binary tree corresponding to the optimal prediction structure under each GOP length can be easily found out. Any optimal binary tree must be composed of recursive optimal binary subtrees. For example, GOP length L=8, corresponding to the root of the binary tree is 8, there are 4 possible child nodes under it: (1, 7), (2, 6), (3, 5), (4, 4), as shown in the figure 5. For each possibility, an optimal subtree rooted at a child node is formed. Therefore, when the optimal structures with GOP lengths from 2 to L-1 have been obtained, the optimal structure with GOP length L can be deduced conveniently. Adding the cost of one or two optimal subtrees corresponding to the child node of the root (when the child node is 1, there is no subtree) to the cost of the pair of child nodes, the result of this division can be obtained total cost. from here

Find the best one in the classification method, that is, get the optimal binary tree with GOP length L and the optimal prediction structure. Since adding logarithms is equal to multiplying and then taking the logarithms, multiplication can be used directly when calculating the cost. Let C _i be the cost of the optimal tree when the GOP length is i. The specific calculation method for the optimal tree with the GOP length L is as follows:

(1) Start recursively from the GOP length of 2, at this time there is only one B picture, and only one prediction structure, C ₂ =1·1;

(2) When C _i (i=2...L-1) has been obtained, C _L can be obtained by selecting the minimum value from the following values: 1·(L-1)·C _L-1 , 2·(L- 2)·C ₂ ·C _L-2 ，...，(L/2)·(L/2)·C _L/2 ·C _L/2 (when L is an even number) or ((L-1)/ 2)·((L+1)/2)·C _(L-1)/2 ·C _(L+1)/2 (when L is an odd number).

Using this dynamic programming method, the optimal prediction structure with a GOP length of 2 to 16 is obtained, and the roots of the left and right subtrees after the first level division (ie the length of the left and right parts) are listed in the following table:

Table 1 Optimal tree partitioning with GOP lengths from 2 to 16

The lengths of the left and right subtrees in this table are interchangeable. Since the recursive dichotomy is used to configure the hierarchical B prediction structure, the optimal structure with any length of GOP ranging from 2 to 16 can be configured by querying the table. For example, if the length of the GOP is 10, firstly, by querying the table, we can know that the first level is divided into 4 and 6, and then query how to divide the GOP with the length of 4 and 6, so that a complete prediction structure can be configured by recursively going down. As shown in Figure 6. In some specific GOP lengths, the prediction structure found through the optimal tree is the same as the existing prediction structure HierarchicalCoding=2 (for example, when the GOP length is an integer power of 2), but the division criteria are different .

Continuing the process of dynamic programming can introduce the optimal hierarchical B prediction structure with larger GOP length. By summarizing the division rules of the optimal hierarchical B forecasting structure derived from the above models, a method for configuring the hierarchical B forecasting structure with high compression performance is obtained.

The configuration method of the present invention can be realized by adding C language code in H.264/AVC as an additional HierarchicalCoding option; it can also be realized by setting HierarchicalCoding to 3 and manually configuring the level of B-picture in the ExplicitHierarchyFormat parameter. In terms of the difficulty of configuration and the regularity of division, this configuration method is also equivalent to the existing configuration method. Use H.264/AVC reference software version JM11.0 to encode, and compare with two kinds of existing hierarchical B prediction structures that HierarchicalCoding is set to 1 and 2, the experimental results show that the configuration method of the present invention is better than each GOP length Existing prediction structures have better average rate-distortion performance. Realize result comparison

Table 2 Comparison of experimental results between the new configuration method and the original configuration method

The above table lists 8 sequences with different resolutions, and the experimental results when the GOP length is 7, 11, and 15. In the table, HC1 represents the hierarchical B prediction structure with HierarchicalCoding set to 1, and HC2 represents the hierarchical B prediction structure with HierarchicalCoding set to 2. When the GOP length is 7, the configuration method of the present invention is similar to the prediction structure of HC1, but is quite different from the prediction structure of HC2, and has an average gain of 0.1dB; when the GOP length is 11 and 15, the configuration method of the present invention It is similar to the prediction structure of HC2, but there is still gain on average. Compared with the prediction structure of HC1, the difference is getting bigger and bigger. When the GOP length is 15, there is a gain of 0.1-0.4dB. Generally speaking, the configuration method of the present invention has better compression performance.

Claims (1)

1. configuration method of layered B forecasting structure with high compression performance is characterized in that comprising:

The optimum prediction structure is to carry out on the basis of mathematical modeling at the compression performance to layered B forecasting structures, obtains through the Dynamic Programming search;

Adopt the dichotomy of recurrence that layered B forecasting structures is configured, promptly at first image sets is divided into two number of sub images groups with a B figure, the image of these B figure employing image sets two ends as a reference, then again this two number of sub images group is divided into two littler subimage groups with a B figure respectively, the image that adopts part two ends separately respectively so down carries out as a reference until not dividing;

If image sets or subimage group length are L, L＞1 then is divided into this image sets or subimage group one of them distance of this image sets of B map distance of two littler subimage groups or subimage group head and the tail image and should be 2 integral number power D ₁=2 ^m, m is that positive integer and obtaining value method are: order

Representative rounds downwards, if L 〉=2 ⁿ* 3, m=n+1 then, otherwise m=n, this B figure distance of another image in the head and the tail image is D ₂=L-D ₁

Images